Add support for multiple LLMs simultaneously for cost savings and expanding to a team-level automation (and beyond)

[AlexanderDorofeev]

Dear OpenHands Team,

I am truly impressed by the OpenHands project compared to Devin (YouTube real demos). You've once again demonstrated the power of the open-source community.

I'd like to propose an enhancement that aligns OpenHands with real-world development workflows. Currently, the project focuses on single-developer automation. I believe it would be beneficial to expand its vision to accommodate team-level automation, IT business units, and IT business domains.

Here's my proposal:

Implement a team of agents with different expertise and model sizes:
Utilize inexpensive, even local models like Qwen 2.5 Coder (which is excellent at 14B parameters) for simple tasks
Escalate to more advanced models when issues arise or for review purposes
Develop an automated and manual escalation workflow:
Drive digital labor from lower-tier models to higher-tier models
Allow manual triggering of high-tier models for assistance or specific domain expertise
Accommodate cross-functional teams:
Set up agents using different models fine-tuned for particular expertise
Explore the use of other modalities
Currently, we can only configure one model, and there's no option to set up a team/hierarchy of agents. While GPT-40-mini works well for me, there's vast room for improvement in the current implementation.

Key benefits of this approach:

Cost savings through efficient resource allocation
Enhanced flexibility to handle various task complexities
Better alignment with real-world team structures and workflows
I believe this enhancement would significantly boost the project's capabilities and appeal to a broader audience. Please consider implementing this feature to take OpenHands to the next level.

Thank you for your dedication to this amazing project!

Best regards, Alex

[AlexanderDorofeev]

Hi Friends!
There is another opportunity to introduce several models simultaneously. It would be great to use reasoner model ChatGpt O1\Deepseek R1 for task planning and decomposition and "action model" like claude-3.5-sonnet to do actual coding

[enyst]

Thank you for the feedback and ideas! Yes, we currently have two proposals to introduce a planning/action paradigm in the execution flow:

• Setup model routing config and plan routing to o1 #6189
• [Feat] Exploring AgentDelegation through Supervisor Agent #4449

To note,

Currently, we can only configure one model, and there's no option to set up a team/hierarchy of agents.

This limitation exists only on the web UI. The backend supports multiple models, and they can be configured one per agent. There's a lot more to be done to make it flexible and easy to use in real-world settings, though.

[AlexanderDorofeev]

Awesome news. Thank you for sharing. I'm not a python developer, but I'm sure that I can use AllHands as a tool to create custom build that suits my needs

[bogzbonny]

This is a great idea, checkout https://github.com/getAsterisk/deepclaude I'd love to try out a sonnet + r1 combination in openhands