Version

At a glance

The community member is creating a multi-tool agent with 3 different tools, using the microsoft/Phi-3-mini-4k-instruct model and the models/gemini-1.5-flash model. They are using llama_index Vllm on top of the HuggingFace language model for faster inference and optimized memory utilization. However, they are facing issues when deploying the agent using Gradio, as there is a version conflict that prevents the installation of llama-index-llms-huggingface. The community member has raised an issue on the llama_index repository and is seeking help from others who have worked on similar tasks.

In the comments, another community member thanks the original poster for the quick reply on Sundays and expresses hope that the version conflicts will be resolved soon. Additionally, a comment mentions that a pull request (https://github.com/run-llama/llama_index/pull/17433) will fix the issue after it is merged.

Useful resources

aanujsahani01

https://github.com/run-llama/llama_index/issues/17428

Hi Folks,
A brief overview of what i am trying to do:
I am making a multi-tool agent which will be having 3 different tools as of now, 2 are QueryEngineTool and 1 is FunctionTool .
I am using open source model -- microsoft/Phi-3-mini-4k-instruct model using llama_index HuggingFace and models/gemini-1.5-flash model.

HuggingFace model will be used by the tool to perfrom the given task.
Gemini -- will be used by the llama_index agent.

For faster inference and optimized memory utiluzation i am using llama_index Vllm on top of huggingface llm.

At the end i am using A CustomReAct agent toperform the given task.

Everything is working fine but the issue arrises when i am deploying it using gradio. There is a version conflict while installing gradio it is not able to find any suitable version, i tried using poetry as well but that too is not working.
I have also raised an issue of llama_index git, if anyone has worked on any such similar task please help me out with this.

I have attached a link that gives a detailed explanation about the version conflict.
Thanks for your time.

2 comments

aanujsahani01

Thank you for your quick reply on Sundays.
I hope you resolve the version conflicts soon so that llama-index-llms-huggingface can be installed without any issues.

LLogan M

After this merges it'll be fixed
https://github.com/run-llama/llama_index/pull/17433

Add a reply

Find answers from the community

Version