LLM

I think together.ai interface is currently not available in llamaindex.

https://docs.llamaindex.ai/en/stable/module_guides/models/llms.html#modules
You can still connect with Together.ai services using the custom LLM class
Just setup the llm and pass it to the service_context https://docs.llamaindex.ai/en/stable/module_guides/models/llms/usage_custom.html#example-using-a-custom-llm-model-advanced

There should be an integration for it soon in the next few days! 👍

Nice!!! @Logan M 🧀🐟 working hard this holiday. We appreciate it

Being will be able to replace ollama run ... with together.ai

Any other integration we can expect ? Together
▶ Anyscale?
▶ Perplexity?
▶ DeepInfra?

It seems like together has the best combo of price , developer experience and speed.

We already have anyscale and perplexity actually

OctoAI seems to have good prices and 10$ free credits (https://octoai.cloud/). Is in your plan to integrate with their API?

Is this merged ?

It has not. Actually, I don't even see the PR anymore lol

But I need to do it anyways before Jan 3rd lol

@Logan M happy new year.

when using the query engine llama pack with ollama we just need to pul the model we want to use only. We dont need the command ollama run mistral correct?

Happy new year!

Yea I think that's correct? It should pull the model if you havent used it already.... I think. I'm not an ollama expert lol

to use these instead of ollama its possibel using the Ollama Query Engine Pack. Is there any guide or documentation? gave away my mac and my laptop is taking 30s on dolphin-phi model...

😢

It should be possible! I think mistral will just automatically download right?

no i ment how do i use perplexity with query engine pack. so the documents local but the llm is run on the cloud from perplexity...

Oh perplexity!

You have to setup the llm, and throw it into the service context

https://docs.llamaindex.ai/en/stable/examples/llm/perplexity.html#setup-llm

Probably will have to edit the pack code to do that

ollama_pack = OllamaQueryEnginePack(model="dolphin-phi", documents=documents)


if we use remote provider aka model="perplexity-mistral-7b-instruct"

and also provide the api key above ...

Maybe im confused lol. Does ollama talk to remote providers? I thought it was all local?

yes its all local. and i was hoping to do the rag example using remote providers instead...

so local document folder for rag and remote provider for the llm

The link I shared above shows how to setup the llm

I really think it's worth learning how to set up the llm and embeddings and service context yourself, and creating the query engine 🙏

We have extensive docs on how to do it 👍

https://github.com/run-llama/llama-hub/tree/8fa06e01f1ae72722cd77372fc1cf23e4ef37e08/llama_hub/llama_packs/ollama_query_engine

ok got it will look into how this works and see if its possible to do local doc with remote provider.