How will I do that for llama2, will I

hey no silly question here:

you can try using huggingface or replicate for open source llms

you'd need to get an api key in order to use their services, but after you have that, you can use our LLM integrations for either of the two

• https://replicate.com/meta/llama-2-70b-chat (in llama-index, use with Replicate)
• https://huggingface.co/meta-llama/Llama-2-70b-hf (in llama-index, use with HuggingFaceInferenceAPI)

What does replicate do exactly?

Is it a cloud compute service for LLMs?

Thanks for replying btw

Yup, they're an inference service for LLMs https://replicate.com/

they even offer fine-tuning services too

so instead of hitting openai's API for example to query an LLM, you can use replicate in similar ways to query an open-sourced LLM of your choosing (by providing a model name to any of the models they support)