The only way to create the index in via

Llama Index supports a ton of 3rd-party APIs (cohere, etc.), just have to pass in the LLM like we currently do with OpenAI: https://gpt-index.readthedocs.io/en/latest/how_to/custom_llms.html#example-changing-the-underlying-llm

Also, there is support for custom embedding models: https://gpt-index.readthedocs.io/en/latest/how_to/embeddings.html#custom-embeddings

And any custom LLM: https://gpt-index.readthedocs.io/en/latest/how_to/custom_llms.html#example-using-a-custom-llm-model

Are there any free one that we could use? Sorry that might be obvious

@Logan M thanks for the help

Sadly, the only "free" ones are available from huggingface (I.e. the last two links I sent)

But this assumes you have the hardware needed to run the models, which in most cases is pretty expensive (I.e. you'd need a 3090 minimum to run a decent LLM at a good speed)

Here's a list of all the possible (paid) 3rd party options as well: https://langchain.readthedocs.io/en/latest/modules/llms/integrations.html

Ahhh that make sense now, I was searching a way around the opeapi, not to wrack up the price 😂.

If it's just personal experimentation, OpenAI is not too bad. I think I spent about $20 last month lol

At scale, then things maybe get a little pricey

Honestly they should think of renaming their name to CloseAI 😂

one question regarding the response, why does it sometime cut off like it doesn't finish the full sentence? Is there any way around that such that it finish the sentence or continues from that point onward?

By default, OpenAI sets the max_tokens to 256, you can change that like this: https://gpt-index.readthedocs.io/en/latest/how_to/custom_llms.html#example-changing-the-number-of-output-tokens-for-openai-cohere-ai21

Then, use a prompt helper to make sure you leave room for the adjusted output tokens (change num_output in this example): https://gpt-index.readthedocs.io/en/latest/how_to/custom_llms.html#example-changing-the-number-of-output-tokens-for-openai-cohere-ai21

(Under the hood, the OpenAI model just predicts words until it predicts a special "stop" word, or it reaches the max length)

thanks mate 😄