Find answers from the community

Updated 2 months ago

The only way to create the index in via

The only way to create the index in via openapi? or there are other tools that could be used for indexing
L
V
14 comments
Are there any free one that we could use? Sorry that might be obvious
@Logan M thanks for the help
Sadly, the only "free" ones are available from huggingface (I.e. the last two links I sent)

But this assumes you have the hardware needed to run the models, which in most cases is pretty expensive (I.e. you'd need a 3090 minimum to run a decent LLM at a good speed)
Here's a list of all the possible (paid) 3rd party options as well: https://langchain.readthedocs.io/en/latest/modules/llms/integrations.html
Ahhh that make sense now, I was searching a way around the opeapi, not to wrack up the price πŸ˜‚.
If it's just personal experimentation, OpenAI is not too bad. I think I spent about $20 last month lol
At scale, then things maybe get a little pricey
Honestly they should think of renaming their name to CloseAI πŸ˜‚
one question regarding the response, why does it sometime cut off like it doesn't finish the full sentence? Is there any way around that such that it finish the sentence or continues from that point onward?
By default, OpenAI sets the max_tokens to 256, you can change that like this: https://gpt-index.readthedocs.io/en/latest/how_to/custom_llms.html#example-changing-the-number-of-output-tokens-for-openai-cohere-ai21

Then, use a prompt helper to make sure you leave room for the adjusted output tokens (change num_output in this example): https://gpt-index.readthedocs.io/en/latest/how_to/custom_llms.html#example-changing-the-number-of-output-tokens-for-openai-cohere-ai21
(Under the hood, the OpenAI model just predicts words until it predicts a special "stop" word, or it reaches the max length)
thanks mate πŸ˜„
Add a reply
Sign up and join the conversation on Discord