Find answers from the community

Updated last year

here my actual test code i want to limit

At a glance

here my actual test code, i want to limit the GPT responses for only my docs, for example if i ask what is vesuvio? it response me with a correct response but it was not in my docs

16 comments

LLogan M

Make sure you pass in the service context when loading from disk

query_engne = load_index_from_storage(storage_context, service_context=service_context).as_query_engine()

You LLM definition is not quite correct. Temperature can only be between 0 and 1, and you are using the incorrect langchain class for gpt-3.5. Try this instead

Plain Text

from llama_index.llms import OpenAI
_llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.5, model="gpt-3.5-turbo", max_tokens=num_outputs))

ssl33p

really thanks for the help Logan

ssl33p

i will try this now

ssl33p

i really appreciate that

LLogan M

If it still doesn't help, we can set an extra system prompt in the llm predictor to try and control the LLM a bit more 🙂

ssl33p

here is my actual code i have mesh yours suggestions + gpt4 + custom prompt and now work as expected

ssl33p

if you have time and want to suggest me any ideas to improve that i would offer you a coffee hahahha

ssl33p

the next steps i will follow are:

ssl33p

online db for index (mongo + policone)

ssl33p

build rest api with flask

LLogan M

Heads up, are you using the LLM class from llama-index? The correct kwarg in that case is model="gpt-4" not model_name="gpt-4"

ssl33p

do frontend with react

ssl33p

ok sure thanks

ssl33p

what model was the app using with the wrong args?

LLogan M

it was defaulting to gpt-3.5

ssl33p

Add a reply