Find answers from the community

Updated 2 years ago

I also just wanted to start a discussion

I also just wanted to start a discussion about the new ChatGPT LLM predictor, it seems like even with temperature 0 it seems unreliable for use in gpt-index's query pipelines, what's the plan for this in the future? https://github.com/jerryjliu/gpt_index/issues/590 Is this something that others have noticed too? Are there any things I can change (q&a prompt, etc) that might help?

11 comments

jjerryjliu0

yeah it's something we're taking a look at - have you been able to find any prompt improvemetns ?

jjerryjliu0

we might start looking at model-dependent prompts

KKaveen

Unfortunately no not yet, it's seeming really difficult to get chatgpt to be friendly in the query pipeline

KKaveen

especially if we're looking through multiple nodes like >3

KKaveen

I think so, any chatgpt prompt I made broke davinci3 prompting

KKaveen

Attachments

KKaveen

some more examples of the gpt turbo behavior

KKaveen

The main issue seems to be that if the refinement step doesn't improve the answer it just loses the information and doesn't echo it back (the already-good answer from the last step)

jjerryjliu0

yeah

jjerryjliu0

this seems to be a pretty common issue

jjerryjliu0

chatgpt drops more context than text-davinci-003

Add a reply