Maker

Code

Hi, I am loading GitHub repos into llamaindex and using GPTVectorStoreIndex for indexing. I am trying to get the GPT to help me answer questions about the code. I am getting okay results, but it could be a lot better. It works well if the question about the code is present in the documentation, but outside of documented code it isn't doing a good job.

Would you happen to know if there is a better way of doing this? 🙏

I thought ListIndex might be better, but it takes 5+ minutes to get an answer and I am also running into context limit errors, I would probably have to use some larger 32K context model for example. But that would get quite expensive.

6 comments

MMaker

Insert

would appreciate the input of the man, the legend

2 comments

MMaker

Max tokens

Hi! 🙂

I am running into an issue where if I set service_context=service_context in query_engine like so:

Plain Text

llm_predictor = ChatGPTLLMPredictor(llm=ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo", streaming=False))
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor)
    query_engine = index.as_query_engine(text_qa_template=CHAT_QA_PROMPT,
                                         refine_template=CHAT_REFINE_PROMPT,
                                         similarity_top_k=3,
                                         streaming=False, 
                                         service_context=service_context)

then the chatGPT does NOT have access to General Knowledge.

However, when I do NOT set service_context=service_context in query_engine like so:

Plain Text

    query_engine = index.as_query_engine(text_qa_template=CHAT_QA_PROMPT,
                                         refine_template=CHAT_REFINE_PROMPT,
                                         similarity_top_k=3)

then I do have access to General Knowledge, but the ChatGPT response is getting cut off when it writes longer text response.

How do I achieve both access to General Knowledge AND no cut off of longer text responses?

Thank you!

8 comments

MMaker

LLM Fine-tuning

Hi,

I followed the LLM fine-tuning guide https://gpt-index.readthedocs.io/en/latest/examples/finetuning/openai_fine_tuning.html, but I used my own query engine

Plain Text

query_engine = general_index.as_query_engine(text_qa_template=CHAT_TEXT_QA_PROMPT, refine_template=CHAT_REFINE_PROMPT, similarity_top_k=10, streaming=False, service_context=service_context, node_postprocessors=[rerank])

I attached a screenshot with the finetuning_events.jsonl results. My question is, are these results okay? Should the

{"messages": [{"role": "user", "content": "You are an expert Q&A system that strictly operates in two modeswhen refining existing answers:\n1. **Rewrite** an original answer using the new context ....

be present in the finetuning_events.jsonl ? Is this correct or wrong?

Thank you!

3 comments

MMaker

Templates

Hi I am migrating my code to newest llama index version and getting an error ValueError: Invalid message type: <class 'langchain.schema.messages.SystemMessage'>

I believe this is due to the query_engine = index.as_query_engine(text_qa_template=CHAT_QA_PROMPT, refine_template=CHAT_REFINE_PROMPT I assume this is outdated code. What would be the newer way of doing this? (see screenshot) thank you! 👍 @Logan M

10 comments

MMaker

Migratw

Hi, I updated to the latest llamaindex and I am having trouble migrating my code. This was my code before migration:

Plain Text

llm_predictor = ChatGPTLLMPredictor(llm=ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo-0613", streaming=False, max_tokens=1000))
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper, callback_manager=callback_manager)

and now this is example code that is giving me an error after migrating to the latest llamaindex version:

Plain Text

llm = OpenAI(temperature=0, model="gpt-3.5-turbo")
service_context = ServiceContext.from_defaults(llm=llm)
storage_context = StorageContext.from_defaults(persist_dir="./storage")
index = load_index_from_storage(storage_context)
query_engine = index.as_query_engine()
response = query_engine.query("hi")
print(response)

The error I get is AttributeError: 'ServiceContext' object has no attribute 'llm' What am I doing wrong here please? 😅 I tried following this guide but unsuccessfully https://gpt-index.readthedocs.io/en/latest/how_to/customization/llms_migration_guide.html

9 comments

MMaker

Mmr or not

Hi @Logan M,

I am torn between using query_engine = index.as_query_engine(vector_store_query_mode="mmr") or not. Sometimes I get better answer with vector_store_query_mode="mmr" and sometimes I get better answer without it.

Is it possible for me to route the query to both query engines (one with mmr and one without) and have the LLM decide which answer is better and output that?

The only thing I found in the documentation is this https://gpt-index.readthedocs.io/en/latest/examples/query_engine/RouterQueryEngine.html but this has me specifying "description" over the query engine, but the description would be the same, so it doesn't fit what I am looking for.

Thank you 🙂

3 comments

MMaker

System prompt

hi, please how do I specify a base prompt? I want to specify the chatGPT and give it some information on what it's name is, how to act etc. I found this example in the docs, but I assume this is not what I am looking for, since when I specify the context_str variable I get an error, so I guess there is another way to do this? Thanks https://gpt-index.readthedocs.io/en/latest/how_to/customization/custom_prompts.html

6 comments

Find answers from the community

Code

Insert

Max tokens

LLM Fine-tuning

Templates

Migratw

Mmr or not

System prompt