Hello everyone I m doing a simple QA

jjoao_pt

Hello everyone, I'm doing a simple QA using below structure:
0) "Context text: (45,000 characters)"

1) Indexing context text using:
-- -- llama_index.indices.list.base.ListIndex (GPTListIndex)
-- -- service_context: OpenAI("GPT-3.5-Turbo") + callback_manager(LlamaDebugHandler)
-- -- Creating a query engine with QuestionAnswerPrompt + RefinePrompt

2) With the created Query Engine on point 1):
-- performing a query

The response is not working as well as I wished in some cases.
In order to understand better I tried to dig into the structure and debug. One of the things that jumped into my attention is that the response has the nodes sent to the GPT (14 nodes) and they don't have any score on any of the nodes.
So the question is... is it normal that (llama_index NodeWithScore) doesn't contain any score in any of the nodes returned?

8 comments

LLogan M

yea that's normal. You are using a list index, so there is no similarity search being done (and therefore no scoring)

A list index always sends every node to the LLM

jjoao_pt

@Logan M thanks for your reply!
So I guess I would need to use for example: VectorStore or KnowledgeGraph, ...

Any hint on which could be better for my case?

I'm doing: "structured text extraction" from a Medical Text.

LLogan M

If you are doing structured extraction, maybe running a pydantic program is better? Like, if you know what you want to extract ahead of time, you can setup a class that defines that info, and then run the program over chunks of your text
https://github.com/jerryjliu/llama_index/blob/main/docs/examples/output_parsing/openai_pydantic_program.ipynb

Only downside is the process is a little manual. But we did this to extract info on issues from our github haha
https://github.com/jerryjliu/llama_index/blob/main/docs/examples/usecases/github_issue_analysis.ipynb

jjoao_pt

I tried with pydantic... somehow the query_engine was getting better results

jjoao_pt

but thanks for the hints!

jjoao_pt

I'm getting a nice 94% score of what I want to get and what I'm getting from LLM

jjoao_pt

just trying to improve on the 6 remaining %

jjoao_pt

saw as well a possibility to try:

Query Transformation
PostProcessor

Add a reply

Find answers from the community

Hello everyone I m doing a simple QA