Parsing

iisr43l86

hello... greetings. I have a problem when I make an analysis of several documents using SubQuestionQueryEngine and Llama2-7b model by meta. Can someone give an idea of what is wrong with my code. The error i got is the following
"
JSONDecodeError Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/llama_index/output_parsers/utils.py in parse_json_markdown(text)
43 try:
---> 44 json_obj = json.loads(json_string)
45 except json.JSONDecodeError as e_json:

17 frames
JSONDecodeError: Extra data: line 15 column 1 (char 326)

During handling of the above exception, another exception occurred:

ScannerError Traceback (most recent call last)
ScannerError: while scanning for the next token
found character '`' that cannot start any token
in "<unicode string>", line 15, column 1:
```
^
"

4 comments

LLogan M

Yea this is an error in parsing whatever the model wrote

We did make some upgrades to this parsing recently, but at the end of the day it depends on the LLM to write proper json

Which version of llama-index do you have? You could try updating to make sure you have those improvements I mentioned?

If that doesn't work, it basically means the LLM is just deciding not to output json at all 😅

iisr43l86

Thanks a lot @Logan M i have "llama-index 0.8.5.post2" version btw. if you know any version that you can recommend me to use with llama2 I would appreciate it. For now I will try with openai as in the tutorial.

LLogan M

Yea I can try debugging myself locally, but I suspect that this LLM is just not well suited to the selector task 😅

iisr43l86

thanks... i'm using the model "meta-llama/Llama-2-7b-chat-hf" of llama loaded from "HuggingFaceLLM" btw... please try it out 🙂

Add a reply

Find answers from the community

Parsing