Find answers from the community

Updated 3 months ago

Was it able to understand column values

Was it able to understand column values or just doing embedding focused processing
L
H
10 comments
There as a pandas index you might be interested in!
https://github.com/jerryjliu/llama_index/blob/main/examples/struct_indices/PandasIndexDemo.ipynb

Otherwise, you might have to do some pre-processing on the csv/excel file text to make it easier to understand by the LLM (like splitting each row into a document, but include the headers as sort of section labels?)
Thanks Logan I will take a look, is there a way to combine the model so that the answer would not ONLY based on the text provided?
For that, and since you are wanting a chatbot, using llama index as a tool in langchain will enable that

Basically, langchain let's the model decide when to use the tool. So normal conversation will flow as normal
thanks Logan, is there an example for that as how to do it? for example I have a big document about say Enya (well i actually do its all her lyrics lol https://enyacore.com/), but if i ask a question then the chat says, there is no content about this in the document when it is public knowledge
Thanks Logan!

One more question, is there a way to summarize tabular data or llama index or gpt is just language embedding model and does not apply to numeric data
Ooo interesting. It might depend on the data. How would you personally summarize a table? Like a description of what the table is?
Ye think excel highest mean average etc etc but maybe that’s not for language model haha
I do have a table that’s descriptions of items and their costs. I might experiment to see what comes out of that
Yea the pandas and sql indexes provide some basic text2sql or text2pandas capabilities, so you could ask those questions directly. Might be helpful!

I have a cool demo of this actually: https://github.com/logan-markewich/llama_index_starter_pack/tree/main/streamlit_sql_sandbox
Add a reply
Sign up and join the conversation on Discord