Find answers from the community

Updated 2 years ago

Aside from making the storage larger and

At a glance

Aside from making the storage larger and slower to perform, I'm not seeing an accuracy change up or down - but I'm also lacking metrics to say definitively, or if there's under-the-hood processes that would take advantage of both sources of data in different ways.

6 comments

LLogan M

I think throwing everything into a single vector store should work fine. But I would make sure the q/a pairs are clearly structured as documents

Something like

Plain Text

Example Question: ...
Answer: ...

LLogan M

That's just my initial impression though lol

FFred Bliss

Yep - I basically have a column for the question, and one for the answer.

Are you saying to actually structure them as documents as in llamaindex document objects?

FFred Bliss

and yeah, my impression is somewhere around the same. it's duplicating text in different ways, but it seems to make it better able to generalize rather than trying to match the exact question/answer format when the user asks it an open ended question

FFred Bliss

maybe somewhat akin to creating different indices for different types of queries in a database? duplicative but helpful in the end

LLogan M

Yea, I think structuring them a bit will help the LLM understand better what it's looking at 🤔

Add a reply