can somebody throw some light on the

ssjhakab

can somebody throw some light on the architecture that should be used to build a chatbot that reads data from JSON file that contains instances metadata, provides relevant info to user when qn asked for that aws account?

4 comments

WWhiteFang_Jr

To read data from a JSON file, You can use the loader from llama-hub: https://llamahub.ai/l/file-json
You can add more metadata to the documents. Here is the ref to Document customization: https://docs.llamaindex.ai/en/stable/core_modules/data_modules/documents_and_nodes/usage_documents.html#customizing-documents
Currently Llamaindex is available in Python and Typsescript. Here is the documentation ref: https://docs.llamaindex.ai/en/stable/index.html
Opensource models requires GPU. So you'll have to make sure you have good amount of GPU to run LLMs. LlamaIndex suppport opensource as well as closed LLMs. Ref guide can be found here for LLM setup: https://docs.llamaindex.ai/en/stable/core_modules/model_modules/llms/root.html

ssjhakab

@WhiteFang_Jr , thanks a lot for replying.
but if i have to do qn answering over large dataset then how is that possible?
bcz i got this openai for obvious reasons
raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x13b10d460 state=finished raised RateLimitError>]

WWhiteFang_Jr

You may have set chunk size to a very small value. Try with Default i.e 1024.
As LLM make multiple calls during response synthesizing process.

ssjhakab

@WhiteFang_Jr can you advice the architecture or workflow to be adopted to get the following.
"the question can be natural language like how many distinct albs are present and for each alb what is the count of instance."
the bot/ system should be able to analyze the metadata of account which is stored in json or any other file and give responses for the questions asked in natural language.
The file might be updated day by day or something like that.

Add a reply

Find answers from the community

can somebody throw some light on the