I have, what I think is a fairly simple use case yet I am struggling to find a way to implement this with llamaindex.
I want to query (JSON) data that is being returned from a fixed set of APIs. I have found that the JSONalyzer Query Engine works pretty well for the querying part. I tested this by first
manually calling one specific API and then passing the resulting JSON to the QueryEngine. I can then successfully chat with the QueryEngine. This is akin to the example in the documentation where the json_list is hard coded (the only difference is that I am getting the JSON from an API endpoint).
This seems to be a common theme on
every example I see on the
https://docs.llamaindex.ai which starts with "here's the data you want to query", and that data is effectively fixed (.pdf document, hard coded JSON etc.)
What I want to do is, depending on the question the user asks, have llamaindex determine which API to call and then pass the resulting JSON to the JSONalyzer Query Engine so that I can chat with it. I do not, under any circumstances, want llamaindex to pass the JSON to the LLM as it contains proprietary information (and it might be too large for the LLM Context window). The JSONalyzer Query Engine beautifully circumvents this by only sending the DB Schema to the LLM and ask it to generate a SQL Query, which it then runs on the in-memory DB it created.
More than anything, I think I am getting overwhelmed by the multitude of components I can pick from. It seems this is a great use case for a (custom) Agent, that has a set of tools that can run the APIs and retrieve the JSON, but then how do I instruct it to pass that JSON to the JSONalyzer Query Engine so that I can query that data?
At the moment, I am focusing on just 1 API call actually, so the use case is pretty simple: When the user asks a question:
- Call the API and return the JSON
- instantiate the JSONalyzer Query Engine with that JSON
- Ask the question and return a response
- goto 1
