Hey all, anyone here using `llama-

PPwnosaurusRex

Hey all, anyone here using llama-create for a starting point? I did for a quick test of making a backend and it worked great! I'm looking for some advice. What's the easiest way to add the nodes and corresponding metadata that was returned? I want to maintain the streaming response, and don't think I can modify index.as_chat_engine...Just build a custom chat message object like this?

https://docs.llamaindex.ai/en/stable/module_guides/deploying/chat_engines/usage_pattern/#low-level-composition-api

2 comments

LLogan M

the nodes should already be on the response object, but you might need to modify how the API endpoint responds in order to include it in the API response

PPwnosaurusRex

I'm still struggling here...to simplify what I'm after, I basically want to make an openai chat completions compatible endpoint, but have it use my vectordb...for /API/chat I can't figure out how to modify the out of the box one from llama-create to essentially emulate it. Any ideas? https://github.com/run-llama/create-llama/

Add a reply

Find answers from the community

Hey all, anyone here using `llama-