Is there a way to add in the chat

Depends on how you want it to use the chat history

The query engine will always query the index. You could modify the prompt template to include the chat history before every query, but tbh I'm not sure how useful this is

Another option is using one of our new agents, which does a simple react loop using the new openai function calling api

If you just want it for retrieval/embeddings, there's an option I can explain for that as well

like if I want to have the bot be able to use the chat history to find relevant docs and answer the question being asked (so it probably will be important for both the retrieval and also just answering the question asked), basically like a chatbot

What approach would be appropriate for this usecase?