Can you give an example using Llama index and querying ...

TTennisPrestigious624

Can you give an example using Llama index and querying a single GPTVectorStoreIndex using gpt-3.5-turbo and using memory

13 comments

kkapa.ai

Please stand by @TennisPrestigious624 as I search the knowledge base. We are experiencing especially high traffic right now and might take longer than usual. :wave:

kkapa.ai

----------------------
Sure! Here's an example of using LlamaIndex with GPTVectorStoreIndex and gpt-3.5-turbo. This example uses an in-memory vector store, which is the default option for GPTVectorStoreIndex.

First, you need to import the necessary libraries and set up logging:

Plain Text

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

from llama_index import GPTVectorStoreIndex, SimpleDirectoryReader, LLMPredictor, ServiceContext
from langchain.chat_models import ChatOpenAI

Next, load the documents using SimpleDirectoryReader:

Plain Text

documents = SimpleDirectoryReader('data').load_data()

Create an LLM Predictor with gpt-3.5-turbo and a ServiceContext:

Plain Text

llm_predictor = LLMPredictor(llm=ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo", streaming=True))
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, chunk_size_limit=512)

Build the GPTVectorStoreIndex using the documents and the service_context:

Plain Text

index = GPTVectorStoreIndex.from_documents(documents, service_context=service_context)

Finally, query the GPTVectorStoreIndex:

Plain Text

query_engine = index.as_query_engine()
response = query_engine.query("What did the author do growing up?")
print(response)

This example is based on the information provided in the SimpleIndexDemo-ChatGPT (https://gpt-index.readthedocs.io/en/latest/examples/customization/llms/SimpleIndexDemo-ChatGPT.html) notebook.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------

TTennisPrestigious624

where is the memory?

TTennisPrestigious624

@kapa.ai where is the memory?

kkapa.ai

Let me check the knowledge base for you. One moment! :books:

kkapa.ai

----------------------
In the example provided, the memory refers to the in-memory vector store used by the GPTVectorStoreIndex. By default, GPTVectorStoreIndex uses an in-memory SimpleVectorStore, which is initialized as part of the default storage context. You don't need to explicitly specify it in the code, as it's used automatically when creating the index.

Here's the relevant code snippet from the example:

Plain Text

index = GPTVectorStoreIndex.from_documents(documents, service_context=service_context)

This information is based on the Vector Store Guide (https://gpt-index.readthedocs.io/en/latest/how_to/index_structs/vector_store_guide.html) and the SimpleIndexDemo-ChatGPT (https://gpt-index.readthedocs.io/en/latest/examples/customization/llms/SimpleIndexDemo-ChatGPT.html) notebook.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------

TTennisPrestigious624

@kapa.ai I meant chat memory, like using redis to remember previous messages in the conversation, how can I do that?

kkapa.ai

Let me check the knowledge base for you. One moment! :books:

kkapa.ai

----------------------
Hmm, I don't know enough to give you a confident answer yet about using Redis for chat memory with LlamaIndex. However, you can refer to the RedisIndexDemo (https://gpt-index.readthedocs.io/en/latest/examples/vector_stores/RedisIndexDemo.html) for an example of using Redis as a vector store with LlamaIndex. This might help you get started with integrating Redis for chat memory.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------

LLogan M

If you want chat memory, right now you'll have to use langchain

Simple/low level example
https://github.com/jerryjliu/llama_index/blob/main/examples/langchain_demo/LangchainDemo.ipynb

High level wrappers
https://gpt-index.readthedocs.io/en/latest/guides/tutorials/building_a_chatbot.html

TTennisPrestigious624

Thanks @Logan M I did tried that, but I would like to use the index 100% of time, and with the tools I just can't ensure it will used all the time, any ideas? so yeah Im currently building it on LangChain:

Plain Text

    template = system_prompt + """\n

    {chat_history}
    Human: {human_input}
    AI:"""

    prompt = PromptTemplate(
        input_variables=["chat_history", "human_input"],
        template=template
    )

    redis_url = os.getenv("REDIS_URL", "redis://localhost:6379/0")
    redis_memory_ttl = os.getenv("REDIS_MEMORY_TTL", 864000)

    message_history = RedisChatMessageHistory(session_id,
                                              ttl=redis_memory_ttl,
                                              url=redis_url,
                                              key_prefix="chat_session:")
    memory = ConversationBufferMemory(return_messages=True, memory_key="chat_history", chat_memory=message_history)

    chat_llm = ChatOpenAI(model_name=llm_model_name, temperature=temp, max_tokens=MAX_TOKENS)

    llm_chain = LLMChain(
        llm=chat_llm,
        prompt=prompt,
        verbose=True,
        memory=memory,
    )

    result = llm_chain.run({"human_input": query})

But I am still missing the index there, not sure how to add it.

LLogan M

Yea I've seen this issue with langchain come up before.

With a langchain agent, you could change the agent prefix to include instructions to always use a certain tool. Not 100% garunteed though

Some point soon, llama index will have a better builtin solution for this though 🙏

TTennisPrestigious624

I will have a look into that.
Yeah it seems like a very common use case, thanks for your help and work on LlamaIndex

Add a reply

Find answers from the community

Can you give an example using Llama index and querying a single GPTVectorStoreIndex using