context_window setting is crucial when working with LLMs in LlamaIndex. As you've noticed, it's often set to 4096, which is a common setting for models like GPT-3.5. However, it's important to adjust this setting based on the specific requirements of your application and the capabilities of the LLM you're using.context_window:from llama_index import (
KeywordTableIndex,
SimpleDirectoryReader,
ServiceContext,
)
from llama_index.llms import OpenAI
documents = SimpleDirectoryReader("data").load_data()
# set context window
context_window = 4096
# set number of output tokens
num_output = 256
# define LLM
llm = OpenAI(
temperature=0,
model="text-davinci-002",
max_tokens=num_output,
)
service_context = ServiceContext.from_defaults(
llm=llm,
context_window=context_window,
num_output=num_output,
)context_window is set to 4096, but you can adjust this value as needed for your specific use case.Index over your data objects (e.g., Document objects) to enable querying. LlamaIndex offers several different index types. A common one mentioned in the extracts is VectorStoreIndex.VectorIndexAutoRetriever class to create a retriever on top of a summarized metadata index.from llama_index import (
VectorStoreIndex,
SimpleDirectoryReader,
load_index_from_storage,
StorageContext,
)
# load documents
documents = SimpleDirectoryReader(
input_files=["paul_graham_essay.txt"]
).load_data()
# create index
index = VectorStoreIndex.from_documents(documents)
# define retriever
retriever = index.as_retriever(similarity_top_k=2)import chromadb
from llama_index import VectorStoreIndex
from llama_index.vector_stores import ChromaVectorStore
from llama_index.storage.storage_context import StorageContext
# initialize client
db = chromadb.PersistentClient(path="./chroma_db")
# get collection
chroma_collection = db.get_or_create_collection("quickstart")
# assign chroma as the vector_store to the context
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
# load your index from stored vectors
index = VectorStoreIndex.from_vector_store(
vector_store, storage_context=storage_context
)chunk_size for embeddings can depend on several factors, including the nature of your data, the specific LLM model you're using, and the capabilities of your vector database. However, the extracts provided do not specify an optimal chunk_size for the combination of OpenAI's text-embedding-3-large, gpt-4-turbo, and ChromaDB.chunk_size can lead to more granular embeddings, which might be beneficial for certain types of queries. On the other hand, a larger chunk_size can capture more context in each embedding, which might be beneficial for other types of queries.chunk_size when initializing a ServiceContext in LlamaIndex:from llama_index import ServiceContext from llama_index.llms import OpenAI # define LLM llm = OpenAI(model="gpt-4") # initialize service context with specific chunk size service_context = ServiceContext.from_defaults(llm=llm, chunk_size=400)
chunk_size is set to 400, but you can adjust this value as needed for your specific use case.chunk_size, you might want to refer to the LlamaIndex documentation () or conduct some experiments with your specific data and use case.text-embedding-3-large model through its OpenAIEmbedding class. text-embedding-3-large with LlamaIndex:from llama_index.embeddings import OpenAIEmbedding
# create embeddings
embed_model = OpenAIEmbedding(model="text-embedding-3-large")
embeddings = embed_model.get_text_embedding(
"Open AI new Embeddings models is great."
)text-embedding-3-large model from OpenAI. from llama_index import VectorStoreIndex, SimpleDirectoryReader
from llama_index.vector_stores import YourVectorStore
from llama_index.storage.storage_context import StorageContext
# Load documents
documents = SimpleDirectoryReader("./data").load_data()
# Define your transformer model (e.g., SentenceTransformer)
# and use it to create embeddings for your documents
# Create a vector store
vector_store = YourVectorStore(your_embeddings)
# Create a storage context
storage_context = StorageContext.from_defaults(vector_store=vector_store)
# Create a VectorStoreIndex from your documents
index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)YourVectorStore would be a vector store that uses your SentenceTransformer model to create embeddings. You would need to replace YourVectorStore and your_embeddings with the actual code that uses SentenceTransformer to create embeddings and a vector store.VectorStoreIndex and you're unable to find a specific string that you know is present, it could be due to the nature of vector-based search. Vector-based search is based on semantic similarity, not exact string matching. This means that it might not always return a document containing the exact string you're searching for, even if such a document exists in your datastore.# Assume `documents` is a list of all your documents
for doc in documents:
if 'your_string' in doc.text:
print(f"Found in document {doc.id}")VectorStoreIndex and other types of indexes in LlamaIndex, you can refer to the LlamaIndex documentation ().VectorStoreIndex, you can use the get_all_documents method. Here's a general example:# Assume `index` is your VectorStoreIndex documents = index.get_all_documents()
VectorStoreIndex.VectorStoreIndex is not provided in the extracts. For more detailed information, you should refer to the LlamaIndex documentation ().WhereDocument is a type of filter used in LlamaIndex to filter results based on certain conditions. However, the specific usage of WhereDocument is not provided in the extracts. WhereDocument in LlamaIndex, you should refer to the LlamaIndex documentation ().