TokenCountingHandler
in LlamaIndex. This handler tracks each token usage event in an object called a TokenCountingEvent
, which includes the prompt string sent to the LLM or Embedding model.print("prompt: ", token_counter.llm_token_counts[0].prompt[:100], "...\n")
token_counter.llm_token_counts[0].prompt
is the prompt string sent to the LLM. The [:100]
is used to print the first 100 characters of the prompt, and you can adjust this number based on your needs.token_counter
object properly set up and filled with token counting events for this to work.load_data
function. You can use the SimpleDirectoryReader
to load documents from a directory. Here is an example:from llama_index import SimpleDirectoryReader documents = SimpleDirectoryReader('./path_to_your_markdown_files').load_data()
VectorStoreIndex.from_documents
method. This method takes in the documents and builds an index over them. Here is an example:from llama_index import VectorStoreIndex index = VectorStoreIndex.from_documents(documents)
query_engine.query
method. This method takes in a query string and returns the response from the index. Here is an example:query_engine = index.as_query_engine() response = query_engine.query("Your question here") print(response)
'./path_to_your_markdown_files'
with the actual path to your markdown files and "Your question here"
with the actual question you want to ask.DocArrayInMemoryVectorStore
or DocArrayHnswVectorStore
provided by LlamaIndex. DocArrayInMemoryVectorStore
stores documents in memory and is a great starting point for small datasets, where you may not want to launch a database server. Here is an example of how to use it:from llama_index import GPTVectorStoreIndex, SimpleDirectoryReader, Document from llama_index.vector_stores import DocArrayInMemoryVectorStore from llama_index.storage.storage_context import StorageContext # load documents documents = SimpleDirectoryReader("../data/paul_graham").load_data() # Initialization and indexing vector_store = DocArrayInMemoryVectorStore() storage_context = StorageContext.from_defaults(vector_store=vector_store) index = GPTVectorStoreIndex.from_documents(documents, storage_context=storage_context)
DocArrayHnswVectorStore
is a lightweight Document Index implementation that runs fully locally and is best suited for small- to medium-sized datasets. It stores vectors on disk in hnswlib, and stores all other data in SQLite. Here is an example of how to use it:from llama_index import GPTVectorStoreIndex, SimpleDirectoryReader, Document from llama_index.vector_stores import DocArrayHnswVectorStore from llama_index.storage.storage_context import StorageContext # load documents documents = SimpleDirectoryReader("../data/paul_graham").load_data() # Initialization and indexing vector_store = DocArrayHnswVectorStore(work_dir="hnsw_index") storage_context = StorageContext.from_defaults(vector_store=vector_store) index = GPTVectorStoreIndex.from_documents(documents, storage_context=storage_context)
"../data/paul_graham"
with the path to your markdown files.