Find answers from the community

Home
Members
redlizard
r
redlizard
Offline, last seen 3 months ago
Joined September 25, 2024
HI, I've been having trouble getting the persist method to work - I keep getting permission denied errors. i'm able to write text files to the directory just fine, just can't save my vector_store, any suggestions?

from llama_index import SimpleDirectoryReader, VectorStoreIndex

documents = SimpleDirectoryReader(".").load_data()
index = VectorStoreIndex.from_documents(documents)

index.vector_store.persist(persist_path=".")
1 comment
L
i had this working using the openAI api, but now want to move to using a local model - i see different references in the docs so wasn't sure exactly how to point to the local model. I think I'm supposed to set my service_context to 'local' but it's not clear how I then point it to the specific model. This is what I have now... Advice?

#specify a local model
from llama_cpp import llama, Llama
from llama_index import ServiceContext
llm = Llama(model_path=r'C:\Users\erraballiv\Downloads\llama-2-7b.Q5_K_S.gguf')
service_context = ServiceContext.from_defaults(llm='local')



storage_context = StorageContext.from_defaults(persist_dir=r"C:\Users\erraballiv\PycharmProjects\Ll-index-ex1")
loaded_index = load_index_from_storage(storage_context)
query_engine = loaded_index.as_query_engine()
16 comments
L
r