I tried asking KapaGPT,
https://discord.com/channels/1059199217496772688/1194708617564270704, for help and it told me to reach out to the maintainers, so, I'm getting this error when trying to use local LLMs for loading an index, the index persistent directory has been deleted, and then remade using the local models, but when I try to run the exact same code again to access the index I get this:
Traceback (most recent call last):
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\llms\utils.py", line 29, in resolve_llm
validate_openai_api_key(llm.api_key)
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\llms\openai_utils.py", line 379, in validate_openai_api_key
raise ValueError(MISSING_API_KEY_ERROR_MESSAGE)
ValueError: No API key found for OpenAI.
Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization.
API keys can be found or created at https://platform.openai.com/account/api-keys
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "s:\local-indexer\flask_server.py", line 53, in <module>
index = load_index_from_storage(storage_context)
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\loading.py", line 33, in load_index_from_storage
indices = load_indices_from_storage(storage_context, index_ids=index_ids, **kwargs)
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\loading.py", line 78, in load_indices_from_storage
index = index_cls(
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\vector_store\base.py", line 52, in __init__
super().__init__(
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\base.py", line 62, in __init__
self._service_context = service_context or ServiceContext.from_defaults()
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\service_context.py", line 178, in from_defaults
llm_predictor = llm_predictor or LLMPredictor(
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\llm_predictor\base.py", line 109, in __init__
self._llm = resolve_llm(llm)
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\llms\utils.py", line 31, in resolve_llm
raise ValueError(
ValueError:
******
Could not load OpenAI model. If you intended to use OpenAI, please check your OPENAI_API_KEY.
Original error:
No API key found for OpenAI.
Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization.
API keys can be found or created at https://platform.openai.com/account/api-keys
To disable the LLM entirely, set llm=None.
******I followed the tutorials on the docs for all of this:
embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
set_global_tokenizer(
AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-chat-hf").encode
)
model_url = "{url}"
llm = LlamaCPP(
model_url=model_url,
temperature=0.1,
max_new_tokens=256,
context_window=3900,
generate_kwargs={},
model_kwargs={"n_gpu_layers": 41},
messages_to_prompt=messages_to_prompt,
completion_to_prompt=completion_to_prompt,
verbose=True,
)
service_context = ServiceContext.from_defaults(
llm=llm,
embed_model=embed_model,
)
PERSIST_DIR = "storage-data"
if not os.path.exists(PERSIST_DIR):
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents, service_context=service_context)
index.storage_context.persist(persist_dir=PERSIST_DIR)
else:
storage_context = StorageContext.from_defaults(persist_dir=PERSIST_DIR)
index = load_index_from_storage(storage_context)