Find answers from the community

Updated 2 years ago

Api key

Plain Text

 index = GPTVectorStoreIndex.from_documents(documents, service_context=service_context)

Is it possible to use this function without api key

8 comments

LLogan M

Yea, but you'll need a local LLM and local embedding model

https://gpt-index.readthedocs.io/en/latest/how_to/customization/custom_llms.html#example-using-a-huggingface-llm

https://gpt-index.readthedocs.io/en/latest/how_to/customization/embeddings.html#custom-embeddings

PPup

can you explain more how we define both of them in local ?

LLogan M

I think both those links cover it well. There's even two example notebooks for the huggingface llm there

We are so releasing a new tutorial notebook tomorrow, keep your eyes peeled 👀

Anything in particular that's confusing?

PPup

I am following the second link but the error show require openai key

LLogan M

Yea, you need to provide both an llm_predictor and an embed_model, otherwise the one you don't provide defaults to an openai model

LLogan M

If you only want to retrieve relevant nodes, and not send text to an LLM, set your embed model but also set your openai key to something random

Then you can do this

Plain Text

query_engine = index.as_query_engine(response_mode="no_text")
response = index.query(...)
print(response.source_nodes)

This will show the nodes that would have been sent to the llm, without sending them

PPup

Thanks you now I can run it however it have warning that

Plain Text

Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.

LLogan M

Yup, that's fine. That just means it's generating until it predicts a special token or it runs out of room (which is normal)

Add a reply