If index was created with one llm model, and if you load that index with another llm model, would it work? I have some indexes that are mixed up.
Directly looking at the json files, there seems to be no way of knowing which llm model they correspond to. I tried to load those indexes, and tried things like index._service_context.llm_predictor.get_llm_metadata() etc, but they dont seem to indicate which LLM the index was created with.
On a prior thread there, it was suggested that I could define the llm and create a custom service object with that llm and load it into the service object while loading index from storage Like index._service_context.llm_predictor.get_llm_metadata()
While I am doing that, i see error like below (likely because this index was created with another llm than the current one)
File "/home/paperspace/wynk/wynkenv/lib/python3.9/site-packages/torch/nn/functional.py", line 2515, in layer_norm return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled) RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'
Yeah, based on the info you shared on other thread for printing LLM name from the index, I realised the same.
Took me a while to figure this out but incase this helps anyone
1) index generated from one model can be used with another as long as the embedding model used in query and index are the same. 2) the LLM present in the service context when you load an index from storage, is the default LLM of llama index. 3) The LLM isn't stored as meta data in index. 4) If you want to use the same LLM you while querying, as you used for index, create that LLM, create a service context with it, and pass that service context as Param while loading the index from storage.
Thanks @Logan M for answering my many dumb question and helping me figure these out across multiple threads.