How to use with Jupyter Notebook? Getting error trying to load LlamaCPP
UnsupportedOperation Traceback (most recent call last)
c:\LLMs\SharikAI\My_llama_index.py in line 5
33 callback_manager = CallbackManager([llama_debug])
34 context_window=3900
----> 35 llm = LlamaCPP(
36 model_path='C:/LLMs/oobabooga_windows/text-generation-webui/models/openorca-platypus2-13b.Q4_K_M.gguf',
37 temperature=0.3,
38 max_new_tokens=512,
39 context_window=context_window,
40 generate_kwargs={"top_p": 1,},
41 model_kwargs={"n_gpu_layers": 32, "n_batch": 512, "n_threads": 10},
42 # callback_manager = callback_manager,
43 verbose=False,
44 messages_to_prompt=messages_to_prompt, # The function to convert messages to a prompt
45 completion_to_prompt=completion_to_prompt, # The function to convert a completion to a prompt.
46 )
File c:\LLMs\SharikAI.venv\Lib\site-packages\llama_index\llms\llama_cpp.py:110, in LlamaCPP.init(self, model_url, model_path, temperature, max_new_tokens, context_window, messages_to_prompt, completion_to_prompt, callback_manager, generate_kwargs, model_kwargs, verbose)
105 raise ValueError(
106 "Provided model path does not exist. "
107 "Please check the path or provide a model_url to download."
108 )
109 else:
--> 110 self._model = Llama(model_path=model_path, **model_kwargs)
...
370 else:
371 msg = "fileno"
--> 372 raise io.UnsupportedOperation(msg)
UnsupportedOperation: fileno
Add a reply
Sign up and join the conversation on Discord