Find answers from the community

Updated last year

How to use with Jupyter Notebook Getting

How to use with Jupyter Notebook? Getting error trying to load LlamaCPP

UnsupportedOperation Traceback (most recent call last)
c:\LLMs\SharikAI\My_llama_index.py in line 5
33 callback_manager = CallbackManager([llama_debug])
34 context_window=3900
----> 35 llm = LlamaCPP(
36 model_path='C:/LLMs/oobabooga_windows/text-generation-webui/models/openorca-platypus2-13b.Q4_K_M.gguf',
37 temperature=0.3,
38 max_new_tokens=512,
39 context_window=context_window,
40 generate_kwargs={"top_p": 1,},
41 model_kwargs={"n_gpu_layers": 32, "n_batch": 512, "n_threads": 10},
42 # callback_manager = callback_manager,
43 verbose=False,
44 messages_to_prompt=messages_to_prompt, # The function to convert messages to a prompt
45 completion_to_prompt=completion_to_prompt, # The function to convert a completion to a prompt.
46 )

File c:\LLMs\SharikAI.venv\Lib\site-packages\llama_index\llms\llama_cpp.py:110, in LlamaCPP.init(self, model_url, model_path, temperature, max_new_tokens, context_window, messages_to_prompt, completion_to_prompt, callback_manager, generate_kwargs, model_kwargs, verbose)
105 raise ValueError(
106 "Provided model path does not exist. "
107 "Please check the path or provide a model_url to download."
108 )
109 else:
--> 110 self._model = Llama(model_path=model_path, **model_kwargs)
...
370 else:
371 msg = "fileno"
--> 372 raise io.UnsupportedOperation(msg)

UnsupportedOperation: fileno
А
2 comments
It is because verbose=False
Add a reply
Sign up and join the conversation on Discord