Find answers from the community

Updated last year

How to use with Jupyter Notebook Getting

At a glance

The community member is encountering an UnsupportedOperation error when trying to use the LlamaCPP library with Jupyter Notebook. The comments suggest that the issue is caused by setting verbose=False, and that the community member should set verbose=True when using Jupyter Notebook, as mentioned in a related GitHub issue.

Useful resources
How to use with Jupyter Notebook? Getting error trying to load LlamaCPP

UnsupportedOperation Traceback (most recent call last)
c:\LLMs\SharikAI\My_llama_index.py in line 5
33 callback_manager = CallbackManager([llama_debug])
34 context_window=3900
----> 35 llm = LlamaCPP(
36 model_path='C:/LLMs/oobabooga_windows/text-generation-webui/models/openorca-platypus2-13b.Q4_K_M.gguf',
37 temperature=0.3,
38 max_new_tokens=512,
39 context_window=context_window,
40 generate_kwargs={"top_p": 1,},
41 model_kwargs={"n_gpu_layers": 32, "n_batch": 512, "n_threads": 10},
42 # callback_manager = callback_manager,
43 verbose=False,
44 messages_to_prompt=messages_to_prompt, # The function to convert messages to a prompt
45 completion_to_prompt=completion_to_prompt, # The function to convert a completion to a prompt.
46 )

File c:\LLMs\SharikAI.venv\Lib\site-packages\llama_index\llms\llama_cpp.py:110, in LlamaCPP.init(self, model_url, model_path, temperature, max_new_tokens, context_window, messages_to_prompt, completion_to_prompt, callback_manager, generate_kwargs, model_kwargs, verbose)
105 raise ValueError(
106 "Provided model path does not exist. "
107 "Please check the path or provide a model_url to download."
108 )
109 else:
--> 110 self._model = Llama(model_path=model_path, **model_kwargs)
...
370 else:
371 msg = "fileno"
--> 372 raise io.UnsupportedOperation(msg)

UnsupportedOperation: fileno
А
2 comments
It is because verbose=False
Add a reply
Sign up and join the conversation on Discord