How to use with Jupyter Notebook? Getting error trying to load LlamaCPP UnsupportedOperation Traceback (most recent call last) c:\LLMs\SharikAI\My_llama_index.py in line 5 33 callback_manager = CallbackManager([llama_debug]) 34 context_window=3900 ----> 35 llm = LlamaCPP( 36 model_path='C:/LLMs/oobabooga_windows/text-generation-webui/models/openorca-platypus2-13b.Q4_K_M.gguf', 37 temperature=0.3, 38 max_new_tokens=512, 39 context_window=context_window, 40 generate_kwargs={"top_p": 1,}, 41 model_kwargs={"n_gpu_layers": 32, "n_batch": 512, "n_threads": 10}, 42 # callback_manager = callback_manager, 43 verbose=False, 44 messages_to_prompt=messages_to_prompt, # The function to convert messages to a prompt 45 completion_to_prompt=completion_to_prompt, # The function to convert a completion to a prompt. 46 ) File c:\LLMs\SharikAI.venv\Lib\site-packages\llama_index\llms\llama_cpp.py:110, in LlamaCPP.init (self, model_url, model_path, temperature, max_new_tokens, context_window, messages_to_prompt, completion_to_prompt, callback_manager, generate_kwargs, model_kwargs, verbose) 105 raise ValueError( 106 "Provided model path does not exist. " 107 "Please check the path or provide a model_url to download." 108 ) 109 else: --> 110 self._model = Llama(model_path=model_path, **model_kwargs) ... 370 else: 371 msg = "fileno" --> 372 raise io.UnsupportedOperation(msg) UnsupportedOperation: fileno
Add a reply
Sign up and join the conversation on Discord
Join on Discord