LLM
class or CustomLLM
for a simpler interface. You will be responsible for passing the text to the model and returning the newly generated tokens. Here is an example of how to do this using a locally running facebook/OPT model and Huggingface’s pipeline abstraction:``
This example uses the
CustomLLM class to define a new LLM that uses a locally running model. The
complete method is responsible for passing the text to the model and returning the newly generated tokens. The
stream_complete method is not implemented in this example.
Please note that you may have to adjust the internal prompts to get good performance. Even then, you should be using a sufficiently large LLM to ensure it’s capable of handling the complex queries that LlamaIndex uses internally, so your mileage may vary.
For more details, you can refer to the **LlamaIndex documentation** (<https://gpt-index.readthedocs.io/en/latest/how_to/customization/custom_llms.html>).
----------------------
Note: you can ask me a follow up question by @ mentioning me again` :speech_balloon: