Find answers from the community

Updated last year

Hi everyone, how can I use API-URL as a

At a glance

The community member is asking how to use an API-URL as a Large Language Model (LLM) instead of using the Hugging Face library. The comments suggest that the community member can wrap the API requests with a custom LLM, as shown in an example from the LlamaIndex documentation. Another community member confirms that the API-URL can be used in place of the "dummy_response" in the example, and that the community member would need to make an API request using a library like requests. However, there is a question about whether an LLM is required for building the service context, and how the data would look like if an LLM is not used.

Useful resources
Hi everyone, how can I use API-URL as a LLM? Normally this is a post requests. How can I work on this instead of huggingface library model calling?
L
A
7 comments
Hi @Logan M, thanks for reaching out. Here in the place of dummy_response I can give my API-URL, right?
I.e dummy_response= 'HTTP://23.34.56:8001/model'
Am I following right?
Exactly, you would make an api request there
requests.post(url) or what ever your api needs
Thanks, will check that..
@Logan M Btw LLM is not required for building service_context right? If we use LLM how it will develop context, if not used how data will looks like?
Add a reply
Sign up and join the conversation on Discord