LlamaIndex
Log in

Find answers from the community

Home
Members
Punkbit
P
Punkbit
Offline, last seen 3 months ago
Joined September 25, 2024
PPunkbit
·

Where do you get/import the correct type

Where do you get/import the correct type for QueryType?

Plain Text
from llama_index.schema import QueryType


I get the error

Plain Text
"QueryType" is unknown import symbol (reportGeneralTypeIssues)


Obs: I'm definitely not a Python developer, but Python is quite approachable
4 comments
L
P
PPunkbit
·

When using the `llama_index.llms import

When using the llama_index.llms import OpenAI how to set the log to "debug"?

For getting openai logs seen suggested openai.log = "debug", but doesn't work for my setup.

Here's a snippet:

Plain Text
...
from llama_index.llms import OpenAI
import openai

openai.log = "debug"
app = FastAPI()
loader = SitemapReader()

llm = OpenAI(temperature=0.1, model="gpt-3.5-turbo")
service_context = ServiceContext.from_defaults(llm=llm)


Followed the documentation, from most basics simply set an environment variable for OPEN AI api etc. Happens that I extended my setup to include OpenAI, as I wanted to modify the model, otherwise I wouldn't need it.

So, when someone says to openai.log = "debug", I wonder what they mean in llama index context or setup?
6 comments
L
W
P
T
PPunkbit
·

Apologies in advance, as my knowledge

Apologies in advance, as my knowledge regarding OpenAI, LLM, etc, are pretty poor. But as a software developer, given tools like Llama Index, it helps reduce friction.

I've done a few model tests to figure out which provides reasonable computation considering cost. I found that GPT-4 didn't work for me, because of the rate limit–I'd expect to have the ability to make a request at least–, and gpt-4-1106-preview worked but its "preview" with further limitations. In any case, GPT-4 is pricier to GPT-3.5. Although, GPT-3.5 doesn't seem to work that well.

So, this brings me into questioning how to determine the number of TPM my request requires. The use case is scrapping several pages with the help of SimpleWebPageReader, I believe that the number of URL sources might have a big role on this, correct?

For this reason, I'm thinking that it is best to create a single page with FAQ instead and put this instead of the multiple URL sources.

Any hints or suggestions to improve performance?
4 comments
P
T
PPunkbit
·

Regarding Open AI costs when llama index

Regarding Open AI costs when llama index data loading and responding. For my use case I'm using a Restful API to access handle requests.

I've done a quick test of llama index as a simple query endpoint, let's say:

Plain Text
def query(question: Union[str, None] = None):
  documents = SimpleWebPageReader(html_to_text=True)
         .load_data(["https://docs.foobar.com/some-knowledge"])
  index = SummaryIndex.from_documents(documents)
  query_engine = index.as_query_engine() 
  answer = query_engine.query(question)

  return { "answer": str(answer )}


It can be determined that for every GET query request there's an associated cost.

Since each computation over Open AI has a cost, I would like to know how costly the operation is for the query endpoint above.

I'm assuming that the SimpleWebPageReader.load_data and query question goes on a single request to Open AI and not two or more?
7 comments
L
P
T
PPunkbit
·

What type is returned for:

What type is returned for:

Plain Text
res = query_engine.query("My foobar?")


As a non-python dev, decided to do a quick restfyk API for llama index using fastapi.

Plain Text
def query(question: Union[str, None] = None):
  ...

  return { "answer": res }


👆 This causes an error ERROR: Exception in ASGI application, which I believe is related to the fn query return type as tested by hard typing "some random text" as res.

Did a quick check by doing a print for res

Plain Text
print(res)


Where I can assert a valid text response. Does anyone know what I need to look at?
4 comments
P
L
PPunkbit
·

Hey,

Hey,

I'm experimenting with "Replicate" to compare with the OpenAI 3.5 turbo, which seems quite reasonable. My use case is to provide a knowledge base and no prior knowledge to respond to queries.

Which model category should I look into to have the best experience for my use case?

Thank you!
10 comments
P
W
PPunkbit
·

When pairing `SimpleWebPageReader` and

When pairing SimpleWebPageReader and TreeIndex more specifically from_documents where the documents are the load_data from SimpleWebPageReader. Also, including that the query engine update_prompts use a custom template as follows:

Plain Text
query_engine.update_prompts({
  "response_synthesizer:text_qa_template": qa_template
})


Can the response/answer include the document URL where the summary is based from at all?
2 comments
P
L
PPunkbit
·

Custom Prompt

The SummaryIndex supports custom prompts?

In the docs we can find the following sentence "Some indices use different types of prompts during construction (NOTE: the most common ones, VectorStoreIndex and SummaryIndex, don’t use any).".

Let's assume that my business logic is the following:

Plain Text
...

index = SummaryIndex.from_documents(documents)
query_engine = index.as_query_engine()
answer = query_engine.query(question)

...


Can I use a custom prompt at all?
4 comments
P
E
PPunkbit
·

OpenAI

When using Llama Index, do we have control over the model used in Open AI? The reason I'd ask is because I'm clueless about which model my Llama Index setup is using. Is this managed on Open AI side or my application side? Thank you!
4 comments
P
W
PPunkbit
·

Regarding llama-hub packages, finding an

Regarding llama-hub packages, finding an empty package after install and couldn't determine the reason why.

Plain Text
cat /opt/homebrew/lib/python3.11/site-packages/llama_hub/web/sitemap/__init__.py


👆 This file is empty, given:

Plain Text
pip3 install llama-hub


Updated homebrew and upgraded python3 to double-check.
2 comments
P
L
PPunkbit
·

LlamaIndex is great Although have some

LlamaIndex is great! Although have some thoughts.

There are a lot of services online that provide a ChatGPT like experience for own documents, and I notice that all provide a different answer based on the same documentation sources. As most of this services seem to copy each other, and the pricing, thought they're using LlamaIndex or similar but yesterday, when indexed a bunch of markdown files noticed that LlamaIndex + GPT3 responds with what seem to be poorer answers. For example, my markdown has text description and source-code like this

Plain Text
cmd arg1 arg2


👆 Which is included in the answer, but I don't seem to get the same result with LlamaIndex.

How can I improve my LlamaIndex to be closer to the answer I get with third party services?
16 comments
c
P
L