dev_blockchain

What is the format or type of embeddings

What is the format or type of embeddings we got from the OpenAI embeddings? like when we do embeddings using laama_index it provides multiple files docsore, vector and index. I want to read more about them and how they are related and is same approach openAI also using ?

5 comments

ddev_blockchain

Hey guys I am trying to index data which

Hey guys, I am trying to index data which size is around 500MB, is there any way I can do it without any error and i can save a little time(fast indexing), because doing embedding of this large is little hectic .

31 comments

ddev_blockchain

MosaicML as an llm model with llamaindex

Hey guys, has anyone here tried to implement the MosaicML as an llm model with llamaindex?

64 comments

ddev_blockchain

load multiple embeddings files

Hey guys is there a way we can load multiple embeddings files in one go, let's say my index_file is a kind of array or list or a directory.

Plain Text

index = GPTSimpleVectorIndex.load_from_disk(index_file, service_context)

5 comments

ddev_blockchain

Embed model error

Hey @Logan M , What I have done is, I created the embeddings using HuggingFaceEmbeddings and then I passed it to the openAI LLM model for query but it's throwing an error. Do we have any such scenario where we can create embeddings and save them into index.json and then pass this to openAi llm for query?

2 comments

ddev_blockchain

Import error

Hey guys, does anyone encounter this error

Plain Text

ImportError: cannot import name 'GPTSimpleVectorIndex' from 'llama_index' (/usr/local/lib/python3.10/dist-packages/llama_index/__init__.py)

1 comment

ddev_blockchain

Embeddings

Hey @Logan M , can you please help me here, I was searching so much around this. But did not get anything for this. This will really help.

10 comments

ddev_blockchain

```2023 04 28 13 57 57 117 build index

Plain Text

2023-04-28 13:57:57.117 > [build_index_from_nodes] Total LLM token usage: 0 tokens
2023-04-28 13:57:57.117 > [build_index_from_nodes] Total embedding token usage: 56150 tokens
2023-04-28 13:58:42.427 > [build_index_from_nodes] Total LLM token usage: 0 tokens
2023-04-28 13:58:42.427 > [build_index_from_nodes] Total embedding token usage: 56150 tokens
2023-04-28 13:59:45.027 > [query] Total LLM token usage: 6197 tokens
2023-04-28 13:59:45.027 > [query] Total embedding token usage: 6 tokens
2023-04-28 13:59:50.074 > [build_index_from_nodes] Total LLM token usage: 0 tokens
2023-04-28 13:59:50.075 > [build_index_from_nodes] Total embedding token usage: 56150 tokens

Why the code is sending embedding requests again and again.
I have passed my pdf and did the embedding but then while asking the question, it's also creating embedding. Any suggestion or any help

2 comments

ddev_blockchain

So it s like we can t work without

So, it's like we can't work without OPENAI_API_KEY, do we have any open source llm as language engine. ?

3 comments

Find answers from the community

What is the format or type of embeddings

Hey guys I am trying to index data which

MosaicML as an llm model with llamaindex

load multiple embeddings files

Embed model error

Import error

Embeddings

```2023 04 28 13 57 57 117 build index

So it s like we can t work without