Find answers from the community

Updated 3 months ago

some code i used ```# define llm with

some code i used
Plain Text
# define llm with HuggingFaceInferenceAPI
llm = HuggingFaceInferenceAPI(
    model_name="HuggingFaceH4/zephyr-7b-beta",
    token=os.environ.get("HUGGINGFACE_ACCESS_TOKEN")
)

# create QdrantClient with the location set to ":memory:", which means the vector db will be stored in memory
vectordb_client = qdrant_client.QdrantClient(location=":memory:")

# create QdrantVectorStore using QdrantClient and the collection name "wonderful_life"
vector_store = QdrantVectorStore(
    client=vectordb_client, collection_name="wonderful_life"
)

# create StorageContext object using the QdrantVectorStore
storage_context = StorageContext.from_defaults(vector_store=vector_store)

node_parser = SentenceSplitter()

service_context = ServiceContext.from_defaults(
    llm=llm,
    embed_model="local:WhereIsAI/UAE-Large-V1",
    callback_manager=callback_manager
)

# Build agents dictionary
query_engine_tools = []

for idx, wiki_title in enumerate(wiki_titles):
    nodes = node_parser.get_nodes_from_documents(city_docs[wiki_title])

    if not os.path.exists(f"./data/{wiki_title}"):
        # build vector index
        vector_index = VectorStoreIndex(
            nodes, service_context=service_context, callback_manager=callback_manager,storage_context=storage_context,
        )
        vector_index.storage_context.persist(persist_dir=f"./data/{wiki_title}")
    else:
        vector_index = load_index_from_storage(
            StorageContext.from_defaults(persist_dir=f"./data/{wiki_title}"),
            service_context=service_context,
            callback_manager=callback_manager,
        )
    # define query engines
    vector_query_engine = vector_index.as_query_engine()
L
a
6 comments
HuggingfaceInferenceAPI does not have achat() implemented, but llm_compilier requires it in order to run planned tasks concurrently
Solution is to use an API that supports async
Sorry for too many messages, i tried llm = OpenAI(model="gpt-4") but got error:
Plain Text
AttributeError                            Traceback (most recent call last)
<ipython-input-8-ed33bf2e2643> in <cell line: 1>()
----> 1 response = agent.chat(
      2     "Tell me about the demographics of Miami, and compare that with the demographics of Chicago?"
      3 )
      4 print(str(response))

25 frames
/usr/local/lib/python3.10/dist-packages/llama_index/vector_stores/qdrant.py in aquery(self, query, **kwargs)
    663             return self.parse_to_query_result(response[0])
    664         else:
--> 665             response = await self._aclient.search(
    666                 collection_name=self.collection_name,
    667                 query_vector=query_embedding,

AttributeError: 'NoneType' object has no attribute 'search'
you did not provide an async client to qdrant

qdrant has an annoying thing that when using :memory: , the sync and async clients do not share data, which makes using it with :memory: very tricky
maybe use chroma
Awesome, added chromadb and it worked. Thank you very much for all your help. Please forgive for spamming the board with too many questions. Have a Good Day!!
Add a reply
Sign up and join the conversation on Discord