Find answers from the community

Home
Members
heloisy
h
heloisy
Offline, last seen 3 months ago
Joined September 25, 2024
is everything ok with googledrivereader? because in january i use my credentials.json and everything is ok, now i run the same code and have this error:
raise ValueError("Client secrets must be for a web or installed app.") ValueError: Client secrets must be for a web or installed app.
10 comments
K
L
h
N
Hi, im trying do the document management, but dont works:
This is my code (i'm using ingestion pipeline, qdrant as vector store and redis as docstore, the docs are loading by a bucket):

def load_and_index(self): try: docs = self.loader() loaded_doc_ids = {doc.id_ for doc in docs} print(len(docs)) if docs: for doc in docs: nodes = self.pipeline.run(documents=[doc]) print(f"Ingestão de {len(nodes)} Nodes do documento: {doc.metadata['file_name']}") docstore_hashes = set(self.pipeline.docstore.get_all_document_hashes().keys()) print(f"Hashs docstore: {docstore_hashes}") docstore_ids = set(self.pipeline.docstore.get_all_ref_doc_info().keys()) print(f"IDs docstore: {docstore_ids}") docs_exclu = docstore_ids - loaded_docs_ids if self.exclude_docs: for doc_id in docs_exclu: self.pipeline.docstore.delete_document(doc_id) self.index.delete_ref_doc(doc_id, delete_from_docstore=True) print(f"Doc delete: {doc_id}")

I think the code is not working because when i print(f"IDs docstore: {docstore_ids}"), the output is: IDs docstore: set()
Any help?
21 comments
L
h
Hey, is it possible use retrieval with batch?
4 comments
L
W
h
Hi. I have a question for beginners. I was indexing a collection in Qdrant, everything was fine. I went to index a v2 collection, with the same data, but now with text as aggregated metadata, and I came across the following problem: the tokens in my embedding model reached their maximum. So I had a question: does the metadata linked to the document also become an embedding?
2 comments
h
W