I ve been using the GPTWeaviateIndex

bbradcohn

I've been using the GPTWeaviateIndex previously but after my Weaviate database was wiped I've been unable to reupload everything. Curious if this is a bug or I'm just trying to upload too large of an index at one time. Here's the error I'm getting.

Plain Text

File ~/projects/GPeaT/backend/venv/lib/python3.8/site-packages/requests/sessions.py:701, in Session.send(self, request, **kwargs)
    698 start = preferred_clock()
    700 # Send the request
--> 701 r = adapter.send(request, **kwargs)
    703 # Total elapsed time of the request (approximately)
    704 elapsed = preferred_clock() - start

File ~/projects/GPeaT/backend/venv/lib/python3.8/site-packages/requests/adapters.py:578, in HTTPAdapter.send(self, request, stream, timeout, verify, cert, proxies)
    576     raise SSLError(e, request=request)
    577 elif isinstance(e, ReadTimeoutError):
--> 578     raise ReadTimeout(e, request=request)
    579 elif isinstance(e, _InvalidHeader):
    580     raise InvalidHeader(e, request=request)

ReadTimeout: HTTPSConnectionPool(host='hv6yv9irrmanj7ha36vffa.gcp-b.weaviate.cloud', port=443): Read timed out. (read timeout=60)

5 comments

bbradcohn

My index when stored locally is about 100mb

bbradcohn

Is there some way I can chunk the upload to Weaviate? If I upload one or two documents at a time it seems to be successful.

bbradcohn

Scratch that, even when I make my batches much smaller they still don't upload with this error:

bbradcohn

Plain Text

File ~/projects/GPeaT/backend/venv/lib/python3.8/site-packages/weaviate/batch/crud_batch.py:1012, in Batch._flush_in_thread(self, data_type, batch_request)
    992 """
    993 Flush BatchRequest in current thread/process.
    994 
   (...)
   1008     The request response and number of items sent with the BatchRequest as tuple.
   1009 """
   1011 if len(batch_request) != 0:
-> 1012     response = self._create_data(
   1013         data_type=data_type,
   1014         batch_request=batch_request,
   1015     )
   1016     return response, len(batch_request)
   1017 return None, 0

File ~/projects/GPeaT/backend/venv/lib/python3.8/site-packages/weaviate/batch/crud_batch.py:662, in Batch._create_data(self, data_type, batch_request)
    660             break
    661 except RequestsConnectionError as conn_err:
--> 662     raise RequestsConnectionError("Batch was not added to weaviate.") from conn_err
    663 except ReadTimeout:
    664     message = (
    665         f"The '{data_type}' creation was cancelled because it took "
    666         f"longer than the configured timeout of {self._connection.timeout_config[1]}s. "
    667         f"Try reducing the batch size (currently {len(batch_request)}) to a lower value. "
    668         "Aim to on average complete batch request within less than 10s"
    669     )

ConnectionError: Batch was not added to weaviate.

RRadoslav Kuchár

any luck solving this issue, please?

Add a reply

Find answers from the community

I ve been using the GPTWeaviateIndex