Find answers from the community

H
HeyC
Offline, last seen 3 months ago
Joined September 25, 2024
H
HeyC
·

0.8.x

Embeddings Fine-tuning. issue when generating dataset.

Hi. I decided to use the "outdated" approach for generating the dataset to be used for embeddings fine-tuning:

https://github.com/run-llama/finetune-embedding/blob/main/generate_dataset.ipynb

I am getting the error
module 'openai' has no attribute 'error'
when performing the step :
train_queries, train_relevant_docs = generate_queries(train_corpus)

I am aware that the newest guidelines are available at
https://docs.llamaindex.ai/en/latest/examples/finetuning/embeddings/finetune_embedding/
but I like more the idea of being able to get control over the prompt (not using ENG in my case), which is not anymore there.

Any idea which version could help running it smoothly ?
The llama-index version I am using is 0.8.5.post2.

The specific error is thrown at

......./llama_index/llms/openai_utils.py:119, in _create_retry_decorator(max_retries) 111 max_seconds = 10 112 # Wait 2^x * 1 second between each retry starting with 113 # 4 seconds, then up to 10 seconds, then 10 seconds afterwards 114 return retry( 115 reraise=True, 116 stop=stop_after_attempt(max_retries), 117 wait=wait_exponential(multiplier=1, min=min_seconds, max=max_seconds), 118 retry=( --> 119 retry_if_exception_type(openai.error.Timeout)

Thanks.
6 comments
H
L