Hi everyone, I have a huge knowledge

mmrcngrcki

Hi everyone, I have a huge knowledge base about different business processes in confluence. The trick is that it's in Polish. I did a simple llamaindex code to do embedding with openai ada002 and asking gpt4 to answer but it still hallucinates sometimes or cannot answer correctly, especially when it comes to abbreviations (very often it translates it to english and then again to polish which doesn't make any sense). What solution would you reccomend to test out? Other embedding model? Better data retrieval? (i'm using confluence connector from llama-hub)

9 comments

TTeemu

Here you can see the embedding models ranking for Polish, just change the language selector: https://huggingface.co/spaces/mteb/leaderboard

It doesn't include ada-002 though which is also multilingual

You can also play around with the language of the prompts you're using, it can confuse the model sometimes when they're in a different language

Maybe you can debug by printing out the response source nodes and seeing whether it's a retrieval or synthesis issue

jjeffreyip

I build this thing that allows you to iterate towards the optimal hyperparameters with greater clarify, thought it would be useeful (https://github.com/confident-ai/deepeval) (its free)

mmrcngrcki

Thanks a lot 🙂 I tried with intfloat/multilingual-e5-large but after sending question, I'm getting "ValueError: shapes (1536,) and (1024,) not aligned: 1536 (dim 0) != 1024 (dim 0)". I guess there's simple solution but i tried to search online and didn't find any answer. I'm using gpt-4 llm

TTeemu

Yeah 1536 are the embedding dimensions for ada-002, 1024 is for the open source one you're using. Are you trying to use your OpenAI ones?

mmrcngrcki

Plain Text

        embed_model = HuggingFaceEmbedding(model_name="intfloat/multilingual-e5-large")

        # llm model
        print('defining llm model')
        llm_predictor = LLMPredictor(llm=ChatOpenAI(temperature=0, model_name="gpt-4", streaming=True))
        service_context = ServiceContext.from_defaults(
            llm_predictor=llm_predictor,
            embed_model=embed_model
        )


    confluence_index = VectorStoreIndex.from_documents(docs, service_context=service_context, show_progress=True)
    confluence_index.set_index_id("vector_index")
    confluence_index.storage_context.persist(data_path)



    QA_TEMPLATE = Prompt(TEMPLATE_STR)

    print('defining chat engine')
    storage_context = StorageContext.from_defaults(persist_dir=data_path)
    index = load_index_from_storage(storage_context)
    chat_engine = index.as_chat_engine(
        chat_mode='condense_question',
        streaming=True,
        text_qa_template=QA_TEMPLATE
    )

    while True:
        print('waiting for question...')
        message = input()
        if message == 'RESET':
            chat_engine.reset()
        else:
            response = chat_engine.chat(message)
            print(response)

mmrcngrcki

I'm using open source embedding and then openai gpt-4 LLM, is it ok?

mmrcngrcki

bump

TTeemu

Yeah that's fine, you just need to make sure you're loading your new embeddings instead of ones made with OpenAI

mmrcngrcki

but I am, aren't I? in this example I'm saving it to local directory and reading it again

Add a reply

Find answers from the community

Hi everyone, I have a huge knowledge