The community member was using ollama for property graph construction, but it was too slow. They then got an exl2 quant, installed exllama, and installed tabbyAPI. When they tried to use llm = OpenAILike(model="text2cypher-codestral-exl2-4.0bpw", api_base="http://127.0.0.1:5000/v1/", api_key="fake", temperature=0.1, top_p=0.1, top_k=40, repetition_penalty=1.18), only the temperature parameter was passed, and the model ran forever. They also had issues with kg_extractor = SchemaLLMPathExtractor() and index = PropertyGraphIndex.from_documents(), where the code ran forever without making any API calls. A community member suggested using additional_kwargs={...} for other parameters.
I was using ollama to do the property graph construction, but it was too slow. I got an exl2 quant, installed exllama, and installed tabbyAPI. when I try to use llm = OpenAILike(model="text2cypher-codestral-exl2-4.0bpw", api_base="http://127.0.0.1:5000/v1/", api_key="fake", temperature=0.1, top_p=0.1, top_k=40, repetition_penalty=1.18), it only passes the temperature, nothing else. also, it seems that the model runs forever when using this (if I say what is your name, it will start asking itself multiple questions after that; I assume this is a settings issue?? I did not have this problem with ollama).
index = PropertyGraphIndex.from_documents( documents, embed_model=embed_model, kg_extractors=[kg_extractor], property_graph_store=graph_store, show_progress=True, ) it just runs forever and no api calls are made.
anyone have any experience with this openailike endpoint?
