Hi Everyone, Currently I’m solving the

At a glance

The community member is working on a use case where they can chat with a BigQuery database. They are using the deepseek-coder-6.7B-instruct-GGUF model and have provided a code snippet of their implementation. Another community member has commented that the LLM setup seems to be not quite configured properly, as the response is including [INST] tokens. There is no explicitly marked answer.

SSristi Agrawal

Hi Everyone, Currently I’m solving the use case where you can chat with your bigquery database. I’m using deepseek-coder-6.7B-instruct-GGUF model.

Here’s the snippet of my implementation

from llama_index.indices.struct_store import SQLTableRetrieverQueryEngine
from llama_index import VectorStoreIndex

obj_index = ObjectIndex.from_objects(
table_schema_objs,
table_node_mapping,
VectorStoreIndex,
service_context=service_context
)

query_engine = SQLTableRetrieverQueryEngine(
sql_database, obj_index.as_retriever(similarity_top_k=1), service_context=service_context,
)

response = query_engine.query("List all the unique countries present in item master.")

1 comment

LLogan M

How did you setup your LLM? seems like its not quite configured properly if its writing [INST] tokens in the response

Add a reply

Find answers from the community

Hi Everyone, Currently I’m solving the