Find answers from the community

Updated 2 months ago

bigcode/starencoder · Hugging Face

Hi there!
I am thinking of doing RAG plugin on VS Code. There is the question of embedding choice for code and retrieval strategy.
Do you guys have good feedback on using RAG for code and what model / retrieval were used? I saw that HF has an embedding for code for instance but wondering if there are other smaller options https://huggingface.co/bigcode/starencoder

8 comments

TTeemu

Not sure about the best open source model for code but Llamaindex does have the CodeSplitter for better code chunking:

https://docs.llamaindex.ai/en/stable/core_modules/data_modules/node_parsers/usage_pattern.html#text-splitter-customization

https://github.com/run-llama/llama_index/blob/main/llama_index/text_splitter/code_splitter.py

DDHuynh95

Cool!

DDHuynh95

Is there an end to end example of querying a codebase with it?

LLogan M

Not that I've seen actually. Chunking is one step, but querying a code base can be quite difficult from what I've seen 🤔

DDHuynh95

I see! I think I can get something quick and dirty running and see how it works 😄
Have you seen good resource on the topic?

LLogan M

Sweep has some interesting blogs on the subject

Here's one example

https://docs.sweep.dev/blogs/chunking-improvements

DDHuynh95

Super cool! Will try to hack something this weekend

DDHuynh95

Thanks @Logan M 🙂

Add a reply