Is there a way to use a raptor tree in

You can certainly download the llama-pack for raptor and edit to your needs (this is actually the intention behind llama-packs 💪 )

Thanks @Logan M. I saw this example
https://github.com/run-llama/llama_index/blob/main/llama-index-packs/llama-index-packs-raptor/examples/raptor.ipynb

And was wondering if I should only replace the vector_store with a Vector3B or are there other recommended ways to filter based on meta data? Also I would need to decompose the query first to understand what filter should be applied in the meta data.
I assume it should be a common case so I was wondering if there are any recommendations or best practices

You could swap in the vector store with any vector store integration that support metadata filtering yea

To infer filters on the fly, you'd have to throw something like the auto retriever in, but this is somewhat less reliable (and requires a smart LLM)