Llamaindex workflow compatibility with Redpanda and Confluent

I've never used redpanda -- not sure what that would look like

It implements the Kafka protocol but is might lighter weight.

C++ and Raft impl of Kafka standard.

hmm, workflows don't use a message queue. Its all just single-process asyncio queues 👀

redpanda could be a llama-deploy integration in the future though, if the existing kafka message queue doesn't work with it

Ok

So do you need to install Kafka broker for it to work?

What's the minimum setup for Llama workflow that uses Kafka ?

I don't see any reason for a docker deployment why RedPanda could not substitute here https://github.com/run-llama/llama_deploy/blob/main/docker/docker-compose-kafka.yml

Because it implements the Kafka protocol. Can you please check this. Thanks

I meant the code under the hood here is using kafka pypi libraries
https://github.com/run-llama/llama_deploy/blob/3d156c89599b1be6bfedc943adc54ebda2fb18d7/llama_deploy/message_queues/apache_kafka.py#L112

assuming the protocol is the same, i'd assume it just works 🤷

Yes, that's what I am hearing from RedPanda as well. I think it should work