@kapa.ai
Hey! I'm currently working on a chat app. App is based on websockets (fastapi lib).
I have a problem with paralelism. Once I call one instance the second one is waiting for the first one to be done.
I've looked around but I don't seem to find a solution. Tried to implement
instead of the sync one -
,
but that does not seem to work for me either (I straight up could not get astream_chat to work). I wonder if this is why it waits for the first instance to be done or not.
Anybody has any idea? (As a model wrapper I use Ollama - it is not single threaded, tried it in cmd)
