Question for anyone who uses Flask or

At a glance

The community members are discussing the correct way to share a mutable resource, such as an index, in a Flask (or similar Python API framework) application. The main points are:

- The community member wants to have an API with two endpoints: one for querying the index, and one for inserting documents into the index. They need to ensure that the insert operations are handled sequentially to avoid race conditions.

- Some community members suggest using a mutex to prevent race conditions, but others mention that in-memory mutexes may not work in a threaded Flask application. Instead, they recommend using a file-based mutex for a single server instance, or a distributed mutex (e.g., Redis) or centralized mutex (e.g., Postgres advisory locks) for a more general solution.

There is no explicitly marked answer in the comments.

Useful resources

LLogan M

Question for anyone who uses Flask (or similar python API frameworks).

What's the correct way to share a mutable resource? I.e. if I create an index, how do I let API calls insert documents synchronously? Or am I overthinking this and it's not a problem to let multiple threads insert into a global index?

I can think of some hacky solutions, but nothing feels "correct" lol

6 comments

LLogan M

Hmm yea, that makes sense 🤔 The use case is I want to have an example API with two endpoints, query and insert, where each endpoint performs either operation on an existing index that is in memory already.

But if two insert requests come in, I need to ensure those are handled sequentially, whereas a query can happen whenever since it isn't modifying the index

LLogan M

Ah, this seems to be the best way to do it. Yay more config lol https://stackoverflow.com/questions/28423069/store-large-data-or-a-service-connection-per-flask-session/28426819#28426819

wwalid

@Logan M Why not add a mutex over the index handler to prevent race conditions?

LLogan M

@walid people on stackoverflow where mentioning those wont work when an application/server is threaded or something? Something related to how memory is allocated in flask

If you use gunicorn, there's this preload option that makes it work, but I wanted a solution a bit more general

wwalid

I have little Flask knowledge but in general an in-memory mutex won't work but you can either do a mutex on a file if you have one server instance, or ultimately, use a distributed mutex (Redis supports this) or centralized mutex (like Postgres advisory locks)

wwalid

So it really depends on your setup.

If you have one instance of a Flask server running, try doing a mutex over a file

Add a reply

Find answers from the community

Question for anyone who uses Flask or