Flask: Background thread sees a non-empty queue as empty Flask: Background thread sees a non-empty queue as empty flask flask

Flask: Background thread sees a non-empty queue as empty


From the Things to Know documenation page:

uWSGI tries to (ab)use the Copy On Write semantics of the fork() call whenever possible. By default it will fork after having loaded your applications to share as much of their memory as possible. If this behavior is undesirable for some reason, use the lazy option. This will instruct uWSGI to load the applications after each worker’s fork(). Lazy mode changes the way graceful reloading works: instead of reloading the whole instance, each worker is reloaded in chain. If you want “lazy app loading”, but want to maintain the standard uWSGI reloading behaviour, starting from 1.3 you can use the lazy-apps option.

Your Flask app is started when uWSGI starts, then the one worker process is forked. On forking, the Queue object is empty, and no longer shared with the original process. The thread isn't taken along.

Try setting the lazy-apps option to delay the loading of the Flask app until the worker is started.


The documentation link in @Martijn Pieters' answer notes that lazy-apps may consume more memory than preforking. If you're concerned about this, you may also wish to consider the @postfork decorator to have more granular control over what gets run after forking. You could create your Queue inside a @postfork-decorated function and it will get created in each worker.