Python async and CPU-bound tasks? Python async and CPU-bound tasks? flask flask

Python async and CPU-bound tasks?


It should be thread-safe to do something like the following to separate cpu intensive tasks into asynchronous threads:

from threading import Threaddef send_async_email(msg):    mail.send(msg)def send_email(subject, sender, recipients, text_body, html_body):    msg = Message(subject, sender = sender, recipients = recipients)    msg.body = text_body    msg.html = html_body    thr = Thread(target = send_async_email, args = [msg])    thr.start()

IF you need something more complicated, then perhaps Flask-Celery or Multiprocessing library with "Pool" might be useful to you.

I'm not too familiar with gevent though I can't imagine what more complexity you might need or why.

I mean if you're attempting to have efficiency of a major world-website, then I'd recommend building C++ applications to do your CPU-intensive work, and then use Flask-celery or Pool to run that process. (this is what YouTube does when mixing C++ & Python)


How about simply using ThreadPool and Queue? You can then process your stuff in a seperate thread in a synchronous manner and you won't have to worry about blocking at all. Well, Python is not suited for CPU bound tasks in the first place, so you should also think of spawning subprocesses.