Celery task that runs more tasks

To answer your opening questions: As of version 2.0, Celery provides an easy way to start tasks from other tasks. What you are calling "secondary tasks" are what it calls "subtasks". See the documentation for Sets of tasks, Subtasks and Callbacks, which @Paperino was kind enough to link to.

For version 3.0, Celery changed to using groups for this, and other, types of behavior.

Your code shows that you are already familiar with this interface. Your actual question seems to be, "Why am I getting a 'Socket Closed' IOError when I try to run my set of subtasks?" I don't think anyone can answer that, because you have not provided enough information about your program. Your excerpt cannot be run as-is, so we cannot examine the problem you're having for ourselves. Please post the stacktrace provided with the IOError, and with any luck, someone that can help you with your crasher will come along.

You can use something like this (Support in 3.0 )

g = group(compute_for_user.s(user.id) for user in users)g.apply_async()

And since version 3.0 'TaskSet' isn't the term anymore... Groups, Chains and Chords as a special type of subtask is the new thing, see http://docs.celeryproject.org/en/3.1/whatsnew-3.0.html#group-chord-chain-are-now-subtasks