Threads in twisted... how to use them properly?

You didn't explain why you actually need threads here. If you had, I might have been able to explain why you don't need them. ;)

That aside, I can confirm that your basic understanding of things is correct. One possible misunderstanding I can clear up, though, is the notion that "python threads" and "Twisted threads" are at all different from each other. They're not. Python provides a threading library. All of Twisted's thread APIs are implemented in terms of Python's threading library. Only the API is different.

As far as shutdown goes, you have two options.

• Start your run-forever thread using Python's threading APIs directly and make the thread a daemon. Your process can exit even while daemon threads are still running. A possible problem with this solution is that some versions of Python have issues with daemon threads that will lead to a crash at shutdown time.
• Create your thread using either Twisted's APIs or the stdlib threading APIs but also add a Twisted shutdown hook using reactor.addSystemEventTrigger('before', 'shutdown', f). In that hook, communicate with the work thread and tell it to shut down. For example, you could share a threading.Event between the Twisted thread and your work thread and have the hook set it. The work thread can periodically check to see if it has been set and exit when it notices that it has been. Aside from not crashing, this gives another advantage over daemon threads - it will let you run some cleanup or finalization code in your work thread before the process exits.

Assuming that your main is relatively non-blocking:

import randomfrom twisted.internet import taskclass MyProcess:  def __init__(self):    self.stats = []    self.lp = None  def myloopingCall(self):    print "I have %s stats" % len(self.stats)  def myMainFunction(self,reactor):    self.stats.append(random.random())    reactor.callLater(0,self.myMainFunction,reactor)  def start(self,reactor):    self.lp = task.LoopingCall(self.myloopingCall)    self.lp.start(2)    reactor.callLater(0,self.myMainFunction,reactor)  def stop(self):    if self.lp is not None:      self.lp.stop()    print "I'm done"if __name__ == '__main__':  myproc = MyProcess()  from twisted.internet import reactor  reactor.callWhenRunning(myproc.start,reactor)  reactor.addSystemEventTrigger('during','shutdown',myproc.stop)  reactor.callLater(10,reactor.stop)  reactor.run()

$ python bleh.pyI have 0 statsI have 33375 statsI have 66786 statsI have 100254 statsI have 133625 statsI'm done