Queue.Queue vs. collections.deque
Queue.Queue
and collections.deque
serve different purposes. Queue.Queue is intended for allowing different threads to communicate using queued messages/data, whereas collections.deque
is simply intended as a datastructure. That's why Queue.Queue
has methods like put_nowait()
, get_nowait()
, and join()
, whereas collections.deque
doesn't. Queue.Queue
isn't intended to be used as a collection, which is why it lacks the likes of the in
operator.
It boils down to this: if you have multiple threads and you want them to be able to communicate without the need for locks, you're looking for Queue.Queue
; if you just want a queue or a double-ended queue as a datastructure, use collections.deque
.
Finally, accessing and manipulating the internal deque of a Queue.Queue
is playing with fire - you really don't want to be doing that.
If all you're looking for is a thread-safe way to transfer objects between threads, then both would work (both for FIFO and LIFO). For FIFO:
Note:
- Other operations on
deque
might not be thread safe, I'm not sure. deque
does not block onpop()
orpopleft()
so you can't base your consumer thread flow on blocking till a new item arrives.
However, it seems that deque has a significant efficiency advantage. Here are some benchmark results in seconds using CPython 2.7.3 for inserting and removing 100k items
deque 0.0747888759791Queue 1.60079066852
Here's the benchmark code:
import timeimport Queueimport collectionsq = collections.deque()t0 = time.clock()for i in xrange(100000): q.append(1)for i in xrange(100000): q.popleft()print 'deque', time.clock() - t0q = Queue.Queue(200000)t0 = time.clock()for i in xrange(100000): q.put(1)for i in xrange(100000): q.get()print 'Queue', time.clock() - t0
For information there is a Python ticket referenced for deque thread-safety (https://bugs.python.org/issue15329).Title "clarify which deque methods are thread-safe"
Bottom line here: https://bugs.python.org/issue15329#msg199368
The deque's append(), appendleft(), pop(), popleft(), and len(d) operations are thread-safe in CPython. The append methods have a DECREF at the end (for cases where maxlen has been set), but this happens after all of the structure updates have been made and the invariants have been restored, so it is okay to treat these operations as atomic.
Anyway, if you are not 100% sure and you prefer reliability over performance, just put a like Lock ;)