What exactly are iterator, iterable, and iteration?
Iteration is a general term for taking each item of something, one after another. Any time you use a loop, explicit or implicit, to go over a group of items, that is iteration.
In Python, iterable and iterator have specific meanings.
An iterable is an object that has an
__iter__ method which returns an iterator, or which defines a
__getitem__ method that can take sequential indexes starting from zero (and raises an
IndexError when the indexes are no longer valid). So an iterable is an object that you can get an iterator from.
An iterator is an object with a
next (Python 2) or
__next__ (Python 3) method.
Whenever you use a
for loop, or
map, or a list comprehension, etc. in Python, the
next method is called automatically to get each item from the iterator, thus going through the process of iteration.
A good place to start learning would be the iterators section of the tutorial and the iterator types section of the standard types page. After you understand the basics, try the iterators section of the Functional Programming HOWTO.
Here's the explanation I use in teaching Python classes:
An ITERABLE is:
- anything that can be looped over (i.e. you can loop over a string or file) or
- anything that can appear on the right-side of a for-loop:
for x in iterable: ...or
- anything you can call with
iter()that will return an ITERATOR:
- an object that defines
__iter__that returns a fresh ITERATOR,or it may have a
__getitem__method suitable for indexed lookup.
An ITERATOR is an object:
- with state that remembers where it is during iteration,
- with a
- returns the next value in the iteration
- updates the state to point at the next value
- signals when it is done by raising
- and that is self-iterable (meaning that it has an
__iter__method that returns
__next__method in Python 3 is spelt
nextin Python 2, and
- The builtin function
next()calls that method on the object passed to it.
'cat' # s is an ITERABLE # s is a str object that is immutable # s has no state # s has a __getitem__() method t = iter(s) # t is an ITERATOR # t has state (it starts by pointing at the "c" # t has a next() method and an __iter__() methodnext(t) # the next() function returns the next value and advances the state'c'next(t) # the next() function returns the next value and advances'a'next(t) # the next() function returns the next value and advances't'next(t) # next() raises StopIteration to signal that iteration is completeTraceback (most recent call last):...StopIterationiter(t) is t # the iterator is self-iterables =
The above answers are great, but as most of what I've seen, don't stress the distinction enough for people like me.
Also, people tend to get "too Pythonic" by putting definitions like "X is an object that has
__foo__() method" before. Such definitions are correct--they are based on duck-typing philosophy, but the focus on methods tends to get between when trying to understand the concept in its simplicity.
So I add my version.
In natural language,
- iteration is the process of taking one element at a time in a row of elements.
iterable is an object that is, well, iterable, which simply put, means thatit can be used in iteration, e.g. with a
forloop. How? By using iterator.I'll explain below.
... while iterator is an object that defines how to actually do theiteration--specifically what is the next element. That's why it must have
Iterators are themselves also iterable, with the distinction that their
__iter__() method returns the same object (
self), regardless of whether or not its items have been consumed by previous calls to
So what does Python interpreter think when it sees
for x in obj: statement?
forloop. Looks like a job for an iterator... Let's get one. ... There's this
objguy, so let's ask him.
obj, do you have your iterator?" (... calls
iter(obj), which calls
obj.__iter__(), which happily hands out a shiny new iterator
OK, that was easy... Let's start iterating then. (
x = _i.next()...
x = _i.next()...)
obj succeeded in this test (by having certain method returning a valid iterator), we reward him with adjective: you can now call him "iterable Mr.
However, in simple cases, you don't normally benefit from having iterator and iterable separately. So you define only one object, which is also its own iterator. (Python does not really care that
_i handed out by
obj wasn't all that shiny, but just the
This is why in most examples I've seen (and what had been confusing me over and over),you can see:
class IterableExample(object): def __iter__(self): return self def next(self): pass
class Iterator(object): def next(self): passclass Iterable(object): def __iter__(self): return Iterator()
There are cases, though, when you can benefit from having iterator separated from the iterable, such as when you want to have one row of items, but more "cursors". For example when you want to work with "current" and "forthcoming" elements, you can have separate iterators for both. Or multiple threads pulling from a huge list: each can have its own iterator to traverse over all items. See @Raymond's and @glglgl's answers above.
Imagine what you could do:
class SmartIterableExample(object): def create_iterator(self): # An amazingly powerful yet simple way to create arbitrary # iterator, utilizing object state (or not, if you are fan # of functional), magic and nuclear waste--no kittens hurt. pass # don't forget to add the next() method def __iter__(self): return self.create_iterator()
I'll repeat again: iterator is not iterable. Iterator cannot be used asa "source" in
forloop primarily needs is
__iter__()(that returns something with
foris not the only iteration loop, so above applies to some otherconstructs as well (
next()can throw StopIteration to stop iteration. Does not have to,though, it can iterate forever or use other means.
In the above "thought process",
_idoes not really exist. I've made up that name.
There's a small change in Python 3.x:
next()method (not the built-in) nowmust be called
__next__(). Yes, it should have been like that all along.
You can also think of it like this: iterable has the data, iterator pulls the nextitem
Disclaimer: I'm not a developer of any Python interpreter, so I don't really know what the interpreter "thinks". The musings above are solely demonstration of how I understand the topic from other explanations, experiments and real-life experience of a Python newbie.