第十四章 Iterations and Comprehensions

“iterable objects”：an object is considered iterable if it is either a physically stored sequence, or an object that produces one result at a time in the context of an iteration tool like a for loop。

iterable objects include both physical sequences and virtual sequences computed on demand.

读取文件时，可以使用文件的readline()方法，到文件结尾返回空字符串。

也可使用__next__方法，但在文件结尾时产生 built-in StopIteration exception

Any object with a __next__ method to advance to a next result, which raisesStopIteration at the end of the series of results, is considered an iterator in Python.

all iteration tools normally work internally by calling __next__ on each iteration and catching the StopIteration exception to determine when to exit.

>>> for line in open('script2.py'): # Use file iterators to read by lines
... print(line.upper(), end='') # Calls __next__, catches StopIteration
...
IMPORT SYS
PRINT(SYS.PATH)
X = 2
PRINT(X ** 32)  #最佳读文件方法：it’s the simplest to code, might be the quickest to run, and is the best in terms of memory usage.

Manual Iteration: iter and next：

>>> f = open('script2.py')
>>> f.__next__() # Call iteration method directly
'import sys\n'
>>> f.__next__()
'print(sys.path)\n'


>>> f = open('script2.py')
>>> next(f) # The next(f) built-in calls f.__next__() in 3.X
'import sys\n'
>>> next(f) # next(f) => [3.X: f.__next__()], [2.X: f.next()]
'print(sys.path)\n'

• The iterable object you request iteration for, whose __iter__ is run by iter
• The iterator object returned by the iterable that actually produces values during the iteration, whose __next__ is run by next and raises StopIteration when finished producing results
  

>>> L = [1, 2, 3]
>>> I = iter(L) # Obtain an iterator object from an iterable
>>> I.__next__() # Call iterator's next to advance to next item
1
>>> I.__next__() # Or use I.next() in 2.X, next(I) in either line
2
>>> I.__next__()
3
>>> I.__next__()
...error text omitted...
StopIteration

a file object is its own iterator，support just one iteration

>>> f = open('script2.py')
>>> iter(f) is f
True
>>> iter(f) is f.__iter__()
True
>>> f.__next__()
'import sys\n'

Lists and many other built-in objects, though, are not their own iterators，support multiple open iterations

>>> L = [1, 2, 3]
>>> iter(L) is L
False
>>> L.__next__()
AttributeError: 'list' object has no attribute '__next__'
>>> I = iter(L)
>>> I.__next__()
1
>>> next(I) # Same as I.__next__()
2

Manual iteration：

>>> L = [1, 2, 3]
>>>
>>> for X in L: # Automatic iteration
... print(X ** 2, end=' ') # Obtains iter, calls __next__, catches exceptions
...
1 4 9



>>> I = iter(L) # Manual iteration: what for loops usually do
>>> while True:
... try: # try statement catches exceptions
... X = next(I) # Or call I.__next__ in 3.X
... except StopIteration:
... break
... print(X ** 2, end=' ')
...
1 4 9

Other Built-in Type Iterables（shelves，the results from os.popen，enumerate）

>>> D = {'a':1, 'b':2, 'c':3}
>>> for key in D.keys():
... print(key, D[key])
...
a 1
b 2
c 3



>>> I = iter(D)
>>> next(I)
'a'
>>> next(I)
'b'
>>> next(I)
'c'
>>> next(I)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration



>>> for key in D:
... print(key, D[key])
...
a 1
b 2
c 3

1.List Comprehensions: A First Detailed Look

>>> L = [1, 2, 3, 4, 5]
>>> for i in range(len(L)):   #原位置改变其值
... L[i] += 10
...
>>> L
[11, 12, 13, 14, 15]


>>> L = [x + 10 for x in L]  #生成新L
>>> L
[21, 22, 23, 24, 25]

Extended List Comprehension Syntax

Filter clauses: if

>>> lines = [line.rstrip() for line in open('script2.py') if line[0] == 'p']
>>> lines
['print(sys.path)', 'print(x ** 32)']

Nested loops: for

>>> [x + y for x in 'abc' for y in 'lmn']
['al', 'am', 'an', 'bl', 'bm', 'bn', 'cl', 'cm', 'cn']

2.Other Iteration Contexts

>>> uppers = [line.upper() for line in open('script2.py')]
>>> uppers
['IMPORT SYS\n', 'PRINT(SYS.PATH)\n', 'X = 2\n', 'PRINT(X ** 32)\n']


>>> map(str.upper, open('script2.py')) # map is itself an iterable in 3.X
<map object at 0x00000000029476D8>
>>> list(map(str.upper, open('script2.py')))
['IMPORT SYS\n', 'PRINT(SYS.PATH)\n', 'X = 2\n', 'PRINT(X ** 32)\n']

>>> sorted(open('script2.py'))
['import sys\n', 'print(sys.path)\n', 'print(x ** 32)\n', 'x = 2\n']
>>> list(zip(open('script2.py'), open('script2.py')))
[('import sys\n', 'import sys\n'), ('print(sys.path)\n', 'print(sys.path)\n'),
('x = 2\n', 'x = 2\n'), ('print(x ** 32)\n', 'print(x ** 32)\n')]
>>> list(enumerate(open('script2.py')))
[(0, 'import sys\n'), (1, 'print(sys.path)\n'), (2, 'x = 2\n'),
(3, 'print(x ** 32)\n')]
>>> list(filter(bool, open('script2.py'))) # nonempty=True
['import sys\n', 'print(sys.path)\n', 'x = 2\n', 'print(x ** 32)\n']
>>> import functools, operator
>>> functools.reduce(operator.add, open('script2.py'))
'import sys\nprint(sys.path)\nx = 2\nprint(x ** 32)\n'

essentially everything in Python’s built-in toolset that scans an object from left to right is defined to use the iteration protocol on the subject object.

>>> list(open('script2.py'))
['import sys\n', 'print(sys.path)\n', 'x = 2\n', 'print(x ** 32)\n']
>>> tuple(open('script2.py'))
('import sys\n', 'print(sys.path)\n', 'x = 2\n', 'print(x ** 32)\n')
>>> '&&'.join(open('script2.py'))
'import sys\n&&print(sys.path)\n&&x = 2\n&&print(x ** 32)\n'

>>> def f(a, b, c, d): print(a, b, c, d, sep='&')
...
>>> f(1, 2, 3, 4)
1&2&3&4
>>> f(*[1, 2, 3, 4]) # Unpacks into arguments
1&2&3&4

>>> X = (1, 2)
>>> Y = (3, 4)
>>>
>>> list(zip(X, Y)) # Zip tuples: returns an iterable
[(1, 3), (2, 4)
>>> A, B = zip(*zip(X, Y)) # Unzip a zip!
>>> A
(1, 2)
>>> B
(3, 4)

3.New Iterables in Python 3.X

The range Iterable：

C:\code> c:\python33\python
>>> R = range(10) # range returns an iterable, not a list
>>> R
range(0, 10)
>>> I = iter(R) # Make an iterator from the range iterable
>>> next(I) # Advance to next result
0 # What happens in for loops, comprehensions, etc.
>>> next(I)
1
>>> next(I)
2
>>> list(range(10)) # To force a list if required
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

range objects in 3.X support only iteration,indexing, and the len function.

>>> len(R) # range also does len and indexing, but no others
10
>>> R[0]
0
>>> R[-1]
9
>>> next(I) # Continue taking from iterator, where left off
3
>>> I.__next__() # .next() becomes .__next__(), but use new next()
4

The map, zip, and filter Iterables：

>>> M = map(abs, (-1, 0, 1)) # map returns an iterable, not a list
>>> M
<map object at 0x00000000029B75C0>
>>> next(M) # Use iterator manually: exhausts results
1 # These do not support len() or indexing
>>> next(M)
0
>>> next(M)
1
>>> next(M)
StopIteration
>>> for x in M: print(x) # map iterator is now empty: one pass only
...
>>> M = map(abs, (-1, 0, 1)) # Make a new iterable/iterator to scan again
>>> for x in M: print(x) # Iteration contexts auto call next()
...
1
0
1
>>> list(map(abs, (-1, 0, 1))) # Can force a real list if needed
[1, 0, 1]

>>> Z = zip((1, 2, 3), (10, 20, 30)) # zip is the same: a one-pass iterator
>>> Z
<zip object at 0x0000000002951108>
>>> list(Z)
[(1, 10), (2, 20), (3, 30)]
>>> for pair in Z: print(pair) # Exhausted after one pass
...
>>> Z = zip((1, 2, 3), (10, 20, 30))
>>> for pair in Z: print(pair) # Iterator used automatically or manually
...
(1, 10)
(2, 20)
(3, 30)
>>> Z = zip((1, 2, 3), (10, 20, 30)) # Manual iteration (iter() not needed)
>>> next(Z)
(1, 10)
>>> next(Z)
(2, 20)

>>> filter(bool, ['spam', '', 'ni'])
<filter object at 0x00000000029B7B70>
>>> list(filter(bool, ['spam', '', 'ni']))
['spam', 'ni']

Multiple Versus Single Pass Iterators：

>>> R = range(3) # range allows multiple iterators
>>> next(R)
TypeError: range object is not an iterator
>>> I1 = iter(R)
>>> next(I1)
0
>>> next(I1)
1
>>> I2 = iter(R) # Two iterators on one range
>>> next(I2)
0
>>> next(I1) # I1 is at a different spot than I2
2

multiple iterators are usually supported by returning new objects for the iter call;

a single iterator generally means an object returns itself

>>> Z = zip((1, 2, 3), (10, 11, 12))
>>> I1 = iter(Z)
>>> I2 = iter(Z) # Two iterators on one zip
>>> next(I1)
(1, 10)
>>> next(I1)
(2, 11)
>>> next(I2) # (3.X) I2 is at same spot as I1!
(3, 12)


>>> M = map(abs, (-1, 0, 1)) # Ditto for map (and filter)
>>> I1 = iter(M); I2 = iter(M)
>>> print(next(I1), next(I1), next(I1))
1 0 1
>>> next(I2) # (3.X) Single scan is exhausted!
StopIteration



>>> R = range(3) # But range allows many iterators
>>> I1, I2 = iter(R), iter(R)
>>> [next(I1), next(I1), next(I1)]
[0 1 2]
>>> next(I2) # Multiple active scans, like 2.X lists
0

Dictionary View Iterables：

in Python 3.X the dictionary keys, values, and items methods return iterable view objects that generate result items one at a time

>>> D = dict(a=1, b=2, c=3)
>>> D
{'a': 1, 'b': 2, 'c': 3}
>>> K = D.keys() # A view object in 3.X, not a list
>>> K
dict_keys(['a', 'b', 'c'])
>>> next(K) # Views are not iterators themselves
TypeError: dict_keys object is not an iterator
>>> I = iter(K) # View iterables have an iterator,
>>> next(I) # which can be used manually,
'a' # but does not support len(), index
>>> next(I)
'b'
>>> for k in D.keys(): print(k, end=' ') # All iteration contexts use auto
...
a b c

3.X dictionaries still are iterables themselves, with an iterator that returns successive keys：

>>> D # Dictionaries still produce an iterator
{'a': 1, 'b': 2, 'c': 3} # Returns next key on each iteration
>>> I = iter(D)
>>> next(I)
'a'
>>> next(I)
'b'
>>> for key in D: print(key, end=' ') # Still no need to call keys() to iterate
... # But keys is an iterable in 3.X too!
a b c

4. Other Iteration Topics

• User-defined functions can be turned into iterable generator functions, with yield statements.
• List comprehensions morph into iterable generator expressions when coded in parentheses.
• User-defined classes are made iterable with __iter__ or __getitem__ operator overloading.