Python Generator - How to create or iterate a generator

The Python generator (expression) is declared on the fly as follows:

a = [1, 3, 5]

g = (i * i for i in a)

print(g)
# <generator object <genexpr> at 0x11940ef90>

for x in g:
    print(x)

# 1
# 9
# 25

Precisely, g is a generator expression called "generator", which is fuzzy and not the form or expression of a list-like object. g is a generator, not a list (nor list comprehension). A generator generates values in the for loop but doesn't save those in memory. Meanwhile, all the elements of a list are saved in memory. Generator uses memory efficiently.

You can't get each element a generator generates by an index.

a = [1, 3, 5]

g = (i * i for i in a)

print(g[0])
# TypeError: 'generator' object is not subscriptable

You can iterate what a generator generates.

a = [2, 3, 4]

g = (i + 100 for i in a)

for x in g:
    print(x)

# 102
# 103
# 104

Another example:

g = (i * i for i in range(6))

for x in g:
    print(x)

# 0
# 1
# 4
# 9
# 16
# 25

An empty list can be an "empty" generator.

a = []

g = (i * i for i in a)

for x in g:
    print(x)

#

A generator generates values once

g = (i * i for i in range(6))

for v in g:
    print(v)

print('***')

for w in g:
    print(w)

# 0
# 1
# 4
# 9
# 16
# 25
# ***

v are iterated and the square numbers (0-25) are printed. But w in g isn't iterated. The generator generates all values in the first loop. The next example shows a generator can stop generating in the loop and start again in the next loop.

g = (i * i for i in range(6))

j = 0
for v in g:
    j += 1
    if j < 4:
        print(v)
    else:
        break

print('***')

for w in g:
    print(w)

# 0
# 1
# 4
# ***
# 16
# 25

Generator and next

You can get the item of a generator expression using next(), which is a Python built-in function and returns "next value" of generator.

g = (i * i for i in range(6))

print(next(g))  # 0
print(next(g))  # 1
print(next(g))  # 4
print(next(g))  # 9
print(next(g))  # 16
print(next(g))  # 25
print(next(g))  # StopIteration

First, next() returns 0 because it's the first time to generate i * i. 6th next() returns 25 but the next doesn't exist. So the last next() raises a StopIteration.

Convert a Python generator to a Python list or set

You can convert a Python generator to a list.

a = [1, 2, 3]

g = (i * i for i in a)

b = list(g)

print(b)
# [1, 4, 9]

Also you can convert a generator to a set.

a = [1, 2, 3]

g = (i * i for i in a)

b = set(g)

print(b)
# {1, 4, 9}

Length of generator

The len() can't take a Python generator.

g = (i * i for i in range(5))

c = len(g)  # TypeError: object of type 'generator' has no len()

So, to get the length of a generator, convert it to a list.

g = (i * i for i in range(5))

s = list(g)

c = len(s)  # TypeError: object of type 'generator' has no len()

print(s)  # [0, 1, 4, 9, 16]
print(c)  # 5

Sum of values generated

g = (i * i for i in range(5))

m = sum(g)

print(m)  # 30

The sum() sums all the elements generated. You don't need to convert a generator to a list to sum.

g = (i * i for i in range(5))

s = list(g)

m = sum(s)