Skip to content
Hello Python
1/7

Generators & Iterators

Topic 1 of 7, with 3 concept checks. The iterator protocol, yield, and lazy evaluation

Move through data one value at a time

Lazy traversal

Distinguish an iterable from its iterator, follow the next protocol to exhaustion, and use generators when producing values lazily is clearer than building a collection first.

Core lesson 01

An iterable implements __iter__ and can produce a fresh iterator each time. An iterator implements __next__ and is consumed once, tracking its own position.

Every for loop calls iter(obj) to get an iterator, then repeatedly calls next() on it until StopIteration. Lists, strings, and dicts are iterables — each for loop over them gets a brand new iterator starting from the beginning. An iterator is also iterable (its __iter__ returns itself), but it has state — once exhausted, it stays exhausted even if you loop over it again.

Python example
nums = [1, 2, 3]
it = iter(nums)
print(next(it))   # 1
print(next(it))   # 2
print(list(it))   # [3]  -- resumes where it left off
print(list(it))   # []   -- exhausted, no reset

What to remember

What's the difference between an iterable and an iterator?

Common footguns

  • Passing an already-exhausted iterator to a second for-loop and getting nothing, silently, instead of an error.
  • Assuming list(some_iterable) gives the same result twice — true for lists, false for a generator or file object.

Core lesson 02

Calling a function containing yield returns a generator object immediately without running any code. Each next() call resumes execution until the next yield, preserving local state in between.

return ends a function and hands back one value permanently. yield pauses a function, hands back one value, and remembers the entire local state (variables, loop position) so execution can resume exactly there on the next next() call. A function containing yield is a generator function; calling it doesn't execute any code — it returns a generator object that runs the body lazily, one step at a time.

Python example
def countdown(n):
    print("starting")
    while n > 0:
        yield n
        n -= 1

g = countdown(3)
print("created, nothing printed yet")
print(next(g))   # prints 'starting', then 1
print(next(g))   # 2
print(next(g))   # 3
# next(g) again -> StopIteration

What to remember

What does yield do differently from return, and what is a generator function?

Common footguns

  • Forgetting generators are single-use — once exhausted you must call the generator function again for a fresh one.
  • Mixing yield with return VALUE in the same function — return x inside a generator sets StopIteration's value, it does not yield x.
def countdown(n):        g = countdown(3)   <- no code runs yet
    yield n               next(g) -> runs until yield, returns n, PAUSES
    n -= 1                next(g) -> resumes right after yield, continues

Core lesson 03

A list comprehension builds the entire list in memory immediately. A generator expression produces values lazily, one at a time, using O(1) memory regardless of size.

Swapping [] for () around a comprehension changes it from eager to lazy. [x**2 for x in range(10_000_000)] allocates a list of ten million integers immediately. (x**2 for x in range(10_000_000)) creates a generator that computes each value only when asked — ideal for pipelines where you only need to iterate once, or don't need every value at once.

Python example
import sys
list_comp = [x for x in range(100000)]
gen_exp = (x for x in range(100000))
print(sys.getsizeof(list_comp))  # large, grows with size
print(sys.getsizeof(gen_exp))    # tiny, constant regardless of range size

What to remember

Why use a generator expression (x for x in range(n)) instead of a list comprehension for large data?

Common footguns

  • Trying to index a generator (gen[0]) or check its length with len() — neither works, since it doesn't hold all values at once.
  • Iterating a generator expression twice — like any generator, it's exhausted after one full pass.

Python lab

Browser Python lab

Runtime · idle

Python loads on your first run. Your code stays in this browser.

Best practices

  • Prefer generator expressions over list comprehensions when you only need to iterate once and don't need indexing or len().
  • Use itertools (islice, chain, groupby) instead of manually managing iterator state.
  • Document that a function returns a generator if the name isn't obvious — callers need to know it's single-use.
  • Convert to list() explicitly when you truly need to iterate more than once or need random access.

References

Python Docs — GeneratorsReal Python — Iterators and Iterables

Apply the concept in Interview practice

Peeking IteratormediumLeetCode #284 · O(1) per operation

Cache the next value ahead of time so peek() can return it without consuming it, then re-fetch on the next call to next().

Open problem
Flatten Nested List IteratormediumLeetCode #341 · O(n) total across all calls

Use a stack initialized with the reversed top-level items; whenever the top is a list, pop it and push its contents reversed, so the top of the stack is always the next integer.

Open problem
Binary Search Tree IteratormediumLeetCode #173 · O(1) amortized per next()

Use a stack to simulate in-order traversal: push all left children going down; on next(), pop a node, then push its right subtree's left chain.

Open problem
Design Circular QueuemediumLeetCode #622 · O(1) per operation

Use a fixed-size array with head/tail indices computed mod capacity, tracking count separately to distinguish full from empty.

Open problem

Concept checks

Q01

What's the difference between an iterable and an iterator?

Q02

What does yield do differently from return, and what is a generator function?

Q03

Why use a generator expression (x for x in range(n)) instead of a list comprehension for large data?