Skip to content
Hello Python
6/7

functools & itertools Deep Dive

Topic 6 of 7, with 3 concept checks. reduce, partial, chain, groupby, and friends

Compose lazy transformations from small tools

Functional building blocks

Combine functools and itertools primitives while tracking laziness, iterator consumption, cache boundaries, and the point where a direct loop would communicate intent better.

Core lesson 01

reduce(func, iterable) applies func cumulatively to the items of iterable, collapsing a sequence into a single value — but for common cases like sum or max, the dedicated built-ins are clearer.

reduce(function, [a, b, c]) computes function(function(a, b), c) — each step combines the running result with the next item. It's genuinely useful for custom combining logic with no dedicated built-in (composing a chain of functions, merging dicts), but for things Python already has built-ins for (sum, max, min, any, all) those are more readable and usually faster.

Python example
from functools import reduce

nums = [1, 2, 3, 4]
product = reduce(lambda acc, x: acc * x, nums)
print(product)         # 24

# clearer alternative for a case that HAS a built-in:
total = sum(nums)       # prefer this over reduce(lambda a,b: a+b, nums)
print(total)            # 10

What to remember

What does functools.reduce do, and when is a plain built-in clearer?

Common footguns

  • Reaching for reduce() out of habit when sum(), max(), any(), or a simple comprehension already expresses the same thing more clearly.

Core lesson 02

functools.partial(func, *args, **kwargs) returns a new callable with some arguments already fixed — similar to a lambda wrapper, but keeps the original function's identity and works with multiprocessing/pickling, which most lambdas don't.

You could write lambda x: my_func(fixed_arg, x) to achieve something similar, but partial is more explicit about intent, preserves useful introspection (partial objects expose .func, .args, .keywords), and — critically — can be pickled, which lambdas generally cannot. This matters when passing a partially-applied function to multiprocessing.Pool, which needs to pickle callables to send them to worker processes.

Python example
from functools import partial

def power(base, exponent):
    return base ** exponent

square = partial(power, exponent=2)
cube = partial(power, exponent=3)
print(square(5))   # 25
print(cube(2))     # 8
print(square.func, square.keywords)   # introspectable, unlike a lambda

What to remember

What does functools.partial do, and how is it different from a lambda?

Common footguns

  • Trying to pass a lambda to multiprocessing.Pool.map() and getting a pickling error — use functools.partial (or a module-level function) instead.

Core lesson 03

itertools.chain(*iterables) lazily iterates several iterables as if they were one. itertools.groupby(iterable, key) groups consecutive elements sharing a key — the input usually needs to be pre-sorted by that key first.

chain is straightforward: it walks each iterable in turn without concatenating them into a new list first, staying lazy and memory-efficient. groupby is more subtle — it only groups elements that are consecutive and share the same key; if equal keys aren't adjacent (data isn't sorted), you'll get multiple separate groups for the same key instead of one. This trips up almost everyone the first time.

Python example
from itertools import chain, groupby

print(list(chain([1, 2], [3, 4], [5])))   # [1, 2, 3, 4, 5]

data = [('a', 1), ('a', 2), ('b', 3), ('a', 4)]
# NOT sorted by key -- groupby will NOT merge the two 'a' groups
for key, group in groupby(data, key=lambda x: x[0]):
    print(key, list(group))
# a [('a', 1), ('a', 2)]
# b [('b', 3)]
# a [('a', 4)]   <- separate group! data wasn't sorted first

What to remember

What do itertools.chain and itertools.groupby do?

Common footguns

  • Calling groupby on unsorted data and being surprised the same key appears in multiple separate groups — sort by the same key first.

Python lab

Browser Python lab

Runtime · idle

Python loads on your first run. Your code stays in this browser.

Best practices

  • Sort data by the grouping key before using itertools.groupby, or you'll silently get fragmented groups.
  • Prefer built-ins (sum, any, all, max, min) over functools.reduce when they directly express the same operation.
  • Use functools.partial instead of a lambda when the callable needs to be pickled (e.g. for multiprocessing).
  • Reach for itertools.islice instead of slicing a generator directly (generators don't support slicing).

References

Python Docs — itertoolsPython Docs — functools

Apply the concept in Interview practice

PermutationsmediumLeetCode #46 · O(n·n!) time

Backtrack by swapping elements into place one position at a time — itertools.permutations solves this directly, but implementing it builds the same intuition.

Open problem
SubsetsmediumLeetCode #78 · O(n·2^n) time

Either backtrack including/excluding each element, or iteratively double the result by adding the next element to a copy of every existing subset — mirrors itertools.combinations-style enumeration.

Open problem
Combination SummediumLeetCode #39 · O(2^target) worst case

Backtrack while allowing the same number to be reused, pruning branches once the running sum exceeds the target.

Open problem

Concept checks

Q01

What does functools.reduce do, and when is a plain built-in clearer?

Q02

What does functools.partial do, and how is it different from a lambda?

Q03

What do itertools.chain and itertools.groupby do?