Python List Aggregate Operations

An aggregate operation turns a whole collection into one value. Examples are the number of elements, their sum, the biggest element, or all elements combined into one result.

Python gives you the common ones directly as built-in functions. len() counts the elements, min() and max() find the smallest and the biggest element, and sum() adds them up.

Be careful: min() and max() raise a ValueError on an empty list and the program stops. Check len(s) > 0 first when the list might be empty.

For combining elements one by one there is functools.reduce(). It keeps a running value, the accumulator, and calls your function with the accumulator and each element in turn. One function covers what many languages split into reduce and fold: without a third argument it starts with the first element, with one it starts with that initial value.

The most common aggregates are count, minimum, maximum, sum and average.

• len(numbers) gives the count.
• min(numbers) and max(numbers) give the smallest and the biggest element. They work for numbers and strings.
• sum(numbers) adds every element up. The average is the sum divided by the count; division with / always keeps the decimal part.

Sometimes the smallest or biggest element should be picked by a rule, not by the value itself. min() and max() take a key= function for that: the element with the smallest or biggest key wins, and the element itself is returned.

Sometimes every element must be converted with a function first and the results added up. sum() takes a generator expression, so sum(f(n) for n in numbers) does it in one line.

For the numbers 6, 42, 10 and 4 print the count with len(), the biggest and smallest element with max() and min(), and the sum with sum(). The average is 62 / 4 = 15.5.

min() with key=lambda n: n % 3 compares the remainders. The remainders of 5, 42, 10 and 4 are 2, 0, 1 and 1, so 42 wins. max() with key=len compares string lengths and finds "three".

sum() adds up whatever the generator expression produces. Doubling 5, 42, 10 and 4 gives 122. Halving them with / gives 30.5.

Fold and reduce combine all elements into one value, step by step. They keep a running value called the accumulator. For each element they call a function with the accumulator and the element, and the function returns the new accumulator.

The difference is the starting value, and in Python it is simply the third argument of functools.reduce():

• reduce(f, items, initial) is a fold: it starts with the initial value you give it. The function runs once for every element.
• reduce(f, items) starts with the first element. The function runs for the second element onward, so the first element is never passed through it.

This matters when the function changes the element. If it doubles every element, reduce() without an initial value leaves the first element as it is.

Python has no reduce-from-the-right. To fold from the right, run reduce() over reversed(items): the elements arrive last to first, and the function still gets the accumulator first and the element second.

reduce() without an initial value takes 5 as the start and adds 2, 10 and 4 to it. The result is 21.

With the initial value 0, reduce() starts at 0 and adds each element times 2. Every element is doubled: 10 + 4 + 20 + 8 = 42.

Both calls use the same function, accumulator plus element times 2. Without an initial value the result is 37, because the first element 5 is the start and is not doubled. With the initial value 0 it is 42.

reversed(numbers) hands the elements to reduce() from the last one to the first. Addition does not care about the order, so the result is again 42.

Sometimes the position of an element matters, not only its value.

In Python a plain loop with enumerate() gives you the index. for idx, element in enumerate(numbers) walks from the start. for idx, element in reversed(list(enumerate(numbers))) walks the same index-element pairs from the end. Inside the loop an if checks the index and decides whether to add the element.

Only the elements at even indexes are added: index 0 holds 5 and index 2 holds 10, so the sum is 15.

The same rule, but the loop starts at index 3 and goes down to 0. The result is still 15.

reduce() needs a first element to start with. On an empty list it raises a TypeError and the program stops.

The Python way to say "there is no result" is None. A small helper checks the list first: when it is empty, it returns None instead of calling reduce(). The caller then checks for None, just like with dict.get() for a missing key.

reduce_or_none returns 21 for the numbers 5, 2, 10 and 4, and None for an empty list.

A running fold or reduce does not throw away the steps in between. It saves the accumulator after every element, so the result is a list of all the running values.

Python has this built into itertools.accumulate(). By default it adds, and it yields every running sum. For a running reduce the result starts with the first element. With the initial= keyword it becomes a running fold: the result starts with the initial value and has one more element than the input.

The numbers 0 to 5 are added one by one. accumulate() keeps every sum along the way: [0, 1, 3, 6, 10, 15].

The same with initial=10. The result begins with 10 itself and has one more element than the input: [10, 10, 11, 13, 16, 20, 25].