arundhaj

all that is technology

Why Some Python Built-in Functions Return Objects and Others Return Lists

 

If you are new to Python, this output looks odd:

>>> map(str, [1, 2, 3])
<map object at 0x7f3a2c1d0>

>>> sorted([3, 1, 2])
[1, 2, 3]

>>> sum([1, 2, 3])
6

Three built-ins, three different kinds of result. This isn't inconsistency. Each function returns the cheapest form that can honestly represent its work, and most built-ins fall into one of three groups.

Lazy iterators

map, filter, zip, enumerate and reversed return an iterator. No work happens until you ask for the next item.

>>> squares = map(lambda n: n * n, range(10**9))
>>> next(squares)
0
>>> next(squares)
1

This returns instantly, even with a billion inputs, because only one item is computed at a time. These functions can work this way because each output item depends on just the current input.

The <map object> you see is literally true: map is a class, not a function, so calling it gives you an instance.

>>> type(map)
<class 'type'>

This wasn't always the design. In Python 2, map, filter and zip returned lists, and Python 3 switched them to iterators. You can always go from lazy to eager with list(), but an eager function has already spent the memory by the time you get the result.

The trade-off is that an iterator can only be consumed once.

>>> pairs = zip("ab", [1, 2])
>>> list(pairs)
[('a', 1), ('b', 2)]
>>> list(pairs)
[]

range looks similar but is different. It is a lazy sequence, so it supports len() and indexing, and you can loop over it as many times as you like.

Direct collections

sorted, list, tuple, set, dict and dir build the whole result up front.

>>> sorted("python")
['h', 'n', 'o', 'p', 't', 'y']

Compare sorted with reversed. To reverse a list, Python can simply walk it backwards, so it stays lazy. To sort, Python must see every item before it can even tell you which one comes first. The work can't be deferred, so the result is stored and returned as a list.

The constructors list, tuple, set and dict are direct by definition, since building a collection is their whole purpose. That is also why they are the usual way to materialize a lazy iterator.

Math and reduction functions

These return a single value, so there is nothing to be lazy about.

Math functions work on plain values:

>>> abs(-5)
5
>>> round(3.14159, 2)
3.14
>>> pow(2, 10)
1024
>>> divmod(17, 5)
(3, 2)

Reduction functions collapse an iterable into one result:

>>> sum([1, 2, 3])
6
>>> max([1, 2, 3])
3
>>> any(n > 2 for n in [1, 2, 3])
True

They pair nicely with lazy iterators, because the items flow through one at a time without ever building a list in memory.

>>> sum(n * n for n in range(1000000))
333332833333500000

any and all go one step further and stop as soon as the answer is known.

Quick reference

Style Functions Returns Work happens
Lazy iterators map, filter, zip, enumerate, reversed Iterator object One item at a time, on demand
Direct collections sorted, list, tuple, set, dict, dir Full collection Immediately
Math abs, round, pow, divmod A number, or a tuple for divmod Immediately
Reduction sum, min, max, any, all Single value Immediately

So when you meet a new built-in, ask what it needs to produce its output. If it needs one item at a time, expect an iterator. If it needs all the items and keeps them all, expect a collection. If it boils them down to an answer, expect a single value.

Hope this helps!

  Python

Comments