Question

How do I compose or chain multiple function in Python

When writing a program, I often want to perform a series of steps on a piece of data (usually a list or string). For example I might want to call a function which returns a string, convert it to a list, map over the list, and then reduce to get a final result.

In programming languages I've used in the past, I would expect to be able to do something like one of the following:

compose(
  getSomeString,
  list,
  map(someMapFunction)
  reduce(someReduceFunction)
)()
getSomeString()
  .list()
  .map(someMapFunction)
  .reduce(someReduceFunction)
getSomeFunction
  => list
  => map(someMapFunction)
  => reduce(someReduceFunction)()

However, I can't figure out a clean and compact way to do composition/chaining in Python. The one exception I've found is that a list comprehension works for the case where I want to compose a map and a filter.

What is the Pythonic way of writing composition oriented code without either having tons of nesting that makes my code look like Lisp:

reduce(
  someReduceFunction,
  map(
    someMapFunction,
    list(
      getSomeString()
    )
  )
)

Or creating intermediate values for everything:

myString = getSomeString()
stringAsList = list(myString)
mappedString = map(someMapFunction, stringAsList)
reducedString = reduce(someReduceFunction, mappedString)
 3  51  3
1 Jan 1970

Solution

 3

If you don't mind writing your own utility function something like this will do:

def compose(*functions):
  def pipe(value):
    for f in functions:
      value = f(value)
    return value

  return pipe

It works by producing an inner function that iterates over your given functions and passes the output of one to the input of the next.

And then you can use it as:

from functools import partial

compose(
  getSomeString,
  list,
  partial(map, someMapFunction),
  partial(reduce, someReduceFunction)
)("input")

The downside is that it can only pass around single return values. but you could fix this with lambdas:

compose(
  returns_a_dict,
  # Use a lambda to rewrite the single argument the way you want.
  lambda d: takes_kw_args(**d)
)("input")

(PS: I used functools.partial to freeze map together with its map function (call it map_f), so that the pipe function calls f(value) which equals partial(map, map_f)(value) which equals map(map_f, value))

2024-07-19
Robin De Schepper