Question
Let's say I have a number of iterables:
[[1, 2], [3, 4, 5, 6], [7, 8, 9], [10, 11, 12, 13, 14]]
How can I get only each element that is the first to appear at its index in any of the iterables? In this case:
[1, 2, 5, 6, 14]
Visualized:
[1, 2]
[_, _, 5, 6]
[_, _, _]
[_, _, _, _, 14]
Solution
You can do just about anything with a generator:
def chain_suffixes(iters):
n=0
for it in iters:
for x in itertools.islice(it,n,None):
yield x
n+=1
Solution
can it done in more functional style?
Sure, but I wouldn't. Davis Herring's approach is already lovely. Here's a "more functional" way, but more obscure to my eyes:
from itertools import zip_longest
SKIP = object()
def chain_suffixes(*iters):
return (next(a for a in s if a is not SKIP)
for s in zip_longest(*iters, fillvalue=SKIP))
print(list(chain_suffixes(
[1, 2], [3, 4, 5, 6], [7, 8, 9], [10, 11, 12, 13, 14])))
which prints
[1, 2, 5, 6, 14]
BTW, "more functional" may be in the eye of the beholder. To my eyes, various forms of generator comprehension are just as "functional" as other ways of spelling it. At an extreme, it's possible to rewrite the above without using comprehensions at all, or even typing "def".
from itertools import zip_longest
SKIP = object()
not_SKIP = lambda x: x is not SKIP
first_not_SKIP = lambda s: next(filter(not_SKIP, s))
chain_suffixes = lambda *iters: map(
first_not_SKIP,
zip_longest(*iters, fillvalue=SKIP))
Solution
Here is one way to do it:
out = []
for sub_list in m:
out.extend(sub_list[len(out):])
# out = [1, 2, 5, 6, 14]
For each sub list, we only select those elements that starts with index len(out) to the end.
Solution
from itertools import islice
def chain_suffixes(iters):
result = []
for it in iters:
result += islice(it, len(result), None)
return result
iterables = [[1, 2], [3, 4, 5, 6], [7, 8, 9], [10, 11, 12, 13, 14]]
print(chain_suffixes(iterables))
Another, using zip to wrap elements in singleton tuples and the empty tuple as fill value:
from itertools import *
def chain_suffixes(iters):
return map(next, map(chain.from_iterable, zip_longest(*map(zip, iters), fillvalue=())))
iterables = [[1, 2], [3, 4, 5, 6], [7, 8, 9], [10, 11, 12, 13, 14]]
print(*chain_suffixes(iterables))
Another, without itertools:
def chain_suffixes(iters):
iters = list(map(iter, iters))
while iters:
for x in iters.pop(0):
yield x
for it in iters:
next(it, None)
iterables = [[1, 2], [3, 4, 5, 6], [7, 8, 9], [10, 11, 12, 13, 14]]
print(*chain_suffixes(iterables))
Solution
Here's a fun one using islice to get the tails, chain to chain them, and compress and count to count the produced elements in order to get the islice starts. It runs without any Python interpretation (after setting it up).
def chain_suffixes6(iters):
counter = count()
starts = map(sub, counter, count())
tails = map(islice, iters, starts, repeat(None))
return compress(chain.from_iterable(tails), counter)
Explanation: Imagine you had a magical way to get the start indices for the tails. Then it would simply be this:
def chain_suffixes(iters):
starts = <magic here>
tails = map(islice, iters, starts, repeat(None))
return chain.from_iterable(tails)
The start index for each iterable's tail is simply the count of how many elements we outputted so far, from all the previous iterables. To count them, I use a count iterator and use compress to move it along:
def chain_suffixes6(iters):
counter = count()
...
return compress(chain.from_iterable(tails), counter)
After the first iterable, we could just ask the counter for its next element. That would tell us how long that was. But then after the second iterable, the counter wouldn't just have counted the produced elements from the first iterable and the tail of the second. It would also have counted that one request for the current count! So we need to subtract 1. And the next time we ask, we need to subtract 2, etc. I do this compensation by subtracting another count():
starts = map(sub, counter, count())
This actually begins with start = 0 - 0 for the first iterable's "tail". Note that this also already consumes the 0 from counter, so that in
compress(chain.from_iterable(tails), counter)
it only provides positive numbers, which are true, so that compress yields all elements from the chain of tails.
That solution is also the last version in this sequence of solutions. Each evolves from the one before it and they get more and more unusual, so it might be best to read them in order. (Though I btw wrote version 3 right away and then 4 and 5, then wrote 1 and 2 to perhaps help understanding, and then 6 when I realized I can "compress/count" the chain of tails instead of each tail.)
from itertools import *
from operator import sub
def chain_suffixes1(iters):
start = 0
for it in iters:
counter = count(1)
tail = islice(it, start, None)
yield from compress(tail, counter)
start += next(counter) - 1
def chain_suffixes2(iters):
counter = count()
for i, it in enumerate(iters):
start = next(counter) - i
tail = islice(it, start, None)
yield from compress(tail, counter)
def chain_suffixes3(iters):
counter = count()
starts = map(sub, counter, count())
for it in iters:
tail = islice(it, next(starts), None)
yield from compress(tail, counter)
def chain_suffixes4(iters):
counter = count()
starts = map(sub, counter, count())
return chain.from_iterable(
compress(tail, counter)
for it in iters
for tail in [islice(it, next(starts), None)]
)
def chain_suffixes5(iters):
counter = count()
starts = map(sub, counter, count())
tails = map(islice, iters, starts, repeat(None))
tails_counted = map(compress, tails, repeat(counter))
return chain.from_iterable(tails_counted)
def chain_suffixes6(iters):
counter = count()
starts = map(sub, counter, count())
tails = map(islice, iters, starts, repeat(None))
return compress(chain.from_iterable(tails), counter)
iterables = [[1, 2], [3, 4, 5, 6], [7, 8, 9], [10, 11, 12, 13, 14]]
for f in chain_suffixes1, chain_suffixes2, chain_suffixes3, chain_suffixes4, chain_suffixes5, chain_suffixes6:
print(*f(iterables), f.__name__)
Silly oneliner version, I was just curious how that would look:
def chain_suffixes7(iters):
return compress(chain.from_iterable(map(islice, iters, map(sub, counter := count(), count()), repeat(None))), counter)
And spreading that over multiple physical lines:
def chain_suffixes8(iters):
return compress(
chain.from_iterable(
map(islice,
iters,
map(sub, counter := count(), count()),
repeat(None))),
counter
)
Solution
You can iterate over the lists in reverse and replace a slice each iteration
lists = [[1, 2], [3, 4, 5, 6], [7, 8, 9], [10, 11, 12, 13, 14]]
result = lists[-1]
for lst in lists[-1::-1]:
result[:len(lst)] = lst
print(result) # [1, 2, 5, 6, 14]