It’s almost Christmas, which means the 2017 edition of Advent of Code is officially upon us! While studying other people’s solutions to the first few problems, Sasa Juric‏’s use of streams stood out to me.

The resource/3 and the closely related iterate/2 and unfold/2 functions in the Stream module were completely unknown to me and seemed worthy of some deeper study.

After spending time researching and banging out a few examples, I realized that these functions are extremely useful and underrated tools for generating complex enumerable sequences in Elixir.

Let’s dive into a few examples to find out why!

Simple Sequences

The resource/3, iterate/2, and unfold/2 functions in the Stream module are all designed to emit some new values based on previous information. While they all accomplish the same task, they each have their own nuances.

Stream.iterate/2 is best suited for generating simple sequences. We’ll define a “simple sequence” as a sequence who’s next value can entirely be determined by its preview value.

For example, we can implement a simple incrementing sequence:


Stream.iterate(0, &(&1 + 1))

Taking the first 5 values from this stream (|> Enum.take(5)) gives us [0, 1, 2, 3, 4].

We ramp the complexity up a notch and generate a Mandelbrot sequence for a given value of c:


defmodule Mandelbrot do
  def sequence({cr, ci}) do
    Stream.iterate({0, 0}, fn {r, i} -> 
      # z₂ = z₁² + c
      {(r * r - i * i) + cr, (i * r + r * i) + ci}
    end)
  end
end

The value of z₂ is entirely determined by the value of z₁ and some constant.

Sequences with Accumulators

There are times when the next value in a sequence can’t be generated with the previous value alone. Sometimes we need to use other information we’ve built up along the way to generate the next value in our sequence.

A perfect example of this type of sequence is the Fibonacci sequence. To generate the nth value in the Fibonacci sequence, we need to know the previous value, n-1, and also the value before that, n-2.

This is where Stream.unfold/2 shines. The unfold/2 function lets us build up an accumulator as we generate each value in our sequence.

Here’s an stream that generates the Fibonacci sequence:


Stream.unfold({0, 1}, fn {a, b} -> {a, {b, a + b}} end)

Stream.unfold/2 takes the initial value of our accumulator and a next_fun function. Our next_fun function returns a tuple who’s first element is the value emitted by our stream, and who’s second element is the accumulator that will be passed into the next call to next_fun.

Taking the first 10 elements of this stream gives us a familiar sequence of numbers:


[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

For a more complex example of the power of Stream.unfold/2, check out how I implemented a stream that generates an integer spiral as part of an Advent of Code challenge.

Multiple Values per Accumulator

Sometimes we’re faced with a true doozy of a sequence, and even an accumulator isn’t enough to accurately describe the next value. For example, what if we need to emit multiple sequential values for each invocation of our next_fun?

Stream.resource/3 to the rescue!

The Stream.resource/3 function was originally intended to consume external resources, hence its name and start_fun and after_fun functions, but it also works beautifully for building sufficiently complex sequences.

Wolfram-style cellular automata, such as Rule 30, is a perfect example of this type of complex sequence.

To generate our Rule 30 sequence, we’ll start with an initial list of values (usually just [1]). Every time we call our next_fun, we want to calculate the entire next layer in one go (which would be [1, 1, 1]). Unfortunately, the size of these layers continues to grow (the next would be [1, 1, 0, 0, 1], etc…).

How would we write a stream that outputs each cell of each layer in order, and tells us the cell’s value and depth?


defmodule Rule30 do
  def stream(values) do
    Stream.resource(fn -> {values, 0} end, &next_fun/1, fn _ -> :ok end)
  end

  defp next_fun({values, layer}) do
    next = ([0, 0] ++ values ++ [0, 0])
    |> Enum.chunk_every(3, 1, :discard)
    |> Enum.map(&rule/1)
    {values |> Enum.map(&{layer, &1}), {next, layer + 1}}
  end

  defp rule([1, 1, 1]), do: 0
  defp rule([1, 1, 0]), do: 0
  defp rule([1, 0, 1]), do: 0
  defp rule([1, 0, 0]), do: 1
  defp rule([0, 1, 1]), do: 1
  defp rule([0, 1, 0]), do: 1
  defp rule([0, 0, 1]), do: 1
  defp rule([0, 0, 0]), do: 0
end

Stream.resource/3 makes short work of this potentially difficult problem. We start with our initial set of values, and a depth of 0. Each call to our next_fun tacks on a few buffering 0s to each side of our values list, and applies the rule function to each chunk of 3 values. Once we’ve finished calculating our next layer, we emit each value grouped with the current depth along with our updated accumulator tuple.

We can take the first 9 values from this stream and notice that they match up perfectly with what we would expect from Rule 30:


Rule30.stream([1])
|> Enum.take(9)
|> IO.inspect
# [                {0, 1},
#          {1, 1}, {1, 1}, {1, 1}, 
#  {2, 1}, {2, 1}, {2, 0}, {2, 0}, {2, 1}]

Final Thoughts

While we’ve mostly talked about mathematical sequences here, these and techniques ideas apply to any type of enumerative data that needs to be generated.

In closing, here are a few rules to remember when working with Stream.iterate/2, Stream.unfold/2, and Stream.resource/3:

I highly encourage you to check out Elixir streams if you haven’t. The Stream module is full of ridiculously useful tools to help streamline your data pipeline.