1173 words, 6 min read

Some workflows are slow, not because the work is hard, but because we do it one step at a time. A typical example is a pipeline that calls an LLM (or any slow external service) several times: clean up a body of text, generate a title, produce a social media blurb, and suggest tags. Each call takes seconds, none depends on another's output, and yet they run back to back.

Elixir makes concurrency cheap, so the fix is obvious: run the calls in parallel. The interesting part is doing so without changing how errors behave. This post covers a tiny helper module, Concurrent, that we added to do exactly that.

The problem

The first instinct is to reach for Task.async/1 and Task.await_many/2:

[body, title, social, tags] =
  [
    fn -> clean_body(post) end,
    fn -> generate_title(post) end,
    fn -> generate_social(post) end,
    fn -> suggest_tags(post) end
  ]
  |> Enum.map(&Task.async/1)
  |> Task.await_many()

This works on the happy path and cuts the total latency from the sum of the steps to roughly the slowest step. But it has a sharp edge: Task.async/1 links the task to the caller. If any function raises, the task process crashes, the link propagates the exit signal, and the caller dies with it.

That is a very different failure mode from sequential code. With sequential code, you can write:

try do
  run_pipeline(post)
rescue
  e in SomeError -> handle(e)
end

and it works. After naively parallelising, the rescue never fires. The caller gets killed by an exit signal instead, and your error handling, retries, and logging around the call are bypassed. In a web request or a background job, this typically shows up as a confusing crash rather than the cleanly handled error you had before.

The solution

We wanted the speed-up of concurrency with the ergonomics of a normal function call. The idea is simple:

  1. Catch exceptions inside each task and return them as data.
  2. Wait for all tasks to finish.
  3. Re-raise the first error in the caller, with the original stacktrace.
defmodule Concurrent do
  @moduledoc """
  Small helpers for running independent work concurrently.
  """

  @default_timeout :timer.minutes(2)

  @doc """
  Runs all funs concurrently and returns their results in order.

  A raise inside a fun is caught in its task and re-raised in the caller with
  the original stacktrace, so callers can still `rescue` it.
  """
  @spec all([(-> term())], timeout()) :: [term()]
  def all(funs, timeout \\ @default_timeout) do
    funs
    |> Enum.map(&Task.async(fn -> run_safely(&1) end))
    |> Task.await_many(timeout)
    |> Enum.map(fn
      {:ok, value} -> value
      {:error, error, stacktrace} -> reraise error, stacktrace
    end)
  end

  defp run_safely(fun) do
    {:ok, fun.()}
  rescue
    error -> {:error, error, __STACKTRACE__}
  end
end

Usage is a drop-in replacement for the sequential version:

[body, title, social, tags] =
  Concurrent.all([
    fn -> clean_body(post) end,
    fn -> generate_title(post) end,
    fn -> generate_social(post) end,
    fn -> suggest_tags(post) end
  ])

Why it works

Errors become values inside the task. run_safely/1 wraps the function in an implicit try/rescue and returns either {:ok, value} or {:error, error, stacktrace}. Because the task never crashes, the link never fires and the caller is never killed.

Errors become exceptions again in the caller. After Task.await_many/2 returns, we map over the results. A success unwraps to its value; a failure is re-raised with reraise/2, using the original stacktrace. The stacktrace points at the code that actually failed, not at the helper, which keeps your error tracker useful.

Results keep their order. Task.await_many/2 returns results in the same order as the tasks were started, so destructuring the result list is safe.

There's a sensible timeout. Task.await_many/2 defaults to 5 seconds, which is too short for LLM calls and many network operations. The helper defaults to two minutes and lets the caller override it.

Design decisions and trade-offs

  • No process without a runtime reason. This is a plain module with plain functions. There is no GenServer, supervisor, or state. The tasks are short-lived and exist only for the duration of the call, which is exactly what Task is for.
  • Only independent work. The helper is valid only when the steps do not depend on each other. When we introduced it, we also had to examine the data flow: one step was waiting for the output of a previous, slow step when it only needed a cheaper, earlier intermediate result. Removing that artificial dependency made parallelism possible in the first place.
  • Only rescue, not catch or exits. The helper handles exceptions. Throws and exits inside a task will still crash it and, because of the link, the caller. That's a deliberate simplification; if you need to handle those, extend run_safely/1.
  • First error wins. If several functions fail, the first failing one in list order is re-raised. The others are simply discarded, since all tasks have already completed by then.
  • Not a replacement for Task.Supervisor. For fire-and-forget work, or when you want isolation from the caller's lifecycle, use Task.Supervisor.async_nolink/3 and handle the results explicitly. Concurrent.all/2 is for the "do these N things and give me all the answers" case.

Testing it

The behaviour worth pinning down is small and easy to test:

test "returns results in order" do
  assert [1, 2, 3] = Concurrent.all([fn -> 1 end, fn -> 2 end, fn -> 3 end])
end

test "runs concurrently" do
  {micros, _} =
    :timer.tc(fn ->
      Concurrent.all(List.duplicate(fn -> Process.sleep(100) end, 5))
    end)

  assert micros < 300_000
end

test "re-raises errors in the caller" do
  assert_raise RuntimeError, "boom", fn ->
    Concurrent.all([fn -> :ok end, fn -> raise "boom" end])
  end
end

The third test is important: it proves that a failure surfaces as a normal, rescuable exception in the calling process.

Takeaways

  • Parallelising independent, slow calls is often the cheapest performance win available, and in Elixir it takes only a few lines.
  • Task.async/1 is linked. To make failures behave as they do in sequential code, convert them to data inside the task and re-raise them in the caller.
  • Keep the abstraction tiny. Thirty lines and no processes to supervise are easier to reason about than a framework.
  • Before you parallelise, examine the data flow. Often the real win is removing an unnecessary dependency between steps.