Async/Await for the Monadic Programmer

Adam Rosien @arosien / @arosien@fosstodon.org

Inner Product LLC @InnerProductLLC

⁠

ScalaDays Seattle, 6 Jun 2023

⁠

Welcome to Seattle!

⁠

Remember the Chinese spy balloon?

What do I do?

I’m a Principal at Inner Product LLC, along with my partner Noel Welsh.

⁠

We’re consultants who write code,
train and mentor your staff.

We also write books.

⁠

We’ve both used Scala for over 10 years—we’ve seen a lot!

⁠

arosien detail NoelWelshSq

Styles: monadic vs. direct

⁠

My original dilemma

public async Task<ActionResult<bool>> GetRedacted(...) {
  var user = HttpContext.User as RedactedPrincipal;
  if (...) return BadRequest(...);

  var results = ...;
  if (!results.IsValid) return BadRequest(results.ToString());

  try {
    Responses<RedactedResponse> redacted = await _redacted.Redacted(...);

    if (metadata ?? false) {
      return Ok(...);
    }
    else { return Ok(...); }
  }
  catch (ArgumentException e) {
    return BadRequest(e.ToString());
  }
}

☜ this is C#

⁠

This direct style mixes:

asynchronous execution
conditional logic / branching
early returns

into a (mostly) linear sequence.

⁠

How do I translate (functional, monadic(?)) Scala into this style?

asynchronous execution
conditional logic
early returns

⁠

☜ Future, IO…
☜? hopefully some cats combinators
☜? monads

⁠

But why is this translation so complicated?

⁠

That is, why is it bugging me so much??

Outline

Using monads
Using async/await
Reconciling these styles

Using Monads

⁠

for {

values ← effects

} yield expr

⁠

* All effects must have the same type; e.g., all Option, all Future, etc.

"happy-path" programming

Monads abstract over errors (usually). This is (usually) a good thing.

⁠

def doSomething(req: Request): F[Response] =
  for {
    a <- serviceA(toARequest(req))
    b <- serviceB(toBRequest(a))
  } yield toResponse(a, b)

What does this really do?

def doSomething(req: Request): F[Response] =
  for {
    a <- serviceA(...)
    b <- serviceB(...)
  } yield toResponse(a, b)

def doSomething(req: Request): F[Response] =

  serviceA(...).flatMap { a =>
    serviceB(...).map { b =>
      toResponse(a, b)
    }
  }

Tip: introduce local values via `=`

def doSomething(req: Request): F[Response] =
  for {
    a <- serviceA(toARequest(req))
    rb = toBRequest(a)
    b <- serviceB(rb)
    res = toResponse(a, b)
  } yield res

Fail-fast

def doSomething(req: Request): F[Response] =
  for {
    a <- serviceA(toARequest(req)) 🧨
    b <- serviceB(toBRequest(a))
  } yield toResponse(a, b)

⁠

The call to serviceA fails.
No a is produced, so no call to serviceB, etc. Everything stops.
doSomething returns the error.

Fail-fast

Monad success type error type

Monad	success type	error type
`Option[A]`	`A`	`Unit`
`Either[E, A]`	`A`	`E`
`Try[A]`	`A`	`Throwable`
`Future[A]`	`A`	`Throwable`
`IO[A]`	`A`	`Throwable`
…	…	…

Option[A]

A

Unit

Either[E, A]

A

E

Try[A]

A

Throwable

Future[A]

A

Throwable

IO[A]

A

Throwable

…

Error handling

def doSomething(req: Request): F[Response] =
  for {
    a <- serviceA(toARequest(req))     🧨
           .handleError(_ => defaultA) 🙈
    b <- serviceB(toBRequest(a))
  } yield toResponse(a, b)

⁠

Note: we don’t use built-in conditional logic (if/else) with monads. They need to be into the monad, usually as combinators.

⁠

See also: MonadError typeclass for error-handling combinators.

Early return

def doSomething(req: Request): F[Response] =
  serviceA(toARequest(req))               👍
    .ensure(new RuntimeException("meh!")) 🧨
      (a => a.isValid)                    🧐
    .handleError(_ => defaultResponse)    ⏏️
    .flatMap {                            🥳
      for {
        b <- serviceB(toBRequest(a))
      } yield toResponse(a, b)
    }

⁠

I don’t actually know a good way to do this monadically. This is my best guess.

⁠

See also: MonadError typeclass for error-handling combinators.

Critique

People struggle with for-comprehensions. It’s a wholly different syntax.
On the other hand, perhaps they are conceptually simpler? Less syntax: no val keyword required, etc.
Conditionals, error handling and early return are all handled as combinators—that you need to know—on the monad. These use typeclasses and other more advanced patterns.

Using Monads

⁠

Monads abstract over the sequencing of effects: what happens next.
Monads fail-fast. We can also handle errors using monad-specific or monad-agnostic combinators (via Typelevel cats, etc.).
for-comprehensions provide a different + better syntax for monadic sequencing. But it is different than using vals and normal function calls, and so requires more training.

Using Async/Await

Async/Await example

val io: IO[Int] = ???

val program: IO[Int] =
  async[IO] {           (1)
    io.await + io.await (2)
  }

1	The `async` "capability" creates a scope where waiting may occur.
2	We can `await` effects and act on their results as regular values.

⁠

This is using the Typelevel Cats Effect async/await support.

What does this really do?

val io: IO[Int] = ???

val program: IO[Int] =
  async[IO] {
    io.await + io.await
  }

val io: IO[Int] = ???

val program: IO[Int] =
  io.flatMap { x =>
    io.flatMap { y =>
      IO.pure(x + y)
    }

⁠

Other systems and libraries create something similar.

Error handling

def doSomething(req: Request): F[Response] =
  async {
    val a = await(serviceA(toARequest(req))) 🧨
              .?(defaultResponse)            ⏏️
    val b = await(serviceB(toBRequest(a)))   🥳

    toResponse(a, b)
  }

⁠

There are proposals to add a early return operator (? in this example) which can optionally supply a default value in case of an error.

Early return (alternative)

def doSomething(req: Request): F[Response] =
  async {
    val a = await(serviceA(toARequest(req)))

    if (!a.isValid) defaultResponse
    else {
      val b = await(serviceB(toBRequest(a)))

      toResponse(a, b)
    }
  }

⁠

This does validation and early return.

Critique

Async/await is great for programmers who are new to concurrent programming.
But it is non-referentially-transparent. This really bugs functional programmers; refactoring is not safe
It has limited support for higher-order functions—like map or traverse--because it’s a macro.

Using Async/Await

⁠

It’s new in the Scala ecosystem, but can be a good on-ramp for those new to the language.
At the same time, programmers can use the most common parts of the languge (keywords like if, else, etc.) and only need to learn a very small set of new features in order to use asynchronous programming.

Reconciling these styles

Back to my grumbling…

⁠

I was looking for some theory to explain
my mental mismatch when translating
between monadic and direct styles.

⁠

linear

tree-like

lists

algebraic data types

statements

expressions

imperative

functional

Structure

linear

tree-like

Data structures

lists (iterables?)

algebraic data types

Unit of expression

statements

expressions

Paradigm

imperative

functional

Branching

if/else

MonadError typeclass

Error Handling

try/catch, ?-like capability methods

MonadError typeclass

⁠

These start to mix and match.

A path forward

Async/await is fantastic for pure → effectful refactoring.

+ async {
    ...

-   if (isFeatureFlag(FOO)) somethingLocal()
+   if (isFeatureFlag(FOO)) await(somethingRemote())
    else somethingElse()

    ...
+ }

⁠

You don’t have to change the (general) structure of the code.

Source: Li Haoyi

A path forward

Async/await is fantastic for pure → effectful refactoring.

⁠

You can then use the more full-featured monadic style in subsequent refactorings.

Summary

⁠

Async/await vs. monads is not a technological choice. It’s about programmers' needs.

⁠

And different programmers have different needs.

⁠

Multiple styles can coexist, and we can work on providing guidance on how to do reduce the friction.

⁠

References

async/await:

What Color is Your Function? Classic introduction to the subject.
dotty-cps-asyc "This is the implementation of async/await transformation for Scala 3 (Dotty), based on an optimized version of CPS (Continuation Passing Style) transformation, where the continuation is ‘pushed’ to the monad."
Towards A New Base Library for Asynchronous Computing "This is a proof of concept for a base library for asynchronous computing in direct style."
A giant debate on reddit about Scala async/await wherein Li Haoyi advocates async/await, amongst other threads.

⁠

Extra thanks to:

Gabriella Gonzalez "I’ve been trying to write a blog post explaining the difference between functional and imperative programming and I’ve failed multiple times due to writer’s block"
impurepics, as always.
Other internet sources for the tweets, images, wisdom and vibes that I’ve failed to accurately cite.