Runtime implementation(s)

[MateuszKowalewski]

As I understand it (please correct me if I'm wrong!) Libretto is at its core "just" an abstract (domain specific) language. It has its own evaluation semantics but isn't tied to a runtime (and associated semantics). That's one of the more interesting parts about it imho as it would make switching the runtime implementation which works underneath quite simple. One "just" needs to (re)implement the DSL in terms of the desired runtime system.

Libretto has currently a runtime implemented based on Scala's Future. But other runtimes / concrete implementations of the DSL are not only possible but even something planed.

Building a performant, portable, and feature complete async runtime is quite an undertaking OTOH.

So how about basing this thing for the time being on something proven?

I've seen some ZIO integration, which seems a good starting point. Another idea I had was to use Pekko as the low-level runtime.

I understand that Libretto was actually invented to get rid of things like ZIO, Akka/Pekko, and Futures in general. But this won't happen overnight. If Libretto wants to succeed it needs to run in the environments that are already deployed out there. I guess we just can't wait for hardware which executes Libretto programs directly… 😁

So I think actors could be in fact for now a nice base as a Libretto runtime. They're quite low-level and model very well "little machines" which react to streams of messages (and can have some internal state). At the same time something like Akka/Pekko is know to scale very well (vertically and horizontally!), is performant, and runs reasonably well even in demanding production environments. It's definitely a solid solution in this space. Only not one you would like to touch directly as "end user" due to all the unconstrained effects (spooky action at a distance) everywhere.

Another idea of my was to review co-routines as a more lightweight (just local) base runtime implementation for Libretto. Co-routines can be very efficient and performant but at the same time "reasonably simple" to implement (as shown for example by the decision to include them in C++ lately).

Does this make any sense? What do you think about the ideas?

Also I suspect the current effort at EPFL to build an async runtime is related here. But I'm very unsure how exactly Libretto and Scala's "new Future" match up (or don't).

Would having proper continuations on the language level make anything better for Libretto? If so, I guess it would make sense to start to talk to the people over there so Scala could take care of Libretto's needs and input form this project.

[TomasMikula]

As I understand it (please correct me if I'm wrong!) Libretto is at its core "just" an abstract (domain specific) language. It has its own evaluation semantics but isn't tied to a runtime (and associated semantics). That's one of the more interesting parts about it imho as it would make switching the runtime implementation which works underneath quite simple. One "just" needs to (re)implement the DSL in terms of the desired runtime system.

Correct.

Libretto has currently a runtime implemented based on Scala's Future. But other runtimes / concrete implementations of the DSL are not only possible but even something planed.

Building a performant, portable, and feature complete async runtime is quite an undertaking OTOH.

Correct.

So how about basing this thing for the time being on something proven?

Nothing against something proven. But it has to be clear what that something is going to get us. Some obvious things that come to mind:

• Convenience. Does it let us express some aspects of Libretto especially nicely and concisely?
• Performance. Does it bring non-trivial performance improvement?
  By my guess, the most wasteful parts of the current implementation are:
  • The tasks (Futures) submitted to the executor are micro-tasks. Like, wait for a couple Futures, do something trivial to produce a new Future and complete a Promise in the process. It's likely that the overhead of Futures (scheduling, synchronization, ...) dwarfs everything else.
  • The above process produces a lot of garbage (short-lived heap-allocated objects).
• Scalability. Does it eliminate some scalability bottleneck of the current implementation?
  Surely the scheduler and garbage collector are bottlenecks of the current implementation.
• <what else?>

Clearly, replacing Futures by ZIOs or actors to run the micro-tasks is not going to eliminate the performance drain or scalability bottlenecks.
Neither do I see those as being convenient for Libretto implementation.

There needs to be a good way to do larger chunks of work synchronously to avoid that concurrency overhead.

To that end, I think it makes sense to not try to interpret the Libretto program directly, but to elaborate it to some intermediate representation. That representation would again be a data structure, but this time a live, mutable data structure. Different regions of that structure could then be assigned to different threads/actors for executing rewrites in that region. Akka, ZIO, etc could then be considered to run these regions, but I don't see them as saving us the hard work. (Though I could just be blind.)

I've seen some ZIO integration, which seems a good starting point. Another idea I had was to use Pekko as the low-level runtime.

I understand that Libretto was actually invented to get rid of things like ZIO, Akka/Pekko, and Futures in general. But this won't happen overnight. If Libretto wants to succeed it needs to run in the environments that are already deployed out there.

There are 2 different things here:

1. Interface with other Scala code.
   To this end, Libretto provides an unsafe, imperative interface to interact with the surrounding Scala program. The unsafe interface can sometimes be wrapped to provide a safe integration. That is what was done for integrating with ZIO streams. Note that both the Libretto part and the ZIO part run in their respective runtimes.
2. Base Libretto implementation on some other runtime.
   Here, again, it needs to be justified by what it brings us (see above).

I guess we just can't wait for hardware which executes Libretto programs directly… 😁

💯

So I think actors could be in fact for now a nice base as a Libretto runtime. They're quite low-level and model very well "little machines" which react to streams of messages (and can have some internal state). At the same time something like Akka/Pekko is know to scale very well (vertically and horizontally!), is performant, and runs reasonably well even in demanding production environments. It's definitely a solid solution in this space. Only not one you would like to touch directly as "end user" due to all the unconstrained effects (spooky action at a distance) everywhere.

Another idea of my was to review co-routines as a more lightweight (just local) base runtime implementation for Libretto. Co-routines can be very efficient and performant but at the same time "reasonably simple" to implement (as shown for example by the decision to include them in C++ lately).

Does this make any sense? What do you think about the ideas?

Also I suspect the current effort at EPFL to build an async runtime is related here. But I'm very unsure how exactly Libretto and Scala's "new Future" match up (or don't).

Would having proper continuations on the language level make anything better for Libretto? If so, I guess it would make sense to start to talk to the people over there so Scala could take care of Libretto's needs and input form this project.

My lack of practical experience with delimited continuations obviously limits my imagination, but I don't see them as a big win for Libretto implementation.

If it's about direct style, well Libretto is already direct style. Having also the implementation in direct style is an unimportant detail.

If it's about being lightweight, I'll take that.

But at this moment, I don't see a big expressivity gain with delimited continuations.

[MateuszKowalewski]

Sorry for the long delay.

I want to have some code prepared for further discussions. But there are still some distractions in here I want to address first before moving to the more funny parts. I'll be back, soon!