Use of Subflows (E.g. splitWhen + mergeSubstreams) vs. separate stream materialization for every element

Akka Libraries Akka Streams & Alpakka
1

I’m curious if there are any general rules of thumb regarding the performance between those two approaches.

My use case calls for incorporating multiple Flows, which I do not control whether or not they throw exceptions, into my overall stream. My goal is to insulate my overall stream from failure due to a thrown exception from one of these “unsafe” flows.

I’ve explored two approaches:

  1. use splitWhen and mergeSubstreams, with the unsafe flow in between, in order to run each element through it’s own subflow. This requires also including a recover stage in there to ensure the overall stream cannot be failed by an exception in the subflow.

  2. much simpler to just use a mapAsync stage that materializes the “unsafe” flow on every element. eg: mapAsync(in => Source.single(in).via(unsafeFlow).runWith(Sink.head))

The 2nd approach is much simpler. So I’m just wondering if there is a significant performance difference between the two. With #2 I understand that the unsafeFlow is materialized for every element, but I believe the same is true in approach #1 yes? Or is there perhaps some significant optimization internally in akka that might make #1 a more performant approach?

thanks

2

You are right, both of them will cause one materialization of the subflow per element so from that perspective neither will likely be more performant. There is one difference with mapAsync allocating and running an additional stage (Sink.head) and a Future though. May or may not make a difference for your application, benchmark and see!

Also, note that mergeSubstreams out of the box is more like mapAsyncUnordered, since it will emit whatever result comes first through the sub streams downstream - mapAsync will always emit the elements in the order the triggering element came in, potentially causing head of line blocking.

1 Like