Are there any best practices on how to avoid cycles in streams?

Akka
1

Hi,

I am trying to implement a flow that has some cyclic logic. Current flow design looks like following:

     *        +------------------------------------------------ 0 +-----------+
     *        |                                                   | partition |
     *        |      +-----------------+                          |           | 1 ------>
     *        +--> 0 |                 |     +-----------+ 0 ---->+-----------+
     * >---------> 3 |MergePrioritized | --> |           |       +-----+
     *        +--> 1 |                 |     | Partition | 1 --->| Flow|----+
     *        |+-> 2 |                 |     |           |       +-----+    |
     *        ||     +-----------------+     +-----------+ 2 --->+-------+  |
     *        |+-------------------------------------------------|  Flow |  |
     *        |                                                  +-------+  |
     *        +-------------------------------------------------------------+

However, I faced a lot of troubles related to undefined code behaviour, deadlocks etc. Could you please tell me if there are some recommendations on how to avoid cycles in the stream if you have a repeatable logic that is usually implemented as a loop?
PS: I am using Java as a main language.

Thanks.

– Sergii

2

Recently there was a contribution to akka-stream-samples that makes it easier and safer to do feedback loops. Your graph is a bit more complex, than usecases in the PR, but you might find the ideas from that PR still interesting.

Usually the deadlocks occur, when messages are being merged in to the feedback loop, and never leave the loop until eventually there are too many messages in the loop so the loop starts to backpressure and eventually deadlocks. Have you seen the documentation page about the deadlocks in the graph cycles?

3

Hi Martynas,

Thank you very much! Yes, I have seen the documentation. In my case it was not pure deadlock, but eagerComplete configuration. When I set it to false my flow never ends, but when it is true it seems to be ending earlier than it should.

I was even thinking about streaming “loop” data into a separate Sink, then convert it into a source like explained here: Create Source from Sink (and vice versa) and merge newly created Source with the initial one. In this case I will have less problems with back-pressure and eagerComplete.

– Sergii