Akka Persistence and efficient querying for potential target Actor

Akka Persistence / Event Sourcing
1

Hello,

I am currently evaluating Akka with Akka Persistence for new project in our company. I am new to DDD and builiding Microservices the reactive way (with actors and without), but I like what I have seen so far.

As of now, I have worked through the IoT example as well as the platform guide with the shopping cart and am now trying to build a minimal proof-of-concept with only a few commands / events and a single EventSourcedBehaviour.

At the moment, I am a bit lost how to implement a specfic requirement with Akka Persistence; one that is actually beautifully showcased in the IoT example. For illustration purposes, I have also created a small tactical design diagram (see attachment).

In summary, I need to:

  • Be able to receive a lot (millions) of events of type ‘Part received’ from an external system within a very short time (eg during inital setup for a new customer). They are ‘Parts (Entity)’ of a ‘Device (RootEntity)’. Parts can reference each other and basically form a unidirectonal graph. Parts are received in random order. There can also be a lot of devices (lets say also a few 100k up to a million).
  • When I receive a part I need to be able to ask if there is already a ‘Device’ that contains a ‘Part’ that is referenced from the incoming ‘Part’. If yes the incoming ‘Part’ should be attached to the device. If no, the incoming ‘Part’ should create a new ‘Device’ and attach itself to it.
  • I need strong consistency for all of this.

So my question would be how to actually implement this efficient ‘ask-if-there-is-a-Device-Actor-that-should-handle-this’ with Akka Persistence and EventSourcedBehaviour. I am pretty sure I am missing something very obvious here.

What I have checked so far:

  • The IoT example deals with this in general simply by having a hierarchy of actors where the parents just query their children. I could imagine to somehow also group Parts and do it the same way, but it seems that using child actors is not recommended when using event sourcing (Is it common to stop child actors in event sourcing? - #2 by octonato)
  • The Shopping Cart example showcases the usage of the interaction pattern ‘Request-Response with ask from outside an Actor’, but this would require me to already have an ActorRef at hand.
  • To discover Actors, there seems to be the ‘Receptions’, but documentation suggests that I can only go looking with a known ServiceKey.

Any pointers into the right direction would be greatly appreciated.

Thx

Part Creation System
Part Creation System1068×936 101 KB

2

So this suggests that you want Part to be an aggregate, tracking which Device it’s a component of and which other parts it’s related to. If anything in world operates on parts without knowing what device the part is “part of”, then part can’t be a part of the device aggregate (the device aggregate can refer to parts by their ID).

By unidirectional graph, do you mean directed acyclic, i.e. that following the reference chain from some part will always eventually reach a part which refers to no other parts?

It seems that the ReceivePart command which is generated based on a PartReceived event from that external system (it’s generally a good practice, IME, to consider the external system’s events (facts which that external system cannot deny) as commands (which may be denied by this system)) has at least zero referenced parts. In Scala:

case class ReceivePart(part: PartId, references: Set[PartId])

So the ingestion process (e.g. an Akka Stream consuming from Kafka or Pulsar or whatever) will resolve a Part aggregate actor (e.g. via cluster sharding) and send a command saying “you’re received and you reference these other parts” (there could be other facts about the part from the external system which could be included here, too).

In the initial state of the Part aggregate actor, it will add the referenced part IDs to its relations. If the relations are now empty, it will also create and associate itself with a device. Otherwise, it will ask all of its relations “are you associated with a device?” (this might not necessarily be the ask pattern). Until it gets responses back from its relations, this part itself is not associated with a device.

In response to an “are you associated?” command (to keep with the imperative phrasing of a command, ReportDeviceAssociation might be a useful name) the Part aggregate actor responds with the associated device ID and adds the ID of the requestor to its set of relations if it’s associated. If it’s not associated, it adds the ID of the requestor to its set of relations and records that whenever it discovers what device it’s associated with, it should inform the requestor.

When a Part aggregate actor finds out from a relative that it’s part of a device, it then tells informs its relatives of this fact or resolves the conflict by beginning the device merge process.

It may be useful to, rather than delete the obsolete devices, keep them around as forwarders for the device they got merged into, BTW.

3

Hi Levi,

thx for the fast response.

So this suggests that you want Part to be an aggregate, tracking which Device it’s a component of and which other parts it’s related to. If anything in world operates on parts without knowing what device the part is “part of”, then part can’t be a part of the device aggregate (the device aggregate can refer to parts by their ID).

Alright, this is already very good input. It seems I still need to get my head around the basics of the modelling part.

By unidirectional graph, do you mean directed acyclic, i.e. that following the reference chain from some part will always eventually reach a part which refers to no other parts?

Yes, exactly.

It seems that the ReceivePart command which is generated based on a PartReceived event from that external system (it’s generally a good practice, IME, to consider the external system’s events (facts which that external system cannot deny) as commands (which may be denied by this system)) has at least zero referenced parts. In Scala:

case class ReceivePart(part: PartId, references: Set[PartId])

Ah yes this (it’s generally a good practice ... as commands (which may be denied by this system)) makes sense .

With the remainders of your suggestions I think I should be able to make good progress with proof-of-concept - thanks for the very detailed outline what needs to be done.

1 Like