Distribution of ShardedDaemonProcess Actors

Akka Akka Cluster
1

I ran into some distribution issues with ShardedDaemonProcess. For example, let’s say you need to build 3 projections and you want to have 3 workers for each projection. So for each projection, we will make a call to shardedDaemonProcess.init("projection-name", 3, _ => projectionBuildingBehavior()). So in total we will start 9 daemon processes, but the problem is that if you scale the cluster to 9 nodes, 6 of them will be sitting completely idle, because the entity ID for each process is just a number and process #1 for each .init call will end up in the same node, because all of them will do "1".hashCode % 3 to determine the shard region.

One workaround would be to just have a single .init call for all projections and then based on the process number determine what projection worker to spawn. But having separate .init calls is much more intuitive.

I wonder, should ShardedDaemonProcess be modified to have overall better distribution, or is that not really possible? In which case it would still be good to update the docs about this behavior.

2

One thing you could do is to implement your own shard allocation strategy which is either aware of all projections and allocates based on that, or just some clever logic that will not end up with the same allocations for all entity types.

From a quick look at the ShardedDaemonProcess APIs it seems like we are currently missing a way to specify a custom shard allocation strategy for it though. I created #31320 to track that.

1 Like