Role of xyzEntity Sharing Coodrinators

Hi all,

In the messages table, I see some persistence ids with the following names:

/sharding/CustomerEntityCoordinator

And the events are like of this shape:

qakka.tcp://application@10-8-6-133.xxxx-zxz.pod.cluster.local:10001/system/sharding/CustomerEntity#793149114

What’s the role of these entity coordinators and why are they logging to the messages table? Also, is it okay to delete them? I deleted these messages and restarted the service, seems like nothing went wrong so far.

Thanks,

Hi,

these entries are persisted states of Cluster Sharding when state store mode is persistence.
From Lagom 1.4.x new state store modeddata is available to use.

Br,
Alan

1 Like

Thanks Alan for the answer.

I believe deleting their messages won’t make any differences to the services because the coordinators will behave the same way how the entities will behave when it doesn’t see any events from itself? i.e. just starting from the default state

unfortunately my knowledge about this is limited so i can not confirm your statement.
I suggest you do not delete it manually.
I also suggest migrating to ddata. Migration is trivial, you just need to avoid rolling updates.

Thanks again Alan.

Just two more things:

  1. So you mean just for the migration, we cannot use rolling updates. It does not mean that we cannot use rolling updates once we move to ddata, am I right?

  2. I don’t quite understand the real upside of going to ddata from persistence. Do you mind just sharing a bit on this please?

Thank you very much Alan!

  1. You are right. Non rolling update is required only once. Check this for more details.

  2. From Cluster Sharding:

The functionality when using the two modes is the same. If your sharded entities are not using **Akka Persistence** themselves it is more convenient to use the Distributed Data mode, since then you don’t have to setup and operate a separate data store (e.g. Cassandra) for persistence. Aside from that, there are no major reasons for using one mode over the the other.
For Lagom there is no major difference because Akka persistence is used in any way.

But from Lagom 1.5 MigrationGuide:

If you still haven’t adopted ddata as the cluster sharding mode and your application can tolerate a *one time only* downtime upgrade, **we recommend** you to enable ddata

There is no explanation why it is recommended but, from my opinion, major reason is to be independent of the store access.

Br,
Alan

2 Likes

Thanks a lot Alan. Appreciate your answer.

There’s no real need for the shard coordinator state to survive full restarts of the cluster. It is only used to coordinate among nodes of a running cluster. If you shut down the cluster fully, it’s safe to remove the persistent data.

This is why we recommend the ddata mode: it’s unnecessary to store the data persistently in a database and it can cause additional problems in the case of a split brain scenario that causes the shard coordinator event data to become corrupted. In Akka 2.6 (and the next version of Lagom), persistence mode will be deprecated:

I raised an issue to clarify the Akka documentation:

2 Likes

Thanks @TimMoore - if I read the migration guide correctly, migration from persistence to ddata does NOT require downtime if my clusters are already on Lagom 1.4.x, am I right?

@lejoow, no that’s not correct. You do need downtime when making the change from persistence to ddata — you can’t run some nodes in one mode and some in the other at the same time. You need to stop all nodes, then make the configuration change, then start them up again. You can then delete the persistent shard coordinator data from the database later, while the services are up… or not… it doesn’t hurt anything to keep it there, as it will be ignored by ddata mode.

If this isn’t clear in the migration guide, it would be great if you could suggest an update to anything that is confusing.

1 Like