In Singleton Actor, Scheduler schedule method not running after 1 or 2 runs

The region will repeatedly retry the registration until it succeeds, with a pretty short interval, it will only start logging after 5 tries though. The presence of that warning one or a few times does not necessarily mean it permanently failed, just that it did not succeed in interacting with the coordinator multiple times.

If it keeps logging the warning infinitely it means it can’t communicate with the oldest node for some reason, or sharding was never started on the oldest node.

One more observation was that we also tried deploying the previous build (Rollback version of app) with the same strategy (Oldest node at last, once deployment of other nodes are success). At that time also we faced same WARN message.
So at the end we had to kill all nodes to clear the cluster state.

It was almost infinitely only had seen more than 200 (Total [3] buffered messages).