# Cluster Sharding messages to /system/sharding/replicator not delivered

**URL:** <https://discuss.akka.io/t/cluster-sharding-messages-to-system-sharding-replicator-not-delivered/7669>\
**Category:** Akka Cluster\
**Tags:** akka-cluster, configuration\
**Created:** [January 2, 2021, 10:48pm UTC](https://discuss.akka.io/t/cluster-sharding-messages-to-system-sharding-replicator-not-delivered/7669 "2021-01-02T22:48:37Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sfosborn](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@sfosborn](https://discuss.akka.io/u/sfosborn)\
**Post date:** [January 2, 2021, 10:48pm UTC](https://discuss.akka.io/t/cluster-sharding-messages-to-system-sharding-replicator-not-delivered/7669/1 "2021-01-02T22:48:37Z")

</div>

Akka version 2.6.10  
Components:  
akka-management\_2.12  
akka-management-cluster-bootstrap\_2.12  
akka-discovery\_2.12  
akka-discovery-aws-api\_2.12  
akka-cluster\_2.12  
akka-persistence-typed\_2.12  
akka-cluster-sharding-typed\_2.12  
…

I am trying to use EventSourcedBehavior with Akka Cluster Sharding running on AWS using EC2 tag-based discovery with my service running in docker (one container per node).

Relevant config:

```auto
  discovery {
    aws-api-ec2-tag-based {
      tag-key = "platform:role"
      filters = "tag:MyFilterTag=myfiltertag"
    }
  }

  management {
    cluster.bootstrap.contact-point-discovery {
      service-name = "msg-orchestrator"
      discovery-method = "aws-api-ec2-tag-based"
    }

    http.hostname = ${PRIVATE_IP}
    http.port = 8558
    http.bind-hostname = 0.0.0.0
    http.bind-port = 8558
  }

  remote.artery {
    canonical.port = 25520
    canonical.hostname = ${PRIVATE_IP}
    bind.hostname = 0.0.0.0 # internal (bind) hostname
    bind.port = 25520 # internal (bind) port
  }

  cluster {
    # seed-nodes = ["akka://MsgOrchestrator@127.0.0.1:25520"] - using bootstrap
    sharding {
      number-of-shards = 1000
      downing-provider-class = "akka.cluster.sbr.SplitBrainResolverProvider"
      state-store-mode = ddata
      min-nr-of-members = 2 
    }
  }

```

Startup code:

```auto
        final ClusterSharding sharding = ClusterSharding.get(actorSystem);
        AkkaManagement.get(actorSystem).start();
        ClusterBootstrap.get(actorSystem).start();

```

Running two nodes (separate boxes), I have gotten to the point where a Leader is elected and both the leader and the other box join the cluster. Both boxes are logging `Received gossip status` messages.

Where I am stuck now is that, when the non-leader executes a Command (instantiating an EntityRef), the ShardRegion actor starts logging

`Trying to register to coordinator at [ActorSelection[Anchor(akka://MsgOrchestrator@<leader_ip>:25520/), Path(/system/sharding/MoMeetingCoordinator/singleton/coordinator)]], but no acknowledgement.`

and the leader starts logging

`Message [akka.cluster.ddata.Replicator$Internal$Status] from Actor[akka://MsgOrchestrator@<non_leader_ip>:25520/system/sharding/replicator#-502570199] to Actor[akka://MsgOrchestrator/system/sharding/replicator] was not delivered`

If I send a Command to the leader that pertains to the EntityRef created on the non-leader, the leader doesn’t know about it.

Any help much appreciated.

UPDATE: Once the “oldest” member of the cluster receives any Command, the ShardCoordinator seems to be initialized, and from thereon, it works as expected. Short of sending a dummy command on cluster startup, is there a correct way to ensure this is initialized? I guess I would have expected the BootstrapCoordinator to take care of this.

---

<div class="post-metadata">

**Author:** ![johanandren](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.akka.io/johanandren/32/85_2.png) [@johanandren](https://discuss.akka.io/u/johanandren)\
**Post date:** [January 18, 2021, 4:27pm UTC](https://discuss.akka.io/t/cluster-sharding-messages-to-system-sharding-replicator-not-delivered/7669/2 "2021-01-18T16:27:44Z")

</div>

You need to “start” the specific shard type by calling `ClusterSharding#init(…) during startup on each node. See third sample snippet down [in the docs](https://doc.akka.io/docs/akka/current/typed/cluster-sharding.html#basic-example).

---

<div class="post-metadata">

**Author:** ![sfosborn](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@sfosborn](https://discuss.akka.io/u/sfosborn)\
**Post date:** [January 20, 2021, 11:28pm UTC](https://discuss.akka.io/t/cluster-sharding-messages-to-system-sharding-replicator-not-delivered/7669/3 "2021-01-20T23:28:33Z")

</div>

Thanks - finally figured out that the .init method was global and not to start up a single Behavior instance.