Running ClusterSingleton with different processes

Hello everyone,

right now I am in the process of getting the hang of Akka framework and managed to run few sample examples especially the ClusterSingleton example involving a frontend and a backend with workers. Everything runs fine when I run the example within a single process.e.g starting frontend,backend plus workers (in maven context: mvn compile exec:java -Dexec.mainClass=“worker.Main”)

However, if I attempt to run the same example with different processes, the first process gives a warning;

[WARN] [02/21/2019 23:35:25.900] [ClusterSystem-akka.remote.default-remote-dispatcher-6] [akka.tcp://ClusterSystem@127.0.0.1:2551/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FClusterSystem%40127.0.0.1%3A2552-0] Association with remote system [akka.tcp://ClusterSystem@127.0.0.1:2552] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://ClusterSystem@127.0.0.1:2552]] Caused by: [Connection refused: /127.0.0.1:2552]

The second process does not start giving the following exception ;

[ERROR] [02/21/2019 23:37:25.223] [ClusterSystem-akka.remote.default-remote-dispatcher-7] [akka.tcp://ClusterSystem@127.0.0.1:2552/system/endpointManager/endpointWriter-akka.tcp%3A%2F%2FClusterSystem%40127.0.0.1%3A2551-5] AssociationError [akka.tcp://ClusterSystem@127.0.0.1:2552] <- [akka.tcp://ClusterSystem@127.0.0.1:2551]: Error [[B cannot be cast to [C] [
java.lang.ClassCastException: [B cannot be cast to [C
at akka.remote.artery.FastHash$.ofString(LruBoundedCache.scala:18)
at akka.remote.serialization.ActorRefResolveCache.hash(ActorRefResolveCache.scala:61)
at akka.remote.serialization.ActorRefResolveCache.hash(ActorRefResolveCache.scala:55)
at akka.remote.artery.LruBoundedCache.getOrCompute(LruBoundedCache.scala:110)
at akka.remote.RemoteActorRefProvider.resolveActorRef(RemoteActorRefProvider.scala:403)
at akka.cluster.pubsub.protobuf.DistributedPubSubMessageSerializer.akka$cluster$pubsub$protobuf$DistributedPubSubMessageSerializer$$resolveActorRef(DistributedPubSubMessageSerializer.scala:167)

Am i missing something with the concept of ClusterSingleton? Below is my application.conf and worker.conf respectively;

akka {

actor.provider = “akka.cluster.ClusterActorRefProvider”

remote.netty.tcp.port=0
remote.netty.tcp.hostname=127.0.0.1

cluster {
seed-nodes = [
“akka.tcp://ClusterSystem@127.0.0.1:2551”,
“akka.tcp://ClusterSystem@127.0.0.1:2552”]

auto-down-unreachable-after = 10s

}

extensions = [“akka.cluster.client.ClusterClientReceptionist”]

persistence {
journal.plugin = “akka.persistence.journal.leveldb-shared”
journal.leveldb-shared.store {
# DO NOT USE ‘native = off’ IN PRODUCTION !!!
native = off
dir = “target/shared-journal”
}
snapshot-store.plugin = “akka.persistence.snapshot-store.local”
snapshot-store.local.dir = “target/snapshots”
}

}

worker.conf

akka {

actor.provider = “akka.remote.RemoteActorRefProvider”

remote.netty.tcp.port=0
remote.netty.tcp.hostname=127.0.0.1

cluster.client {
initial-contacts = [
“akka.tcp://ClusterSystem@127.0.0.1:2551/system/receptionist”,
“akka.tcp://ClusterSystem@127.0.0.1:2552/system/receptionist”
]
}

}

Appreciate your help in this regard.

ClassCastException doesn’t look good. Haven’t seen that before. Is this with latest Akka version 2.5.21? Please create an issue in Github akka/akka describing how and what you run.

maybe a dumb question, as there is no bind error:
Are the two processes trying to bind same 2551 and 2552 port on same machine twice?

@patriknw … yes, this is with the latest version 2.5.21. I will launch a ticket soon as I will double-check the code just to be sure that I did not miss anything.
@rahat, no. I double checked it. I start the first process on port 2551 and waited for atleast 10 minutes (just to be sure) and then started the second process for the port 2552.