# Nodes not rejoining after cluster spilt

**URL:** https://discuss.akka.io/t/nodes-not-rejoining-after-cluster-spilt/721
**Category:** Akka Cluster
**Tags:** akka-cluster
**Created:** [April 16, 2018, 11:43am UTC](https://discuss.akka.io/t/nodes-not-rejoining-after-cluster-spilt/721 "2018-04-16T11:43:57Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![makesh](https://avatars.discourse-cdn.com/v4/letter/m/b782af/32.png) [@makesh](https://discuss.akka.io/u/makesh)
#### Post date: [April 16, 2018, 11:43am UTC](https://discuss.akka.io/t/nodes-not-rejoining-after-cluster-spilt/721/1 "2018-04-16T11:43:57Z")

</div>

Hi,

We use akka v2.4.10 and have 4 nodes in an akka cluster (ex. A, B, C, D) and we observe the below cluster split scenario.

1. When the actor system from a node (i.e D) gets restarted, we received “Associate timed out after [15000 ms]” message when tried connecting with other nodes.  
The cluster.state().members() contains only node D when the actor system is completely started.

Is there a way to get the node D back into the cluster (where A, B and C present) ?

1. We receive “Associate timed out after [15000 ms]” message during Association failure for a node from akka.remote.ReliableDeliverySupervisor.

Thanks,  
Makesh

---

<div class="post-metadata">

### Author: ![patriknw](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.akka.io/patriknw/32/89_2.png) [@patriknw](https://discuss.akka.io/u/patriknw)
#### Post date: [April 17, 2018, 5:23pm UTC](https://discuss.akka.io/t/nodes-not-rejoining-after-cluster-spilt/721/2 "2018-04-17T17:23:37Z")

</div>

Do you mean that you see this when D tries to join to ABC? That shouldn’t happen. Difficult to guess what could be causing it without inspecting logs. Could it be that your network is not configured for peer-to-peer, so that connections can only be opened in one direction (firewalls).

BTW you are using a very old version that is end-of-life.

---

<div class="post-metadata">

### Author: ![makesh](https://avatars.discourse-cdn.com/v4/letter/m/b782af/32.png) [@makesh](https://discuss.akka.io/u/makesh)
#### Post date: [April 18, 2018, 1:26pm UTC](https://discuss.akka.io/t/nodes-not-rejoining-after-cluster-spilt/721/3 "2018-04-18T13:26:41Z")

</div>

Hi Patrik,

Thanks for your response.

In our case, when we start D node, it tries to join the cluster(ABC). But ABC remains unreachable due to some network unavailability. So D forms a separate cluster.  
At this point, the Cluster.state().members() on D has only D node and not other nodes.

No firewalls present at our end and even when the network becomes stable at later time, D remains in a separate cluster.  
Now to recover from this, we restart the node D (actor system) manually to join the ABC cluster.

So the query is, any option available in Akka to rejoin these cluster islands without restarting the nodes?

Regards,  
Makesh

---

<div class="post-metadata">

### Author: ![patriknw](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.akka.io/patriknw/32/89_2.png) [@patriknw](https://discuss.akka.io/u/patriknw)
#### Post date: [April 18, 2018, 3:09pm UTC](https://discuss.akka.io/t/nodes-not-rejoining-after-cluster-spilt/721/4 "2018-04-18T15:09:48Z")

</div>

Then you have configured node D as the first element in the seed-nodes list, otherwise it wouldn’t join itself.

Some reading:

- [https://doc.akka.io/docs/akka/current/cluster-usage.html#joining-to-seed-nodes](https://doc.akka.io/docs/akka/current/cluster-usage.html#joining-to-seed-nodes)
- [https://developer.lightbend.com/docs/akka-management/current/bootstrap.html](https://developer.lightbend.com/docs/akka-management/current/bootstrap.html)
