Why the remote Akka nodes become unreachable however the network is reachable?

[zhangmroy (zhangmeng)]

Hi, I would like to rebuild my ODL clusters on the overlay network.

the cluster before: these nodes connect each other on the underlay network(11.128.85.27, 11.128.85.58, 11.128.85.79), and the cluster works well

the cluster network overlay: I started three dockers which are assigned with overlay ips(192.168.10.1 ,192.168.10.2, 192.168.10.3), and use veth peer to connect the docker to Host, use vxlan vtep to connect eath Host.

the overlay network seems ok as I can ping each other in the docker

But when I started the ODL cluster, the log tells me the akka peer becomes unreachable and cluster starts failed, the message is following

when I use underlay Ip , it turns ok ,

Can anybody tell me why the peer node become reachable?

[johanandren (Johan Andrén)]

This looks it might be more a question for the opendaylight community and their forums.

In general Akka cluster nodes being marked unreachable could be about issues with IP networking between nodes, firewalls, or misconfigurations (the canonical IP of each node must be reachable from all other nodes) but it can also happen if nodes are overloaded and not able to respond in a timely fashion to messages from other cluster nodes.

Note that unreachable is not a final state, if the nodes end up being able to talk again an unreachable node will recover.

[davidogren]

I suspect that it’s the canonical IP config.

Essentially each node is identifying itself when it registers itself to the cluster. So even if it can be pinged on 11.128.85.58 (for example), if the node advertising the internal IP (192.168.10.2) then all of the other nodes will be trying to communicate with that node on an IP that can’t be routed.

[zhangmroy (zhangmeng)]

Thanks for reply， The reason is MTU of the network interface in docker is too big

[zhangmroy (zhangmeng)]

The reason is MTU of the network interface in docker is too big, some packets was lost