After upgrading Lagom from 1.4.x to 1.5.x, we are seeing a significant increase of the Scheduled sending of heartbeat was delayed. message. Do you have any guidance on how we can investigate this or why this would happen after the upgrade?
I don’t know why this would happen more after upgrading, but it indicates that the Akka default dispatcher is being starved of CPU time.
There are a few common reasons for this:
- Frequent blocking I/O calls are being made on the default dispatcher
- Non-yielding, CPU-intensive functions are executing on the default dispatcher
- Lengthy garbage collection cycles are pausing the entire VM
I hope that gives some some leads to investigate. If you have a Lightbend Platform Subscription, you can also use the Akka Thread Starvation Detector to help diagnose these types of problems.
Hello,
I am also facing the issues with the delayed heartbeats. It is very strange because it is happening even without load. I also tried to use a custom dispatcher for the cluster with no luck. I created a repo, which you can use to reproduce it. Link : https://github.com/CostasChaitas/Akka-Demo
Any help would be much appreciated. Thanks
1 Like