We are observing high latency intermittently while using akka http 10.1.10.
The client is also passing invalid header requests to the AKKA HTTP server which we can identify using application logs below.
{"@timestamp":“2021-12-24T06:12:58.380+00:00”,"@version":1,“message”:“Illegal header: Illegal ‘content-type’ header: Invalid input ‘,’, expected tchar, OWS, ws or ‘EOI’ (line 1, column 32): application/json; charset=UTF-8,application/json\n ^”,“logger_name”:“akka.actor.ActorSystemImpl”} ,
One common pattern that we can see on the server-side is , whenever the frequency of the above warning log is high, only then we are observing latent behavior at server side.
We tried to check possible areas for any resource crunch on the server side but we could not find any.
Dependencies --com.typesafe.akka:akka-actor_2.11:2.5.23