Akka Http closes connection during response processing

Akka Libraries Akka HTTP
1

I am facing issue with closed connection while using akka http client and processing response with huge file(100MB+) using Http().singleRequest(). Akka-http version 10.2.4

I am making 2 concurrent requests to download the same file from the same host (e.g. http://ipv4.download.thinkbroadband.com/100MB.zip) but having for the processing of the response only one thread, so that response.dataBytes from the responses are being processed consequently. The requests are HTTP/1.1 and the responses contain Connection: keep-alive headers.

The processing of the first response is always successful, but the second one is always failing with ErrorInfo(Entity stream truncation. The HTTP parser was receiving an entity when the underlying connection was closed unexpectedly., (details omitted), )
I have tried all possible timeouts(client and host-connection-pool) to prevent the connection from being closed, but no go.

When I try to make two requests, but each request goes to a different host, then everything works fine. Both responses are being processed successfully.
Having 2 threads also fixes the problem.
The setting host-connection-pool.max-connections = 1 (default is 1024) also helps. But, I think that it is an ugly workaround.

Any ideas how to fix closed connection without workarounds?

2

Why do you have only one thread available? That sounds as if you are running two requests but can only handle one response at a time so the server is probably just timing out after a while.

Using streams and other asynchronous means it should well be possible to handle more than response at the same time.

3

Well, actually it was just an experiment during development. In prod we gonna configure 50 threads to handle responses.
I stumbled upon the issue accidentally.

Do you have an idea why it works with the requests which are being sent to different hosts?

4

It might be a timing issue. The most likely reason for the original problem is that you run two requests on the same connection but the server times out the connection before it is able to send a response.

With two different target hosts you use two different connections which may lead to different behavior.

You can try to enable debug logging to see how the connection pools try to dispatch the requests.