We recently experienced some issues regarding uploading files in a streaming fashion to S3, that cause Exceptions during the upload, due to an invalid encoding.
We use Alpakka AWS S3 for this and notice that when this happens, the Akka Http client that is used by Alpakka, does not close the connection with S3 properly (does not read or discard the chunked response from S3)
Simply put, when you would run a stream that stops due to an Exception, we get the following WARN:
2022-02-01 08:17:39,721 WARN akka.http.impl.engine.client.PoolId [object-storage-prod/Pool(shared->https://<our-s3-instance>.amazonaws.com:443)] - [0 (WaitingForResponseEntitySubscription)]Response entity was not subscribed after 1 second. Make sure to read the response `entity` body or call `entity.discardBytes()` on it -- in case you deal with `HttpResponse`, use the shortcut `response.discardEntityBytes()`. POST /e69b058c-a5b6-4dcf-8e4a-c1bbe0188c45 Strict(0 bytes) -> 200 OK Chunked
Code to reproduce:
Source(Seq(ByteString("A"), ByteString("B")))
.map { b =>
if (b == ByteString("B")) {
throw new RuntimeException("Boom!")
}
b
}
.runWith(S3.multipartUpload("myBucket", "fileA", ContentTypes.`text/plain(UTF-8)`))
I’m trying to figure out whether this is caused by a bug in the library itself, or by our own code. Any thoughts?
Any help is much appreciated,
Roy