Hi all,
I’m currently playing around with a streaming implementation of a consumer for a paginated HTTP resource. I built the page crawling mechanism by using Source.unfoldAsync, but I’m facing some issues with backpressure (the HTTP requests are going out too fast and the downstream can’t subscribe to the returned HttpResponse fast enough). A more detailed description of the problem including some code snippets can be found here: https://stackoverflow.com/questions/55316741/akka-streams-backpressure-for-source-unfoldasync
The behavior I have in mind is that Source.unfoldAsync backpressures if the downstream can’t process the HttpResponses fast enough. What I observed though is that there are several outgoing requests, no matter what the downstream (in my case it’s only Unmarshalling, splitting up the Multipart document of the response and some pretty printing) does.
I would love to hear your thoughts and opinions on that!