Zip multiple files as a stream (on the fly)

Akka Libraries Akka Streams & Alpakka
1

Hy there!

I have multiple akka-http endpoints with streaming csv-s:

override def bytes(data: Data)(
                implicit lang: Lang,
                messages: MessageCache,
                executionContext: ExecutionContext
            ): Source[ByteString, NotUsed] = {
              Source
                .single(csvEv.header)
                .concat(
                  Source(data.toList).map(_.toCsv)
                )
                .via(CsvFormatting.format())
            }

The customer wants a new endpoint where they can get all of the csv-s as a big zip.

Is there any prebuild solution in the streams or the alpakka that can be used without hacking around? (Checked the alpakka-files not so helpful for this, also checked the streams docs)

I found that we have Compression.gzip which is a Bytestring-Bytestring flow, but that will not work if I simply concat the sources one by one. Somewhat I should inject file-headers to the stream I think.

Other method would be to write down the csv-s as files zip locally and give back the zip, but I think that is a super ineffective solution…

I’ll trying to came up with a useable solution but would be nice if sb could give me directions.

2

Hi @tg44

Such a thing has recently been contributed to Alpakka File: https://doc.akka.io/docs/alpakka/snapshot/file.html#zip-archive

We plan to cut an Alpakka 2.0.0-M1 release later this week which will contain it. Until than you may resort to the snapshot build.

Cheers,
Enno.