First, thanks for all of your work ! I’m really impressed how well the system works :)
I’m currently using akka-stream-alpakka-hdfs_2.12:3.0.3 in order to dump a stream.
When the source terminates, I’d like the rotation to happen (in other words, the file in temporary directory should be moved to the correct location in HDFS).
Afaik, the source is correctly closed currently.
Is there an approach to do so?
Thanks in advance for your answers, I hope I’ve sufficiently searched for an answer before posting
I’m not familiar with specifics of HDFS but given that you’d feed the flow into something downstream you can either do something after the stream completes, given that the Sink in the end of the stream materializes a Future/CompletionStage value, like Sink.ignore (Sink.ignore • Akka Documentation) or Sink.seq (Sink.seq • Akka Documentation) does for example, or if the sink you use does not materialize any such future you can use the .watchTermination (watchTermination • Akka Documentation) operator to get a corresponding future to add an on complete-callback to with arbitrary logic for moving the file to its final destination.
