Hi All,
Problem: Messages are published to kafka, but are not received even once with AtLeastOnce consumer config after node crash
Details: We are using a single service to publish events (no partition strategy used) and then receive those events in same service. Through kafka lenses and app-logs we can see event getting published but it is not received.
Kafka Broker used: lagom-javadsl-kafka-broker_2.13:1.6.4
myService.myMoneyTopic().subscribe()
.withGroupId("myMoney")
.atLeastOnce(Flow.<myEvent>create().mapAsync(1, event -> {
LOGGER.debug("Received event: [{}]", event);
//Some processing here
}));
Config used: (we are not using any other config besides this for producer or consumer)
akka.kafka.consumer {
poll-interval = 400ms
poll-timeout = 1s
commit-timeout = 15s
commit-time-warning = 300ms
commit-refresh-interval = infinite
use-dispatcher = "akka.kafka.default-dispatcher"
kafka-clients {
enable.auto.commit = true
auto.commit.interval.ms = 100
}
wait-close-partition = 500ms
position-timeout = 5s
offset-for-times-timeout = 5s
metadata-request-timeout = 5s
}
What could be the possible reasons we can investigate for these missing events?