I understand after reading the code of the logstash-input-kafka plugin, that the offset it being committed after the batch has been transferred to the logstash internal queue, but not actually sent to the output (In my case Elasticsearch output).
Is there a wait to force logstash to wait for the batch to be fully committed to Elasticsearch before committing the offset to Kafka?
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext