[jira] [Resolved] (GOBBLIN-684) Ensure buffered messages are flushed before close() in KafkaProducerPusher

["Hung Tran (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GOBBLIN-684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hung Tran resolved GOBBLIN-684.
-------------------------------
    Resolution: Fixed

Issue resolved by pull request #2556
[https://github.com/apache/incubator-gobblin/pull/2556]

> Ensure buffered messages are flushed before close() in KafkaProducerPusher
> --------------------------------------------------------------------------
>
>                 Key: GOBBLIN-684
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-684
>             Project: Apache Gobblin
>          Issue Type: Improvement
>          Components: gobblin-metrics
>    Affects Versions: 0.15.0
>            Reporter: Sudarshan Vasudevan
>            Assignee: Sudarshan Vasudevan
>            Priority: Major
>             Fix For: 0.15.0
>
>          Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Currently, when KafkaProducerPusher is closed, it invokes KafkaProducer#close(). However,close() only guarantees delivery of in-flight messages, not the messages in the producer buffer waiting to be sent out. This results in data loss.
> The fix ensures that we call flush() before close(). As a result, any buffered messages are immediately pushed out and we block until the messages are acked. 
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)