[jira] [Resolved] (SAMZA-2526) Azure blob system producer: do not commit blobs if avro DataFileWriter.close fails

["Lakshmi Manasa Gaduputi (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-2526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Lakshmi Manasa Gaduputi resolved SAMZA-2526.
--------------------------------------------
    Resolution: Fixed

> Azure blob system producer: do not commit blobs if avro DataFileWriter.close fails
> ----------------------------------------------------------------------------------
>
>                 Key: SAMZA-2526
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2526
>             Project: Samza
>          Issue Type: Bug
>            Reporter: Lakshmi Manasa Gaduputi
>            Assignee: Lakshmi Manasa Gaduputi
>            Priority: Minor
>          Time Spent: 1h
>  Remaining Estimate: 0h
>
> Currently, when Avro DataFileWriter.close fails, the blob is committed with blocks uploaded so far for that blob. This could lead to invalid blobs because, the DataFileWriter.close does a flush which writes some magic bytes (the sync marker to signal end of block) which when absent might lead to invalid blob.
> Hence, prevent commit of blob in this scenario. This will lead to uploaded blocks being discarded. Hence, the user of the AzureBlobSystemProducer should now be aware of this exception bubbling up and retry producing the whole blob again. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)