[jira] [Created] (SAMZA-2526) Azure blob system producer: do not commit blobs if avro DataFileWriter.close fails

["Lakshmi Manasa Gaduputi (Jira)" <ji...@apache.org>]

Lakshmi Manasa Gaduputi created SAMZA-2526:
----------------------------------------------

             Summary: Azure blob system producer: do not commit blobs if avro DataFileWriter.close fails
                 Key: SAMZA-2526
                 URL: https://issues.apache.org/jira/browse/SAMZA-2526
             Project: Samza
          Issue Type: Bug
            Reporter: Lakshmi Manasa Gaduputi
            Assignee: Lakshmi Manasa Gaduputi


Currently, when Avro DataFileWriter.close fails, the blob is committed with blocks uploaded so far for that blob. This could lead to invalid blobs because, the DataFileWriter.close does a flush which writes some magic bytes (the sync marker to signal end of block) which when absent might lead to invalid blob.

Hence, prevent commit of blob in this scenario. This will lead to uploaded blocks being discarded. Hence, the user of the AzureBlobSystemProducer should now be aware of this exception bubbling up and retry producing the whole blob again. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)