[jira] [Commented] (AVRO-3013) Avro files should allow fsync-ing files to disk in Python

["Michael A. Smith (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253809#comment-17253809 ] 

Michael A. Smith commented on AVRO-3013:
----------------------------------------

Welcome!

bq. closing a file does not guarantee to write data to disk.

This would seem to be the crux of the problem, but it's a curious assertion. What kind of python io object does not have this guarantee? TempFile objects seem to proxy TextFileWrappers, which appear to execute flush before close.

Can you provide evidence or show the code where the file handle might not get written before close?

> Avro files should allow fsync-ing files to disk in Python
> ---------------------------------------------------------
>
>                 Key: AVRO-3013
>                 URL: https://issues.apache.org/jira/browse/AVRO-3013
>             Project: Apache Avro
>          Issue Type: New Feature
>            Reporter: He Chen
>            Priority: Major
>
> I am new to Apache, but here I am...
> In our use case, we need to constantly update an existing avro file. The way we did it is that we copy the old avro file to a temporary file, append data to the temporary file, close the temporary file, and rename the temporary file to the original avro file. This is problematic since closing a file does not guarantee to write data to disk. The bug caused by this is hard to track since it's hard to reproduce.
> I noticed that there is a ticket that addresses this for the Java client https://issues.apache.org/jira/browse/AVRO-1388. Why isn't it implemented for the Python client? If there are no objections, I'd like to submit a patch. Or perhaps I am missing something here? Please let me know!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)