You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Ari Pollak (Created) (JIRA)" <ji...@apache.org> on 2012/04/16 22:44:19 UTC

[jira] [Created] (AVRO-1061) Add sync interval option to Avro commandline tools

Add sync interval option to Avro commandline tools
--------------------------------------------------

                 Key: AVRO-1061
                 URL: https://issues.apache.org/jira/browse/AVRO-1061
             Project: Avro
          Issue Type: Improvement
          Components: java
    Affects Versions: 1.7.0
            Reporter: Ari Pollak
            Priority: Trivial


It would be nice to expose the sync interval to the avro commandline writer tools, since I've seen a 20%+ decrease in file size using deflate compression and a 64K+ sync interval instead of the default of 16K.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (AVRO-1061) Add sync interval option to Avro commandline tools

Posted by "Ari Pollak (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AVRO-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ari Pollak updated AVRO-1061:
-----------------------------

    Attachment: AVRO-1061.patch
    
> Add sync interval option to Avro commandline tools
> --------------------------------------------------
>
>                 Key: AVRO-1061
>                 URL: https://issues.apache.org/jira/browse/AVRO-1061
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>    Affects Versions: 1.7.0
>            Reporter: Ari Pollak
>            Priority: Trivial
>         Attachments: AVRO-1061.patch
>
>
> It would be nice to expose the sync interval to the avro commandline writer tools, since I've seen a 20%+ decrease in file size using deflate compression and a 64K+ sync interval instead of the default of 16K.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (AVRO-1061) Add sync interval option to Avro commandline tools

Posted by "Doug Cutting (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255174#comment-13255174 ] 

Doug Cutting commented on AVRO-1061:
------------------------------------

This looks good to me.

We should probably add a test to TestDataFileTools that checks this.

Should we also increase the default sync interval to 64k?



                
> Add sync interval option to Avro commandline tools
> --------------------------------------------------
>
>                 Key: AVRO-1061
>                 URL: https://issues.apache.org/jira/browse/AVRO-1061
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>    Affects Versions: 1.7.0
>            Reporter: Ari Pollak
>            Priority: Trivial
>         Attachments: AVRO-1061.patch
>
>
> It would be nice to expose the sync interval to the avro commandline writer tools, since I've seen a 20%+ decrease in file size using deflate compression and a 64K+ sync interval instead of the default of 16K.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira