You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@pig.apache.org by "Rohini Palaniswamy (JIRA)" <ji...@apache.org> on 2013/10/08 02:16:42 UTC

[jira] [Updated] (PIG-3505) Make AvroStorage sync interval take default from io.file.buffer.size

     [ https://issues.apache.org/jira/browse/PIG-3505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rohini Palaniswamy updated PIG-3505:
------------------------------------

    Attachment: PIG-3505-1.patch

> Make AvroStorage sync interval take default from io.file.buffer.size 
> ---------------------------------------------------------------------
>
>                 Key: PIG-3505
>                 URL: https://issues.apache.org/jira/browse/PIG-3505
>             Project: Pig
>          Issue Type: Improvement
>    Affects Versions: 0.11
>            Reporter: Rohini Palaniswamy
>            Assignee: Rohini Palaniswamy
>             Fix For: 0.13.0
>
>         Attachments: PIG-3505-1.patch
>
>
>    The default sync interval is 16K which is very bad for bzip compression which can take bigger chunk of data for compression. Hadoop's Bzip2code uses io.file.buffer.size as the buffer size. Most tuned environments have it set to 128K which gives better compression. 



--
This message was sent by Atlassian JIRA
(v6.1#6144)