You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Joe Crobak (JIRA)" <ji...@apache.org> on 2010/12/23 19:59:45 UTC
[jira] Created: (AVRO-719) AvroOutputFormat: Expose sync interval
configuration
AvroOutputFormat: Expose sync interval configuration
----------------------------------------------------
Key: AVRO-719
URL: https://issues.apache.org/jira/browse/AVRO-719
Project: Avro
Issue Type: Improvement
Components: java
Affects Versions: 1.4.1
Environment: Mac, Java 6
Reporter: Joe Crobak
Priority: Minor
AvroOutputFormat supports setting deflate level for the underlying DataFileWriter, but not the sync interval. As I've seen performance improvements using larger-than-default sync intervals, it would be nice to make this option configurable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-719) AvroOutputFormat: Expose sync interval
configuration
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977514#action_12977514 ]
Doug Cutting commented on AVRO-719:
-----------------------------------
I will commit this soon unless there are objections.
> AvroOutputFormat: Expose sync interval configuration
> ----------------------------------------------------
>
> Key: AVRO-719
> URL: https://issues.apache.org/jira/browse/AVRO-719
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.4.1
> Environment: Mac, Java 6
> Reporter: Joe Crobak
> Assignee: Joe Crobak
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: AVRO-719.patch, AVRO-719.patch
>
>
> AvroOutputFormat supports setting deflate level for the underlying DataFileWriter, but not the sync interval. As I've seen performance improvements using larger-than-default sync intervals, it would be nice to make this option configurable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-719) AvroOutputFormat: Expose sync interval
configuration
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-719:
------------------------------
Fix Version/s: 1.5.0
Assignee: Joe Crobak
> AvroOutputFormat: Expose sync interval configuration
> ----------------------------------------------------
>
> Key: AVRO-719
> URL: https://issues.apache.org/jira/browse/AVRO-719
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.4.1
> Environment: Mac, Java 6
> Reporter: Joe Crobak
> Assignee: Joe Crobak
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: AVRO-719.patch
>
>
> AvroOutputFormat supports setting deflate level for the underlying DataFileWriter, but not the sync interval. As I've seen performance improvements using larger-than-default sync intervals, it would be nice to make this option configurable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-719) AvroOutputFormat: Expose sync interval
configuration
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-719:
------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Joe!
> AvroOutputFormat: Expose sync interval configuration
> ----------------------------------------------------
>
> Key: AVRO-719
> URL: https://issues.apache.org/jira/browse/AVRO-719
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.4.1
> Environment: Mac, Java 6
> Reporter: Joe Crobak
> Assignee: Joe Crobak
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: AVRO-719.patch, AVRO-719.patch
>
>
> AvroOutputFormat supports setting deflate level for the underlying DataFileWriter, but not the sync interval. As I've seen performance improvements using larger-than-default sync intervals, it would be nice to make this option configurable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-719) AvroOutputFormat: Expose sync interval
configuration
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-719:
------------------------------
Attachment: AVRO-719.patch
Patch looks good. Here's a version that avoid copying DEFAULT_SYNC_INTERVAL by making DataFileConstants public.
Does this look reasonable?
> AvroOutputFormat: Expose sync interval configuration
> ----------------------------------------------------
>
> Key: AVRO-719
> URL: https://issues.apache.org/jira/browse/AVRO-719
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.4.1
> Environment: Mac, Java 6
> Reporter: Joe Crobak
> Assignee: Joe Crobak
> Priority: Minor
> Fix For: 1.5.0
>
> Attachments: AVRO-719.patch, AVRO-719.patch
>
>
> AvroOutputFormat supports setting deflate level for the underlying DataFileWriter, but not the sync interval. As I've seen performance improvements using larger-than-default sync intervals, it would be nice to make this option configurable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-719) AvroOutputFormat: Expose sync interval
configuration
Posted by "Joe Crobak (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joe Crobak updated AVRO-719:
----------------------------
Attachment: AVRO-719.patch
Patch that creates static setSyncInterval method for AvroOutputFormat. Small test case to be sure that the setter does what it is expected.
> AvroOutputFormat: Expose sync interval configuration
> ----------------------------------------------------
>
> Key: AVRO-719
> URL: https://issues.apache.org/jira/browse/AVRO-719
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.4.1
> Environment: Mac, Java 6
> Reporter: Joe Crobak
> Priority: Minor
> Attachments: AVRO-719.patch
>
>
> AvroOutputFormat supports setting deflate level for the underlying DataFileWriter, but not the sync interval. As I've seen performance improvements using larger-than-default sync intervals, it would be nice to make this option configurable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-719) AvroOutputFormat: Expose sync interval
configuration
Posted by "Joe Crobak (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joe Crobak updated AVRO-719:
----------------------------
Status: Patch Available (was: Open)
> AvroOutputFormat: Expose sync interval configuration
> ----------------------------------------------------
>
> Key: AVRO-719
> URL: https://issues.apache.org/jira/browse/AVRO-719
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.4.1
> Environment: Mac, Java 6
> Reporter: Joe Crobak
> Priority: Minor
> Attachments: AVRO-719.patch
>
>
> AvroOutputFormat supports setting deflate level for the underlying DataFileWriter, but not the sync interval. As I've seen performance improvements using larger-than-default sync intervals, it would be nice to make this option configurable.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.