You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ankit Kamboj (JIRA)" <ji...@apache.org> on 2012/06/20 00:52:44 UTC
[jira] [Updated] (MAPREDUCE-4354) Performance improvement with
compressor object reinit restriction
[ https://issues.apache.org/jira/browse/MAPREDUCE-4354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ankit Kamboj updated MAPREDUCE-4354:
------------------------------------
Attachment: codec_reinit_diff
> Performance improvement with compressor object reinit restriction
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-4354
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4354
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 0.20.205.0
> Reporter: Ankit Kamboj
> Labels: performance
> Fix For: 0.20.205.0
>
> Attachments: codec_reinit_diff
>
>
> HADOOP-5879 patch aimed at picking the conf (instead of default) settings for GzipCodec. It also involved re-initializing the recycled compressor object.
> On our performance tests, this re-initialization led to performance degradation of 15% for LzoCodec because re-initialization for Lzo involves reallocation of buffers. LzoCodec takes the initial settings from config so it is not necessary to re-initialize it. This patch checks for the codec class and calls reinit only if the codec class is Gzip. This led to significant performance improvement of 15% for LzoCodec.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira