You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Siddharth Seth (JIRA)" <ji...@apache.org> on 2013/09/25 22:48:03 UTC

[jira] [Created] (TEZ-500) RLE in IFile does not seem to work correctly

Siddharth Seth created TEZ-500:
----------------------------------

             Summary: RLE in IFile does not seem to work correctly
                 Key: TEZ-500
                 URL: https://issues.apache.org/jira/browse/TEZ-500
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Siddharth Seth


The compressed length reported by the writer is typically larger than the Uncompressed length. The size of the output file generated matches the uncompressed length.
The Shuffle fetchers allocate buffers based on the compressed length, and pull that much data. As a result the entire contents are not pulled in.

Also, even if the entire content is pulled in - nextRawKey, nextRawValue ends up failing the moment a repeated key is hit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira