[jira] [Created] (TEZ-2339) randomwriter generates sequence file which does not have "SEQ6" in the beginning (causes exceptions while reading)

["Rajesh Balamohan (JIRA)" <ji...@apache.org>]

Rajesh Balamohan created TEZ-2339:
-------------------------------------

             Summary: randomwriter generates sequence file which does not have "SEQ6" in the beginning (causes exceptions while reading)
                 Key: TEZ-2339
                 URL: https://issues.apache.org/jira/browse/TEZ-2339
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Rajesh Balamohan


Scenario:
=======
yarn jar ./dist/tez/tez-tests-0.7.0-SNAPSHOT.jar randomwriter "-Dmapreduce.randomwriter.totalbytes=10000000" "-Dmapreduce.framework.name=yarn-tez"  /user/rajesh/tez-test-rw/

Job finishes, however the data generated by the above command does not have "SEQ6" which is needed for reading back the data.  It contains some garbage values like "99,51,32,102" instead of "83,69,81,6" (SEQ6) .  This causes exceptions while reading back the data.

For e.g 
yarn jar ./dist/tez/tez-tests-0.7.0-SNAPSHOT.jar sort "-Dmapreduce.framework.name=yarn-tez"  -r 10 /user/rajesh/tez-test-rw/ /user/rajesh/tez-test-rw-sort/

would fail (exception at org.apache.hadoop.io.SequenceFile$Reader.init) reading the data from /user/rajesh/tez-test-rw/ 

This is observed even in 0.5.4 branch. Haven't checked in branch 0.4



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)