Terasort error when running Hadoop over PVFS2

[Wantao <li...@qq.com>]

Hi guys,

I am trying to run Hadoop0.21.0 with PVFS2. Following the email thread http://www.mail-archive.com/core-user@hadoop.apache.org/msg04434.html, I managed to make my Hadoop cluster up and run wordcount example successfully. Then I plan to perform some test for Terasort example. The data generator program Teragen is OK, but I got some errors from Terasort:

11/04/06 21:40:14 INFO mapreduce.Job: Task Id : attempt_201104062136_0002_m_000016_0, Status : FAILED
java.lang.IllegalArgumentException: can't read paritions file
    at org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.setConf(TeraSort.java:217)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)
    at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:569)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:636)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:742)
    at org.apache.hadoop.mapred.Child.main(Child.java:211)
Caused by: java.io.FileNotFoundException: File _partition.lst does not exist.
    at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:423)
    at org.apache.


 The relevant configuration I made in core-site.xml is:
<property>
       <name>fs.default.name</name>
       <value>file:///pvfs/hadoop</value>
</property>

and the configuration in mapred-site is:
<property>
      <name>mapreduce.jobtracker.system.dir</name>
      <value>file:///pvfs/hadoop/mapred/system</value>
    </property>
    <property>
      <name>mapreduce.cluster.temp.dir</name>
      <value>file:///pvfs/hadoop/mapred/temp</value>
    </property>
    <property>
      <name>mapreduce.jobtracker.staging.root.dir</name>
      <value>file:///pvfs/hadoop/mapred/staging</value>
    </property>
    <property>
      <name>mapreduce.cluster.local.dir</name>
      <value>/root/hadoop_local</value>
    </property>

Does anybody know how to fix this problem?

Thanks for any advice.

Wantao