You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by 张伟 <zh...@126.com> on 2013/08/19 14:15:43 UTC

hadoop rumen question

Now I want to use Gridmix to do some tests. Firstly I should use rumen to generate a job trace file. 


I use hadoop 1.0.4. However, when I use the command 
hadoop@mastertest:~/hadoop-scheduler/hadoop-1.0.4$ ./bin/hadoop jar hadoop-tools-1.0.4.jar org.apache.hadoop.tools.rumen.TraceBuilder file:///home/hadoop/job-trace.json file:///home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done


I got the following errors. Can someone give me some guidance which part brings this problem?






13/08/18 04:03:49 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/08/18 04:03:49 WARN snappy.LoadSnappy: Snappy native library not loaded
java.io.FileNotFoundException: /home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done (Is a directory)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
        at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:188)
        at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:184)
        at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:255)
        at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:236)
        at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:335)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:381)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:555)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:536)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:443)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:435)
        at org.apache.hadoop.tools.rumen.DefaultOutputter.init(DefaultOutputter.java:48)
        at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:194)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:120)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

Re: hadoop rumen question

Posted by Thomas Graves <tg...@yahoo-inc.com>.
The usage of Tracebuilder has 3 parameters. I believe you are only putting in 2 and its expecting the second parameter to be a file not a directory.


java org.apache.hadoop.tools.rumen.TraceBuilder [options] <jobtrace-output> <topology-output> <inputs>

See http://hadoop.apache.org/docs/r1.1.1/rumen.html.

Tom

From: 张伟 <zh...@126.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Monday, August 19, 2013 7:15 AM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: hadoop rumen question

Now I want to use Gridmix to do some tests. Firstly I should use rumen to generate a job trace file.

I use hadoop 1.0.4. However, when I use the command
hadoop@mastertest:~/hadoop-scheduler/hadoop-1.0.4$ ./bin/hadoop jar hadoop-tools-1.0.4.jar org.apache.hadoop.tools.rumen.TraceBuilder file:///home/hadoop/job-trace.json file:///home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done

I got the following errors. Can some! one give me some guidance which part brings this problem?



13/08/18 04:03:49 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/08/18 04:03:49 WARN snappy.LoadSnappy: Snappy native library not loaded
java.io.FileNotFoundException: /home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done (Is a directory)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
        at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:188)
        at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:184)
        at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:255! )
        at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:236)
        at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:335)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:381)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:555)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:536)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:443)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:435)
        at org.apache.hadoop.tools.rumen.DefaultOutputter.init(D! efaultOut putter.java:48)
        at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:194)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:120)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

!

Re: hadoop rumen question

Posted by Thomas Graves <tg...@yahoo-inc.com>.
The usage of Tracebuilder has 3 parameters. I believe you are only putting in 2 and its expecting the second parameter to be a file not a directory.


java org.apache.hadoop.tools.rumen.TraceBuilder [options] <jobtrace-output> <topology-output> <inputs>

See http://hadoop.apache.org/docs/r1.1.1/rumen.html.

Tom

From: 张伟 <zh...@126.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Monday, August 19, 2013 7:15 AM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: hadoop rumen question

Now I want to use Gridmix to do some tests. Firstly I should use rumen to generate a job trace file.

I use hadoop 1.0.4. However, when I use the command
hadoop@mastertest:~/hadoop-scheduler/hadoop-1.0.4$ ./bin/hadoop jar hadoop-tools-1.0.4.jar org.apache.hadoop.tools.rumen.TraceBuilder file:///home/hadoop/job-trace.json file:///home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done

I got the following errors. Can some! one give me some guidance which part brings this problem?



13/08/18 04:03:49 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/08/18 04:03:49 WARN snappy.LoadSnappy: Snappy native library not loaded
java.io.FileNotFoundException: /home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done (Is a directory)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
        at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:188)
        at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:184)
        at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:255! )
        at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:236)
        at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:335)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:381)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:555)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:536)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:443)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:435)
        at org.apache.hadoop.tools.rumen.DefaultOutputter.init(D! efaultOut putter.java:48)
        at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:194)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:120)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

!

Re: hadoop rumen question

Posted by Thomas Graves <tg...@yahoo-inc.com>.
The usage of Tracebuilder has 3 parameters. I believe you are only putting in 2 and its expecting the second parameter to be a file not a directory.


java org.apache.hadoop.tools.rumen.TraceBuilder [options] <jobtrace-output> <topology-output> <inputs>

See http://hadoop.apache.org/docs/r1.1.1/rumen.html.

Tom

From: 张伟 <zh...@126.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Monday, August 19, 2013 7:15 AM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: hadoop rumen question

Now I want to use Gridmix to do some tests. Firstly I should use rumen to generate a job trace file.

I use hadoop 1.0.4. However, when I use the command
hadoop@mastertest:~/hadoop-scheduler/hadoop-1.0.4$ ./bin/hadoop jar hadoop-tools-1.0.4.jar org.apache.hadoop.tools.rumen.TraceBuilder file:///home/hadoop/job-trace.json file:///home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done

I got the following errors. Can some! one give me some guidance which part brings this problem?



13/08/18 04:03:49 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/08/18 04:03:49 WARN snappy.LoadSnappy: Snappy native library not loaded
java.io.FileNotFoundException: /home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done (Is a directory)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
        at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:188)
        at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:184)
        at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:255! )
        at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:236)
        at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:335)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:381)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:555)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:536)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:443)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:435)
        at org.apache.hadoop.tools.rumen.DefaultOutputter.init(D! efaultOut putter.java:48)
        at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:194)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:120)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

!

Re: hadoop rumen question

Posted by Thomas Graves <tg...@yahoo-inc.com>.
The usage of Tracebuilder has 3 parameters. I believe you are only putting in 2 and its expecting the second parameter to be a file not a directory.


java org.apache.hadoop.tools.rumen.TraceBuilder [options] <jobtrace-output> <topology-output> <inputs>

See http://hadoop.apache.org/docs/r1.1.1/rumen.html.

Tom

From: 张伟 <zh...@126.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Monday, August 19, 2013 7:15 AM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: hadoop rumen question

Now I want to use Gridmix to do some tests. Firstly I should use rumen to generate a job trace file.

I use hadoop 1.0.4. However, when I use the command
hadoop@mastertest:~/hadoop-scheduler/hadoop-1.0.4$ ./bin/hadoop jar hadoop-tools-1.0.4.jar org.apache.hadoop.tools.rumen.TraceBuilder file:///home/hadoop/job-trace.json file:///home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done

I got the following errors. Can some! one give me some guidance which part brings this problem?



13/08/18 04:03:49 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/08/18 04:03:49 WARN snappy.LoadSnappy: Snappy native library not loaded
java.io.FileNotFoundException: /home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done (Is a directory)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
        at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:188)
        at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:184)
        at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:255! )
        at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:236)
        at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:335)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:381)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:555)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:536)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:443)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:435)
        at org.apache.hadoop.tools.rumen.DefaultOutputter.init(D! efaultOut putter.java:48)
        at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:194)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:120)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

!