You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by 张伟 <zh...@126.com> on 2013/08/19 14:15:43 UTC
hadoop rumen question
Now I want to use Gridmix to do some tests. Firstly I should use rumen to generate a job trace file.
I use hadoop 1.0.4. However, when I use the command
hadoop@mastertest:~/hadoop-scheduler/hadoop-1.0.4$ ./bin/hadoop jar hadoop-tools-1.0.4.jar org.apache.hadoop.tools.rumen.TraceBuilder file:///home/hadoop/job-trace.json file:///home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done
I got the following errors. Can someone give me some guidance which part brings this problem?
13/08/18 04:03:49 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/08/18 04:03:49 WARN snappy.LoadSnappy: Snappy native library not loaded
java.io.FileNotFoundException: /home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done (Is a directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:188)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:184)
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:255)
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:236)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:335)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:381)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:555)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:536)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:443)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:435)
at org.apache.hadoop.tools.rumen.DefaultOutputter.init(DefaultOutputter.java:48)
at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:194)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Re: hadoop rumen question
Posted by Thomas Graves <tg...@yahoo-inc.com>.
The usage of Tracebuilder has 3 parameters. I believe you are only putting in 2 and its expecting the second parameter to be a file not a directory.
java org.apache.hadoop.tools.rumen.TraceBuilder [options] <jobtrace-output> <topology-output> <inputs>
See http://hadoop.apache.org/docs/r1.1.1/rumen.html.
Tom
From: 张伟 <zh...@126.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Monday, August 19, 2013 7:15 AM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: hadoop rumen question
Now I want to use Gridmix to do some tests. Firstly I should use rumen to generate a job trace file.
I use hadoop 1.0.4. However, when I use the command
hadoop@mastertest:~/hadoop-scheduler/hadoop-1.0.4$ ./bin/hadoop jar hadoop-tools-1.0.4.jar org.apache.hadoop.tools.rumen.TraceBuilder file:///home/hadoop/job-trace.json file:///home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done
I got the following errors. Can some! one give me some guidance which part brings this problem?
13/08/18 04:03:49 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/08/18 04:03:49 WARN snappy.LoadSnappy: Snappy native library not loaded
java.io.FileNotFoundException: /home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done (Is a directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:188)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:184)
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:255! )
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:236)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:335)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:381)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:555)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:536)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:443)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:435)
at org.apache.hadoop.tools.rumen.DefaultOutputter.init(D! efaultOut putter.java:48)
at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:194)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
!
Re: hadoop rumen question
Posted by Thomas Graves <tg...@yahoo-inc.com>.
The usage of Tracebuilder has 3 parameters. I believe you are only putting in 2 and its expecting the second parameter to be a file not a directory.
java org.apache.hadoop.tools.rumen.TraceBuilder [options] <jobtrace-output> <topology-output> <inputs>
See http://hadoop.apache.org/docs/r1.1.1/rumen.html.
Tom
From: 张伟 <zh...@126.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Monday, August 19, 2013 7:15 AM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: hadoop rumen question
Now I want to use Gridmix to do some tests. Firstly I should use rumen to generate a job trace file.
I use hadoop 1.0.4. However, when I use the command
hadoop@mastertest:~/hadoop-scheduler/hadoop-1.0.4$ ./bin/hadoop jar hadoop-tools-1.0.4.jar org.apache.hadoop.tools.rumen.TraceBuilder file:///home/hadoop/job-trace.json file:///home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done
I got the following errors. Can some! one give me some guidance which part brings this problem?
13/08/18 04:03:49 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/08/18 04:03:49 WARN snappy.LoadSnappy: Snappy native library not loaded
java.io.FileNotFoundException: /home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done (Is a directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:188)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:184)
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:255! )
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:236)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:335)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:381)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:555)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:536)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:443)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:435)
at org.apache.hadoop.tools.rumen.DefaultOutputter.init(D! efaultOut putter.java:48)
at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:194)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
!
Re: hadoop rumen question
Posted by Thomas Graves <tg...@yahoo-inc.com>.
The usage of Tracebuilder has 3 parameters. I believe you are only putting in 2 and its expecting the second parameter to be a file not a directory.
java org.apache.hadoop.tools.rumen.TraceBuilder [options] <jobtrace-output> <topology-output> <inputs>
See http://hadoop.apache.org/docs/r1.1.1/rumen.html.
Tom
From: 张伟 <zh...@126.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Monday, August 19, 2013 7:15 AM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: hadoop rumen question
Now I want to use Gridmix to do some tests. Firstly I should use rumen to generate a job trace file.
I use hadoop 1.0.4. However, when I use the command
hadoop@mastertest:~/hadoop-scheduler/hadoop-1.0.4$ ./bin/hadoop jar hadoop-tools-1.0.4.jar org.apache.hadoop.tools.rumen.TraceBuilder file:///home/hadoop/job-trace.json file:///home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done
I got the following errors. Can some! one give me some guidance which part brings this problem?
13/08/18 04:03:49 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/08/18 04:03:49 WARN snappy.LoadSnappy: Snappy native library not loaded
java.io.FileNotFoundException: /home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done (Is a directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:188)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:184)
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:255! )
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:236)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:335)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:381)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:555)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:536)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:443)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:435)
at org.apache.hadoop.tools.rumen.DefaultOutputter.init(D! efaultOut putter.java:48)
at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:194)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
!
Re: hadoop rumen question
Posted by Thomas Graves <tg...@yahoo-inc.com>.
The usage of Tracebuilder has 3 parameters. I believe you are only putting in 2 and its expecting the second parameter to be a file not a directory.
java org.apache.hadoop.tools.rumen.TraceBuilder [options] <jobtrace-output> <topology-output> <inputs>
See http://hadoop.apache.org/docs/r1.1.1/rumen.html.
Tom
From: 张伟 <zh...@126.com>>
Reply-To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Date: Monday, August 19, 2013 7:15 AM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: hadoop rumen question
Now I want to use Gridmix to do some tests. Firstly I should use rumen to generate a job trace file.
I use hadoop 1.0.4. However, when I use the command
hadoop@mastertest:~/hadoop-scheduler/hadoop-1.0.4$ ./bin/hadoop jar hadoop-tools-1.0.4.jar org.apache.hadoop.tools.rumen.TraceBuilder file:///home/hadoop/job-trace.json file:///home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done
I got the following errors. Can some! one give me some guidance which part brings this problem?
13/08/18 04:03:49 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/08/18 04:03:49 WARN snappy.LoadSnappy: Snappy native library not loaded
java.io.FileNotFoundException: /home/hadoop/hadoop-scheduler/hadoop-1.0.4/logs/history/done (Is a directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:188)
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:184)
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:255! )
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:236)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:335)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:381)
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:364)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:555)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:536)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:443)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:435)
at org.apache.hadoop.tools.rumen.DefaultOutputter.init(D! efaultOut putter.java:48)
at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:194)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
!