You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Bingbing Liu <ru...@gmail.com> on 2010/12/02 03:26:13 UTC
how can i ran the word count example on windows?
i don't know how to set the command line
is there a word_count.bat ? like the word_count in the bin on linux?
2010-12-02
Bingbing Liu
Re: Re: how can i ran the word count example on windows?
Posted by CassUser CassUser <ca...@gmail.com>.
You can install cygwin in order to run mapred job locally on windows
On Thu, Dec 2, 2010 at 5:57 AM, Bingbing Liu <ru...@gmail.com> wrote:
> hi ,arron ,thanks for your reply
>
> i run a single node cassandra instance on linux (IP:10.37.17.231)
>
> but i run the word count in the eclipse on windows with seting the host ip
> = 10.37.17.231
>
>
> 2010-12-02
> ------------------------------
> Bingbing Liu
> ------------------------------
> *发件人：* aaron morton
> *发送时间：* 2010-12-02 19:04:13
> *收件人：* user
> *抄送：*
> *主题：* Re: how can i ran the word count example on windows?
> chmod is used on linux to change the file permissions. It seems like
> hadoop links your on linux.
>
> What command are you running to get this error ?
>
> Aaron
>
> On 2 Dec 2010, at 21:03, Bingbing Liu wrote:
>
> i use the same configuration in eclispe both on windows and linux , on
> linux ,it goes well, but on windows , it fails
>
> here is the exception information:
>
>
>
> 10/12/02 10:53:40 INFO mapreduce.MRSelect: output reducer type: filesystem
>
> 10/12/02 10:53:40 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
>
> Exception in thread "main" java.io.IOException: Cannot run program "chmod": CreateProcess error=2, ϵͳÕҲ»µ½ָ¶
> at java.lang.ProcessBuilder.start(Unknown Source)
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
> at org.apache.hadoop.util.Shell.run(Shell.java:134)
>
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
>
> at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:481)
>
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:473)
>
> at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:280)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:261)
>
> at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:573)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
> at org.clouddb.cassandra.mapreduce.MRSelect.run(MRSelect.java:209)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.clouddb.cassandra.mapreduce.MRSelect.main(MRSelect.java:54)
> Caused by: java.io.IOException: CreateProcess error=2, ϵͳÕҲ»µ½ָ¶
> at java.lang.ProcessImpl.create(Native Method)
> at java.lang.ProcessImpl.<init>(Unknown Source)
> at java.lang.ProcessImpl.start(Unknown Source)
> ... 17 more
>
> 2010-12-02
> ------------------------------
> Bingbing Liu
> ------------------------------
> *发件人：* Jeremy Hanna
> *发送时间：* 2010-12-02 11:13:04
> *收件人：* user
> *抄送：*
> *主题：* Re: how can i ran the word count example on windows?
> There isn't currently, but perhaps you could contribute one :). If you
> take a look at the sh script in the bin directory of the word count example,
> it shouldn't be terribly difficult to mimic the behavior. It's mostly just
> setting up the classpath and executing the Java class with some arguments.
>
> On Dec 1, 2010, at 8:26 PM, Bingbing Liu wrote:
>
> i don't know how to set the command line
>
> is there a word_count.bat ? like the word_count in the bin on linux?
>
> 2010-12-02
> ------------------------------
> Bingbing Liu
>
>
>
>
>
Re: Re: Re: how can i ran the word count example on windows?
Posted by Bingbing Liu <ru...@gmail.com>.
yes , i can resolve the problem using cygwin
but i want to know the reason for the problem , such as :
the MR can't recognize the type of OS or may be i should
configure it in the my word count?
2010-12-03
Bingbing Liu
发件人： CassUser CassUser
发送时间： 2010-12-03 07:38:32
收件人： user
抄送：
主题： Re: Re: how can i ran the word count example on windows?
You can install cygwin in order to run mapred job locally on windows
On Thu, Dec 2, 2010 at 5:57 AM, Bingbing Liu <ru...@gmail.com> wrote:
hi ,arron ,thanks for your reply
i run a single node cassandra instance on linux (IP:10.37.17.231)
but i run the word count in the eclipse on windows with seting the host ip = 10.37.17.231
2010-12-02
Bingbing Liu
发件人： aaron morton
发送时间： 2010-12-02 19:04:13
收件人： user
抄送：
主题： Re: how can i ran the word count example on windows?
chmod is used on linux to change the file permissions. It seems like hadoop links your on linux.
What command are you running to get this error ?
Aaron
On 2 Dec 2010, at 21:03, Bingbing Liu wrote:
i use the same configuration in eclispe both on windows and linux , on linux ,it goes well, but on windows , it fails
here is the exception information:
10/12/02 10:53:40 INFO mapreduce.MRSelect: output reducer type: filesystem
10/12/02 10:53:40 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.io.IOException: Cannot run program "chmod": CreateProcess error=2, ϵͳÕҲ»µ½ָ¶
at java.lang.ProcessBuilder.start(Unknown Source)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
at org.apache.hadoop.util.Shell.run(Shell.java:134)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:481)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:473)
at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:280)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:261)
at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:573)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.clouddb.cassandra.mapreduce.MRSelect.run(MRSelect.java:209)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.clouddb.cassandra.mapreduce.MRSelect.main(MRSelect.java:54)
Caused by: java.io.IOException: CreateProcess error=2, ϵͳÕҲ»µ½ָ¶
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(Unknown Source)
at java.lang.ProcessImpl.start(Unknown Source)
... 17 more
2010-12-02
Bingbing Liu
发件人： Jeremy Hanna
发送时间： 2010-12-02 11:13:04
收件人： user
抄送：
主题： Re: how can i ran the word count example on windows?
There isn't currently, but perhaps you could contribute one :). If you take a look at the sh script in the bin directory of the word count example, it shouldn't be terribly difficult to mimic the behavior. It's mostly just setting up the classpath and executing the Java class with some arguments.
On Dec 1, 2010, at 8:26 PM, Bingbing Liu wrote:
i don't know how to set the command line
is there a word_count.bat ? like the word_count in the bin on linux?
2010-12-02
Bingbing Liu
Re: Re: how can i ran the word count example on windows?
Posted by Bingbing Liu <ru...@gmail.com>.
hi ,arron ,thanks for your reply
i run a single node cassandra instance on linux (IP:10.37.17.231)
but i run the word count in the eclipse on windows with seting the host ip = 10.37.17.231
2010-12-02
Bingbing Liu
发件人： aaron morton
发送时间： 2010-12-02 19:04:13
收件人： user
抄送：
主题： Re: how can i ran the word count example on windows?
chmod is used on linux to change the file permissions. It seems like hadoop links your on linux.
What command are you running to get this error ?
Aaron
On 2 Dec 2010, at 21:03, Bingbing Liu wrote:
i use the same configuration in eclispe both on windows and linux , on linux ,it goes well, but on windows , it fails
here is the exception information:
10/12/02 10:53:40 INFO mapreduce.MRSelect: output reducer type: filesystem
10/12/02 10:53:40 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.io.IOException: Cannot run program "chmod": CreateProcess error=2, ϵͳÕҲ»µ½ָ¶
at java.lang.ProcessBuilder.start(Unknown Source)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
at org.apache.hadoop.util.Shell.run(Shell.java:134)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:481)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:473)
at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:280)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:261)
at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:573)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.clouddb.cassandra.mapreduce.MRSelect.run(MRSelect.java:209)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.clouddb.cassandra.mapreduce.MRSelect.main(MRSelect.java:54)
Caused by: java.io.IOException: CreateProcess error=2, ϵͳÕҲ»µ½ָ¶
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(Unknown Source)
at java.lang.ProcessImpl.start(Unknown Source)
... 17 more
2010-12-02
Bingbing Liu
发件人： Jeremy Hanna
发送时间： 2010-12-02 11:13:04
收件人： user
抄送：
主题： Re: how can i ran the word count example on windows?
There isn't currently, but perhaps you could contribute one :). If you take a look at the sh script in the bin directory of the word count example, it shouldn't be terribly difficult to mimic the behavior. It's mostly just setting up the classpath and executing the Java class with some arguments.
On Dec 1, 2010, at 8:26 PM, Bingbing Liu wrote:
i don't know how to set the command line
is there a word_count.bat ? like the word_count in the bin on linux?
2010-12-02
Bingbing Liu
Re: how can i ran the word count example on windows?
Posted by aaron morton <aa...@thelastpickle.com>.
chmod is used on linux to change the file permissions. It seems like hadoop links your on linux.
What command are you running to get this error ?
Aaron
On 2 Dec 2010, at 21:03, Bingbing Liu wrote:
> i use the same configuration in eclispe both on windows and linux , on linux ,it goes well, but on windows , it fails
>
> here is the exception information:
>
>
> 10/12/02 10:53:40 INFO mapreduce.MRSelect: output reducer type: filesystem
> 10/12/02 10:53:40 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
> Exception in thread "main" java.io.IOException: Cannot run program "chmod": CreateProcess error=2, ϵͳÕҲ»µ½ָ¶
> at java.lang.ProcessBuilder.start(Unknown Source)
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
> at org.apache.hadoop.util.Shell.run(Shell.java:134)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
> at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:481)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:473)
> at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:280)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:261)
> at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:573)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
> at org.clouddb.cassandra.mapreduce.MRSelect.run(MRSelect.java:209)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.clouddb.cassandra.mapreduce.MRSelect.main(MRSelect.java:54)
> Caused by: java.io.IOException: CreateProcess error=2, ϵͳÕҲ»µ½ָ¶
> at java.lang.ProcessImpl.create(Native Method)
> at java.lang.ProcessImpl.<init>(Unknown Source)
> at java.lang.ProcessImpl.start(Unknown Source)
> ... 17 more
>
> 2010-12-02
> Bingbing Liu
> 发件人： Jeremy Hanna
> 发送时间： 2010-12-02 11:13:04
> 收件人： user
> 抄送：
> 主题： Re: how can i ran the word count example on windows?
> There isn't currently, but perhaps you could contribute one :). If you take a look at the sh script in the bin directory of the word count example, it shouldn't be terribly difficult to mimic the behavior. It's mostly just setting up the classpath and executing the Java class with some arguments.
>
> On Dec 1, 2010, at 8:26 PM, Bingbing Liu wrote:
>
>> i don't know how to set the command line
>>
>> is there a word_count.bat ? like the word_count in the bin on linux?
>>
>> 2010-12-02
>> Bingbing Liu
>
>
Re: Re: how can i ran the word count example on windows?
Posted by Bingbing Liu <ru...@gmail.com>.
i use the same configuration in eclispe both on windows and linux , on linux ,it goes well, but on windows , it fails
here is the exception information:
10/12/02 10:53:40 INFO mapreduce.MRSelect: output reducer type: filesystem
10/12/02 10:53:40 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.io.IOException: Cannot run program "chmod": CreateProcess error=2, ϵͳÕҲ»µ½ָ¶
at java.lang.ProcessBuilder.start(Unknown Source)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
at org.apache.hadoop.util.Shell.run(Shell.java:134)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:481)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:473)
at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:280)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:261)
at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:573)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.clouddb.cassandra.mapreduce.MRSelect.run(MRSelect.java:209)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.clouddb.cassandra.mapreduce.MRSelect.main(MRSelect.java:54)
Caused by: java.io.IOException: CreateProcess error=2, ϵͳÕҲ»µ½ָ¶
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(Unknown Source)
at java.lang.ProcessImpl.start(Unknown Source)
... 17 more
2010-12-02
Bingbing Liu
发件人： Jeremy Hanna
发送时间： 2010-12-02 11:13:04
收件人： user
抄送：
主题： Re: how can i ran the word count example on windows?
There isn't currently, but perhaps you could contribute one :). If you take a look at the sh script in the bin directory of the word count example, it shouldn't be terribly difficult to mimic the behavior. It's mostly just setting up the classpath and executing the Java class with some arguments.
On Dec 1, 2010, at 8:26 PM, Bingbing Liu wrote:
i don't know how to set the command line
is there a word_count.bat ? like the word_count in the bin on linux?
2010-12-02
Bingbing Liu
Re: how can i ran the word count example on windows?
Posted by Jeremy Hanna <je...@gmail.com>.
There isn't currently, but perhaps you could contribute one :). If you take a look at the sh script in the bin directory of the word count example, it shouldn't be terribly difficult to mimic the behavior. It's mostly just setting up the classpath and executing the Java class with some arguments.
On Dec 1, 2010, at 8:26 PM, Bingbing Liu wrote:
> i don't know how to set the command line
>
> is there a word_count.bat ? like the word_count in the bin on linux?
>
> 2010-12-02
> Bingbing Liu