You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Chad Bad <ch...@gmail.com> on 2016/03/28 23:22:50 UTC

nutch 1.11 with cygwin

Does nutch 1.11 work on windows with cygwin?

I get the following error:

 ERROR util.Shell - Failed to locate the winutils binary in the hadoop
binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in
the Hadoop binaries.
at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326)
at
org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:432)
at
org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:478)
at
org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
at
org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
at org.apache.nutch.crawl.Injector.main(Injector.java:369)

Re: nutch 1.11 with cygwin

Posted by Sebastian Nagel <wa...@googlemail.com>.
Hi Chad,

that's a problem of how to set up Hadoop on Windows.
Nutch 1.11 is based on Hadoop 2.4 while Nutch 1.10
used Hadoop 1.2. I'm no expert for this topic but
many users of various Hadoop-based tools have faced
this problem:
 http://stackoverflow.com/search?q=hadoop+winutils

I hope you'll find the right answer.

Thanks,
Sebastian


On 03/28/2016 11:22 PM, Chad Bad wrote:
> Does nutch 1.11 work on windows with cygwin?
> 
> I get the following error:
> 
>  ERROR util.Shell - Failed to locate the winutils binary in the hadoop
> binary path
> java.io.IOException: Could not locate executable null\bin\winutils.exe in
> the Hadoop binaries.
> at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
> at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
> at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326)
> at
> org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:432)
> at
> org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:478)
> at
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
> at
> org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
> at org.apache.nutch.crawl.Injector.main(Injector.java:369)
>