You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by Lewis John Mcgibbney <le...@gmail.com> on 2014/11/20 19:35:10 UTC
Re: [nsf-polar-usc-students] Nutch in Windows: Failed to set
permissions of path
This is not a good workaround at all.
There are many reasons why this is not a good idea.
If I were you, I would seriously suggest you download and work with
VirtualBox on a Linux image. It will make your life so much easier anf the
barrier to entry is very low these days.
Lewis
On Thu, Nov 20, 2014 at 10:29 AM, MengYing Wang <me...@gmail.com>
wrote:
> Hi everyone,
>
> If you run the Nutch on Windows using the Cygwin, it may fail due to a
> permission error.
>
> $./crawl urls crawlId http://localhost:8983/solr/collection1 2
> 2014-11-17 15:39:25,041 ERROR security.UserGroupInformation -
> PriviledgedActionException as:YangLu cause:java.io.IOException: Failed to
> set permissions of path:
> \tmp\hadoop-YangLu\mapred\staging\YangLu534937598\.staging to 0700
> 2014-11-17 15:39:25,046 ERROR crawl.Injector - Injector:
> java.io.IOException: Failed to set permissions of path:
> \tmp\hadoop-YangLu\mapred\staging\YangLu534937598\.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:691)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:664)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:514)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:349)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:193)
> at
> org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:126)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:942)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1353)
> at org.apache.nutch.crawl.Injector.inject(Injector.java:324)
> at org.apache.nutch.crawl.Injector.run(Injector.java:380)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.nutch.crawl.Injector.main(Injector.java:370)
>
> To solve the problem, you should download Hadoop Core 0.20.2
> <http://mvnrepository.com/artifact/org.apache.hadoop/hadoop-core/0.20.2> from
> the MVN repository to your (nutch-home)/lib directory. For the details,
> please refer to
> http://stackoverflow.com/questions/15188050/nutch-in-windows-failed-to-set-permissions-of-path.
> Thanks.
>
> --
> Best,
> Mengying (Angela) Wang
>
> --
> You received this message because you are subscribed to the Google Groups
> "nsf-polar-usc-students" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to nsf-polar-usc-students+unsubscribe@googlegroups.com.
> To post to this group, send email to
> nsf-polar-usc-students@googlegroups.com.
> Visit this group at http://groups.google.com/group/nsf-polar-usc-students.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/nsf-polar-usc-students/CAJX%3DLAu0DWxq-DzA3jipKq81KfxvjDy1-kgSbOQKQhYARvscOg%40mail.gmail.com
> <https://groups.google.com/d/msgid/nsf-polar-usc-students/CAJX%3DLAu0DWxq-DzA3jipKq81KfxvjDy1-kgSbOQKQhYARvscOg%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
*Lewis*
Re: [nsf-polar-usc-students] Nutch in Windows: Failed to set permissions
of path
Posted by Sebastian Nagel <wa...@googlemail.com>.
Hi,
that's an Hadoop 1.x problem on Windows 7:
https://issues.apache.org/jira/browse/HADOOP-7682
http://mail-archives.apache.org/mod_mbox/nutch-user/201307.mbox/%3C51DB1853.3040504@googlemail.com%3E
Indeed, using Linux may be the simplest solution,
simpler than to down/upgrade Hadoop.
Sebastian
On 11/20/2014 07:35 PM, Lewis John Mcgibbney wrote:
> This is not a good workaround at all.
> There are many reasons why this is not a good idea.
> If I were you, I would seriously suggest you download and work with VirtualBox on a Linux image. It
> will make your life so much easier anf the barrier to entry is very low these days.
> Lewis
>
> On Thu, Nov 20, 2014 at 10:29 AM, MengYing Wang <mengyingwang1@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi everyone,
>
> If you run the Nutch on Windows using the Cygwin, it may fail due to a permission error.
>
> $./crawl urls crawlId http://localhost:8983/solr/collection1 2
> 2014-11-17 15:39:25,041 ERROR security.UserGroupInformation - PriviledgedActionException
> as:YangLu cause:java.io.IOException: Failed to set permissions of path:
> \tmp\hadoop-YangLu\mapred\staging\YangLu534937598\.staging to 0700
> 2014-11-17 15:39:25,046 ERROR crawl.Injector - Injector: java.io.IOException: Failed to set
> permissions of path: \tmp\hadoop-YangLu\mapred\staging\YangLu534937598\.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:691)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:664)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:514)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:349)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:193)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:126)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:942)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1353)
> at org.apache.nutch.crawl.Injector.inject(Injector.java:324)
> at org.apache.nutch.crawl.Injector.run(Injector.java:380)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.nutch.crawl.Injector.main(Injector.java:370)
>
> To solve the problem, you should download Hadoop Core 0.20.2
> <http://mvnrepository.com/artifact/org.apache.hadoop/hadoop-core/0.20.2> from the MVN repository
> to your (nutch-home)/lib directory. For the details, please refer
> to http://stackoverflow.com/questions/15188050/nutch-in-windows-failed-to-set-permissions-of-path.
> Thanks.
>
> --
> Best,
> Mengying (Angela) Wang
>
> --
> You received this message because you are subscribed to the Google Groups
> "nsf-polar-usc-students" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> nsf-polar-usc-students+unsubscribe@googlegroups.com
> <ma...@googlegroups.com>.
> To post to this group, send email to nsf-polar-usc-students@googlegroups.com
> <ma...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/nsf-polar-usc-students.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/nsf-polar-usc-students/CAJX%3DLAu0DWxq-DzA3jipKq81KfxvjDy1-kgSbOQKQhYARvscOg%40mail.gmail.com
> <https://groups.google.com/d/msgid/nsf-polar-usc-students/CAJX%3DLAu0DWxq-DzA3jipKq81KfxvjDy1-kgSbOQKQhYARvscOg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> /Lewis/