[jira] [Resolved] (NUTCH-2383) Wrong FS exception in Fetcher

["Sebastian Nagel (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NUTCH-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sebastian Nagel resolved NUTCH-2383.
------------------------------------
    Resolution: Not A Problem

Thanks [~yossi] for reporting this problem. Closing this as it can hardly be solved inside Nutch: it's clear that the default value "local" of {{mapreduce.framework.name}} does not allow to access hdfs:// paths. It's defined in [mapred-default.xml|https://hadoop.apache.org/docs/r2.7.2/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml] and should be set appropriately in mapred-site.xml which is not controlled by Nutch. It needs to be configured when setting up the Hadoop cluster. Please reopen if you see any option to fix this inside Nutch. Thanks!

> Wrong FS exception in Fetcher
> -----------------------------
>
>                 Key: NUTCH-2383
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2383
>             Project: Nutch
>          Issue Type: Bug
>          Components: fetcher
>    Affects Versions: 1.13
>         Environment: Hadoop 2.8 and Hadoop 2.7.2
>            Reporter: Yossi Tamari
>            Priority: Major
>         Attachments: crawl output.txt
>
>
> Running bin/crawl on either Hadoop 2.7.2 or Hadoop 2.8, the Injector and Generator succeed, but the Fetcher throws: {code}java.lang.IllegalArgumentException: Wrong FS: hdfs://localhost:9000/user/root/crawl/segments/20170430084337/crawl_fetch, expected: file:///{code}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)