You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Sujen Shah (JIRA)" <ji...@apache.org> on 2016/10/20 22:40:58 UTC
[jira] [Created] (NUTCH-2331) REST API Fetch fails to retrieve HDFS
path on distributed mode
Sujen Shah created NUTCH-2331:
---------------------------------
Summary: REST API Fetch fails to retrieve HDFS path on distributed mode
Key: NUTCH-2331
URL: https://issues.apache.org/jira/browse/NUTCH-2331
Project: Nutch
Issue Type: Bug
Components: fetcher, REST_api
Reporter: Sujen Shah
Assignee: Sujen Shah
Currently in the REST API, if the user does not specify the absolute path of the segment to fetch and only the crawlId, then the fetcher would find the latest segment generated and use that.
But as of now, the above functionality will only work in local mode as per https://github.com/apache/nutch/blob/master/src/java/org/apache/nutch/fetcher/Fetcher.java#L562-L573.
Need to update these lines to enable fetcher to read the directory and list files from an hdfs system.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)