[jira] Updated: (NUTCH-407) Make Nutch crawling parent directories for file protocol configurable

["Thorsten Scherler (JIRA)" <ji...@apache.org>]

     [ http://issues.apache.org/jira/browse/NUTCH-407?page=all ]

Thorsten Scherler updated NUTCH-407:
------------------------------------

    Attachment: 407.fix.diff

This patch fixed the issue by letting the user decide whether or not to crawl parent dirs. The new property is file.crawl.parent and defined in nutch-default.xml.

I did not change the default behavior, meaning by default nutch will crawl the parents for the file protocol. 

> Make Nutch crawling parent directories for file protocol configurable
> ---------------------------------------------------------------------
>
>                 Key: NUTCH-407
>                 URL: http://issues.apache.org/jira/browse/NUTCH-407
>             Project: Nutch
>          Issue Type: Improvement
>    Affects Versions: 0.8
>            Reporter: Thorsten Scherler
>         Attachments: 407.fix.diff
>
>
> http://www.mail-archive.com/nutch-user@lucene.apache.org/msg06698.html
> I am looking into fixing some very weird behavior of the file protocol.
> I am using 0.8.
> Researching this topic I found 
> http://www.mail-archive.com/nutch-user@lucene.apache.org/msg06536.html
> and
> http://www.folge2.de/tp/search/1/crawling-the-local-filesystem-with-nutch
> I am on Ubuntu but I have the same problem that nutch is going down the
> tree (including parents) and not up (including children from the root
> url).
> Further I would vote to make the fetch-parents optional and defined per
> a property whether I would like this not very intuitive "feature".

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira