[jira] [Resolved] (NUTCH-1574) Crawling parent directories for http(s) protocol

["Sebastian Nagel (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NUTCH-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sebastian Nagel resolved NUTCH-1574.
------------------------------------
    Resolution: Not A Bug

(sorry, this issue was left unnoticed since long) I can think of two solutions:
 * use a different URL filter configuration during indexing to filter the file listings away (assumed they're identifiable by URL)
 * use exchanges NUTCH-2412

> Crawling parent directories for http(s) protocol 
> -------------------------------------------------
>
>                 Key: NUTCH-1574
>                 URL: https://issues.apache.org/jira/browse/NUTCH-1574
>             Project: Nutch
>          Issue Type: Bug
>    Affects Versions: 1.6
>            Reporter: Antoinette
>            Priority: Major
>
> I am looking for a fix to prevent indexing the list of files crawled via http(s) protocol. For example: I have 10 files in a directory. Nutch finds and Solr indexes 11, the first being a list of the other 10 files.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)