You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@nutch.apache.org by "Julien Nioche (JIRA)" <ji...@apache.org> on 2016/10/24 10:32:58 UTC

[jira] [Resolved] (NUTCH-1531) URL filtering takes long time for very long URLs

     [ https://issues.apache.org/jira/browse/NUTCH-1531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Julien Nioche resolved NUTCH-1531.
----------------------------------
    Resolution: Duplicate

No follow up on this one + same functionality discussed elsewhere

> URL filtering takes long time for very long URLs
> ------------------------------------------------
>
>                 Key: NUTCH-1531
>                 URL: https://issues.apache.org/jira/browse/NUTCH-1531
>             Project: Nutch
>          Issue Type: Improvement
>    Affects Versions: 1.6, 2.1, 1.7, 2.2
>            Reporter: Fırat KÜÇÜK
>            Priority: Minor
>         Attachments: max_url_length.diff, test_case.txt
>
>
> Filtering very long urls (such as base64 image generators) take long time (hours). On reducing phase it locks down all the system for hours. Therefore some URL length limitation needed. We attached a little patch for this improvement.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)