You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Andrzej Bialecki (JIRA)" <ji...@apache.org> on 2008/03/16 21:06:24 UTC
[jira] Commented: (NUTCH-620) BasicURLNormalizer should collapse
runs of slashes with a single slash
[ https://issues.apache.org/jira/browse/NUTCH-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579243#action_12579243 ]
Andrzej Bialecki commented on NUTCH-620:
-----------------------------------------
Is this an often occurring problem? I'm wary of extending the rules in basic normalizer, because they will affect a lot of users, and running each new rule adds certain cost in terms of performance and possible lock-ups of the regex engine. If this is a problem experienced by many users, we should add this rule to the default rule set, but if it's specific just to our environment I think we shouldn't add it.
> BasicURLNormalizer should collapse runs of slashes with a single slash
> ----------------------------------------------------------------------
>
> Key: NUTCH-620
> URL: https://issues.apache.org/jira/browse/NUTCH-620
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 0.9.0
> Environment: JDK 1.6 update 5, Tomcat 6, Windows Server 2003,
> Reporter: Mark DeSpain
> Priority: Minor
> Fix For: 1.0.0
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> The BasicURLNormalizer should collapse runs of slash characters '/' with a single slash.
> For example, the following URLs should be normalized to http://lucene.apache.org/nutch/about.html
> * http://lucene.apache.org/nutch//about.html
> * http://lucene.apache.org//nutch/about.html
> * http://lucene.apache.org/////nutch////about.html (an exaggerated example)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.