You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Stefan Neufeind (JIRA)" <ji...@apache.org> on 2006/05/22 15:10:29 UTC
[jira] Created: (NUTCH-279) Additions for regex-normalize
Additions for regex-normalize
-----------------------------
Key: NUTCH-279
URL: http://issues.apache.org/jira/browse/NUTCH-279
Project: Nutch
Type: Improvement
Versions: 0.8-dev
Reporter: Stefan Neufeind
Imho needed:
1) Extend normalize-rules to commonly used session-id's etc.
2) Ship a checker to check rules easily by hand
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (NUTCH-279) Additions for regex-normalize
Posted by "Stefan Neufeind (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-279?page=all ]
Stefan Neufeind updated NUTCH-279:
----------------------------------
Attachment: regex-normalize2.patch
New patch with just one session-ID-regex extended (also including . - , now), since I came across those extra chars while used on a common German website (www.bahn.de).
> Additions for regex-normalize
> -----------------------------
>
> Key: NUTCH-279
> URL: http://issues.apache.org/jira/browse/NUTCH-279
> Project: Nutch
> Type: Improvement
> Versions: 0.8-dev
> Reporter: Stefan Neufeind
> Attachments: regex-normalize.patch, regex-normalize2.patch
>
> Imho needed:
> 1) Extend normalize-rules to commonly used session-id's etc.
> 2) Ship a checker to check rules easily by hand
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (NUTCH-279) Additions for regex-normalize
Posted by "Stefan Neufeind (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-279?page=all ]
Stefan Neufeind updated NUTCH-279:
----------------------------------
Attachment: regex-normalize.patch
1) Incorporates jsessionid-normalization from NUTCH-255
2) Adds further normalizations
3) Adds a commandline-checker. Start with:
bin/nutch org.apache.nutch.net.RegexUrlNormalizerChecker
> Additions for regex-normalize
> -----------------------------
>
> Key: NUTCH-279
> URL: http://issues.apache.org/jira/browse/NUTCH-279
> Project: Nutch
> Type: Improvement
> Versions: 0.8-dev
> Reporter: Stefan Neufeind
> Attachments: regex-normalize.patch
>
> Imho needed:
> 1) Extend normalize-rules to commonly used session-id's etc.
> 2) Ship a checker to check rules easily by hand
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira