[jira] [Updated] (NUTCH-3107) QueryString normalizer to support per-host removal of qstr params

["Sebastian Nagel (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NUTCH-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sebastian Nagel updated NUTCH-3107:
-----------------------------------
    Fix Version/s: 1.22
                       (was: 1.21)

> QueryString normalizer to support per-host removal of qstr params
> -----------------------------------------------------------------
>
>                 Key: NUTCH-3107
>                 URL: https://issues.apache.org/jira/browse/NUTCH-3107
>             Project: Nutch
>          Issue Type: Improvement
>          Components: urlnormalizer
>            Reporter: Markus Jelsma
>            Assignee: Markus Jelsma
>            Priority: Minor
>             Fix For: 1.22
>
>         Attachments: NUTCH-3107-1.patch, NUTCH-3107-2.patch, NUTCH-3107.patch
>
>
> QueryString Normalizer now only does sorting of query string key/value pairs. It could also support removal of per-host configurable keys.
> Normally this can be done in normalizer regex, but having a few million XML entries in the config parsed everytime, and millions of regular expressions executed is not very convenient.
> Updated patch also adds support for global ignorable params, and some other checks on query string keys.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)