You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Marcin Okraszewski (JIRA)" <ji...@apache.org> on 2009/05/28 23:13:45 UTC
[jira] Created: (NUTCH-740) Configuration option to override
default language for fetched pages.
Configuration option to override default language for fetched pages.
--------------------------------------------------------------------
Key: NUTCH-740
URL: https://issues.apache.org/jira/browse/NUTCH-740
Project: Nutch
Issue Type: Improvement
Components: fetcher
Affects Versions: 1.0.0, 0.9.0
Reporter: Marcin Okraszewski
Fix For: 1.0.0
By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-740) Configuration option to override
default language for fetched pages.
Posted by "Otis Gospodnetic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Otis Gospodnetic updated NUTCH-740:
-----------------------------------
Priority: Minor (was: Major)
Affects Version/s: (was: 0.9.0)
Fix Version/s: (was: 1.0.0)
1.1
Assignee: Otis Gospodnetic
> Configuration option to override default language for fetched pages.
> --------------------------------------------------------------------
>
> Key: NUTCH-740
> URL: https://issues.apache.org/jira/browse/NUTCH-740
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Affects Versions: 1.0.0
> Reporter: Marcin Okraszewski
> Assignee: Otis Gospodnetic
> Priority: Minor
> Fix For: 1.1
>
> Attachments: AcceptLanguage.patch
>
>
> By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-740) Configuration option to override
default language for fetched pages.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12848537#action_12848537 ]
Hudson commented on NUTCH-740:
------------------------------
Integrated in Nutch-trunk #1104 (See [http://hudson.zones.apache.org/hudson/job/Nutch-trunk/1104/])
Configuration option to override default language for fetched pages
> Configuration option to override default language for fetched pages.
> --------------------------------------------------------------------
>
> Key: NUTCH-740
> URL: https://issues.apache.org/jira/browse/NUTCH-740
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Affects Versions: 1.0.0
> Reporter: Marcin Okraszewski
> Assignee: Julien Nioche
> Priority: Minor
> Fix For: 1.1
>
> Attachments: AcceptLanguage.patch, AcceptLanguage_trunk_2009-06-09.patch, NUTCH-740.patch
>
>
> By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (NUTCH-740) Configuration option to override default
language for fetched pages.
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julien Nioche closed NUTCH-740.
-------------------------------
Resolution: Fixed
Assignee: Julien Nioche
Committed in rev 926003
Thanks Marcin for contributing this patch
> Configuration option to override default language for fetched pages.
> --------------------------------------------------------------------
>
> Key: NUTCH-740
> URL: https://issues.apache.org/jira/browse/NUTCH-740
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Affects Versions: 1.0.0
> Reporter: Marcin Okraszewski
> Assignee: Julien Nioche
> Priority: Minor
> Fix For: 1.1
>
> Attachments: AcceptLanguage.patch, AcceptLanguage_trunk_2009-06-09.patch, NUTCH-740.patch
>
>
> By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-740) Configuration option to override
default language for fetched pages.
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12845886#action_12845886 ]
Julien Nioche commented on NUTCH-740:
-------------------------------------
A nice contribution but should not this be applied to the *protocol-http* plugin as well e.g. in HttpResponse?
> Configuration option to override default language for fetched pages.
> --------------------------------------------------------------------
>
> Key: NUTCH-740
> URL: https://issues.apache.org/jira/browse/NUTCH-740
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Affects Versions: 1.0.0
> Reporter: Marcin Okraszewski
> Assignee: Otis Gospodnetic
> Priority: Minor
> Fix For: 1.1
>
> Attachments: AcceptLanguage.patch, AcceptLanguage_trunk_2009-06-09.patch
>
>
> By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-740) Configuration option to override
default language for fetched pages.
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julien Nioche updated NUTCH-740:
--------------------------------
Attachment: NUTCH-740.patch
Slightly modified version of the patch with modifs for protocol-http.
will commit shortly
> Configuration option to override default language for fetched pages.
> --------------------------------------------------------------------
>
> Key: NUTCH-740
> URL: https://issues.apache.org/jira/browse/NUTCH-740
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Affects Versions: 1.0.0
> Reporter: Marcin Okraszewski
> Priority: Minor
> Fix For: 1.1
>
> Attachments: AcceptLanguage.patch, AcceptLanguage_trunk_2009-06-09.patch, NUTCH-740.patch
>
>
> By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-740) Configuration option to override
default language for fetched pages.
Posted by "Marcin Okraszewski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcin Okraszewski updated NUTCH-740:
-------------------------------------
Attachment: AcceptLanguage_trunk_2009-06-09.patch
It does apply, but with Fuzz factor set to 2. Here is the ported patch.
> Configuration option to override default language for fetched pages.
> --------------------------------------------------------------------
>
> Key: NUTCH-740
> URL: https://issues.apache.org/jira/browse/NUTCH-740
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Affects Versions: 1.0.0
> Reporter: Marcin Okraszewski
> Assignee: Otis Gospodnetic
> Priority: Minor
> Fix For: 1.1
>
> Attachments: AcceptLanguage.patch, AcceptLanguage_trunk_2009-06-09.patch
>
>
> By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-740) Configuration option to override
default language for fetched pages.
Posted by "Marcin Okraszewski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcin Okraszewski updated NUTCH-740:
-------------------------------------
Attachment: AcceptLanguage.patch
The patch which allows overriding of "Accept-Language" header. The patch is done on 1.0 code.
> Configuration option to override default language for fetched pages.
> --------------------------------------------------------------------
>
> Key: NUTCH-740
> URL: https://issues.apache.org/jira/browse/NUTCH-740
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Affects Versions: 0.9.0, 1.0.0
> Reporter: Marcin Okraszewski
> Fix For: 1.0.0
>
> Attachments: AcceptLanguage.patch
>
>
> By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-740) Configuration option to override
default language for fetched pages.
Posted by "Doğacan Güney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717061#action_12717061 ]
Doğacan Güney commented on NUTCH-740:
-------------------------------------
+1 for the idea but your patch does not apply to trunk.
> Configuration option to override default language for fetched pages.
> --------------------------------------------------------------------
>
> Key: NUTCH-740
> URL: https://issues.apache.org/jira/browse/NUTCH-740
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Affects Versions: 1.0.0
> Reporter: Marcin Okraszewski
> Assignee: Otis Gospodnetic
> Priority: Minor
> Fix For: 1.1
>
> Attachments: AcceptLanguage.patch
>
>
> By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-740) Configuration option to override
default language for fetched pages.
Posted by "Otis Gospodnetic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Otis Gospodnetic updated NUTCH-740:
-----------------------------------
Assignee: (was: Otis Gospodnetic)
> Configuration option to override default language for fetched pages.
> --------------------------------------------------------------------
>
> Key: NUTCH-740
> URL: https://issues.apache.org/jira/browse/NUTCH-740
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Affects Versions: 1.0.0
> Reporter: Marcin Okraszewski
> Priority: Minor
> Fix For: 1.1
>
> Attachments: AcceptLanguage.patch, AcceptLanguage_trunk_2009-06-09.patch
>
>
> By default "Accept-Language" HTTP request header is set to English. Unfortunately this value is hard coded and seems there is no way to override it. As a result you may index English version of pages even though you would prefer it in different language.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.