You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Claudio Martella (JIRA)" <ji...@apache.org> on 2011/01/17 11:56:43 UTC
[jira] Created: (NUTCH-958) Httpclient scheme priority order fix
Httpclient scheme priority order fix
------------------------------------
Key: NUTCH-958
URL: https://issues.apache.org/jira/browse/NUTCH-958
Project: Nutch
Issue Type: Bug
Components: fetcher
Affects Versions: 1.3
Reporter: Claudio Martella
Fix For: 1.3
Attachments: httpclient.diff
Httpclient will try to authenticate in this order by default: ntlm, digest, basic.
If you set as default a scheme that comes in this list after a scheme that is negotiated by the server, and this authentication fails, the default scheme will not be tried.
I.e. if you set digest as default scheme but the server negotiates ntlm, the client will still try ntlm and fail.
The fix sets the default scheme as the only possible scheme for authentication for the given realm by setting the authentication priorities of httpclient.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-958) Httpclient scheme priority order fix
Posted by "Markus Jelsma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008422#comment-13008422 ]
Markus Jelsma commented on NUTCH-958:
-------------------------------------
Claudio, i am not sure if this workaround should be committed at all. If the devs agree then it should:
- be patched for 2.0 as well
- add a configuration option to enable your workaround so to prevent breaking other user's HTTP authentication methods
> Httpclient scheme priority order fix
> ------------------------------------
>
> Key: NUTCH-958
> URL: https://issues.apache.org/jira/browse/NUTCH-958
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 1.3
> Reporter: Claudio Martella
> Fix For: 1.3
>
> Attachments: httpclient.diff
>
>
> Httpclient will try to authenticate in this order by default: ntlm, digest, basic.
> If you set as default a scheme that comes in this list after a scheme that is negotiated by the server, and this authentication fails, the default scheme will not be tried.
> I.e. if you set digest as default scheme but the server negotiates ntlm, the client will still try ntlm and fail.
> The fix sets the default scheme as the only possible scheme for authentication for the given realm by setting the authentication priorities of httpclient.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-958) Httpclient scheme priority order fix
Posted by "Claudio Martella (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008432#comment-13008432 ]
Claudio Martella commented on NUTCH-958:
----------------------------------------
this workaround was necessary for my work and introduced an expected behavior. I understand it's not clean, but the actual behavior of nutch isn't correct either. Maybe it can be useful for somebody else and maybe it's enough to keep it here so people can find it and apply the patch if they like, so that it doesn't have to be commited.
The "right" way would probably just pass through moving to httpclient4.
> Httpclient scheme priority order fix
> ------------------------------------
>
> Key: NUTCH-958
> URL: https://issues.apache.org/jira/browse/NUTCH-958
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 1.3
> Reporter: Claudio Martella
> Fix For: 1.3
>
> Attachments: httpclient.diff
>
>
> Httpclient will try to authenticate in this order by default: ntlm, digest, basic.
> If you set as default a scheme that comes in this list after a scheme that is negotiated by the server, and this authentication fails, the default scheme will not be tried.
> I.e. if you set digest as default scheme but the server negotiates ntlm, the client will still try ntlm and fail.
> The fix sets the default scheme as the only possible scheme for authentication for the given realm by setting the authentication priorities of httpclient.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-958) Httpclient scheme priority order fix
Posted by "Claudio Martella (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008408#comment-13008408 ]
Claudio Martella commented on NUTCH-958:
----------------------------------------
that is the problem. right now the system does not allow the default scheme to be used as a fallback, which is the reason i wrote this patch. that comes because of a bug in httpclient.
So, in order to have some control over the kind of authentication is used, which is the expected behavior you also describe, the only way is through this workaround.
> Httpclient scheme priority order fix
> ------------------------------------
>
> Key: NUTCH-958
> URL: https://issues.apache.org/jira/browse/NUTCH-958
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 1.3
> Reporter: Claudio Martella
> Fix For: 1.3
>
> Attachments: httpclient.diff
>
>
> Httpclient will try to authenticate in this order by default: ntlm, digest, basic.
> If you set as default a scheme that comes in this list after a scheme that is negotiated by the server, and this authentication fails, the default scheme will not be tried.
> I.e. if you set digest as default scheme but the server negotiates ntlm, the client will still try ntlm and fail.
> The fix sets the default scheme as the only possible scheme for authentication for the given realm by setting the authentication priorities of httpclient.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-958) Httpclient scheme priority order fix
Posted by "Claudio Martella (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008511#comment-13008511 ]
Claudio Martella commented on NUTCH-958:
----------------------------------------
yes, go on.
> Httpclient scheme priority order fix
> ------------------------------------
>
> Key: NUTCH-958
> URL: https://issues.apache.org/jira/browse/NUTCH-958
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 1.3
> Reporter: Claudio Martella
> Fix For: 1.3
>
> Attachments: httpclient.diff
>
>
> Httpclient will try to authenticate in this order by default: ntlm, digest, basic.
> If you set as default a scheme that comes in this list after a scheme that is negotiated by the server, and this authentication fails, the default scheme will not be tried.
> I.e. if you set digest as default scheme but the server negotiates ntlm, the client will still try ntlm and fail.
> The fix sets the default scheme as the only possible scheme for authentication for the given realm by setting the authentication priorities of httpclient.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (NUTCH-958) Httpclient scheme priority order fix
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julien Nioche resolved NUTCH-958.
---------------------------------
Resolution: Won't Fix
See comments. This patch fixes a bug in the underlying httpclient library which will be upgraded later anyway
> Httpclient scheme priority order fix
> ------------------------------------
>
> Key: NUTCH-958
> URL: https://issues.apache.org/jira/browse/NUTCH-958
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 1.3
> Reporter: Claudio Martella
> Fix For: 1.3
>
> Attachments: httpclient.diff
>
>
> Httpclient will try to authenticate in this order by default: ntlm, digest, basic.
> If you set as default a scheme that comes in this list after a scheme that is negotiated by the server, and this authentication fails, the default scheme will not be tried.
> I.e. if you set digest as default scheme but the server negotiates ntlm, the client will still try ntlm and fail.
> The fix sets the default scheme as the only possible scheme for authentication for the given realm by setting the authentication priorities of httpclient.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (NUTCH-958) Httpclient scheme priority order fix
Posted by "Claudio Martella (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claudio Martella updated NUTCH-958:
-----------------------------------
Attachment: httpclient.diff
> Httpclient scheme priority order fix
> ------------------------------------
>
> Key: NUTCH-958
> URL: https://issues.apache.org/jira/browse/NUTCH-958
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 1.3
> Reporter: Claudio Martella
> Fix For: 1.3
>
> Attachments: httpclient.diff
>
>
> Httpclient will try to authenticate in this order by default: ntlm, digest, basic.
> If you set as default a scheme that comes in this list after a scheme that is negotiated by the server, and this authentication fails, the default scheme will not be tried.
> I.e. if you set digest as default scheme but the server negotiates ntlm, the client will still try ntlm and fail.
> The fix sets the default scheme as the only possible scheme for authentication for the given realm by setting the authentication priorities of httpclient.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Closed] (NUTCH-958) Httpclient scheme priority order fix
Posted by "Markus Jelsma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Markus Jelsma closed NUTCH-958.
-------------------------------
Bulk close of resolved issues:
http://www.lucidimagination.com/search/document/2738eeb014805854/clean_up_open_legacy_issues_in_jira
> Httpclient scheme priority order fix
> ------------------------------------
>
> Key: NUTCH-958
> URL: https://issues.apache.org/jira/browse/NUTCH-958
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 1.3
> Reporter: Claudio Martella
> Fix For: 1.3
>
> Attachments: httpclient.diff
>
>
> Httpclient will try to authenticate in this order by default: ntlm, digest, basic.
> If you set as default a scheme that comes in this list after a scheme that is negotiated by the server, and this authentication fails, the default scheme will not be tried.
> I.e. if you set digest as default scheme but the server negotiates ntlm, the client will still try ntlm and fail.
> The fix sets the default scheme as the only possible scheme for authentication for the given realm by setting the authentication priorities of httpclient.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-958) Httpclient scheme priority order fix
Posted by "Markus Jelsma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008401#comment-13008401 ]
Markus Jelsma commented on NUTCH-958:
-------------------------------------
Hi Claudio. Is this desired behaviour? Shouldn't the default be used as fallback if the negotiated schema fails instead forcing default as only scheme?
> Httpclient scheme priority order fix
> ------------------------------------
>
> Key: NUTCH-958
> URL: https://issues.apache.org/jira/browse/NUTCH-958
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 1.3
> Reporter: Claudio Martella
> Fix For: 1.3
>
> Attachments: httpclient.diff
>
>
> Httpclient will try to authenticate in this order by default: ntlm, digest, basic.
> If you set as default a scheme that comes in this list after a scheme that is negotiated by the server, and this authentication fails, the default scheme will not be tried.
> I.e. if you set digest as default scheme but the server negotiates ntlm, the client will still try ntlm and fail.
> The fix sets the default scheme as the only possible scheme for authentication for the given realm by setting the authentication priorities of httpclient.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-958) Httpclient scheme priority order fix
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008508#comment-13008508 ]
Julien Nioche commented on NUTCH-958:
-------------------------------------
I had a look at upgrading to a more recent version of httpclient but it was a substantial job as most of the API had changed. We'll definitely do that for Nutch 2.0 at some point.
What about marking this issue as won't fix and move it out of 1.3? As you said people will find your patch here if they have the same problem and can easily apply it.
> Httpclient scheme priority order fix
> ------------------------------------
>
> Key: NUTCH-958
> URL: https://issues.apache.org/jira/browse/NUTCH-958
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 1.3
> Reporter: Claudio Martella
> Fix For: 1.3
>
> Attachments: httpclient.diff
>
>
> Httpclient will try to authenticate in this order by default: ntlm, digest, basic.
> If you set as default a scheme that comes in this list after a scheme that is negotiated by the server, and this authentication fails, the default scheme will not be tried.
> I.e. if you set digest as default scheme but the server negotiates ntlm, the client will still try ntlm and fail.
> The fix sets the default scheme as the only possible scheme for authentication for the given realm by setting the authentication priorities of httpclient.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira