You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Wei-Chiu Chuang <we...@cloudera.com> on 2016/02/16 18:20:20 UTC
Replacing Commons-httpclient and bumping httpclient version
Fellow Hadoop developers,
Hadoop codebase depends on commons-httpclient, and its latest version, 3.1.2, is EOL nearly 5 years ago. But because its API is not compatible with its successor, httpclient 4, the community seem to have been reluctant to upgrade.
However, a lot of evidence indicates that commons-httpclient has a number of security vulnerabilities which are never addressed, including CVE-2012-6153. To make Hadoop less susceptible to existing and future vulnerabilities, we should seriously consider replacing commons-httpclient with httpclient 4.x.
There are a few Hadoop JIRAs that have patches available to address that, but they really need more attention to get them committed:
HADOOP-10105 <https://issues.apache.org/jira/browse/HADOOP-10105> (remove httpclient dependency) is the umbrella JIRA for all.
Other efforts includes HADOOP-11613 <https://issues.apache.org/jira/browse/HADOOP-11613> (Remove httpclient dependency from hadoop-azure), HADOOP-11614 <https://issues.apache.org/jira/browse/HADOOP-11614> (Remove httpclient dependency from hadoop-openstack), HADOOP-12710 <https://issues.apache.org/jira/browse/HADOOP-12710> (Remove dependency on commons-httpclient for TestHttpServerLogs), HADOOP-12711 <https://issues.apache.org/jira/browse/HADOOP-12711> (Remove dependency on commons-httpclient for ServletUtil). I’d also like to urge the community to reject patches that imports commons-httpclient in the future.
Additionally, Hadoop trunk depends on httpclient 4.2.5, which is known to suffer from several security vulnerabilities as well, including CVE-2012-6153, CVE-2011-4461, CVE-2014-3577, CVE-2015-5262. HADOOP-12767 <https://issues.apache.org/jira/browse/HADOOP-12767> (update apache httpclient version to the latest 4.5 for security) has a patch that bumps the version to 4.5.1. But I’d like to ask the community whether we should do it or not, and the implication of bump the latest version.
Best regards,
Wei-Chiu Chuang
A very happy Clouderan
Re: Replacing Commons-httpclient and bumping httpclient version
Posted by Wei-Chiu Chuang <we...@cloudera.com>.
Thanks every one for the feedbacks and attention to the related patches for replacing cnmmons-httpclient.
The second part of my question is how do people feel about bumping httpclient version? httpclient 4.2.5 used by current Hadoop also has a few security vulnerabilities. Fortunately in this case, we can easily bump its version to address the security vulnerabilities.
This refers to HADOOP-12767 <https://issues.apache.org/jira/browse/HADOOP-12767> (update apache httpclient version to the latest 4.5 for security)
Thanks again,
Wei-Chiu Chuang
A very happy Clouderan
> On Feb 18, 2016, at 6:50 PM, Brahma Reddy Battula <br...@huawei.com> wrote:
>
> Thanks Wei-Chiu Chuang for initiating discussion here.
>
> I'm +1 too to clean up dependency on commons-httpclient.
>
> -----Original Message-----
> From: Masatake Iwasaki [mailto:iwasakims@oss.nttdata.co.jp]
> Sent: 17 February 2016 22:52
> To: common-dev@hadoop.apache.org
> Subject: Re: Replacing Commons-httpclient and bumping httpclient version
>
> Thanks for the suggestion, Wei-Chiu Chuang.
>
> I'm +1 too to clean up dependency on commons-httpclient.
>
> Your suggestion reminded me of HADOOP-12552 which seems to depends on HADOOP-12710 and HADOOP-12711 now.
> I will revisit it.
>
> Masatake Iwasaki
>
> On 2/17/16 03:59, Colin P. McCabe wrote:
>> +1 for updating the dependencies in trunk.
>>
>> best,
>> Colin
>>
>> On Tue, Feb 16, 2016 at 9:20 AM, Wei-Chiu Chuang <we...@cloudera.com> wrote:
>>> Fellow Hadoop developers,
>>>
>>> Hadoop codebase depends on commons-httpclient, and its latest version, 3.1.2, is EOL nearly 5 years ago. But because its API is not compatible with its successor, httpclient 4, the community seem to have been reluctant to upgrade.
>>> However, a lot of evidence indicates that commons-httpclient has a number of security vulnerabilities which are never addressed, including CVE-2012-6153. To make Hadoop less susceptible to existing and future vulnerabilities, we should seriously consider replacing commons-httpclient with httpclient 4.x.
>>>
>>> There are a few Hadoop JIRAs that have patches available to address that, but they really need more attention to get them committed:
>>> HADOOP-10105 <https://issues.apache.org/jira/browse/HADOOP-10105> (remove httpclient dependency) is the umbrella JIRA for all.
>>> Other efforts includes HADOOP-11613 <https://issues.apache.org/jira/browse/HADOOP-11613> (Remove httpclient dependency from hadoop-azure), HADOOP-11614 <https://issues.apache.org/jira/browse/HADOOP-11614> (Remove httpclient dependency from hadoop-openstack), HADOOP-12710 <https://issues.apache.org/jira/browse/HADOOP-12710> (Remove dependency on commons-httpclient for TestHttpServerLogs), HADOOP-12711 <https://issues.apache.org/jira/browse/HADOOP-12711> (Remove dependency on commons-httpclient for ServletUtil). I’d also like to urge the community to reject patches that imports commons-httpclient in the future.
>>>
>>> Additionally, Hadoop trunk depends on httpclient 4.2.5, which is known to suffer from several security vulnerabilities as well, including CVE-2012-6153, CVE-2011-4461, CVE-2014-3577, CVE-2015-5262. HADOOP-12767 <https://issues.apache.org/jira/browse/HADOOP-12767> (update apache httpclient version to the latest 4.5 for security) has a patch that bumps the version to 4.5.1. But I’d like to ask the community whether we should do it or not, and the implication of bump the latest version.
>>>
>>> Best regards,
>>> Wei-Chiu Chuang
>>> A very happy Clouderan
>>>
>
RE: Replacing Commons-httpclient and bumping httpclient version
Posted by Brahma Reddy Battula <br...@huawei.com>.
Thanks Wei-Chiu Chuang for initiating discussion here.
I'm +1 too to clean up dependency on commons-httpclient.
-----Original Message-----
From: Masatake Iwasaki [mailto:iwasakims@oss.nttdata.co.jp]
Sent: 17 February 2016 22:52
To: common-dev@hadoop.apache.org
Subject: Re: Replacing Commons-httpclient and bumping httpclient version
Thanks for the suggestion, Wei-Chiu Chuang.
I'm +1 too to clean up dependency on commons-httpclient.
Your suggestion reminded me of HADOOP-12552 which seems to depends on HADOOP-12710 and HADOOP-12711 now.
I will revisit it.
Masatake Iwasaki
On 2/17/16 03:59, Colin P. McCabe wrote:
> +1 for updating the dependencies in trunk.
>
> best,
> Colin
>
> On Tue, Feb 16, 2016 at 9:20 AM, Wei-Chiu Chuang <we...@cloudera.com> wrote:
>> Fellow Hadoop developers,
>>
>> Hadoop codebase depends on commons-httpclient, and its latest version, 3.1.2, is EOL nearly 5 years ago. But because its API is not compatible with its successor, httpclient 4, the community seem to have been reluctant to upgrade.
>> However, a lot of evidence indicates that commons-httpclient has a number of security vulnerabilities which are never addressed, including CVE-2012-6153. To make Hadoop less susceptible to existing and future vulnerabilities, we should seriously consider replacing commons-httpclient with httpclient 4.x.
>>
>> There are a few Hadoop JIRAs that have patches available to address that, but they really need more attention to get them committed:
>> HADOOP-10105 <https://issues.apache.org/jira/browse/HADOOP-10105> (remove httpclient dependency) is the umbrella JIRA for all.
>> Other efforts includes HADOOP-11613 <https://issues.apache.org/jira/browse/HADOOP-11613> (Remove httpclient dependency from hadoop-azure), HADOOP-11614 <https://issues.apache.org/jira/browse/HADOOP-11614> (Remove httpclient dependency from hadoop-openstack), HADOOP-12710 <https://issues.apache.org/jira/browse/HADOOP-12710> (Remove dependency on commons-httpclient for TestHttpServerLogs), HADOOP-12711 <https://issues.apache.org/jira/browse/HADOOP-12711> (Remove dependency on commons-httpclient for ServletUtil). I’d also like to urge the community to reject patches that imports commons-httpclient in the future.
>>
>> Additionally, Hadoop trunk depends on httpclient 4.2.5, which is known to suffer from several security vulnerabilities as well, including CVE-2012-6153, CVE-2011-4461, CVE-2014-3577, CVE-2015-5262. HADOOP-12767 <https://issues.apache.org/jira/browse/HADOOP-12767> (update apache httpclient version to the latest 4.5 for security) has a patch that bumps the version to 4.5.1. But I’d like to ask the community whether we should do it or not, and the implication of bump the latest version.
>>
>> Best regards,
>> Wei-Chiu Chuang
>> A very happy Clouderan
>>
Re: Replacing Commons-httpclient and bumping httpclient version
Posted by Masatake Iwasaki <iw...@oss.nttdata.co.jp>.
Thanks for the suggestion, Wei-Chiu Chuang.
I'm +1 too to clean up dependency on commons-httpclient.
Your suggestion reminded me of HADOOP-12552 which
seems to depends on HADOOP-12710 and HADOOP-12711 now.
I will revisit it.
Masatake Iwasaki
On 2/17/16 03:59, Colin P. McCabe wrote:
> +1 for updating the dependencies in trunk.
>
> best,
> Colin
>
> On Tue, Feb 16, 2016 at 9:20 AM, Wei-Chiu Chuang <we...@cloudera.com> wrote:
>> Fellow Hadoop developers,
>>
>> Hadoop codebase depends on commons-httpclient, and its latest version, 3.1.2, is EOL nearly 5 years ago. But because its API is not compatible with its successor, httpclient 4, the community seem to have been reluctant to upgrade.
>> However, a lot of evidence indicates that commons-httpclient has a number of security vulnerabilities which are never addressed, including CVE-2012-6153. To make Hadoop less susceptible to existing and future vulnerabilities, we should seriously consider replacing commons-httpclient with httpclient 4.x.
>>
>> There are a few Hadoop JIRAs that have patches available to address that, but they really need more attention to get them committed:
>> HADOOP-10105 <https://issues.apache.org/jira/browse/HADOOP-10105> (remove httpclient dependency) is the umbrella JIRA for all.
>> Other efforts includes HADOOP-11613 <https://issues.apache.org/jira/browse/HADOOP-11613> (Remove httpclient dependency from hadoop-azure), HADOOP-11614 <https://issues.apache.org/jira/browse/HADOOP-11614> (Remove httpclient dependency from hadoop-openstack), HADOOP-12710 <https://issues.apache.org/jira/browse/HADOOP-12710> (Remove dependency on commons-httpclient for TestHttpServerLogs), HADOOP-12711 <https://issues.apache.org/jira/browse/HADOOP-12711> (Remove dependency on commons-httpclient for ServletUtil). I’d also like to urge the community to reject patches that imports commons-httpclient in the future.
>>
>> Additionally, Hadoop trunk depends on httpclient 4.2.5, which is known to suffer from several security vulnerabilities as well, including CVE-2012-6153, CVE-2011-4461, CVE-2014-3577, CVE-2015-5262. HADOOP-12767 <https://issues.apache.org/jira/browse/HADOOP-12767> (update apache httpclient version to the latest 4.5 for security) has a patch that bumps the version to 4.5.1. But I’d like to ask the community whether we should do it or not, and the implication of bump the latest version.
>>
>> Best regards,
>> Wei-Chiu Chuang
>> A very happy Clouderan
>>
Re: Replacing Commons-httpclient and bumping httpclient version
Posted by "Colin P. McCabe" <cm...@apache.org>.
+1 for updating the dependencies in trunk.
best,
Colin
On Tue, Feb 16, 2016 at 9:20 AM, Wei-Chiu Chuang <we...@cloudera.com> wrote:
> Fellow Hadoop developers,
>
> Hadoop codebase depends on commons-httpclient, and its latest version, 3.1.2, is EOL nearly 5 years ago. But because its API is not compatible with its successor, httpclient 4, the community seem to have been reluctant to upgrade.
> However, a lot of evidence indicates that commons-httpclient has a number of security vulnerabilities which are never addressed, including CVE-2012-6153. To make Hadoop less susceptible to existing and future vulnerabilities, we should seriously consider replacing commons-httpclient with httpclient 4.x.
>
> There are a few Hadoop JIRAs that have patches available to address that, but they really need more attention to get them committed:
> HADOOP-10105 <https://issues.apache.org/jira/browse/HADOOP-10105> (remove httpclient dependency) is the umbrella JIRA for all.
> Other efforts includes HADOOP-11613 <https://issues.apache.org/jira/browse/HADOOP-11613> (Remove httpclient dependency from hadoop-azure), HADOOP-11614 <https://issues.apache.org/jira/browse/HADOOP-11614> (Remove httpclient dependency from hadoop-openstack), HADOOP-12710 <https://issues.apache.org/jira/browse/HADOOP-12710> (Remove dependency on commons-httpclient for TestHttpServerLogs), HADOOP-12711 <https://issues.apache.org/jira/browse/HADOOP-12711> (Remove dependency on commons-httpclient for ServletUtil). I’d also like to urge the community to reject patches that imports commons-httpclient in the future.
>
> Additionally, Hadoop trunk depends on httpclient 4.2.5, which is known to suffer from several security vulnerabilities as well, including CVE-2012-6153, CVE-2011-4461, CVE-2014-3577, CVE-2015-5262. HADOOP-12767 <https://issues.apache.org/jira/browse/HADOOP-12767> (update apache httpclient version to the latest 4.5 for security) has a patch that bumps the version to 4.5.1. But I’d like to ask the community whether we should do it or not, and the implication of bump the latest version.
>
> Best regards,
> Wei-Chiu Chuang
> A very happy Clouderan
>