You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Qiang Zhang <zh...@zte.com.cn> on 2017/02/10 08:20:00 UTC
Re: Review Request 55256: RANGER-1289:Error occured in Ranger KMS
function
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55256/
-----------------------------------------------------------
(Updated \u4e8c\u6708 10, 2017, 8:19 a.m.)
Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
Bugs: RANGER-1289
https://issues.apache.org/jira/browse/RANGER-1289
Repository: ranger
Description (updated)
-------
Steps:
1.Start ranger-kms service
2.Configure KMS Client and restart hdfs
3.Create a key named key0 in the Ranger Web UI
4.Execute the following command in hadoop environment,create an encrypted zone
[xhh@zdh41 ~]$hdfs dfs -mkdir /keyZone
[xhh@zdh41 ~]$hdfs crypto -createZone -keyName key0 -path /keyZone
Error message poped out as below:
ranger-0.7.0-SNAPSHOT-kms/ews/logs/kms.log
2017-01-04 14:27:13,256 ERROR [webservices-driver] - Servlet.service() for servlet [webservices-driver] in context with path [/kms] threw exception
java.lang.NullPointerException
at org.apache.http.client.utils.URLEncodedUtils.parse(URLEncodedUtils.java:235)
at org.apache.hadoop.security.token.delegation.web.ServletUtils.getParameter(ServletUtils.java:48)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.managementOperation(DelegationTokenAuthenticationHandler.java:171)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:514)
at org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
I analyzed the reasons,Ranger-Kms relies on httpclient version 4.5.1, there is a bug, as follows:
org/apache/http/client/utils/URLEncodedUtils.java
{code:java}
public static List<NameValuePair> parse(String s, Charset charset) {
CharArrayBuffer buffer = new CharArrayBuffer(s.length());
buffer.append(s);
return parse(buffer, charset, new char[]{'&', ';'});
}
{code}
When the parameter 's' is null, it will pop out NullPointException.
And in httpclient version 4.5.3, there is no problem. the new code is as follows:
{code:java}
public static List<NameValuePair> parse(final String s, final Charset charset) {
if (s == null) {
return Collections.emptyList();
}
final CharArrayBuffer buffer = new CharArrayBuffer(s.length());
buffer.append(s);
return parse(buffer, charset, QP_SEP_A, QP_SEP_S);
}
{code}
Diffs (updated)
-----
pom.xml 19caacd
Diff: https://reviews.apache.org/r/55256/diff/
Testing (updated)
-------
[xhh@zdh41 ~]$ hadoop key list -metadata
17/02/10 16:20:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Listing keys for KeyProvider: KMSClientProvider[http://zdh41:9292/kms/v1/]
key0 : cipher: AES/CTR/NoPadding, length: 128, description: , created: Thu Feb 09 09:56:51 CST 2017, version: 1, attributes: [key.acl.name=key0]
Thanks,
Qiang Zhang
Re: Review Request 55256: RANGER-1289:Error occured in Ranger KMS
function
Posted by Colm O hEigeartaigh <co...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55256/#review165316
-----------------------------------------------------------
Ship it!
Ship It!
- Colm O hEigeartaigh
On Feb. 13, 2017, 2:23 a.m., Qiang Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55256/
> -----------------------------------------------------------
>
> (Updated Feb. 13, 2017, 2:23 a.m.)
>
>
> Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1289
> https://issues.apache.org/jira/browse/RANGER-1289
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Steps:
> 1.Start ranger-kms service
> 2.Configure KMS Client and restart hdfs
> 3.Create a key named key0 in the Ranger Web UI
> 4.Execute the following command in hadoop environment,create an encrypted zone
> [xhh@zdh41 ~]$hdfs dfs -mkdir /keyZone
> [xhh@zdh41 ~]$hdfs crypto -createZone -keyName key0 -path /keyZone
>
>
> Error message poped out as below:
> ranger-0.7.0-SNAPSHOT-kms/ews/logs/kms.log
>
> 2017-01-04 14:27:13,256 ERROR [webservices-driver] - Servlet.service() for servlet [webservices-driver] in context with path [/kms] threw exception
> java.lang.NullPointerException
> at org.apache.http.client.utils.URLEncodedUtils.parse(URLEncodedUtils.java:235)
> at org.apache.hadoop.security.token.delegation.web.ServletUtils.getParameter(ServletUtils.java:48)
> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.managementOperation(DelegationTokenAuthenticationHandler.java:171)
> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:514)
> at org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)
> at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
> at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
> at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> I analyzed the reasons,Ranger-Kms relies on httpclient version 4.5.1, there is a bug, as follows:
> org/apache/http/client/utils/URLEncodedUtils.java
> {code:java}
> public static List<NameValuePair> parse(String s, Charset charset) {
> CharArrayBuffer buffer = new CharArrayBuffer(s.length());
> buffer.append(s);
> return parse(buffer, charset, new char[]{'&', ';'});
> }
> {code}
> When the parameter 's' is null, it will pop out NullPointException.
> And in httpclient version 4.5.3, there is no problem. the new code is as follows:
> {code:java}
> public static List<NameValuePair> parse(final String s, final Charset charset) {
> if (s == null) {
> return Collections.emptyList();
> }
> final CharArrayBuffer buffer = new CharArrayBuffer(s.length());
> buffer.append(s);
> return parse(buffer, charset, QP_SEP_A, QP_SEP_S);
> }
> {code}
>
>
> Diffs
> -----
>
> pom.xml 19caacd
>
> Diff: https://reviews.apache.org/r/55256/diff/
>
>
> Testing
> -------
>
> [xhh@zdh41 ~]$ hadoop key list -metadata
> 17/02/10 16:20:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> Listing keys for KeyProvider: KMSClientProvider[http://zdh41:9292/kms/v1/]
> key0 : cipher: AES/CTR/NoPadding, length: 128, description: , created: Thu Feb 09 09:56:51 CST 2017, version: 1, attributes: [key.acl.name=key0]
>
>
> Thanks,
>
> Qiang Zhang
>
>
Re: Review Request 55256: RANGER-1289:Error occured in Ranger KMS
function
Posted by Qiang Zhang <zh...@zte.com.cn>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55256/
-----------------------------------------------------------
(Updated \u4e8c\u6708 13, 2017, 2:23 a.m.)
Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
Bugs: RANGER-1289
https://issues.apache.org/jira/browse/RANGER-1289
Repository: ranger
Description
-------
Steps:
1.Start ranger-kms service
2.Configure KMS Client and restart hdfs
3.Create a key named key0 in the Ranger Web UI
4.Execute the following command in hadoop environment,create an encrypted zone
[xhh@zdh41 ~]$hdfs dfs -mkdir /keyZone
[xhh@zdh41 ~]$hdfs crypto -createZone -keyName key0 -path /keyZone
Error message poped out as below:
ranger-0.7.0-SNAPSHOT-kms/ews/logs/kms.log
2017-01-04 14:27:13,256 ERROR [webservices-driver] - Servlet.service() for servlet [webservices-driver] in context with path [/kms] threw exception
java.lang.NullPointerException
at org.apache.http.client.utils.URLEncodedUtils.parse(URLEncodedUtils.java:235)
at org.apache.hadoop.security.token.delegation.web.ServletUtils.getParameter(ServletUtils.java:48)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.managementOperation(DelegationTokenAuthenticationHandler.java:171)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:514)
at org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
I analyzed the reasons,Ranger-Kms relies on httpclient version 4.5.1, there is a bug, as follows:
org/apache/http/client/utils/URLEncodedUtils.java
{code:java}
public static List<NameValuePair> parse(String s, Charset charset) {
CharArrayBuffer buffer = new CharArrayBuffer(s.length());
buffer.append(s);
return parse(buffer, charset, new char[]{'&', ';'});
}
{code}
When the parameter 's' is null, it will pop out NullPointException.
And in httpclient version 4.5.3, there is no problem. the new code is as follows:
{code:java}
public static List<NameValuePair> parse(final String s, final Charset charset) {
if (s == null) {
return Collections.emptyList();
}
final CharArrayBuffer buffer = new CharArrayBuffer(s.length());
buffer.append(s);
return parse(buffer, charset, QP_SEP_A, QP_SEP_S);
}
{code}
Diffs (updated)
-----
pom.xml 19caacd
Diff: https://reviews.apache.org/r/55256/diff/
Testing
-------
[xhh@zdh41 ~]$ hadoop key list -metadata
17/02/10 16:20:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Listing keys for KeyProvider: KMSClientProvider[http://zdh41:9292/kms/v1/]
key0 : cipher: AES/CTR/NoPadding, length: 128, description: , created: Thu Feb 09 09:56:51 CST 2017, version: 1, attributes: [key.acl.name=key0]
Thanks,
Qiang Zhang
Re: Review Request 55256: RANGER-1289:Error occured in Ranger KMS
function
Posted by Qiang Zhang <zh...@zte.com.cn>.
> On \u4e8c\u6708 10, 2017, 10:12 a.m., Colm O hEigeartaigh wrote:
> > As part of this patch, please also change the httpmime version to 4.5.3 for consistency. Also, the httpcore dependency should be changed to 4.4.6, as this is the version that is associated with httpclient 4.5.3.
[~Colm O hEigeartaigh],thanks!
I have done.
- Qiang
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55256/#review165093
-----------------------------------------------------------
On \u4e8c\u6708 13, 2017, 2:23 a.m., Qiang Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55256/
> -----------------------------------------------------------
>
> (Updated \u4e8c\u6708 13, 2017, 2:23 a.m.)
>
>
> Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1289
> https://issues.apache.org/jira/browse/RANGER-1289
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Steps:
> 1.Start ranger-kms service
> 2.Configure KMS Client and restart hdfs
> 3.Create a key named key0 in the Ranger Web UI
> 4.Execute the following command in hadoop environment,create an encrypted zone
> [xhh@zdh41 ~]$hdfs dfs -mkdir /keyZone
> [xhh@zdh41 ~]$hdfs crypto -createZone -keyName key0 -path /keyZone
>
>
> Error message poped out as below:
> ranger-0.7.0-SNAPSHOT-kms/ews/logs/kms.log
>
> 2017-01-04 14:27:13,256 ERROR [webservices-driver] - Servlet.service() for servlet [webservices-driver] in context with path [/kms] threw exception
> java.lang.NullPointerException
> at org.apache.http.client.utils.URLEncodedUtils.parse(URLEncodedUtils.java:235)
> at org.apache.hadoop.security.token.delegation.web.ServletUtils.getParameter(ServletUtils.java:48)
> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.managementOperation(DelegationTokenAuthenticationHandler.java:171)
> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:514)
> at org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)
> at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
> at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
> at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> I analyzed the reasons,Ranger-Kms relies on httpclient version 4.5.1, there is a bug, as follows:
> org/apache/http/client/utils/URLEncodedUtils.java
> {code:java}
> public static List<NameValuePair> parse(String s, Charset charset) {
> CharArrayBuffer buffer = new CharArrayBuffer(s.length());
> buffer.append(s);
> return parse(buffer, charset, new char[]{'&', ';'});
> }
> {code}
> When the parameter 's' is null, it will pop out NullPointException.
> And in httpclient version 4.5.3, there is no problem. the new code is as follows:
> {code:java}
> public static List<NameValuePair> parse(final String s, final Charset charset) {
> if (s == null) {
> return Collections.emptyList();
> }
> final CharArrayBuffer buffer = new CharArrayBuffer(s.length());
> buffer.append(s);
> return parse(buffer, charset, QP_SEP_A, QP_SEP_S);
> }
> {code}
>
>
> Diffs
> -----
>
> pom.xml 19caacd
>
> Diff: https://reviews.apache.org/r/55256/diff/
>
>
> Testing
> -------
>
> [xhh@zdh41 ~]$ hadoop key list -metadata
> 17/02/10 16:20:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> Listing keys for KeyProvider: KMSClientProvider[http://zdh41:9292/kms/v1/]
> key0 : cipher: AES/CTR/NoPadding, length: 128, description: , created: Thu Feb 09 09:56:51 CST 2017, version: 1, attributes: [key.acl.name=key0]
>
>
> Thanks,
>
> Qiang Zhang
>
>
Re: Review Request 55256: RANGER-1289:Error occured in Ranger KMS
function
Posted by Colm O hEigeartaigh <co...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55256/#review165093
-----------------------------------------------------------
As part of this patch, please also change the httpmime version to 4.5.3 for consistency. Also, the httpcore dependency should be changed to 4.4.6, as this is the version that is associated with httpclient 4.5.3.
- Colm O hEigeartaigh
On Feb. 10, 2017, 8:19 a.m., Qiang Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55256/
> -----------------------------------------------------------
>
> (Updated Feb. 10, 2017, 8:19 a.m.)
>
>
> Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1289
> https://issues.apache.org/jira/browse/RANGER-1289
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Steps:
> 1.Start ranger-kms service
> 2.Configure KMS Client and restart hdfs
> 3.Create a key named key0 in the Ranger Web UI
> 4.Execute the following command in hadoop environment,create an encrypted zone
> [xhh@zdh41 ~]$hdfs dfs -mkdir /keyZone
> [xhh@zdh41 ~]$hdfs crypto -createZone -keyName key0 -path /keyZone
>
>
> Error message poped out as below:
> ranger-0.7.0-SNAPSHOT-kms/ews/logs/kms.log
>
> 2017-01-04 14:27:13,256 ERROR [webservices-driver] - Servlet.service() for servlet [webservices-driver] in context with path [/kms] threw exception
> java.lang.NullPointerException
> at org.apache.http.client.utils.URLEncodedUtils.parse(URLEncodedUtils.java:235)
> at org.apache.hadoop.security.token.delegation.web.ServletUtils.getParameter(ServletUtils.java:48)
> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.managementOperation(DelegationTokenAuthenticationHandler.java:171)
> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:514)
> at org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)
> at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
> at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
> at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> I analyzed the reasons,Ranger-Kms relies on httpclient version 4.5.1, there is a bug, as follows:
> org/apache/http/client/utils/URLEncodedUtils.java
> {code:java}
> public static List<NameValuePair> parse(String s, Charset charset) {
> CharArrayBuffer buffer = new CharArrayBuffer(s.length());
> buffer.append(s);
> return parse(buffer, charset, new char[]{'&', ';'});
> }
> {code}
> When the parameter 's' is null, it will pop out NullPointException.
> And in httpclient version 4.5.3, there is no problem. the new code is as follows:
> {code:java}
> public static List<NameValuePair> parse(final String s, final Charset charset) {
> if (s == null) {
> return Collections.emptyList();
> }
> final CharArrayBuffer buffer = new CharArrayBuffer(s.length());
> buffer.append(s);
> return parse(buffer, charset, QP_SEP_A, QP_SEP_S);
> }
> {code}
>
>
> Diffs
> -----
>
> pom.xml 19caacd
>
> Diff: https://reviews.apache.org/r/55256/diff/
>
>
> Testing
> -------
>
> [xhh@zdh41 ~]$ hadoop key list -metadata
> 17/02/10 16:20:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> Listing keys for KeyProvider: KMSClientProvider[http://zdh41:9292/kms/v1/]
> key0 : cipher: AES/CTR/NoPadding, length: 128, description: , created: Thu Feb 09 09:56:51 CST 2017, version: 1, attributes: [key.acl.name=key0]
>
>
> Thanks,
>
> Qiang Zhang
>
>