You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2018/07/27 12:15:40 UTC
[GitHub] shiroari opened a new pull request #4776: Fix
ClassNotFoundException in druid-kerberos extension
shiroari opened a new pull request #4776: Fix ClassNotFoundException in druid-kerberos extension
URL: https://github.com/apache/incubator-druid/pull/4776
Hi,
Here is a one-line fix for `druid-kerberos` extension. There is a problem with class loader for hadoop `Configuration` class. By default, configuration uses the context class loader of the current thread set to the class loader used to load the application. But application class loader doesn't contain any dependencies due to isolation. As a result we can not load any class located in
`hadoop-common` (it contains the most important implementations for configuring hadoop).
There are 2 solutions: use `Configuration` class loader or get class loader of any class in extension. I decided to use module class to make more explicit the fact that we use extension class loader.
Some stacktrace for reference:
```
2017-09-07T13:16:58,928 INFO [main] io.druid.indexing.common.actions.RemoteTaskActionClient - Submitting action for task[index_hadoop_task_2017-09-07T13:16:52.833Z] to overlord[http://host:8090/druid/indexer/v1/action]: LockTryAcquireAction{interval=2017-08-09T00:00:00.000Z/2017-08-11T00:00:00.000Z}
2017-09-07T13:16:59,011 ERROR [main] io.druid.cli.CliPeon - Error when starting up. Failing.
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
at com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler.start(Lifecycle.java:350) ~[java-util-0.27.10.jar:?]
at com.metamx.common.lifecycle.Lifecycle.start(Lifecycle.java:259) ~[java-util-0.27.10.jar:?]
at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:155) ~[druid-api-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]
at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:101) [druid-services-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]
at io.druid.cli.CliPeon.run(CliPeon.java:274) [druid-services-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]
at io.druid.cli.Main.main(Main.java:106) [druid-services-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]
Caused by: com.metamx.common.ISE: Failed to run isReady
at io.druid.indexing.worker.executor.ExecutorLifecycle.start(ExecutorLifecycle.java:173) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]
... 10 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.security.CompositeGroupsMapping not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2273) ~[?:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:99) ~[?:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:95) ~[?:?]
at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:420) ~[?:?]
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:324) ~[?:?]
at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:352) ~[?:?]
at io.druid.security.kerberos.DruidKerberosUtil.authenticateIfRequired(DruidKerberosUtil.java:101) ~[?:?]
at io.druid.security.kerberos.KerberosHttpClient.inner_go(KerberosHttpClient.java:97) ~[?:?]
at io.druid.security.kerberos.KerberosHttpClient.go(KerberosHttpClient.java:67) ~[?:?]
at com.metamx.http.client.AbstractHttpClient.go(AbstractHttpClient.java:14) ~[http-client-1.0.4.jar:?]
at io.druid.indexing.common.actions.RemoteTaskActionClient.submit(RemoteTaskActionClient.java:101) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]
at io.druid.indexing.common.task.HadoopIndexTask.isReady(HadoopIndexTask.java:137) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]
at io.druid.indexing.worker.executor.ExecutorLifecycle.start(ExecutorLifecycle.java:168) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]
... 10 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.security.CompositeGroupsMapping not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2241) ~[?:?]
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2265) ~[?:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:99) ~[?:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:95) ~[?:?]
at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:420) ~[?:?]
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:324) ~[?:?]
at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:352) ~[?:?]
at io.druid.security.kerberos.DruidKerberosUtil.authenticateIfRequired(DruidKerberosUtil.java:101) ~[?:?]
at io.druid.security.kerberos.KerberosHttpClient.inner_go(KerberosHttpClient.java:97) ~[?:?]
at io.druid.security.kerberos.KerberosHttpClient.go(KerberosHttpClient.java:67) ~[?:?]
at com.metamx.http.client.AbstractHttpClient.go(AbstractHttpClient.java:14) ~[http-client-1.0.4.jar:?]
at io.druid.indexing.common.actions.RemoteTaskActionClient.submit(RemoteTaskActionClient.java:101) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]
at io.druid.indexing.common.task.HadoopIndexTask.isReady(HadoopIndexTask.java:137) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]
at io.druid.indexing.worker.executor.ExecutorLifecycle.start(ExecutorLifecycle.java:168) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]
... 10 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.security.CompositeGroupsMapping not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2147) ~[?:?]
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2239) ~[?:?]
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2265) ~[?:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:99) ~[?:?]
at org.apache.hadoop.security.Groups.<init>(Groups.java:95) ~[?:?]
at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:420) ~[?:?]
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:324) ~[?:?]
at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:352) ~[?:?]
at io.druid.security.kerberos.DruidKerberosUtil.authenticateIfRequired(DruidKerberosUtil.java:101) ~[?:?]
at io.druid.security.kerberos.KerberosHttpClient.inner_go(KerberosHttpClient.java:97) ~[?:?]
at io.druid.security.kerberos.KerberosHttpClient.go(KerberosHttpClient.java:67) ~[?:?]
at com.metamx.http.client.AbstractHttpClient.go(AbstractHttpClient.java:14) ~[http-client-1.0.4.jar:?]
at io.druid.indexing.common.actions.RemoteTaskActionClient.submit(RemoteTaskActionClient.java:101) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]
at io.druid.indexing.common.task.HadoopIndexTask.isReady(HadoopIndexTask.java:137) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]
at io.druid.indexing.worker.executor.ExecutorLifecycle.start(ExecutorLifecycle.java:168) ~[druid-indexing-service-0.9.2.2.6.1.0-129.jar:0.9.2.2.6.1.0-129]
... 10 more
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org