You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/10/23 22:24:00 UTC
[jira] [Work logged] (HADOOP-17324) Don't relocate org.bouncycastle
in shaded client jars
[ https://issues.apache.org/jira/browse/HADOOP-17324?focusedWorklogId=504347&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-504347 ]
ASF GitHub Bot logged work on HADOOP-17324:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 23/Oct/20 22:23
Start Date: 23/Oct/20 22:23
Worklog Time Spent: 10m
Work Description: sunchao opened a new pull request #2411:
URL: https://github.com/apache/hadoop/pull/2411
In [HADOOP-15832](https://issues.apache.org/jira/browse/HADOOP-15832) we excluded org.bouncycastle from the shaded client jars but still relocate them to `org.apache.hadoop.shaded`. This caused downstream applications no longer able to use `MiniYARNCluster` from `hadoop-client-minicluster` since there is no way to bring their own org.bouncycastle test dependency.
```
Error: Caused by: sbt.ForkMain$ForkError: java.lang.ClassNotFoundException: org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException
Error: at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
Error: at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
Error: at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
Error: at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
Error: at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:862)
Error: at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
Error: at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1296)
Error: at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:339)
Error: at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
Error: at org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:353)
Error: at org.apache.hadoop.yarn.server.MiniYARNCluster.access$200(MiniYARNCluster.java:127)
Error: at org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceInit(MiniYARNCluster.java:488)
Error: at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
Error: at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:109)
Error: at org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:321)
Error: at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
Error: at org.apache.spark.deploy.yarn.BaseYarnClusterSuite.beforeAll(BaseYarnClusterSuite.scala:94)
Error: at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
Error: at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
Error: at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
Error: at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
Error: at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
Error: at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
Error: at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:413)
Error: at java.util.concurrent.FutureTask.run(FutureTask.java:266)
Error: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
Error: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Error: at java.lang.Thread.run(Thread.java:748)
```
This fixes the issue by excluding org.bouncycastle from relocation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 504347)
Remaining Estimate: 0h
Time Spent: 10m
> Don't relocate org.bouncycastle in shaded client jars
> -----------------------------------------------------
>
> Key: HADOOP-17324
> URL: https://issues.apache.org/jira/browse/HADOOP-17324
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 3.3.0
> Reporter: Chao Sun
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When downstream apps depend on {{hadoop-client-api}}, {{hadoop-client-runtime}} and {{hadoop-client-minicluster}}, it seems the {{MiniYARNCluster}} could have issue because {{org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException}} is not in any of the above jars.
> {code}
> Error: Caused by: sbt.ForkMain$ForkError: java.lang.ClassNotFoundException: org.apache.hadoop.shaded.org.bouncycastle.operator.OperatorCreationException
> Error: at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> Error: at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
> Error: at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> Error: at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
> Error: at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:862)
> Error: at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error: at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1296)
> Error: at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:339)
> Error: at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error: at org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:353)
> Error: at org.apache.hadoop.yarn.server.MiniYARNCluster.access$200(MiniYARNCluster.java:127)
> Error: at org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceInit(MiniYARNCluster.java:488)
> Error: at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error: at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:109)
> Error: at org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:321)
> Error: at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> Error: at org.apache.spark.deploy.yarn.BaseYarnClusterSuite.beforeAll(BaseYarnClusterSuite.scala:94)
> Error: at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
> Error: at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
> Error: at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
> Error: at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61)
> Error: at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
> Error: at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
> Error: at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:413)
> Error: at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> Error: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> Error: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> Error: at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org