You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by "Chingachgook (Jira)" <ji...@apache.org> on 2024/03/21 13:03:00 UTC
[jira] [Commented] (RANGER-4327) Make ranger2.4 hdfs plugin compatible with hadoop 3.3.5 +
[ https://issues.apache.org/jira/browse/RANGER-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829546#comment-17829546 ]
Chingachgook commented on RANGER-4327:
--------------------------------------
I applied the attached patch and the error with jackson went away, but now I get another error
How can this problem be solved?
{code:java}
2024-03-21 15:57:09,993 DEBUG classloader.RangerPluginClassLoader: RangerPluginClassLoader.loadClass(java.util.Iterator): calling childClassLoader.findClass()
2024-03-21 15:57:09,993 DEBUG classloader.RangerPluginClassLoader: <== RangerPluginClassLoader.loadClass(java.util.Iterator): interface java.util.Iterator
2024-03-21 15:57:09,993 DEBUG classloader.RangerPluginClassLoader: ==> RangerPluginClassLoader.deactivate()
2024-03-21 15:57:09,993 DEBUG classloader.RangerPluginClassLoader: <== RangerPluginClassLoader.deactivate()
2024-03-21 15:57:09,993 ERROR namenode.FSNamesystem: FSNamesystem initialization failed.
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:138)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:1026)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:800)
at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:694)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:781)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:1033)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:1008)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1782)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1847)
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
... 8 more
Caused by: java.lang.LinkageError: loader constraint violation: when resolving method 'org.slf4j.ILoggerFactory org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()' the class loader org.apache.ranger.plugin.classloader.RangerPluginClassLoader @36c0d0bd of the current class, org/slf4j/LoggerFactory, and the class loader 'app' for the method's defining class, org/slf4j/impl/StaticLoggerBinder, have different Class objects for the type org/slf4j/ILoggerFactory used in the signature (org.slf4j.LoggerFactory is in unnamed module of loader org.apache.ranger.plugin.classloader.RangerPluginClassLoader @36c0d0bd, parent loader 'bootstrap'; org.slf4j.impl.StaticLoggerBinder is in unnamed module of loader 'app')
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:423)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<clinit>(RangerHdfsAuthorizer.java:83)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:398)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:60)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43)
... 13 more
2024-03-21 15:57:10,038 ERROR blockmanagement.DatanodeManager: Slow peers collection thread interrupted
java.lang.InterruptedException: sleep interrupted
at java.base/java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager$1.run(DatanodeManager.java:395)
at java.base/java.lang.Thread.run(Thread.java:829)
2024-03-21 15:57:10,306 INFO namenode.FSNamesystem: Stopping services started for active state
2024-03-21 15:57:10,306 DEBUG delegation.AbstractDelegationTokenSecretManager: Stopping expired delegation token remover thread
2024-03-21 15:57:10,312 INFO namenode.FSNamesystem: Stopping services started for standby state
2024-03-21 15:57:10,312 DEBUG namenode.FSEditLog: Closing log when already closed
2024-03-21 15:57:10,313 DEBUG namenode.FSEditLog: Closing log when already closed
2024-03-21 15:57:10,313 ERROR namenode.NameNode: Failed to start namenode.
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:138)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:1026)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:800)
at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:694)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:781)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:1033)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:1008)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1782)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1847)
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
... 8 more
Caused by: java.lang.LinkageError: loader constraint violation: when resolving method 'org.slf4j.ILoggerFactory org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()' the class loader org.apache.ranger.plugin.classloader.RangerPluginClassLoader @36c0d0bd of the current class, org/slf4j/LoggerFactory, and the class loader 'app' for the method's defining class, org/slf4j/impl/StaticLoggerBinder, have different Class objects for the type org/slf4j/ILoggerFactory used in the signature (org.slf4j.LoggerFactory is in unnamed module of loader org.apache.ranger.plugin.classloader.RangerPluginClassLoader @36c0d0bd, parent loader 'bootstrap'; org.slf4j.impl.StaticLoggerBinder is in unnamed module of loader 'app')
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:423)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<clinit>(RangerHdfsAuthorizer.java:83)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:398)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:60)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43)
... 13 more
2024-03-21 15:57:10,315 DEBUG util.ExitUtil: Exiting with status 1: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
1: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ExitUtil.terminate(ExitUtil.java:342)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1853)
Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:138)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:1026)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:800)
at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:694)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:781)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:1033)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:1008)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1782)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1847)
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
... 8 more
Caused by: java.lang.LinkageError: loader constraint violation: when resolving method 'org.slf4j.ILoggerFactory org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()' the class loader org.apache.ranger.plugin.classloader.RangerPluginClassLoader @36c0d0bd of the current class, org/slf4j/LoggerFactory, and the class loader 'app' for the method's defining class, org/slf4j/impl/StaticLoggerBinder, have different Class objects for the type org/slf4j/ILoggerFactory used in the signature (org.slf4j.LoggerFactory is in unnamed module of loader org.apache.ranger.plugin.classloader.RangerPluginClassLoader @36c0d0bd, parent loader 'bootstrap'; org.slf4j.impl.StaticLoggerBinder is in unnamed module of loader 'app')
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:423)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<clinit>(RangerHdfsAuthorizer.java:83)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:398)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:60)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43)
... 13 more
2024-03-21 15:57:10,315 INFO util.ExitUtil: Exiting with status 1: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
2024-03-21 15:57:10,339 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************{code}
> Make ranger2.4 hdfs plugin compatible with hadoop 3.3.5 +
> ---------------------------------------------------------
>
> Key: RANGER-4327
> URL: https://issues.apache.org/jira/browse/RANGER-4327
> Project: Ranger
> Issue Type: Improvement
> Components: plugins
> Affects Versions: 2.3.0, 2.4.0
> Reporter: caijialiang
> Assignee: caijialiang
> Priority: Major
> Attachments: image-2023-07-25-17-30-18-136.png, image-2023-07-26-13-40-10-555.png, patch3-RANGER-4327.diff
>
>
> hadoop 3.3.5 has upgraded its jackson dependency and removed all jackson 1.x packages, including the jackson-jaxrs jar. As a result, when starting with the Ranger plugin enabled, Hadoop namenode fails to start due to the inability to load jackson 1.x.
> To address this issue and ensure compatibility with the Hadoop 3.3.5 upgrade, we need to package jackson 1.x jars in the hdfs plugin as well,like the Ozone did https://issues.apache.org/jira/browse/RANGER-2818. This will allow the namenode to start successfully.
> !image-2023-07-26-13-40-10-555.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)