You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ranger.apache.org by "luochong@gdbigdata.com" <lu...@gdbigdata.com> on 2017/06/08 09:34:16 UTC
Hdfs plugin throw exception : java.lang.StackOverflowError
Hi,
Based on the guide of ranger-hdfs (https://cwiki.apache.org/confluence/display/RANGER/Ranger+Installation+Guide#RangerInstallationGuide-Install/ConfigureRangerHDFSPlugin) , I installed the ranger-hdfs-plugin.
Ranger version: 0.7.0
hadoop version: 2.7.3
OS: CentOS 6.6
Java: 1.8.0_40
Hdfs : pseudo distributed mode
But when i start namenode, i get following exception .
17/06/08 02:05:44 ERROR namenode.NameNode: Failed to start namenode.
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:843)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:697)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:985)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1434)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1559)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:132)
... 5 more
Caused by: java.lang.StackOverflowError
at java.lang.Exception.<init>(Exception.java:102)
at java.lang.ReflectiveOperationException.<init>(ReflectiveOperationException.java:89)
at java.lang.reflect.InvocationTargetException.<init>(InvocationTargetException.java:72)
at sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:64)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43)
at sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:64)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43)
at sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Const
...... ....
Thanks & Regards
luochong@gdbigdata.com
Re: Re: Hdfs plugin throw exception : java.lang.StackOverflowError
Posted by "luochong@gdbigdata.com" <lu...@gdbigdata.com>.
Hi Ramesh,
Thanks for your help. Maybe i have solved this question.
Yesterday, i checked the ${HADOOP_HOME}/share/hadoop/hdfs/lib. There was one jar -- ranger-hdfs-plugin-shim-0.7.0.jar in this folder.
I removed this jar , and copied ranger-hdfs-plugin-0.7.0.jar to this folder. Now it is ok. So i want to know where the jar -- ranger-hdfs-plugin-shim-0.7.0.jar is used.
/usr/local/hadoop/share/hadoop/hdfs/lib
[root@h145 lib]# ls
asm-3.2.jar gson-2.2.4.jar javax.persistence-2.1.0.jar mysql-connector-java.jar ranger-plugins-audit-0.7.0.jar
commons-cli-1.2.jar guava-11.0.2.jar jersey-core-1.9.jar netty-3.6.2.Final.jar ranger-plugins-common-0.7.0.jar
commons-codec-1.4.jar htrace-core-3.1.0-incubating.jar jersey-server-1.9.jar netty-all-4.0.23.Final.jar ranger-plugins-cred-0.7.0.jar
commons-daemon-1.0.13.jar httpclient-4.5.1.jar jetty-6.1.26.jar noggit-0.6.jar servlet-api-2.5.jar
commons-io-2.4.jar httpcore-4.4.4.jar jetty-util-6.1.26.jar protobuf-java-2.5.0.jar solr-solrj-5.5.1.jar
commons-lang-2.6.jar httpmime-4.5.2.jar jsr305-3.0.0.jar ranger-hdfs-plugin-0.7.0.jar xercesImpl-2.9.1.jar
commons-logging-1.1.3.jar jackson-core-asl-1.9.13.jar leveldbjni-all-1.8.jar ranger-hdfs-plugin-shim-0.7.0.jar.bak xml-apis-1.3.04.jar
eclipselink-2.5.2.jar jackson-mapper-asl-1.9.13.jar log4j-1.2.17.jar ranger-plugin-classloader-0.7.0.jar xmlenc-0.52.jar
[root@h145 lib]# pwd
/usr/local/hadoop/lib
[root@h145 lib]# ls -l
total 12
drwxr-xr-x. 2 hadoop hadoop 4096 Aug 17 2016 native
lrwxrwxrwx. 1 root root 57 Jun 8 01:37 ranger-hdfs-plugin-impl -> /usr/local/ranger-hdfs-plugin/lib/ranger-hdfs-plugin-impl
lrwxrwxrwx. 1 root root 67 Jun 8 01:37 ranger-hdfs-plugin-shim-0.7.0.jar -> /usr/local/ranger-hdfs-plugin/lib/ranger-hdfs-plugin-shim-0.7.0.jar
lrwxrwxrwx. 1 root root 69 Jun 8 01:37 ranger-plugin-classloader-0.7.0.jar -> /usr/local/ranger-hdfs-plugin/lib/ranger-plugin-classloader-0.7.0.jar
[root@h145 ranger-0.7.0-hdfs-plugin]# pwd
/usr/local/ranger-0.7.0-hdfs-plugin
[root@h145 ranger-0.7.0-hdfs-plugin]# ll
total 80
-rwxr-xr-x. 1 lch lch 22495 Feb 20 08:45 disable-hdfs-plugin.sh
-rwxr-xr-x. 1 lch lch 22495 Feb 20 08:45 enable-hdfs-plugin.sh
drwxr-xr-x. 4 root root 4096 Jun 8 00:04 install
-rwxr-xr-x. 1 lch lch 5403 Jun 8 01:37 install.properties
drwxr-xr-x. 3 root root 4096 Jun 8 01:34 lib
-rwxr-xr-x. 1 lch lch 3087 Feb 20 08:45 ranger_credential_helper.py
-rwxr-xr-x. 1 lch lch 1200 Feb 20 08:45 upgrade-hdfs-plugin.sh
-rwxr-xr-x. 1 lch lch 5685 Feb 20 08:45 upgrade-plugin.py
[root@h145 ranger-hdfs-plugin-impl]# pwd
/usr/local/ranger-0.7.0-hdfs-plugin/lib/ranger-hdfs-plugin-impl
[root@h145 ranger-hdfs-plugin-impl]# ls -l
total 12012
-rw-rw-r--. 1 lch lch 8632395 Jun 5 19:15 eclipselink-2.5.2.jar
-rw-rw-r--. 1 lch lch 190432 Jun 5 19:14 gson-2.2.4.jar
-rw-rw-r--. 1 lch lch 732765 Jun 5 19:15 httpclient-4.5.1.jar
-rw-rw-r--. 1 lch lch 326724 Jun 5 19:17 httpcore-4.4.4.jar
-rw-rw-r--. 1 lch lch 41029 Jun 5 19:15 httpmime-4.5.2.jar
-rw-rw-r--. 1 lch lch 162126 Jun 5 19:15 javax.persistence-2.1.0.jar
lrwxrwxrwx. 1 root root 40 Jun 8 01:37 mysql-connector-java.jar -> /usr/share/java/mysql-connector-java.jar
-rw-rw-r--. 1 lch lch 26720 Jun 5 19:15 noggit-0.6.jar
-rw-r--r--. 1 lch lch 702670 Jun 8 01:28 ranger-hdfs-plugin-0.7.0.jar
-rw-r--r--. 1 lch lch 194316 Jun 8 01:28 ranger-plugins-audit-0.7.0.jar
-rw-r--r--. 1 lch lch 564954 Jun 8 01:28 ranger-plugins-common-0.7.0.jar
-rw-r--r--. 1 lch lch 11664 Jun 8 01:28 ranger-plugins-cred-0.7.0.jar
-rw-rw-r--. 1 lch lch 692590 Jun 5 19:15 solr-solrj-5.5.1.jar
Thanks & Regards
Luochong
luochong@gdbigdata.com
From: Ramesh Mani
Date: 2017-06-09 00:37
To: user@ranger.apache.org
Subject: Re: Hdfs plugin throw exception : java.lang.StackOverflowError
Luochong,
Could you please provide me the list of files in hadoop/lib and hadoop/lib/ranger-hdfs-plugin-impl folder?
The issue looks like some ranger jars are misplaced in these folder and its going into a recursive call.
Thanks,
Ramesh
From: "luochong@gdbigdata.com" <lu...@gdbigdata.com>
Reply-To: "user@ranger.apache.org" <us...@ranger.apache.org>
Date: Thursday, June 8, 2017 at 2:34 AM
To: user <us...@ranger.apache.org>
Subject: Hdfs plugin throw exception : java.lang.StackOverflowError
Hi,
Based on the guide of ranger-hdfs (https://cwiki.apache.org/confluence/display/RANGER/Ranger+Installation+Guide#RangerInstallationGuide-Install/ConfigureRangerHDFSPlugin) , I installed the ranger-hdfs-plugin.
Ranger version: 0.7.0
hadoop version: 2.7.3
OS: CentOS 6.6
Java: 1.8.0_40
Hdfs : pseudo distributed mode
But when i start namenode, i get following exception .
17/06/08 02:05:44 ERROR namenode.NameNode: Failed to start namenode.
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:843)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:697)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:985)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1434)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1559)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:132)
... 5 more
Caused by: java.lang.StackOverflowError
at java.lang.Exception.<init>(Exception.java:102)
at java.lang.ReflectiveOperationException.<init>(ReflectiveOperationException.java:89)
at java.lang.reflect.InvocationTargetException.<init>(InvocationTargetException.java:72)
at sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:64)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43)
at sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:64)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43)
at sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Const
...... ....
Thanks & Regards
luochong@gdbigdata.com
Re: Hdfs plugin throw exception : java.lang.StackOverflowError
Posted by Ramesh Mani <rm...@hortonworks.com>.
Luochong,
Could you please provide me the list of files in hadoop/lib and hadoop/lib/ranger-hdfs-plugin-impl folder?
The issue looks like some ranger jars are misplaced in these folder and its going into a recursive call.
Thanks,
Ramesh
From: "luochong@gdbigdata.com<ma...@gdbigdata.com>" <lu...@gdbigdata.com>>
Reply-To: "user@ranger.apache.org<ma...@ranger.apache.org>" <us...@ranger.apache.org>>
Date: Thursday, June 8, 2017 at 2:34 AM
To: user <us...@ranger.apache.org>>
Subject: Hdfs plugin throw exception : java.lang.StackOverflowError
Hi,
Based on the guide of ranger-hdfs (https://cwiki.apache.org/confluence/display/RANGER/Ranger+Installation+Guide#RangerInstallationGuide-Install/ConfigureRangerHDFSPlugin) , I installed the ranger-hdfs-plugin.
Ranger version: 0.7.0
hadoop version: 2.7.3
OS: CentOS 6.6
Java: 1.8.0_40
Hdfs : pseudo distributed mode
But when i start namenode, i get following exception .
17/06/08 02:05:44 ERROR namenode.NameNode: Failed to start namenode.
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:843)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:697)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:985)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1434)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1559)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:132)
... 5 more
Caused by: java.lang.StackOverflowError
at java.lang.Exception.<init>(Exception.java:102)
at java.lang.ReflectiveOperationException.<init>(ReflectiveOperationException.java:89)
at java.lang.reflect.InvocationTargetException.<init>(InvocationTargetException.java:72)
at sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:64)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43)
at sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:64)
at org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.<init>(RangerHdfsAuthorizer.java:43)
at sun.reflect.GeneratedConstructorAccessor6.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Const
...... ....
Thanks & Regards
________________________________
luochong@gdbigdata.com<ma...@gdbigdata.com>