You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Jeff N <ma...@gmail.com> on 2015/06/15 20:44:12 UTC
JniBasedUnixGroupsMapping.anchorNative()V Error when initializing
MiniAccumuloCluster
I'm using Accumulo 1.6 jar, Hadoop 1.2.2, and Zookeeper 3.4.5
The issue arises when I attempt to initialize the MiniAccumuloCluster to be
used in a Hadoop job test file.
Thread "org.apache.accumulo.server.init.Initialize" died
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.reflect.InvocationTargetException
at org.apache.accumulo.server.init.Initialize.main(Initialize.java:634)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.accumulo.start.Main$1.run(Main.java:141)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException:
java.lang.reflect.InvocationTargetException
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
at org.apache.hadoop.security.Groups.<init>(Groups.java:55)
at
org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:182)
at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:235)
at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:214)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:669)
at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:571)
at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2590)
at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2582)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2448)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:287)
at org.apache.accumulo.core.volume.VolumeImpl.<init>(VolumeImpl.java:43)
at
org.apache.accumulo.core.volume.VolumeConfiguration.create(VolumeConfiguration.java:155)
at
org.apache.accumulo.server.fs.VolumeManagerImpl.get(VolumeManagerImpl.java:414)
at org.apache.accumulo.server.init.Initialize.main(Initialize.java:614)
... 6 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
... 21 more
Caused by: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V
at org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native
Method)
at
org.apache.hadoop.security.JniBasedUnixGroupsMapping.<clinit>(JniBasedUnixGroupsMapping.java:49)
at
org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.<init>(JniBasedUnixGroupsMappingWithFallback.java:38)
... 26 more
The Internet is less than helpful. Thoughts?
- Jeff N.
-----
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/JniBasedUnixGroupsMapping-anchorNative-V-Error-when-initializing-MiniAccumuloCluster-tp14449.html
Sent from the Developers mailing list archive at Nabble.com.
Re: JniBasedUnixGroupsMapping.anchorNative()V Error when initializing MiniAccumuloCluster
Posted by Keith Turner <ke...@deenlo.com>.
The root cause of the problem is UnsatisfiedLinkError for
org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(). What
hadoop shared library proved the implementation for that? Also, I though
the native libs in hadoop were optional?
On Mon, Jun 15, 2015 at 2:44 PM, Jeff N <ma...@gmail.com> wrote:
> I'm using Accumulo 1.6 jar, Hadoop 1.2.2, and Zookeeper 3.4.5
>
> The issue arises when I attempt to initialize the MiniAccumuloCluster to be
> used in a Hadoop job test file.
>
> Thread "org.apache.accumulo.server.init.Initialize" died
> java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.reflect.InvocationTargetException
> at
> org.apache.accumulo.server.init.Initialize.main(Initialize.java:634)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.accumulo.start.Main$1.run(Main.java:141)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> java.lang.reflect.InvocationTargetException
> at
>
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
> at org.apache.hadoop.security.Groups.<init>(Groups.java:55)
> at
>
> org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:182)
> at
>
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:235)
> at
>
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:214)
> at
>
> org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:669)
> at
>
> org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:571)
> at
> org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2590)
> at
> org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2582)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2448)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:287)
> at
> org.apache.accumulo.core.volume.VolumeImpl.<init>(VolumeImpl.java:43)
> at
>
> org.apache.accumulo.core.volume.VolumeConfiguration.create(VolumeConfiguration.java:155)
> at
>
> org.apache.accumulo.server.fs.VolumeManagerImpl.get(VolumeManagerImpl.java:414)
> at
> org.apache.accumulo.server.init.Initialize.main(Initialize.java:614)
> ... 6 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
>
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
>
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
> ... 21 more
> Caused by: java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V
> at
> org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native
> Method)
> at
>
> org.apache.hadoop.security.JniBasedUnixGroupsMapping.<clinit>(JniBasedUnixGroupsMapping.java:49)
> at
>
> org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.<init>(JniBasedUnixGroupsMappingWithFallback.java:38)
> ... 26 more
>
> The Internet is less than helpful. Thoughts?
>
> - Jeff N.
>
>
>
> -----
>
>
>
> --
> View this message in context:
> http://apache-accumulo.1065345.n5.nabble.com/JniBasedUnixGroupsMapping-anchorNative-V-Error-when-initializing-MiniAccumuloCluster-tp14449.html
> Sent from the Developers mailing list archive at Nabble.com.
>
Re: JniBasedUnixGroupsMapping.anchorNative()V Error when initializing MiniAccumuloCluster
Posted by Keith Turner <ke...@deenlo.com>.
On Mon, Jun 15, 2015 at 4:44 PM, Josh Elser <jo...@gmail.com> wrote:
> I had suggested that you rebuild Accumulo, not your project. Unless you
> have an "assemble" profile in your project, of course you would see an
> error that the profile doesn't exist :). Rebuilding your local project does
> not build all of your dependencies.
>
> bq. The root cause of the problem is UnsatisfiedLinkError for
> org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(). What
> hadoop shared library proved the implementation for that? Also, I though
> the native libs in hadoop were optional?
>
> Hadoop noticed that the native libraries were available which is why it
> tried to invoke them. Looking at 2.7.1-SNAPSHOT code, the line numbers
> matched up with what the stacktrace showed.
>
Ok I see, so maybe one part the hadoop code thought the native libs were
available but another part of the hadoop code could not load a specific
native method.
>
> You said earlier that this was in a "Hadoop job test file". What does that
> mean? That's what I was asking previously about how you are running this. I
> assumed you meant a mapreduce job or similar, but below it looks like
> you're just running the program by hand (or via an IDE). Does your maven
> project use the same version of Hadoop that you have installed on your
> nodes (1.2.2) -- see `mvn dependency:tree`?
>
>
> Jeff N wrote:
>
>> I'm using whichever one Maven pulls when you include Accumulo 1.6.0 as a
>> dependency. Yeah, unfortunately it just says
>>
>> "[WARNING] The requested profile "assemble" could not be activated because
>> it does not exist."
>>
>> And then when I try to start the MiniAccumuloCluster it gives me this:
>>
>> [WARNING]
>> java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at
>> org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.lang.RuntimeException: Initialize process returned 1.
>> Check
>> the logs in
>> /tmp/AccumuloMiniCluster(1434399112262)7316723201967391324/logs
>> for errors.
>> at
>>
>> org.apache.accumulo.minicluster.impl.MiniAccumuloClusterImpl.start(MiniAccumuloClusterImpl.java:471)
>> at
>>
>> org.apache.accumulo.minicluster.MiniAccumuloCluster.start(MiniAccumuloCluster.java:72)
>> at
>>
>> ICASA.DAVE.Cloud.Store.AccumuloTestConnection.<init>(AccumuloTestConnection.java:124)
>> at
>>
>> ICASA.DAVE.Cloud.Store.AccumuloTestConnection.getInstance(AccumuloTestConnection.java:56)
>> at
>>
>> ICASA.DAVE.Cloud.Store.AccumuloStore$AccumuloRowIterable.iterator(AccumuloStore.java:179)
>> at ICASA.util.types.io.DataClerk.getEntryMap(DataClerk.java:559)
>> at ICASA.util.types.io.DataClerk.loadObject(DataClerk.java:229)
>> at
>> ICASA.DAVE.Local.LocalResults.loadOperationResults(LocalResults.java:505)
>> at
>>
>> ICASA.DAVE.Local.LocalResults.getOutputsForOperation(LocalResults.java:119)
>> at ICASA.DAVE.Core.Project.buildInputs(Project.java:203)
>> at
>> ICASA.DAVE.Local.Evaluator.LocalEvaluator.submit(LocalEvaluator.java:163)
>> at
>> ICASA.DAVE.Core.Project.scheduleOperationForEvaluation(Project.java:150)
>> at ICASA.DAVE.TestingHadoop.main(TestingHadoop.java:154)
>> ... 6 more
>>
>> I have a 4 node cluster and all the libhadoop.so's seem to be the same on
>> all machines. If there is anything else I can provide please let me know.
>>
>>
>>
>> -----
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-accumulo.1065345.n5.nabble.com/JniBasedUnixGroupsMapping-anchorNative-V-Error-when-initializing-MiniAccumuloCluster-tp14449p14454.html
>> Sent from the Developers mailing list archive at Nabble.com.
>>
>
Re: JniBasedUnixGroupsMapping.anchorNative()V Error when
initializing MiniAccumuloCluster
Posted by Jeff N <ma...@gmail.com>.
Oh dang. That's my bad. I'm testing a hadoop job, but realized that that
information was less relevant since the code wasn't even getting there. The
main method tries to establish a connection to the Accumulo cluster. Maven
does indeed use the same version.
-----
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/JniBasedUnixGroupsMapping-anchorNative-V-Error-when-initializing-MiniAccumuloCluster-tp14449p14457.html
Sent from the Developers mailing list archive at Nabble.com.
Re: JniBasedUnixGroupsMapping.anchorNative()V Error when initializing
MiniAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
I had suggested that you rebuild Accumulo, not your project. Unless you
have an "assemble" profile in your project, of course you would see an
error that the profile doesn't exist :). Rebuilding your local project
does not build all of your dependencies.
bq. The root cause of the problem is UnsatisfiedLinkError for
org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative().
What hadoop shared library proved the implementation for that? Also, I
though the native libs in hadoop were optional?
Hadoop noticed that the native libraries were available which is why it
tried to invoke them. Looking at 2.7.1-SNAPSHOT code, the line numbers
matched up with what the stacktrace showed.
You said earlier that this was in a "Hadoop job test file". What does
that mean? That's what I was asking previously about how you are running
this. I assumed you meant a mapreduce job or similar, but below it looks
like you're just running the program by hand (or via an IDE). Does your
maven project use the same version of Hadoop that you have installed on
your nodes (1.2.2) -- see `mvn dependency:tree`?
Jeff N wrote:
> I'm using whichever one Maven pulls when you include Accumulo 1.6.0 as a
> dependency. Yeah, unfortunately it just says
>
> "[WARNING] The requested profile "assemble" could not be activated because
> it does not exist."
>
> And then when I try to start the MiniAccumuloCluster it gives me this:
>
> [WARNING]
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: Initialize process returned 1. Check
> the logs in /tmp/AccumuloMiniCluster(1434399112262)7316723201967391324/logs
> for errors.
> at
> org.apache.accumulo.minicluster.impl.MiniAccumuloClusterImpl.start(MiniAccumuloClusterImpl.java:471)
> at
> org.apache.accumulo.minicluster.MiniAccumuloCluster.start(MiniAccumuloCluster.java:72)
> at
> ICASA.DAVE.Cloud.Store.AccumuloTestConnection.<init>(AccumuloTestConnection.java:124)
> at
> ICASA.DAVE.Cloud.Store.AccumuloTestConnection.getInstance(AccumuloTestConnection.java:56)
> at
> ICASA.DAVE.Cloud.Store.AccumuloStore$AccumuloRowIterable.iterator(AccumuloStore.java:179)
> at ICASA.util.types.io.DataClerk.getEntryMap(DataClerk.java:559)
> at ICASA.util.types.io.DataClerk.loadObject(DataClerk.java:229)
> at
> ICASA.DAVE.Local.LocalResults.loadOperationResults(LocalResults.java:505)
> at
> ICASA.DAVE.Local.LocalResults.getOutputsForOperation(LocalResults.java:119)
> at ICASA.DAVE.Core.Project.buildInputs(Project.java:203)
> at
> ICASA.DAVE.Local.Evaluator.LocalEvaluator.submit(LocalEvaluator.java:163)
> at ICASA.DAVE.Core.Project.scheduleOperationForEvaluation(Project.java:150)
> at ICASA.DAVE.TestingHadoop.main(TestingHadoop.java:154)
> ... 6 more
>
> I have a 4 node cluster and all the libhadoop.so's seem to be the same on
> all machines. If there is anything else I can provide please let me know.
>
>
>
> -----
>
>
>
> --
> View this message in context: http://apache-accumulo.1065345.n5.nabble.com/JniBasedUnixGroupsMapping-anchorNative-V-Error-when-initializing-MiniAccumuloCluster-tp14449p14454.html
> Sent from the Developers mailing list archive at Nabble.com.
Re: JniBasedUnixGroupsMapping.anchorNative()V Error when
initializing MiniAccumuloCluster
Posted by Jeff N <ma...@gmail.com>.
I'm using whichever one Maven pulls when you include Accumulo 1.6.0 as a
dependency. Yeah, unfortunately it just says
"[WARNING] The requested profile "assemble" could not be activated because
it does not exist."
And then when I try to start the MiniAccumuloCluster it gives me this:
[WARNING]
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Initialize process returned 1. Check
the logs in /tmp/AccumuloMiniCluster(1434399112262)7316723201967391324/logs
for errors.
at
org.apache.accumulo.minicluster.impl.MiniAccumuloClusterImpl.start(MiniAccumuloClusterImpl.java:471)
at
org.apache.accumulo.minicluster.MiniAccumuloCluster.start(MiniAccumuloCluster.java:72)
at
ICASA.DAVE.Cloud.Store.AccumuloTestConnection.<init>(AccumuloTestConnection.java:124)
at
ICASA.DAVE.Cloud.Store.AccumuloTestConnection.getInstance(AccumuloTestConnection.java:56)
at
ICASA.DAVE.Cloud.Store.AccumuloStore$AccumuloRowIterable.iterator(AccumuloStore.java:179)
at ICASA.util.types.io.DataClerk.getEntryMap(DataClerk.java:559)
at ICASA.util.types.io.DataClerk.loadObject(DataClerk.java:229)
at
ICASA.DAVE.Local.LocalResults.loadOperationResults(LocalResults.java:505)
at
ICASA.DAVE.Local.LocalResults.getOutputsForOperation(LocalResults.java:119)
at ICASA.DAVE.Core.Project.buildInputs(Project.java:203)
at
ICASA.DAVE.Local.Evaluator.LocalEvaluator.submit(LocalEvaluator.java:163)
at ICASA.DAVE.Core.Project.scheduleOperationForEvaluation(Project.java:150)
at ICASA.DAVE.TestingHadoop.main(TestingHadoop.java:154)
... 6 more
I have a 4 node cluster and all the libhadoop.so's seem to be the same on
all machines. If there is anything else I can provide please let me know.
-----
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/JniBasedUnixGroupsMapping-anchorNative-V-Error-when-initializing-MiniAccumuloCluster-tp14449p14454.html
Sent from the Developers mailing list archive at Nabble.com.
Re: JniBasedUnixGroupsMapping.anchorNative()V Error when initializing
MiniAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
Huh? What Accumulo 1.6 are you building? As far as I remember, all of
1.6.x had the 'assemble' profile to create a tarball.
Can you give more details on what exactly you're running?
Jeff N wrote:
> Unfortunately recompiling against Hadoop 1.2.2 didn't work, the "assemble"
> profile wasn't found so that didn't run. The AccumuloMiniCluster is still
> failing to initialize and returning 1.
>
> Thoughts?
>
> - Jeff N.
>
>
>
> -----
>
>
>
> --
> View this message in context: http://apache-accumulo.1065345.n5.nabble.com/JniBasedUnixGroupsMapping-anchorNative-V-Error-when-initializing-MiniAccumuloCluster-tp14449p14451.html
> Sent from the Developers mailing list archive at Nabble.com.
Re: JniBasedUnixGroupsMapping.anchorNative()V Error when
initializing MiniAccumuloCluster
Posted by Jeff N <ma...@gmail.com>.
Unfortunately recompiling against Hadoop 1.2.2 didn't work, the "assemble"
profile wasn't found so that didn't run. The AccumuloMiniCluster is still
failing to initialize and returning 1.
Thoughts?
- Jeff N.
-----
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/JniBasedUnixGroupsMapping-anchorNative-V-Error-when-initializing-MiniAccumuloCluster-tp14449p14451.html
Sent from the Developers mailing list archive at Nabble.com.
Re: JniBasedUnixGroupsMapping.anchorNative()V Error when
initializing MiniAccumuloCluster
Posted by Jeff N <ma...@gmail.com>.
Okay so I rebuilt accumulo and not my project and it passed, but it did not
change anything. I still get the exception that the MiniAccumuloCluster
cannot initialize and the log file gives the same exception. You mentioning
a mismatch of hadoop native libraries might be the issue because I've just
recently learned that we're using a custom build of hadoop. I'll keep you
posted as I learn more.
-----
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/JniBasedUnixGroupsMapping-anchorNative-V-Error-when-initializing-MiniAccumuloCluster-tp14449p14487.html
Sent from the Developers mailing list archive at Nabble.com.
Re: JniBasedUnixGroupsMapping.anchorNative()V Error when
initializing MiniAccumuloCluster
Posted by Jeff N <ma...@gmail.com>.
*So here is the ZooKeeper initialization log:*
15/06/18 10:25:29 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=localhost:2489 sessionTimeout=30000
watcher=org.apache.accumulo.fate.zookeeper.ZooSession$ZooWatcher@675cfa96
15/06/18 10:25:29 INFO zookeeper.ClientCnxn: Opening socket connection to
server localhost.localdomain/127.0.0.1:2489. Will not attempt to
authenticate using SASL (unknown error)
15/06/18 10:25:29 INFO zookeeper.ClientCnxn: Socket connection established
to localhost.localdomain/127.0.0.1:2489, initiating session
15/06/18 10:25:29 INFO zookeeper.ClientCnxn: Session establishment complete
on server localhost.localdomain/127.0.0.1:2489, sessionid =
0x14e077d3a570004, negotiated timeout = 30000
*and here is the exception message:*
ERROR security.UserGroupInformation: PriviledgedActionException as:jnapier
cause:java.io.IOException:
org.apache.accumulo.core.client.AccumuloException:
java.lang.RuntimeException: Failed to connect to zookeeper (localhost:2489)
within 2x zookeeper timeout period 1800000000
java.io.IOException: org.apache.accumulo.core.client.AccumuloException:
java.lang.RuntimeException: Failed to connect to zookeeper (localhost:2489)
within 2x zookeeper timeout period 1800000000
at
org.apache.accumulo.core.client.mapreduce.AccumuloOutputFormat.checkOutputSpecs(AccumuloOutputFormat.java:542)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:973)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:550)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.accumulo.core.client.AccumuloException:
java.lang.RuntimeException: Failed to connect to zookeeper (localhost:2489)
within 2x zookeeper timeout period 1800000000
at
org.apache.accumulo.core.client.impl.ServerClient.execute(ServerClient.java:78)
at
org.apache.accumulo.core.client.impl.ConnectorImpl.<init>(ConnectorImpl.java:70)
at
org.apache.accumulo.core.client.ZooKeeperInstance.getConnector(ZooKeeperInstance.java:237)
at
org.apache.accumulo.core.client.mapreduce.AccumuloOutputFormat.checkOutputSpecs(AccumuloOutputFormat.java:538)
... 15 more
Caused by: java.lang.RuntimeException: Failed to connect to zookeeper
(localhost:2489) within 2x zookeeper timeout period 1800000000
at
org.apache.accumulo.fate.zookeeper.ZooSession.connect(ZooSession.java:92)
at
org.apache.accumulo.fate.zookeeper.ZooSession.getSession(ZooSession.java:153)
at
org.apache.accumulo.fate.zookeeper.ZooReader.getSession(ZooReader.java:35)
at
org.apache.accumulo.fate.zookeeper.ZooReader.getZooKeeper(ZooReader.java:39)
at
org.apache.accumulo.fate.zookeeper.ZooCache.getZooKeeper(ZooCache.java:58)
at org.apache.accumulo.fate.zookeeper.ZooCache.retry(ZooCache.java:129)
at org.apache.accumulo.fate.zookeeper.ZooCache.get(ZooCache.java:235)
at org.apache.accumulo.fate.zookeeper.ZooCache.get(ZooCache.java:190)
at
org.apache.accumulo.core.client.ZooKeeperInstance.getInstanceID(ZooKeeperInstance.java:158)
at org.apache.accumulo.core.zookeeper.ZooUtil.getRoot(ZooUtil.java:38)
at
org.apache.accumulo.core.client.impl.ServerClient.getConnection(ServerClient.java:139)
at
org.apache.accumulo.core.client.impl.ServerClient.getConnection(ServerClient.java:129)
at
org.apache.accumulo.core.client.impl.ServerClient.getConnection(ServerClient.java:124)
at
org.apache.accumulo.core.client.impl.ServerClient.executeRaw(ServerClient.java:106)
at
org.apache.accumulo.core.client.impl.ServerClient.execute(ServerClient.java:72)
... 18 more
-----
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/JniBasedUnixGroupsMapping-anchorNative-V-Error-when-initializing-MiniAccumuloCluster-tp14449p14514.html
Sent from the Developers mailing list archive at Nabble.com.
Re: JniBasedUnixGroupsMapping.anchorNative()V Error when initializing
MiniAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
Shouldn't be any reason why it can't. The ZooKeeper process spawned by
MAC is no different than a ZooKeeper instance you'd start for a normal
cluster.
ZooKeeper binds on all interfaces, so localhost should be fine.
Double check that the port matches?
Jeff N wrote:
> So I actually built accumulo 1.6 against hadoop 1.x and that fixed the
> previous issue. New problem though. Can the AccumuloOutputFormat class talk
> to the zookeeper that is spawned from a MiniAccumuloCluster?
>
> During the checkOutputSpecs call from AccumuloOutputFormat it throws and
> exception saying it cannot connect to the zookeeper (localhost:<port>).
>
> Just to refresh, I'm using Accumulo 1.6 built for Hadoop 1.2.1 with
> zookeeper v 3.4.6. The code I'm trying to run involves creating a Hadoop Job
> that talks to a MiniAccumuloCluster for IO. All of this code is run out of a
> main and compiled/executed with Maven.
>
> Thanks.
>
>
>
> -----
>
>
>
> --
> View this message in context: http://apache-accumulo.1065345.n5.nabble.com/JniBasedUnixGroupsMapping-anchorNative-V-Error-when-initializing-MiniAccumuloCluster-tp14449p14511.html
> Sent from the Developers mailing list archive at Nabble.com.
Re: JniBasedUnixGroupsMapping.anchorNative()V Error when
initializing MiniAccumuloCluster
Posted by Jeff N <ma...@gmail.com>.
So I actually built accumulo 1.6 against hadoop 1.x and that fixed the
previous issue. New problem though. Can the AccumuloOutputFormat class talk
to the zookeeper that is spawned from a MiniAccumuloCluster?
During the checkOutputSpecs call from AccumuloOutputFormat it throws and
exception saying it cannot connect to the zookeeper (localhost:<port>).
Just to refresh, I'm using Accumulo 1.6 built for Hadoop 1.2.1 with
zookeeper v 3.4.6. The code I'm trying to run involves creating a Hadoop Job
that talks to a MiniAccumuloCluster for IO. All of this code is run out of a
main and compiled/executed with Maven.
Thanks.
-----
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/JniBasedUnixGroupsMapping-anchorNative-V-Error-when-initializing-MiniAccumuloCluster-tp14449p14511.html
Sent from the Developers mailing list archive at Nabble.com.
Re: JniBasedUnixGroupsMapping.anchorNative()V Error when initializing
MiniAccumuloCluster
Posted by Josh Elser <jo...@gmail.com>.
Might be a compatibility issue with Hadoop 1. 1.6 builds, by default,
against Hadoop 2.
Can you try rebuilding Accumulo against Hadoop 1.2.2?
`mvn clean package -Dhadoop.profile=1 -DskipTests -Passemble`
Looking at the Hadoop code, though, I'm really not certain. It might
also be a mismatch of Hadoop native libraries. Very curious.
Jeff N wrote:
> I'm using Accumulo 1.6 jar, Hadoop 1.2.2, and Zookeeper 3.4.5
>
> The issue arises when I attempt to initialize the MiniAccumuloCluster to be
> used in a Hadoop job test file.
>
> Thread "org.apache.accumulo.server.init.Initialize" died
> java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.reflect.InvocationTargetException
> at org.apache.accumulo.server.init.Initialize.main(Initialize.java:634)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.accumulo.start.Main$1.run(Main.java:141)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException:
> java.lang.reflect.InvocationTargetException
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
> at org.apache.hadoop.security.Groups.<init>(Groups.java:55)
> at
> org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:182)
> at
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:235)
> at
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:214)
> at
> org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:669)
> at
> org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:571)
> at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2590)
> at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2582)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2448)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:287)
> at org.apache.accumulo.core.volume.VolumeImpl.<init>(VolumeImpl.java:43)
> at
> org.apache.accumulo.core.volume.VolumeConfiguration.create(VolumeConfiguration.java:155)
> at
> org.apache.accumulo.server.fs.VolumeManagerImpl.get(VolumeManagerImpl.java:414)
> at org.apache.accumulo.server.init.Initialize.main(Initialize.java:614)
> ... 6 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
> ... 21 more
> Caused by: java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V
> at org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native
> Method)
> at
> org.apache.hadoop.security.JniBasedUnixGroupsMapping.<clinit>(JniBasedUnixGroupsMapping.java:49)
> at
> org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.<init>(JniBasedUnixGroupsMappingWithFallback.java:38)
> ... 26 more
>
> The Internet is less than helpful. Thoughts?
>
> - Jeff N.
>
>
>
> -----
>
>
>
> --
> View this message in context: http://apache-accumulo.1065345.n5.nabble.com/JniBasedUnixGroupsMapping-anchorNative-V-Error-when-initializing-MiniAccumuloCluster-tp14449.html
> Sent from the Developers mailing list archive at Nabble.com.