You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Chen He <ai...@gmail.com> on 2014/12/16 05:31:02 UTC
MRAppMaster running on a docker container failed to load openssl cipher.
Try to run teragen based on hadoop 2.6.0 using docker and met following
error:
2014-12-15 04:15:21,385 DEBUG [main]
org.apache.hadoop.crypto.OpensslCipher: Failed to load OpenSSL Cipher.
java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsOpenssl()Z
at
org.apache.hadoop.util.NativeCodeLoader.buildSupportsOpenssl(Native Method)
at
org.apache.hadoop.crypto.OpensslCipher.<clinit>(OpensslCipher.java:84)
at
org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec.<init>(OpensslAesCtrCryptoCodec.java:50)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
at
org.apache.hadoop.crypto.CryptoCodec.getInstance(CryptoCodec.java:67)
at
org.apache.hadoop.crypto.CryptoCodec.getInstance(CryptoCodec.java:100)
at org.apache.hadoop.fs.Hdfs.<init>(Hdfs.java:91)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.fs.AbstractFileSystem.newInstance(AbstractFileSystem.java:129)
at
org.apache.hadoop.fs.AbstractFileSystem.createFileSystem(AbstractFileSystem.java:157)
at
org.apache.hadoop.fs.AbstractFileSystem.get(AbstractFileSystem.java:242)
at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:334)
at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:331)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:331)
at
org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:448)
at
org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
at
org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.getDefaultFileContext(JobHistoryUtils.java:247)
at
org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.ensurePathInDefaultFileSystem(JobHistoryUtils.java:277)
at
org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.getConfiguredHistoryStagingDirPrefix(JobHistoryUtils.java:191)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.serviceInit(JobHistoryEventHandler.java:147)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:444)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1499)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1496)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1429)
2014-12-15 04:15:21,390 DEBUG [main]
org.apache.hadoop.util.PerformanceAdvisory: Crypto codec
org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec is not available.
Regards!
Chen
RE: MRAppMaster running on a docker container failed to load
openssl cipher.
Posted by "Liu, Yi A" <yi...@intel.com>.
Hi Chen He,
If native is not available, JCE will be used. So you can ignore that debug info. Actually CryptoCodec is not necessary in org.apache.hadoop.fs.Hdfs, and I have created a JIRA to fix it.
Regards,
Yi Liu
-----Original Message-----
From: Chen He [mailto:airbots@gmail.com]
Sent: Tuesday, December 16, 2014 12:31 PM
To: common-dev@hadoop.apache.org
Subject: MRAppMaster running on a docker container failed to load openssl cipher.
Try to run teragen based on hadoop 2.6.0 using docker and met following
error:
2014-12-15 04:15:21,385 DEBUG [main]
org.apache.hadoop.crypto.OpensslCipher: Failed to load OpenSSL Cipher.
java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsOpenssl()Z
at
org.apache.hadoop.util.NativeCodeLoader.buildSupportsOpenssl(Native Method)
at
org.apache.hadoop.crypto.OpensslCipher.<clinit>(OpensslCipher.java:84)
at
org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec.<init>(OpensslAesCtrCryptoCodec.java:50)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
at
org.apache.hadoop.crypto.CryptoCodec.getInstance(CryptoCodec.java:67)
at
org.apache.hadoop.crypto.CryptoCodec.getInstance(CryptoCodec.java:100)
at org.apache.hadoop.fs.Hdfs.<init>(Hdfs.java:91)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.fs.AbstractFileSystem.newInstance(AbstractFileSystem.java:129)
at
org.apache.hadoop.fs.AbstractFileSystem.createFileSystem(AbstractFileSystem.java:157)
at
org.apache.hadoop.fs.AbstractFileSystem.get(AbstractFileSystem.java:242)
at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:334)
at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:331)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:331)
at
org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:448)
at
org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
at
org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.getDefaultFileContext(JobHistoryUtils.java:247)
at
org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.ensurePathInDefaultFileSystem(JobHistoryUtils.java:277)
at
org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.getConfiguredHistoryStagingDirPrefix(JobHistoryUtils.java:191)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.serviceInit(JobHistoryEventHandler.java:147)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:444)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1499)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1496)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1429)
2014-12-15 04:15:21,390 DEBUG [main]
org.apache.hadoop.util.PerformanceAdvisory: Crypto codec org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec is not available.
Regards!
Chen