You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Denes Bodo (JIRA)" <ji...@apache.org> on 2018/03/29 07:48:00 UTC
[jira] [Updated] (OOZIE-3204) Oozie cannot run HBase code in Java
action
[ https://issues.apache.org/jira/browse/OOZIE-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Denes Bodo updated OOZIE-3204:
------------------------------
Description:
After having custom raw file system implementation, HBase (java) action cannot run:
{noformat}
018-03-28 06:55:46,372 WARN HbaseCredentials:523 - SERVER[ctr-e138-1518143905142-137559-01-000003.hwx.site] USER[hrt_qa] GROUP[-] TOKEN[] APP[tpch_query1] JOB[0000002-180328065157516-oozie-oozi-W] ACTION[0000002-180328065157516-oozie-oozi-W@tpch_query1] Exception in receiving hbase credentials
java.io.IOException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
at org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:68)
at org.apache.oozie.action.hadoop.HbaseCredentials$1.run(HbaseCredentials.java:86)
at org.apache.oozie.action.hadoop.HbaseCredentials$1.run(HbaseCredentials.java:84)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
at org.apache.oozie.action.hadoop.HbaseCredentials.obtainToken(HbaseCredentials.java:83)
at org.apache.oozie.action.hadoop.HbaseCredentials.addtoJobConf(HbaseCredentials.java:56)
at org.apache.oozie.action.hadoop.JavaActionExecutor.setCredentialTokens(JavaActionExecutor.java:1338)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1178)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1424)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
at org.apache.oozie.command.XCommand.call(XCommand.java:287)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
... 24 more
Caused by: java.lang.ExceptionInInitializerError
at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)
at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)
at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:903)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:648)
... 29 more
Caused by: java.lang.UnsupportedOperationException: Accessing local file system is not allowed
at org.apache.hadoop.fs.RawLocalFileSystem.initialize(RawLocalFileSystem.java:48)
at org.apache.hadoop.fs.LocalFileSystem.initialize(LocalFileSystem.java:47)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2796)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2830)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2812)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:390)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:179)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:374)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
at org.apache.hadoop.hbase.util.DynamicClassLoader.initTempDir(DynamicClassLoader.java:118)
at org.apache.hadoop.hbase.util.DynamicClassLoader.<init>(DynamicClassLoader.java:98)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.<clinit>(ProtobufUtil.java:251)
... 34 more
{noformat}
Using HBase client as a lib we cannot modify how it uses the file system.
was:
After having custom raw file system implementation, HBase action cannot run:
{noformat}
018-03-28 06:55:46,372 WARN HbaseCredentials:523 - SERVER[ctr-e138-1518143905142-137559-01-000003.hwx.site] USER[hrt_qa] GROUP[-] TOKEN[] APP[tpch_query1] JOB[0000002-180328065157516-oozie-oozi-W] ACTION[0000002-180328065157516-oozie-oozi-W@tpch_query1] Exception in receiving hbase credentials
java.io.IOException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
at org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:68)
at org.apache.oozie.action.hadoop.HbaseCredentials$1.run(HbaseCredentials.java:86)
at org.apache.oozie.action.hadoop.HbaseCredentials$1.run(HbaseCredentials.java:84)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
at org.apache.oozie.action.hadoop.HbaseCredentials.obtainToken(HbaseCredentials.java:83)
at org.apache.oozie.action.hadoop.HbaseCredentials.addtoJobConf(HbaseCredentials.java:56)
at org.apache.oozie.action.hadoop.JavaActionExecutor.setCredentialTokens(JavaActionExecutor.java:1338)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1178)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1424)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
at org.apache.oozie.command.XCommand.call(XCommand.java:287)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
... 24 more
Caused by: java.lang.ExceptionInInitializerError
at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)
at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)
at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:903)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:648)
... 29 more
Caused by: java.lang.UnsupportedOperationException: Accessing local file system is not allowed
at org.apache.hadoop.fs.RawLocalFileSystem.initialize(RawLocalFileSystem.java:48)
at org.apache.hadoop.fs.LocalFileSystem.initialize(LocalFileSystem.java:47)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2796)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2830)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2812)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:390)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:179)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:374)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
at org.apache.hadoop.hbase.util.DynamicClassLoader.initTempDir(DynamicClassLoader.java:118)
at org.apache.hadoop.hbase.util.DynamicClassLoader.<init>(DynamicClassLoader.java:98)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.<clinit>(ProtobufUtil.java:251)
... 34 more
{noformat}
Using HBase client as a lib we cannot modify how it uses the file system.
> Oozie cannot run HBase code in Java action
> -------------------------------------------
>
> Key: OOZIE-3204
> URL: https://issues.apache.org/jira/browse/OOZIE-3204
> Project: Oozie
> Issue Type: Bug
> Components: core
> Affects Versions: 4.3.1
> Reporter: Denes Bodo
> Assignee: Satish Subhashrao Saley
> Priority: Critical
>
> After having custom raw file system implementation, HBase (java) action cannot run:
> {noformat}
> 018-03-28 06:55:46,372 WARN HbaseCredentials:523 - SERVER[ctr-e138-1518143905142-137559-01-000003.hwx.site] USER[hrt_qa] GROUP[-] TOKEN[] APP[tpch_query1] JOB[0000002-180328065157516-oozie-oozi-W] ACTION[0000002-180328065157516-oozie-oozi-W@tpch_query1] Exception in receiving hbase credentials
> java.io.IOException: java.lang.reflect.InvocationTargetException
> at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
> at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
> at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
> at org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:68)
> at org.apache.oozie.action.hadoop.HbaseCredentials$1.run(HbaseCredentials.java:86)
> at org.apache.oozie.action.hadoop.HbaseCredentials$1.run(HbaseCredentials.java:84)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
> at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at org.apache.oozie.action.hadoop.HbaseCredentials.obtainToken(HbaseCredentials.java:83)
> at org.apache.oozie.action.hadoop.HbaseCredentials.addtoJobConf(HbaseCredentials.java:56)
> at org.apache.oozie.action.hadoop.JavaActionExecutor.setCredentialTokens(JavaActionExecutor.java:1338)
> at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1178)
> at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1424)
> at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
> at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
> at org.apache.oozie.command.XCommand.call(XCommand.java:287)
> at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
> at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
> ... 24 more
> Caused by: java.lang.ExceptionInInitializerError
> at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)
> at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)
> at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:903)
> at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:648)
> ... 29 more
> Caused by: java.lang.UnsupportedOperationException: Accessing local file system is not allowed
> at org.apache.hadoop.fs.RawLocalFileSystem.initialize(RawLocalFileSystem.java:48)
> at org.apache.hadoop.fs.LocalFileSystem.initialize(LocalFileSystem.java:47)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2796)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2830)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2812)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:390)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:179)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:374)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
> at org.apache.hadoop.hbase.util.DynamicClassLoader.initTempDir(DynamicClassLoader.java:118)
> at org.apache.hadoop.hbase.util.DynamicClassLoader.<init>(DynamicClassLoader.java:98)
> at org.apache.hadoop.hbase.protobuf.ProtobufUtil.<clinit>(ProtobufUtil.java:251)
> ... 34 more
> {noformat}
> Using HBase client as a lib we cannot modify how it uses the file system.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)