You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/07/18 08:54:44 UTC

[GitHub] [hudi] XuQianJin-Stars opened a new issue, #6131: Exception file does not exist .hoodie/.aux/view_storage_conf.properties

XuQianJin-Stars opened a new issue, #6131:
URL: https://github.com/apache/hudi/issues/6131

   2022-07-18 16:49:53
   org.apache.hudi.exception.HoodieIOException: Could not load filesystem view storage properties from hdfs://qy-pcg-11-v3/user/tdw/warehouse/csig_billing_rt_ods.db/ods_dev_flow_t_operation_flow_ri/.hoodie/.aux/view_storage_conf.properties
       at org.apache.hudi.util.ViewStorageProperties.loadFromProperties(ViewStorageProperties.java:78)
       at org.apache.hudi.util.StreamerUtil.getHoodieClientConfig(StreamerUtil.java:213)
       at org.apache.hudi.util.StreamerUtil.getHoodieClientConfig(StreamerUtil.java:152)
       at org.apache.hudi.util.StreamerUtil.createWriteClient(StreamerUtil.java:376)
       at org.apache.hudi.util.StreamerUtil.createWriteClient(StreamerUtil.java:360)
       at org.apache.hudi.sink.compact.CompactFunction.open(CompactFunction.java:81)
       at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:34)
       at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
       at org.apache.flink.streaming.api.operators.ProcessOperator.open(ProcessOperator.java:56)
       at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:442)
       at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:582)
       at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55)
       at org.apache.flink.streaming.runtime.tasks.StreamTask.executeRestore(StreamTask.java:562)
       at org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:647)
       at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:537)
       at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:759)
       at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
       at java.lang.Thread.run(Thread.java:748)
   Caused by: java.io.FileNotFoundException: File does not exist: /user/tdw/warehouse/csig_billing_rt_ods.db/ods_dev_flow_t_operation_flow_ri/.hoodie/.aux/view_storage_conf.properties
       at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:86)
       at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76)
       at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:156)
       at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1995)
       at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:797)
       at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:290)
       at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
       at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:529)
       at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
       at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1039)
       at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:963)
       at java.base/java.security.AccessController.doPrivileged(Native Method)
       at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:2034)
       at org.apache.hadoop.ipc.Server$Handler.run(Server.java:3047)
   
       at sun.reflect.GeneratedConstructorAccessor71.newInstance(Unknown Source)
       at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
       at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
       at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121)
       at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88)
       at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1015)
       at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1002)
       at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:991)
       at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1229)
       at org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:350)
       at org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:346)
       at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
       at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:358)
       at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:811)
       at org.apache.hudi.util.ViewStorageProperties.loadFromProperties(ViewStorageProperties.java:73)
       ... 17 more
   Caused by: org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): File does not exist: /user/tdw/warehouse/csig_billing_rt_ods.db/ods_dev_flow_t_operation_flow_ri/.hoodie/.aux/view_storage_conf.properties
       at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:86)
       at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76)
       at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:156)
       at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1995)
       at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:797)
       at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:290)
       at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
       at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:529)
       at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
       at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1039)
       at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:963)
       at java.base/java.security.AccessController.doPrivileged(Native Method)
       at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:2034)
       at org.apache.hadoop.ipc.Server$Handler.run(Server.java:3047)
   
       at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1559)
       at org.apache.hadoop.ipc.Client.call(Client.java:1505)
       at org.apache.hadoop.ipc.Client.call(Client.java:1412)
       at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
       at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
       at com.sun.proxy.$Proxy10.getBlockLocations(Unknown Source)
       at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getBlockLocations(ClientNamenodeProtocolTranslatorPB.java:293)
       at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:411)
       at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
       at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
       at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
       at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346)
       at com.sun.proxy.$Proxy11.getBlockLocations(Unknown Source)
       at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1013)
       ... 26 more
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] danny0405 commented on issue #6131: Exception file does not exist .hoodie/.aux/view_storage_conf.properties

Posted by GitBox <gi...@apache.org>.
danny0405 commented on issue #6131:
URL: https://github.com/apache/hudi/issues/6131#issuecomment-1306537414

   Close because it is inactive.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] minihippo commented on issue #6131: Exception file does not exist .hoodie/.aux/view_storage_conf.properties

Posted by GitBox <gi...@apache.org>.
minihippo commented on issue #6131:
URL: https://github.com/apache/hudi/issues/6131#issuecomment-1187328689

   Multiple writers of a table？


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] xushiyan commented on issue #6131: Exception file does not exist .hoodie/.aux/view_storage_conf.properties

Posted by GitBox <gi...@apache.org>.
xushiyan commented on issue #6131:
URL: https://github.com/apache/hudi/issues/6131#issuecomment-1188555314

   @XuQianJin-Stars not sure what case this is for. can you provide versions, configs and scripts to reproduce?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] nsivabalan commented on issue #6131: Exception file does not exist .hoodie/.aux/view_storage_conf.properties

Posted by GitBox <gi...@apache.org>.
nsivabalan commented on issue #6131:
URL: https://github.com/apache/hudi/issues/6131#issuecomment-1229345264

   @XuQianJin-Stars : any updates please.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] danny0405 closed issue #6131: Exception file does not exist .hoodie/.aux/view_storage_conf.properties

Posted by GitBox <gi...@apache.org>.
danny0405 closed issue #6131: Exception file does not exist .hoodie/.aux/view_storage_conf.properties
URL: https://github.com/apache/hudi/issues/6131


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org