You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/10/30 07:21:47 UTC
[GitHub] [hudi] Cherry-Puppy opened a new issue #3895: [SUPPORT] flink hudi sync hive
Cherry-Puppy opened a new issue #3895:
URL: https://github.com/apache/hudi/issues/3895
org.apache.hudi.exception.HoodieException: Failed to read schema from commit metadata
at org.apache.hudi.common.table.TableSchemaResolver.getTableSchemaFromCommitMetadata(TableSchemaResolver.java:271) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.common.table.TableSchemaResolver.getTableSchemaFromCommitMetadata(TableSchemaResolver.java:242) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.common.table.TableSchemaResolver.getTableParquetSchema(TableSchemaResolver.java:200) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.sync.common.AbstractSyncHoodieClient.getDataSchema(AbstractSyncHoodieClient.java:145) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:177) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.hive.HiveSyncTool.doSync(HiveSyncTool.java:131) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:117) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
... 5 more
Caused by: org.apache.hudi.exception.HoodieIOException: Could not read commit details from hdfs://HDFSName/tss/t11/.hoodie/20211030151821.commit
at org.apache.hudi.common.table.timeline.HoodieActiveTimeline.readDataFromPath(HoodieActiveTimeline.java:619) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.common.table.timeline.HoodieActiveTimeline.getInstantDetails(HoodieActiveTimeline.java:233) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.common.table.timeline.HoodieDefaultTimeline.getInstantDetails(HoodieDefaultTimeline.java:339) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.common.table.TableSchemaResolver.getTableSchemaFromCommitMetadata(TableSchemaResolver.java:257) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.common.table.TableSchemaResolver.getTableSchemaFromCommitMetadata(TableSchemaResolver.java:242) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.common.table.TableSchemaResolver.getTableParquetSchema(TableSchemaResolver.java:200) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.sync.common.AbstractSyncHoodieClient.getDataSchema(AbstractSyncHoodieClient.java:145) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:177) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.hive.HiveSyncTool.doSync(HiveSyncTool.java:131) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:117) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
... 5 more
Caused by: java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:817) ~[flink-shaded-hadoop-2-uber-2.7.5-9.0.jar:2.7.5-9.0]
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1531) ~[flink-shaded-hadoop-2-uber-2.7.5-9.0.jar:2.7.5-9.0]
at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:304) ~[flink-shaded-hadoop-2-uber-2.7.5-9.0.jar:2.7.5-9.0]
at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:299) ~[flink-shaded-hadoop-2-uber-2.7.5-9.0.jar:2.7.5-9.0]
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) ~[flink-shaded-hadoop-2-uber-2.7.5-9.0.jar:2.7.5-9.0]
at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:312) ~[flink-shaded-hadoop-2-uber-2.7.5-9.0.jar:2.7.5-9.0]
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:768) ~[flink-shaded-hadoop-2-uber-2.7.5-9.0.jar:2.7.5-9.0]
at org.apache.hudi.common.fs.HoodieWrapperFileSystem.open(HoodieWrapperFileSystem.java:461) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.common.table.timeline.HoodieActiveTimeline.readDataFromPath(HoodieActiveTimeline.java:616) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.common.table.timeline.HoodieActiveTimeline.getInstantDetails(HoodieActiveTimeline.java:233) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.common.table.timeline.HoodieDefaultTimeline.getInstantDetails(HoodieDefaultTimeline.java:339) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.common.table.TableSchemaResolver.getTableSchemaFromCommitMetadata(TableSchemaResolver.java:257) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.common.table.TableSchemaResolver.getTableSchemaFromCommitMetadata(TableSchemaResolver.java:242) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.common.table.TableSchemaResolver.getTableParquetSchema(TableSchemaResolver.java:200) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.sync.common.AbstractSyncHoodieClient.getDataSchema(AbstractSyncHoodieClient.java:145) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:177) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.hive.HiveSyncTool.doSync(HiveSyncTool.java:131) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:117) ~[pool-1-thread-6-1635578219030-hudi-flink_2.11-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on issue #3895: [SUPPORT] flink hudi sync hive
Posted by GitBox <gi...@apache.org>.
xushiyan commented on issue #3895:
URL: https://github.com/apache/hudi/issues/3895#issuecomment-966759689
@Cherry-Puppy can you provide code snippet and env / versions / configs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on issue #3895: [SUPPORT] flink hudi sync hive
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on issue #3895:
URL: https://github.com/apache/hudi/issues/3895#issuecomment-1018630545
@Cherry-Puppy : can we have any updates on your end. If you got the issue resolved, can we close the github issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on issue #3895: [SUPPORT] flink hudi sync hive
Posted by GitBox <gi...@apache.org>.
danny0405 commented on issue #3895:
URL: https://github.com/apache/hudi/issues/3895#issuecomment-966857021
@Cherry-Puppy It seems that you are querying data and actually the data had never been successfully committed before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org