You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/04/16 01:51:56 UTC

[GitHub] [hudi] wk888 opened a new issue #2834: [SUPPORT]org.apache.hudi.exception.TableNotFoundException

wk888 opened a new issue #2834:
URL: https://github.com/apache/hudi/issues/2834


   **_Tips before filing an issue_**
   
   - Have you gone through our [FAQs](https://cwiki.apache.org/confluence/display/HUDI/FAQ)?
   
   - Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.
   
   - If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly.
   hive> set hive.input.format=org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat;
   hive> select * from test.hudu_test_1_rt where dt=2021-04-15;
   WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
   Query ID = root_20210416094020_2f0117b1-4007-4390-ab58-be9ca46a1915
   Total jobs = 1
   Launching Job 1 out of 1
   Number of reduce tasks is set to 0 since there's no reduce operator
   org.apache.hudi.exception.TableNotFoundException: Hoodie table not found in path ofs://qqqqqqq.myqcloud.com/tmp/hive/root/1c7ec12e-4953-4913-bf9f-a09372b51609/.hoodie
           at org.apache.hudi.exception.TableNotFoundException.checkTableValidity(TableNotFoundException.java:53)
           at org.apache.hudi.common.table.HoodieTableMetaClient.<init>(HoodieTableMetaClient.java:110)
           at org.apache.hudi.common.table.HoodieTableMetaClient.<init>(HoodieTableMetaClient.java:71)
           at org.apache.hudi.common.table.HoodieTableMetaClient$Builder.build(HoodieTableMetaClient.java:581)
           at org.apache.hudi.hadoop.utils.HoodieInputFormatUtils.getTableMetaClientForBasePath(HoodieInputFormatUtils.java:327)
           at org.apache.hudi.hadoop.utils.HoodieInputFormatUtils.lambda$getTableMetaClientByBasePath$3(HoodieInputFormatUtils.java:302)
           at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1321)
           at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
           at java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1548)
           at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
           at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
           at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
           at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
           at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
           at org.apache.hudi.hadoop.utils.HoodieInputFormatUtils.getTableMetaClientByBasePath(HoodieInputFormatUtils.java:293)
           at org.apache.hudi.hadoop.utils.HoodieRealtimeInputFormatUtils.getRealtimeSplits(HoodieRealtimeInputFormatUtils.java:66)
           at org.apache.hudi.hadoop.realtime.HoodieParquetRealtimeInputFormat.getSplits(HoodieParquetRealtimeInputFormat.java:66)
           at org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat$HoodieCombineFileInputFormatShim.getSplits(HoodieCombineHiveInputFormat.java:922)
           at org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.getCombineSplits(HoodieCombineHiveInputFormat.java:241)
           at org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.getSplits(HoodieCombineHiveInputFormat.java:363)
           at org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:333)
           at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:324)
           at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)
           at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
           at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)
           at java.security.AccessController.doPrivileged(Native Method)
           at javax.security.auth.Subject.doAs(Subject.java:422)
           at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
           at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)
           at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:578)
           at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:573)
           at java.security.AccessController.doPrivileged(Native Method)
           at javax.security.auth.Subject.doAs(Subject.java:422)
           at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
           at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:573)
           at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:564)
           at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:414)
           at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:151)
           at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
           at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
           at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
   Job Submission failed with exception 'org.apache.hudi.exception.TableNotFoundException(Hoodie table not found in path ofs://qqqqqqq.myqcloud.com/tmp/hive/root/1c7ec12e-4953-4913-bf9f-a09372b51609/.hoodie)'
   FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. Hoodie table not found in path ofs://qqqqqqq.myqcloud.com/tmp/hive/root/1c7ec12e-4953-4913-bf9f-a09372b51609/.hoodie
   
   **Describe the problem you faced**
   
   A clear and concise description of the problem.
   
   **To Reproduce**
   
   Steps to reproduce the behavior:
   
   1.flink  write to hudi
   2.use hive query hudi then throw exception
   
   **Expected behavior**
   
   A clear and concise description of what you expected to happen.
   
   **Environment Description**
   
   * Hudi version :0.8.0
   
   * Flink version :1.12.0
   
   * Hive version :2.3.6
   
   * Hadoop version : cdh2.6.0
   
   * Storage (HDFS/S3/GCS..) : ofs  (Tencent object storage)
   
   * Running on Docker? (yes/no) :no
   
   
   ![image](https://user-images.githubusercontent.com/16316415/114960297-54832d80-9e99-11eb-8303-c35264581c22.png)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] wk888 removed a comment on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
wk888 removed a comment on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-822316390


   > @wk888 Did you sync table information of hoodie to HMS?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] wk888 commented on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
wk888 commented on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-823700483


   @yanghua it seems have no privilege create database not create table ....
   and the table is created successed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] nsivabalan commented on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
nsivabalan commented on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-821764795


   @yanghua : related to Flink. can you (or loop in someone to) assist here.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] yanghua edited a comment on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
yanghua edited a comment on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-823016979


   > Error while compiling statement: No privilege 'Create' found for outputs { }
   
   It seems you do not have hive permission.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] wk888 commented on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
wk888 commented on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-822316390


   > @wk888 Did you sync table information of hoodie to HMS?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] wk888 removed a comment on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
wk888 removed a comment on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-822315896


   > @wk888 Did you sync table information of hoodie to HMS?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] wk888 edited a comment on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
wk888 edited a comment on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-822313322


   > @wk888 Did you sync table information of hoodie to HMS?
   
   @yanghua  yes  .tables  hudu_test_1_rt  and  hudu_test_1_ro is create success by  run_sync_tool.sh
   
   ./hudi-hive-sync/run_sync_tool.sh  \
   > --jdbc-url jdbc:hive2://11.11.11.11:30000 \
   > --user hive \
   > --pass hive \
   > --partitioned-by dt \
   > --partition-value-extractor org.apache.hudi.hive.MultiPartKeysValueExtractor \
   > --base-path ofs://qqqqqqqqq/user/wangkai/test_hudi_1 \
   > --database test \
   > --table hudu_test_1
   
   
   -----------------------------------------logs-------------------------
   2021-04-19 16:26:18,871 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(366)) - Executing SQL create database if not exists test
   2021-04-19 16:26:18,995 WARN  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(140)) - Unable to create database
   org.apache.hudi.hive.HoodieHiveSyncException: Failed in executing SQL create database if not exists test
           at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQL(HoodieHiveClient.java:369)
           at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:137)
           at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:112)
           at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:249)
   Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: No privilege 'Create' found for outputs { }
           at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:267)
           at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:253)
           at org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:313)
           at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:253)
           at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQL(HoodieHiveClient.java:367)
           ... 3 more
   Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: No privilege 'Create' found for outputs { }
           at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:380)
           at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:206)
           at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:290)
           at org.apache.hive.service.cli.operation.Operation.run(Operation.java:320)
           at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:530)
           at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:517)
           at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
           at java.lang.reflect.Method.invoke(Method.java:498)
           at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
           at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
           at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
           at java.security.AccessController.doPrivileged(Native Method)
           at javax.security.auth.Subject.doAs(Subject.java:422)
           at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
           at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
           at com.sun.proxy.$Proxy30.executeStatementAsync(Unknown Source)
           at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:310)
           at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:535)
           at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437)
           at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422)
           at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
           at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
           at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
           at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
           at java.lang.Thread.run(Thread.java:745)
   2021-04-19 16:26:19,050 INFO  [main] fs.CHDFSInputStream (CHDFSInputStream.java:close(73)) - close input stream for file: /user/wangkai/test_hudi_1/.hoodie/20210419160955.deltacommit, inode: 336302421, fd: 0, time-range-ms:[1618820779011 - 1618820779050], used-time-ms:39
   2021-04-19 16:26:19,210 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncSchema(182)) - Hive table hudu_test_1_ro is not found. Creating it
   2021-04-19 16:26:19,215 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:createTable(262)) - Creating table with CREATE EXTERNAL TABLE  IF NOT EXISTS `test`.`hudu_test_1_ro`( `_hoodie_commit_time` string, `_hoodie_commit_seqno` string, `_hoodie_record_key` string, `_hoodie_partition_path` string, `_hoodie_file_name` string, `uuid` string, `name` string, `ts` bigint) PARTITIONED BY (`dt` string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hudi.hadoop.HoodieParquetInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1'
   2021-04-19 16:26:19,215 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(366)) - Executing SQL CREATE EXTERNAL TABLE  IF NOT EXISTS `test`.`hudu_test_1_ro`( `_hoodie_commit_time` string, `_hoodie_commit_seqno` string, `_hoodie_record_key` string, `_hoodie_partition_path` string, `_hoodie_file_name` string, `uuid` string, `name` string, `ts` bigint) PARTITIONED BY (`dt` string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hudi.hadoop.HoodieParquetInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1'
   2021-04-19 16:26:19,864 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(156)) - Schema sync complete. Syncing partitions for hudu_test_1_ro
   2021-04-19 16:26:19,864 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(162)) - Last commit time synced was found to be null
   2021-04-19 16:26:19,864 INFO  [main] common.AbstractSyncHoodieClient (AbstractSyncHoodieClient.java:getPartitionsWrittenToSince(128)) - Last commit time synced is not known, listing all partitions in ofs://qqqqqqq/user/wangkai/test_hudi_1,FS :com.qcloud.chdfs.fs.CHDFSHadoopFileSystemAdapter@3adcc812
   2021-04-19 16:26:19,900 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(164)) - Storage partitions scan complete. Found 2
   2021-04-19 16:26:19,921 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncPartitions(223)) - New Partitions [20210415, 20210419]
   2021-04-19 16:26:19,921 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:addPartitionsToTable(120)) - Adding partitions 2 to table hudu_test_1_ro
   2021-04-19 16:26:19,922 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(366)) - Executing SQL ALTER TABLE `test`.`hudu_test_1_ro` ADD IF NOT EXISTS   PARTITION (`dt`='20210415') LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1/20210415'   PARTITION (`dt`='20210419') LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1/20210419' 
   2021-04-19 16:26:20,285 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncPartitions(226)) - Changed Partitions []
   2021-04-19 16:26:20,285 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updatePartitionsToTable(131)) - No partitions to change for hudu_test_1_ro
   2021-04-19 16:26:20,394 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(169)) - Sync complete for hudu_test_1_ro
   2021-04-19 16:26:20,394 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(131)) - Trying to sync hoodie table hudu_test_1_rt with base path ofs://qqqqqqq/user/wangkai/test_hudi_1 of type MERGE_ON_READ
   2021-04-19 16:26:20,394 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(366)) - Executing SQL create database if not exists test
   2021-04-19 16:26:20,472 WARN  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(140)) - Unable to create database
   org.apache.hudi.hive.HoodieHiveSyncException: Failed in executing SQL create database if not exists test
           at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQL(HoodieHiveClient.java:369)
           at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:137)
           at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:114)
           at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:249)
   Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: No privilege 'Create' found for outputs { }
           at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:267)
           at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:253)
           at org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:313)
           at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:253)
           at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQL(HoodieHiveClient.java:367)
           ... 3 more
   Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: No privilege 'Create' found for outputs { }
           at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:380)
           at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:206)
           at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:290)
           at org.apache.hive.service.cli.operation.Operation.run(Operation.java:320)
           at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:530)
           at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:517)
           at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
           at java.lang.reflect.Method.invoke(Method.java:498)
           at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
           at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
           at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
           at java.security.AccessController.doPrivileged(Native Method)
           at javax.security.auth.Subject.doAs(Subject.java:422)
           at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
           at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
           at com.sun.proxy.$Proxy30.executeStatementAsync(Unknown Source)
           at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:310)
           at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:535)
           at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437)
           at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422)
           at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
           at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
           at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
           at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
           at java.lang.Thread.run(Thread.java:745)
   2021-04-19 16:26:20,516 INFO  [main] fs.CHDFSInputStream (CHDFSInputStream.java:close(73)) - close input stream for file: /user/wangkai/test_hudi_1/.hoodie/20210419160955.deltacommit, inode: 336302421, fd: 0, time-range-ms:[1618820780481 - 1618820780516], used-time-ms:35
   2021-04-19 16:26:20,522 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncSchema(182)) - Hive table hudu_test_1_rt is not found. Creating it
   2021-04-19 16:26:20,522 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:createTable(262)) - Creating table with CREATE EXTERNAL TABLE  IF NOT EXISTS `test`.`hudu_test_1_rt`( `_hoodie_commit_time` string, `_hoodie_commit_seqno` string, `_hoodie_record_key` string, `_hoodie_partition_path` string, `_hoodie_file_name` string, `uuid` string, `name` string, `ts` bigint) PARTITIONED BY (`dt` string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hudi.hadoop.realtime.HoodieParquetRealtimeInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1'
   2021-04-19 16:26:20,522 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(366)) - Executing SQL CREATE EXTERNAL TABLE  IF NOT EXISTS `test`.`hudu_test_1_rt`( `_hoodie_commit_time` string, `_hoodie_commit_seqno` string, `_hoodie_record_key` string, `_hoodie_partition_path` string, `_hoodie_file_name` string, `uuid` string, `name` string, `ts` bigint) PARTITIONED BY (`dt` string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hudi.hadoop.realtime.HoodieParquetRealtimeInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1'
   2021-04-19 16:26:21,154 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(156)) - Schema sync complete. Syncing partitions for hudu_test_1_rt
   2021-04-19 16:26:21,154 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(162)) - Last commit time synced was found to be null
   2021-04-19 16:26:21,154 INFO  [main] common.AbstractSyncHoodieClient (AbstractSyncHoodieClient.java:getPartitionsWrittenToSince(128)) - Last commit time synced is not known, listing all partitions in ofs://qqqqqqq/user/wangkai/test_hudi_1,FS :com.qcloud.chdfs.fs.CHDFSHadoopFileSystemAdapter@3adcc812
   2021-04-19 16:26:21,169 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(164)) - Storage partitions scan complete. Found 2
   2021-04-19 16:26:21,187 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncPartitions(223)) - New Partitions [20210415, 20210419]
   2021-04-19 16:26:21,187 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:addPartitionsToTable(120)) - Adding partitions 2 to table hudu_test_1_rt
   2021-04-19 16:26:21,187 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(366)) - Executing SQL ALTER TABLE `test`.`hudu_test_1_rt` ADD IF NOT EXISTS   PARTITION (`dt`='20210415') LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1/20210415'   PARTITION (`dt`='20210419') LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1/20210419' 
   2021-04-19 16:26:21,574 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncPartitions(226)) - Changed Partitions []
   2021-04-19 16:26:21,574 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updatePartitionsToTable(131)) - No partitions to change for hudu_test_1_rt
   2021-04-19 16:26:21,648 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(169)) - Sync complete for hudu_test_1_rt
   2021-04-19 16:26:21,680 INFO  [main] hive.metastore (HiveMetaStoreClient.java:close(564)) - Closed a connection to metastore, current connections: 0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] wk888 commented on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
wk888 commented on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-823728826


   @yanghua 
   i can find  the hoodie file in hdfs:
   
   ![image](https://user-images.githubusercontent.com/16316415/115488269-cf2ebd00-a28c-11eb-85ac-73ed631b6f31.png)
   
   but from the error log  you can see it find the file from the  /tmp/hive/root/1c7ec12e-4953-4913-bf9f-a09372b51609/.hoodie it seemd like the hive tm directory...so it cant find the .hoodie file
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] n3nash commented on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
n3nash commented on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-854408512


   @wk888 Could you please let @yanghua know if you need further assistance ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] wk888 commented on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
wk888 commented on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-822315896


   > @wk888 Did you sync table information of hoodie to HMS?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] yanghua commented on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
yanghua commented on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-822296834


   @wk888 Did you sync table information of hoodie to HMS?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] yanghua commented on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
yanghua commented on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-823016979


   > Error while compiling statement: No privilege 'Create' found for outputs { }
   
   It seems you do not have hive authorized.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] yanghua commented on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
yanghua commented on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-823705049


   @wk888 OK, I reviewed the code, at `TableNotFoundException.java:53`, the path you provided triggered `FileNotFoundException | IllegalArgumentException`. Did you make sure the path exists?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] wk888 closed issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
wk888 closed issue #2834:
URL: https://github.com/apache/hudi/issues/2834


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] n3nash commented on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
n3nash commented on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-859332953


   Closing this due to inactivity. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] wk888 commented on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
wk888 commented on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-822313322


   > @wk888 Did you sync table information of hoodie to HMS?
   
   yes  .tables  hudu_test_1_rt  and  hudu_test_1_ro is create success by  run_sync_tool.sh
   
   ./hudi-hive-sync/run_sync_tool.sh  \
   > --jdbc-url jdbc:hive2://11.11.11.11:30000 \
   > --user hive \
   > --pass hive \
   > --partitioned-by dt \
   > --partition-value-extractor org.apache.hudi.hive.MultiPartKeysValueExtractor \
   > --base-path ofs://qqqqqqqqq/user/wangkai/test_hudi_1 \
   > --database test \
   > --table hudu_test_1
   
   
   -----------------------------------------logs-------------------------
   2021-04-19 16:26:18,871 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(366)) - Executing SQL create database if not exists test
   2021-04-19 16:26:18,995 WARN  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(140)) - Unable to create database
   org.apache.hudi.hive.HoodieHiveSyncException: Failed in executing SQL create database if not exists test
           at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQL(HoodieHiveClient.java:369)
           at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:137)
           at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:112)
           at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:249)
   Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: No privilege 'Create' found for outputs { }
           at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:267)
           at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:253)
           at org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:313)
           at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:253)
           at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQL(HoodieHiveClient.java:367)
           ... 3 more
   Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: No privilege 'Create' found for outputs { }
           at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:380)
           at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:206)
           at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:290)
           at org.apache.hive.service.cli.operation.Operation.run(Operation.java:320)
           at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:530)
           at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:517)
           at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
           at java.lang.reflect.Method.invoke(Method.java:498)
           at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
           at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
           at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
           at java.security.AccessController.doPrivileged(Native Method)
           at javax.security.auth.Subject.doAs(Subject.java:422)
           at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
           at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
           at com.sun.proxy.$Proxy30.executeStatementAsync(Unknown Source)
           at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:310)
           at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:535)
           at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437)
           at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422)
           at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
           at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
           at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
           at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
           at java.lang.Thread.run(Thread.java:745)
   2021-04-19 16:26:19,050 INFO  [main] fs.CHDFSInputStream (CHDFSInputStream.java:close(73)) - close input stream for file: /user/wangkai/test_hudi_1/.hoodie/20210419160955.deltacommit, inode: 336302421, fd: 0, time-range-ms:[1618820779011 - 1618820779050], used-time-ms:39
   2021-04-19 16:26:19,210 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncSchema(182)) - Hive table hudu_test_1_ro is not found. Creating it
   2021-04-19 16:26:19,215 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:createTable(262)) - Creating table with CREATE EXTERNAL TABLE  IF NOT EXISTS `test`.`hudu_test_1_ro`( `_hoodie_commit_time` string, `_hoodie_commit_seqno` string, `_hoodie_record_key` string, `_hoodie_partition_path` string, `_hoodie_file_name` string, `uuid` string, `name` string, `ts` bigint) PARTITIONED BY (`dt` string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hudi.hadoop.HoodieParquetInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1'
   2021-04-19 16:26:19,215 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(366)) - Executing SQL CREATE EXTERNAL TABLE  IF NOT EXISTS `test`.`hudu_test_1_ro`( `_hoodie_commit_time` string, `_hoodie_commit_seqno` string, `_hoodie_record_key` string, `_hoodie_partition_path` string, `_hoodie_file_name` string, `uuid` string, `name` string, `ts` bigint) PARTITIONED BY (`dt` string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hudi.hadoop.HoodieParquetInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1'
   2021-04-19 16:26:19,864 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(156)) - Schema sync complete. Syncing partitions for hudu_test_1_ro
   2021-04-19 16:26:19,864 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(162)) - Last commit time synced was found to be null
   2021-04-19 16:26:19,864 INFO  [main] common.AbstractSyncHoodieClient (AbstractSyncHoodieClient.java:getPartitionsWrittenToSince(128)) - Last commit time synced is not known, listing all partitions in ofs://qqqqqqq/user/wangkai/test_hudi_1,FS :com.qcloud.chdfs.fs.CHDFSHadoopFileSystemAdapter@3adcc812
   2021-04-19 16:26:19,900 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(164)) - Storage partitions scan complete. Found 2
   2021-04-19 16:26:19,921 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncPartitions(223)) - New Partitions [20210415, 20210419]
   2021-04-19 16:26:19,921 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:addPartitionsToTable(120)) - Adding partitions 2 to table hudu_test_1_ro
   2021-04-19 16:26:19,922 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(366)) - Executing SQL ALTER TABLE `test`.`hudu_test_1_ro` ADD IF NOT EXISTS   PARTITION (`dt`='20210415') LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1/20210415'   PARTITION (`dt`='20210419') LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1/20210419' 
   2021-04-19 16:26:20,285 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncPartitions(226)) - Changed Partitions []
   2021-04-19 16:26:20,285 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updatePartitionsToTable(131)) - No partitions to change for hudu_test_1_ro
   2021-04-19 16:26:20,394 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(169)) - Sync complete for hudu_test_1_ro
   2021-04-19 16:26:20,394 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(131)) - Trying to sync hoodie table hudu_test_1_rt with base path ofs://qqqqqqq/user/wangkai/test_hudi_1 of type MERGE_ON_READ
   2021-04-19 16:26:20,394 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(366)) - Executing SQL create database if not exists test
   2021-04-19 16:26:20,472 WARN  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(140)) - Unable to create database
   org.apache.hudi.hive.HoodieHiveSyncException: Failed in executing SQL create database if not exists test
           at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQL(HoodieHiveClient.java:369)
           at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:137)
           at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:114)
           at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:249)
   Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: No privilege 'Create' found for outputs { }
           at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:267)
           at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:253)
           at org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:313)
           at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:253)
           at org.apache.hudi.hive.HoodieHiveClient.updateHiveSQL(HoodieHiveClient.java:367)
           ... 3 more
   Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: No privilege 'Create' found for outputs { }
           at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:380)
           at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:206)
           at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:290)
           at org.apache.hive.service.cli.operation.Operation.run(Operation.java:320)
           at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:530)
           at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:517)
           at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
           at java.lang.reflect.Method.invoke(Method.java:498)
           at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
           at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
           at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
           at java.security.AccessController.doPrivileged(Native Method)
           at javax.security.auth.Subject.doAs(Subject.java:422)
           at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
           at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
           at com.sun.proxy.$Proxy30.executeStatementAsync(Unknown Source)
           at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:310)
           at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:535)
           at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437)
           at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422)
           at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
           at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
           at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
           at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
           at java.lang.Thread.run(Thread.java:745)
   2021-04-19 16:26:20,516 INFO  [main] fs.CHDFSInputStream (CHDFSInputStream.java:close(73)) - close input stream for file: /user/wangkai/test_hudi_1/.hoodie/20210419160955.deltacommit, inode: 336302421, fd: 0, time-range-ms:[1618820780481 - 1618820780516], used-time-ms:35
   2021-04-19 16:26:20,522 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncSchema(182)) - Hive table hudu_test_1_rt is not found. Creating it
   2021-04-19 16:26:20,522 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:createTable(262)) - Creating table with CREATE EXTERNAL TABLE  IF NOT EXISTS `test`.`hudu_test_1_rt`( `_hoodie_commit_time` string, `_hoodie_commit_seqno` string, `_hoodie_record_key` string, `_hoodie_partition_path` string, `_hoodie_file_name` string, `uuid` string, `name` string, `ts` bigint) PARTITIONED BY (`dt` string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hudi.hadoop.realtime.HoodieParquetRealtimeInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1'
   2021-04-19 16:26:20,522 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(366)) - Executing SQL CREATE EXTERNAL TABLE  IF NOT EXISTS `test`.`hudu_test_1_rt`( `_hoodie_commit_time` string, `_hoodie_commit_seqno` string, `_hoodie_record_key` string, `_hoodie_partition_path` string, `_hoodie_file_name` string, `uuid` string, `name` string, `ts` bigint) PARTITIONED BY (`dt` string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hudi.hadoop.realtime.HoodieParquetRealtimeInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1'
   2021-04-19 16:26:21,154 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(156)) - Schema sync complete. Syncing partitions for hudu_test_1_rt
   2021-04-19 16:26:21,154 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(162)) - Last commit time synced was found to be null
   2021-04-19 16:26:21,154 INFO  [main] common.AbstractSyncHoodieClient (AbstractSyncHoodieClient.java:getPartitionsWrittenToSince(128)) - Last commit time synced is not known, listing all partitions in ofs://qqqqqqq/user/wangkai/test_hudi_1,FS :com.qcloud.chdfs.fs.CHDFSHadoopFileSystemAdapter@3adcc812
   2021-04-19 16:26:21,169 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(164)) - Storage partitions scan complete. Found 2
   2021-04-19 16:26:21,187 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncPartitions(223)) - New Partitions [20210415, 20210419]
   2021-04-19 16:26:21,187 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:addPartitionsToTable(120)) - Adding partitions 2 to table hudu_test_1_rt
   2021-04-19 16:26:21,187 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updateHiveSQL(366)) - Executing SQL ALTER TABLE `test`.`hudu_test_1_rt` ADD IF NOT EXISTS   PARTITION (`dt`='20210415') LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1/20210415'   PARTITION (`dt`='20210419') LOCATION 'ofs://qqqqqqq/user/wangkai/test_hudi_1/20210419' 
   2021-04-19 16:26:21,574 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncPartitions(226)) - Changed Partitions []
   2021-04-19 16:26:21,574 INFO  [main] hive.HoodieHiveClient (HoodieHiveClient.java:updatePartitionsToTable(131)) - No partitions to change for hudu_test_1_rt
   2021-04-19 16:26:21,648 INFO  [main] hive.HiveSyncTool (HiveSyncTool.java:syncHoodieTable(169)) - Sync complete for hudu_test_1_rt
   2021-04-19 16:26:21,680 INFO  [main] hive.metastore (HiveMetaStoreClient.java:close(564)) - Closed a connection to metastore, current connections: 0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] n3nash commented on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
n3nash commented on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-854408512


   @wk888 Could you please let @yanghua know if you need further assistance ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] wk888 commented on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
wk888 commented on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-822218549


   use hive query hudi... would the .hoodie file  copy to /tmp/hive/..???。 how does it work


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] n3nash closed issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
n3nash closed issue #2834:
URL: https://github.com/apache/hudi/issues/2834


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] yanghua commented on issue #2834: [SUPPORT] Help~~~org.apache.hudi.exception.TableNotFoundException

Posted by GitBox <gi...@apache.org>.
yanghua commented on issue #2834:
URL: https://github.com/apache/hudi/issues/2834#issuecomment-824489701


   @wk888 Sorry, I did not find the reason. It seems it was not caused by the flink write client. Can you test the whole workflow via spark?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org