You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2022/01/05 01:30:00 UTC
[jira] [Resolved] (HUDI-3107) Fix HiveSyncTool drop partitions using JDBC
[ https://issues.apache.org/jira/browse/HUDI-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
sivabalan narayanan resolved HUDI-3107.
---------------------------------------
> Fix HiveSyncTool drop partitions using JDBC
> -------------------------------------------
>
> Key: HUDI-3107
> URL: https://issues.apache.org/jira/browse/HUDI-3107
> Project: Apache Hudi
> Issue Type: Bug
> Components: Hive Integration
> Reporter: Yue Zhang
> Assignee: Yue Zhang
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.11.0, 0.10.1
>
>
> ```
> org.apache.hudi.exception.HoodieException: Unable to delete table partitions in /Users/yuezhang/tmp/hudiAfTable/forecast_agg
> at org.apache.hudi.utilities.HoodieDropPartitionsTool.run(HoodieDropPartitionsTool.java:240)
> at org.apache.hudi.utilities.HoodieDropPartitionsTool.main(HoodieDropPartitionsTool.java:212)
> at HoodieDropPartitionsToolTest.main(HoodieDropPartitionsToolTest.java:31)
> Caused by: org.apache.hudi.exception.HoodieException: Got runtime exception when hive syncing forecast_agg
> at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:119)
> at org.apache.hudi.utilities.HoodieDropPartitionsTool.syncHive(HoodieDropPartitionsTool.java:404)
> at org.apache.hudi.utilities.HoodieDropPartitionsTool.syncToHiveIfNecessary(HoodieDropPartitionsTool.java:270)
> at org.apache.hudi.utilities.HoodieDropPartitionsTool.doDeleteTablePartitionsEager(HoodieDropPartitionsTool.java:252)
> at org.apache.hudi.utilities.HoodieDropPartitionsTool.run(HoodieDropPartitionsTool.java:230)
> ... 2 more
> Caused by: org.apache.hudi.hive.HoodieHiveSyncException: Failed to sync partitions for table forecast_agg
> at org.apache.hudi.hive.HiveSyncTool.syncPartitions(HiveSyncTool.java:368)
> at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:202)
> at org.apache.hudi.hive.HiveSyncTool.doSync(HiveSyncTool.java:130)
> at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:116)
> ... 6 more
> Caused by: org.apache.hudi.hive.HoodieHiveSyncException: Failed in executing SQL ALTER TABLE `forecast_agg` DROP PARTITION (20210623/0/20210623)
> at org.apache.hudi.hive.ddl.JDBCExecutor.runSQL(JDBCExecutor.java:64)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at org.apache.hudi.hive.ddl.JDBCExecutor.dropPartitionsToTable(JDBCExecutor.java:149)
> at org.apache.hudi.hive.HoodieHiveClient.dropPartitionsToTable(HoodieHiveClient.java:130)
> at org.apache.hudi.hive.HiveSyncTool.syncPartitions(HiveSyncTool.java:363)
> ... 9 more
> Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: ParseException line 1:43 cannot recognize input near '20210623' '/' '0' in drop partition statement
> at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:256)
> at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:242)
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:254)
> at org.apache.hudi.hive.ddl.JDBCExecutor.runSQL(JDBCExecutor.java:62)
> ... 21 more
> Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: ParseException line 1:43 cannot recognize input near '20210623' '/' '0' in drop partition statement
> at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:380)
> at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:206)
> at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:290)
> at org.apache.hive.service.cli.operation.Operation.run(Operation.java:320)
> at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:530)
> at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:517)
> at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:310)
> at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:530)
> at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437)
> at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.parse.ParseException:line 1:43 cannot recognize input near '20210623' '/' '0' in drop partition statement
> at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:211)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:77)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:70)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1295)
> at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:204)
> ... 15 more
> ```
--
This message was sent by Atlassian Jira
(v8.20.1#820001)