[jira] [Updated] (HUDI-3945) After the async compaction operation is complete, the task should exit.

["Ethan Guo (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-3945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ethan Guo updated HUDI-3945:
----------------------------
    Fix Version/s: 0.11.1

> After the async compaction operation is complete, the task should exit.
> -----------------------------------------------------------------------
>
>                 Key: HUDI-3945
>                 URL: https://issues.apache.org/jira/browse/HUDI-3945
>             Project: Apache Hudi
>          Issue Type: Bug
>            Reporter: YangXuan
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 0.11.1
>
>
> This problem occurs if you perform the following operations:
> 1、Create a mor table and perform the upsert operation on it for five times.
> 2、Gets the timestamp of the last upsert execution, assuming it is 20220318191221.
> 3、Do compaction schedule by spark-submit and the timestamp is 20220318191221 plus 5.
> spark-submit --conf "spark.driver.extraJavaOptions=-Dlog4j.configuration=file:/opt/hudi/error_log4j.properties" --jars /opt/client/Hudi/hudi/lib/hudi-client-common*.jar --class org.apache.hudi.utilities.HoodieCompactor /opt/client/Hudi/hudi/lib/hudi-utilities*.jar --base-path /tmp/testdb/tb_test_mor --table-name tb_test_mor --parallelism 100 --spark-memory 1G --schema-file /tmp/json/compact_tb_base.json --instant-time 20220318191226 --schedule --strategy org.apache.hudi.table.action.compact.strategy.UnBoundedCompactionStrategy
> 4、Run compaction by spark-submit and you will see that the task of running compaction is executed successfully, but the spark task does not exit.
> spark-submit --conf "spark.driver.extraJavaOptions=-Dlog4j.configuration=file:/opt/hudi/error_log4j.properties" --num-executors 4 --jars /opt/client/Hudi/hudi/lib/hudi-client-common-{_}.jar --class org.apache.hudi.utilities.HoodieCompactor /opt/client/Hudi/hudi/lib/hudi-utilities_{_}.jar --base-path /tmp/testdb/tb_test_mor --table-name tb_test_mor --parallelism 100 --spark-memory 1G --schema-file /tmp/json/compact_tb_base.json --instant-time 20220318191226



--
This message was sent by Atlassian Jira
(v8.20.7#820007)