You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2023/03/13 19:33:00 UTC
[jira] [Created] (HUDI-5928) PostWriteTermination Strategy does not shutdown the spark job completely
sivabalan narayanan created HUDI-5928:
-----------------------------------------
Summary: PostWriteTermination Strategy does not shutdown the spark job completely
Key: HUDI-5928
URL: https://issues.apache.org/jira/browse/HUDI-5928
Project: Apache Hudi
Issue Type: Bug
Components: writer-core
Reporter: sivabalan narayanan
When post write termination strategy is used, spark job is not fully shutdown.
{code:java}
23/03/13 11:16:06 INFO DFSPathSelector: Root path => /tmp/input_dataset/ source limit => 30000000
23/03/13 11:16:06 INFO DeltaSync: No new data, source checkpoint has not changed. Nothing to commit. Old checkpoint=(Option{val=1678730775000}). New Checkpoint=(1678730775000)
23/03/13 11:16:06 INFO NoNewDataTerminationStrategy: Shutting down on continuous mode as there is no new data for 3
23/03/13 11:16:12 INFO HoodieAsyncService: Waiting for next instant up to 10 seconds
23/03/13 11:16:22 INFO HoodieAsyncService: Waiting for next instant up to 10 seconds
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 410
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 638
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 567
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 618
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 591
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 577
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 620
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 539
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 424
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 429
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 516
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 622
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 503
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 446
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 460
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 547
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 606
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 435
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 625
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 601
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 630
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 405
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 495
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 500
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 559
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 520
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 621
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 588
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 431
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 423
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 534
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 603
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 631
23/03/13 11:16:31 INFO BlockManagerInfo: Removed broadcast_20_piece0 on 192.168.1.7:60400 in memory (size: 94.3 KB, free: 4.1 GB)
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 537
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 568
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 597
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 562
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 535
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 481
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 611
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 585
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 640
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 515
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 645
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 609
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 581
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 530
23/03/13 11:16:31 INFO BlockManagerInfo: Removed broadcast_23_piece0 on 192.168.1.7:60400 in memory (size: 25.7 KB, free: 4.1 GB)
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 602
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 650
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 469
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 517
23/03/13 11:16:31 INFO ContextCleaner: Cleaned accumulator 542 {code}
Job is just stuck there.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)