You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2022/02/14 20:26:00 UTC

[jira] [Created] (HUDI-3425) Clean up spill path created by Hudi during uneventful shutdown

sivabalan narayanan created HUDI-3425:
-----------------------------------------

             Summary: Clean up spill path created by Hudi during uneventful shutdown
                 Key: HUDI-3425
                 URL: https://issues.apache.org/jira/browse/HUDI-3425
             Project: Apache Hudi
          Issue Type: Task
          Components: compaction
            Reporter: sivabalan narayanan


h1. Hudi spill path not getting cleared when containers getting killed abruptly. 

 

When yarn kills the containers abruptly for any reason while hudi stage is in progress then the spill path created by hudi on the disk is not cleaned and as a result of which the nodes on the cluster start running out of space. We need to clear the spill path manually to free out disk space.

 

Ref issue: https://github.com/apache/hudi/issues/4771



--
This message was sent by Atlassian Jira
(v8.20.1#820001)