You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2022/02/14 20:26:00 UTC

[jira] [Updated] (HUDI-3425) Clean up spill path created by Hudi during uneventful shutdown

     [ https://issues.apache.org/jira/browse/HUDI-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

sivabalan narayanan updated HUDI-3425:
--------------------------------------
    Priority: Critical  (was: Major)

> Clean up spill path created by Hudi during uneventful shutdown
> --------------------------------------------------------------
>
>                 Key: HUDI-3425
>                 URL: https://issues.apache.org/jira/browse/HUDI-3425
>             Project: Apache Hudi
>          Issue Type: Task
>          Components: compaction
>            Reporter: sivabalan narayanan
>            Priority: Critical
>
> h1. Hudi spill path not getting cleared when containers getting killed abruptly. 
>  
> When yarn kills the containers abruptly for any reason while hudi stage is in progress then the spill path created by hudi on the disk is not cleaned and as a result of which the nodes on the cluster start running out of space. We need to clear the spill path manually to free out disk space.
>  
> Ref issue: https://github.com/apache/hudi/issues/4771



--
This message was sent by Atlassian Jira
(v8.20.1#820001)