You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Balaji Varadarajan (Jira)" <ji...@apache.org> on 2020/04/29 19:31:00 UTC

[jira] [Created] (HUDI-846) Turn on incremental cleaning bu default in 0.6.0

Balaji Varadarajan created HUDI-846:
---------------------------------------

             Summary: Turn on incremental cleaning bu default in 0.6.0
                 Key: HUDI-846
                 URL: https://issues.apache.org/jira/browse/HUDI-846
             Project: Apache Hudi (incubating)
          Issue Type: Improvement
          Components: Cleaner
            Reporter: Balaji Varadarajan
             Fix For: 0.6.0


Incremental cleaner will track commits that have happened since the last clean operation to figure out partitions which needs to be scanned for cleaning. This avoids the costly scanning of all partition paths.

Incremental cleaning is currently disabled by default. We need to enable it by default in 0.6.0.

No special handling is required for upgrade/downgrade scenarios as incremental cleaning relies on standard format of commit metadata 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)