You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Sagar Sumit (Jira)" <ji...@apache.org> on 2022/10/03 15:07:00 UTC

[jira] [Created] (HUDI-4974) Avoid creating Configuration copies in Hudi

Sagar Sumit created HUDI-4974:
---------------------------------

             Summary: Avoid creating Configuration copies in Hudi
                 Key: HUDI-4974
                 URL: https://issues.apache.org/jira/browse/HUDI-4974
             Project: Apache Hudi
          Issue Type: Improvement
            Reporter: Sagar Sumit
             Fix For: 0.13.0


See [https://github.com/apache/hudi/issues/6332] for more details.

The reason for creating copies was to avoid ConcurrentModificationException encountered while long-running deltastreamer jobs. To avoid current modification, table meta client and file system get exclusive copies of the configuration. See [#755|https://github.com/apache/hudi/pull/755] for context.

But this breaks when a wrapper configuration object is given to the constructor, e.g. HIveCachingHdfsConfiguration from Presto. Is there a way to avoid creating copies and make it work with whatever the engine provides?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)