You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "vinoyang (Jira)" <ji...@apache.org> on 2020/03/05 08:52:00 UTC

[jira] [Updated] (HUDI-659) Make HoodieCommitArchiveLog spark free

     [ https://issues.apache.org/jira/browse/HUDI-659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

vinoyang updated HUDI-659:
--------------------------
    Description: Currently, {{HoodieCommitArchiveLog}} depends on {{JavaSparkContext}} in its two methods: {{archiveIfRequired}} and {{getInstantsToArchive}}. These two methods pass {{JavaSparkContext}} to get {{HoodieTable}} object. After diving into the call chain, I found we can replace {{JavaSparkContext}} with {{Configuration}} and other some cleanup(e.g. HUDI-658). After that, we can make {{HoodieCommitArchiveLog}} spark free.  (was: Currently, {{HoodieCommitArchiveLog}} depends on {{JavaSparkContext}} in its two methods: {{archiveIfRequired}} and {{getInstantsToArchive}}. These two methods pass {{JavaSparkContext}} to get {{HoodieTable}} object. After diving into the call chain, I found we can replace {{JavaSparkContext}} with {{Configuration}} and other some cleanup. After that, we can make {{HoodieCommitArchiveLog}} spark free.)

> Make HoodieCommitArchiveLog spark free
> --------------------------------------
>
>                 Key: HUDI-659
>                 URL: https://issues.apache.org/jira/browse/HUDI-659
>             Project: Apache Hudi (incubating)
>          Issue Type: Sub-task
>            Reporter: vinoyang
>            Assignee: vinoyang
>            Priority: Major
>
> Currently, {{HoodieCommitArchiveLog}} depends on {{JavaSparkContext}} in its two methods: {{archiveIfRequired}} and {{getInstantsToArchive}}. These two methods pass {{JavaSparkContext}} to get {{HoodieTable}} object. After diving into the call chain, I found we can replace {{JavaSparkContext}} with {{Configuration}} and other some cleanup(e.g. HUDI-658). After that, we can make {{HoodieCommitArchiveLog}} spark free.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)