You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "vinoyang (Jira)" <ji...@apache.org> on 2020/03/05 08:52:00 UTC
[jira] [Updated] (HUDI-659) Make HoodieCommitArchiveLog spark free
[ https://issues.apache.org/jira/browse/HUDI-659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
vinoyang updated HUDI-659:
--------------------------
Description: Currently, {{HoodieCommitArchiveLog}} depends on {{JavaSparkContext}} in its two methods: {{archiveIfRequired}} and {{getInstantsToArchive}}. These two methods pass {{JavaSparkContext}} to get {{HoodieTable}} object. After diving into the call chain, I found we can replace {{JavaSparkContext}} with {{Configuration}} and other some cleanup(e.g. HUDI-658). After that, we can make {{HoodieCommitArchiveLog}} spark free. (was: Currently, {{HoodieCommitArchiveLog}} depends on {{JavaSparkContext}} in its two methods: {{archiveIfRequired}} and {{getInstantsToArchive}}. These two methods pass {{JavaSparkContext}} to get {{HoodieTable}} object. After diving into the call chain, I found we can replace {{JavaSparkContext}} with {{Configuration}} and other some cleanup. After that, we can make {{HoodieCommitArchiveLog}} spark free.)
> Make HoodieCommitArchiveLog spark free
> --------------------------------------
>
> Key: HUDI-659
> URL: https://issues.apache.org/jira/browse/HUDI-659
> Project: Apache Hudi (incubating)
> Issue Type: Sub-task
> Reporter: vinoyang
> Assignee: vinoyang
> Priority: Major
>
> Currently, {{HoodieCommitArchiveLog}} depends on {{JavaSparkContext}} in its two methods: {{archiveIfRequired}} and {{getInstantsToArchive}}. These two methods pass {{JavaSparkContext}} to get {{HoodieTable}} object. After diving into the call chain, I found we can replace {{JavaSparkContext}} with {{Configuration}} and other some cleanup(e.g. HUDI-658). After that, we can make {{HoodieCommitArchiveLog}} spark free.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)