You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Ethan Guo (Jira)" <ji...@apache.org> on 2022/01/15 06:34:00 UTC

[jira] [Resolved] (HUDI-659) Make HoodieCommitArchiveLog spark free

     [ https://issues.apache.org/jira/browse/HUDI-659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ethan Guo resolved HUDI-659.
----------------------------

> Make HoodieCommitArchiveLog spark free
> --------------------------------------
>
>                 Key: HUDI-659
>                 URL: https://issues.apache.org/jira/browse/HUDI-659
>             Project: Apache Hudi
>          Issue Type: Task
>          Components: Writer Core
>            Reporter: vinoyang
>            Assignee: vinoyang
>            Priority: Major
>
> Currently, {{HoodieCommitArchiveLog}} depends on {{JavaSparkContext}} in its two methods: {{archiveIfRequired}} and {{getInstantsToArchive}}. These two methods pass {{JavaSparkContext}} to get {{HoodieTable}} object. After diving into the call chain, I found we can replace {{JavaSparkContext}} with {{Configuration}} and other some cleanup(e.g. HUDI-658). After that, we can make {{HoodieCommitArchiveLog}} spark free.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)