You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2022/02/09 16:33:00 UTC

[jira] [Closed] (HUDI-3367) Add spark scheduling configs support to spark streaming writes

     [ https://issues.apache.org/jira/browse/HUDI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

sivabalan narayanan closed HUDI-3367.
-------------------------------------
    Resolution: Fixed

> Add spark scheduling configs support to spark streaming writes
> --------------------------------------------------------------
>
>                 Key: HUDI-3367
>                 URL: https://issues.apache.org/jira/browse/HUDI-3367
>             Project: Apache Hudi
>          Issue Type: Task
>          Components: spark
>            Reporter: sivabalan narayanan
>            Assignee: sivabalan narayanan
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 0.11.0
>
>
> We have support to set scheduling configs in spark for deltastreamer. this was added so that we can allocate resources efficiently between regular delta syncs and async table services like compaction. We don't have that support in spark streaming writes. we might need the support since spark streaming also supports async table services. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)