You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2022/02/03 13:31:00 UTC

[jira] [Updated] (HUDI-3367) Add spark scheduling configs support to spark streaming writes

     [ https://issues.apache.org/jira/browse/HUDI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

sivabalan narayanan updated HUDI-3367:
--------------------------------------
    Summary: Add spark scheduling configs support to spark streaming writes  (was: Add spark scheduling configs support to spark data source writer)

> Add spark scheduling configs support to spark streaming writes
> --------------------------------------------------------------
>
>                 Key: HUDI-3367
>                 URL: https://issues.apache.org/jira/browse/HUDI-3367
>             Project: Apache Hudi
>          Issue Type: Task
>          Components: spark
>            Reporter: sivabalan narayanan
>            Assignee: sivabalan narayanan
>            Priority: Blocker
>             Fix For: 0.11.0
>
>
> We have support to set scheduling configs in spark for deltastreamer. this was added so that we can allocate resources efficiently between regular delta syncs and async table services like compaction. We don't have that support in spark data source writer. we might need the support to keep it flexible for users. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)