You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2023/10/11 23:03:00 UTC

[jira] [Updated] (SPARK-45504) RocksDB State Store Should Lower RocksDB Background Thread CPU Priority

     [ https://issues.apache.org/jira/browse/SPARK-45504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated SPARK-45504:
-----------------------------------
    Labels: pull-request-available  (was: )

> RocksDB State Store Should Lower RocksDB Background Thread CPU Priority
> -----------------------------------------------------------------------
>
>                 Key: SPARK-45504
>                 URL: https://issues.apache.org/jira/browse/SPARK-45504
>             Project: Spark
>          Issue Type: Task
>          Components: Structured Streaming
>    Affects Versions: 3.5.1
>            Reporter: Siying Dong
>            Priority: Minor
>              Labels: pull-request-available
>
> We can move RocksDB flush and compaction to lower CPU priority. They usually are background tasks and don't need to compete with task execution. For the case where a task may wait for some RocksDB background task to finish, such as checkpointing, or waiting async checkpointing to finish, the task slot is waiting so we are likely to have enough CPU for low pri CPU anyway.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org