[jira] [Created] (SPARK-45504) RocksDB State Store Should Lower RocksDB Background Thread CPU Priority

["Siying Dong (Jira)" <ji...@apache.org>]

Siying Dong created SPARK-45504:
-----------------------------------

             Summary: RocksDB State Store Should Lower RocksDB Background Thread CPU Priority
                 Key: SPARK-45504
                 URL: https://issues.apache.org/jira/browse/SPARK-45504
             Project: Spark
          Issue Type: Task
          Components: Structured Streaming
    Affects Versions: 3.5.1
            Reporter: Siying Dong


We can move RocksDB flush and compaction to lower CPU priority. They usually are background tasks and don't need to compete with task execution. For the case where a task may wait for some RocksDB background task to finish, such as checkpointing, or waiting async checkpointing to finish, the task slot is waiting so we are likely to have enough CPU for low pri CPU anyway.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org