You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2023/10/11 22:50:00 UTC

[jira] [Updated] (SPARK-45503) RocksDB State Store to Use LZ4 Compression

     [ https://issues.apache.org/jira/browse/SPARK-45503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated SPARK-45503:
-----------------------------------
    Labels: pull-request-available  (was: )

> RocksDB State Store to Use LZ4 Compression
> ------------------------------------------
>
>                 Key: SPARK-45503
>                 URL: https://issues.apache.org/jira/browse/SPARK-45503
>             Project: Spark
>          Issue Type: Task
>          Components: Structured Streaming
>    Affects Versions: 3.5.1
>            Reporter: Siying Dong
>            Priority: Minor
>              Labels: pull-request-available
>
> LZ4 is generally faster than Snappy. That's probably we use LZ4 in changelogs and other places by default. However, we don't change RocksDB's default of Snappy compression style. The RocksDB Team recommend LZ4 or ZSTD and the default is kept to Snappy only for backward compatible reason. We should use LZ4 instead.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org