You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Siying Dong (Jira)" <ji...@apache.org> on 2023/10/11 22:48:00 UTC
[jira] [Created] (SPARK-45503) RocksDB State Store to Use LZ4 Compression
Siying Dong created SPARK-45503:
-----------------------------------
Summary: RocksDB State Store to Use LZ4 Compression
Key: SPARK-45503
URL: https://issues.apache.org/jira/browse/SPARK-45503
Project: Spark
Issue Type: Task
Components: Structured Streaming
Affects Versions: 3.5.1
Reporter: Siying Dong
LZ4 is generally faster than Snappy. That's probably we use LZ4 in changelogs and other places by default. However, we don't change RocksDB's default of Snappy compression style. The RocksDB Team recommend LZ4 or ZSTD and the default is kept to Snappy only for backward compatible reason. We should use LZ4 instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org