You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/07/24 12:26:35 UTC
[GitHub] [spark] zzzzming95 commented on pull request #37263: [SPARK-39743][SQL] Unable to set zstd compression level while writing…
zzzzming95 commented on PR #37263:
URL: https://github.com/apache/spark/pull/37263#issuecomment-1193308223
@c21 Thanks Reply~
The default values of `spark.sql.parquet.zstd.level` and `spark.sql.parquet.zstd.worker` are set to 3 and 0 respectively, because this is the default parameter of `parquet-mr`.
detail in `parquet-mr` : https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/README.md
In `parquet-mr`, zstd level and zstd workers correspond to parameters `parquet.compression.codec.zstd.level` and `parquet.compression.codec.zstd.workers` respectively.
When no parameters were specified before, the default parameters for zstd level and zstd workers are 3 and 0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org