[jira] [Commented] (ARROW-9648) [C++] LZ4 compression level not supported

["Antoine Pitrou (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-9648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17448181#comment-17448181 ] 

Antoine Pitrou commented on ARROW-9648:
---------------------------------------

It seems the compression level is only settable for the LZ4 frame format. This wouldn't apply to Parquet therefore (which uses the LZ4 block format), but would apply to Arrow IPC (which uses the LZ4 frame format).

> [C++] LZ4 compression level not supported
> -----------------------------------------
>
>                 Key: ARROW-9648
>                 URL: https://issues.apache.org/jira/browse/ARROW-9648
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++
>            Reporter: Pierre
>            Priority: Minor
>
> Hi,
> I am currently checking the different compression codecs available when writing parquet files.
> Testing LZ4, I got the error message that compression level is not supported.
> {code:python}
> from pyarrow import parquet as pq
> pq.write_table(data, file, compression='LZ4', compression_level=12)
> {code}
> Error message
> {code:bash}
>  File "pyarrow/_parquet.pyx", line 1374, in pyarrow._parquet.ParquetWriter.write_table
> File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
> OSError: Invalid: LZ4 doesn't support setting a compression level.
> {code}
> However man page of LZ4 states the following
> {code:bash}
> Operation modifiers
>  -# Compression level, with # being any value from 1 to 12. Higher
>  values trade compression speed for compression ratio. Values
>  above 12 are considered the same as 12. Recommended values are 1
>  for fast compression (default), and 9 for high compression.
>  Speed/compression trade-off will vary depending on data to com‐
>  press. Decompression speed remains fast at all settings.
> {code}
> Please, can you activate compression levels for LZ4?
> Thanks,
> Bests
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)