[jira] [Updated] (AVRO-4262) avro-tools recodec defaults to level -1 which is an incorrect level for xz and zstd

["Jonathan Lynch (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AVRO-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Lynch updated AVRO-4262:
---------------------------------
    Summary: avro-tools recodec defaults to level -1 which is an incorrect level for xz and zstd  (was: avro-tools recodec defaults to -1 incorrect levels for xz and zstd)

> avro-tools recodec defaults to level -1 which is an incorrect level for xz and zstd
> -----------------------------------------------------------------------------------
>
>                 Key: AVRO-4262
>                 URL: https://issues.apache.org/jira/browse/AVRO-4262
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: tools
>    Affects Versions: 1.12.1
>            Reporter: Jonathan Lynch
>            Priority: Major
>             Fix For: 1.13.0
>
>         Attachments: 0001-use-default-codec-compression-level-for-gzip-xz-and-.patch
>
>
> avro-tools recodec should explicitly set the default codec compression level for the gzip, xz and zstd codecs when the level parameter is not provided. The level defaults to -1 which is fine for gzip but is invalid for xz and this -1 value converts to level 1 for zstd silently which was the most frustrating.
> ```
> {code:java}
> java -jar avro-tools-1.12.1.jar recodec --codec xz input.avro output_conv_xz.avro
> Exception in thread "main" org.tukaani.xz.UnsupportedOptionsException: Unsupported preset: -1
> at org.tukaani.xz.LZMA2Options.setPreset(LZMA2Options.java:196)
> at org.tukaani.xz.LZMA2Options.<init>(LZMA2Options.java:157)
> at org.apache.commons.compress.compressors.xz.XZCompressorOutputStream.<init>(XZCompressorOutputStream.java:148)
> at org.apache.avro.file.XZCodec.compress(XZCodec.java:62)
> at org.apache.avro.file.DataFileStream$DataBlock.compressUsing(DataFileStream.java:384)
> at org.apache.avro.file.DataFileWriter.appendAllFrom(DataFileWriter.java:403)
> at org.apache.avro.tool.RecodecTool.run(RecodecTool.java:80)
> at org.apache.avro.tool.Main.run(Main.java:67)
> at org.apache.avro.tool.Main.main(Main.java:56)
>  
> Working with patch
> java -jar avro-tools-1.13.0-SNAPSHOT.jar recodec --codec xz input.avro output_conv_xz.avro
> java -jar avro-tools-1.13.0-SNAPSHOT.jar getmeta output_conv_xz.avro | grep codec
> avro.codec    xz
> {code}
> ```



--
This message was sent by Atlassian Jira
(v8.20.10#820010)