Re: [PR] ORC-1463: [Java] Support Brotli codec [orc]

["cxzl25 (via GitHub)" <gi...@apache.org>]

cxzl25 commented on PR #1714:
URL: https://github.com/apache/orc/pull/1714#issuecomment-1999090583

   > For Spark community, I'll exclude it officially and add a documentation later to guide the users to add that dependency from application side.
   
   
   Currently, the master branch of Spark has used ORC 2.0.0 version and supports brotli compression encoding. (SPARK-44115)
   However, due to some hardcode checks in Spark, brotli cannot be used normally. 
   
   Can I propose a PR that Spark supports ORC brotli ? 
    https://github.com/cxzl25/spark/commit/0cd5d03116709ec86d9f96e269200f2d05f799ca
   
   ```sql
   set spark.sql.orc.compression.codec=BROTLI;
   ```
   ```
   java.lang.IllegalArgumentException: The value of spark.sql.orc.compression.codec should be one of uncompressed, lz4, lzo, snappy, zlib, none, zstd, but was brotli
   ```
   
   By the way, `netty-codec` declares brotli-related dependencies for multiple OSs.
   
   https://github.com/netty/netty/blob/3cd364107167600e8eb4b0b85553ed895519e2ed/codec/pom.xml#L91-L125
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@orc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org