You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "Neville Dipale (Jira)" <ji...@apache.org> on 2021/01/25 18:20:00 UTC

[jira] [Created] (ARROW-11381) [Rust] [Parquet] LZ4 compressed files written in Rust can't be opened with C++

Neville Dipale created ARROW-11381:
--------------------------------------

             Summary: [Rust] [Parquet] LZ4 compressed files written in Rust can't be opened with C++
                 Key: ARROW-11381
                 URL: https://issues.apache.org/jira/browse/ARROW-11381
             Project: Apache Arrow
          Issue Type: Bug
    Affects Versions: 3.0.0
            Reporter: Neville Dipale


Parquet files that are written with LZ4 compression, cannot be read from pyarrow. It seems that the issue might be the LZ4 block vs frame, which we're also seeing in ARROW-8767.

I'll update this JIRA with more info, as I'm struggling to get pyspark up on MacOS (Rosetta 2 issues)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)