You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@arrow.apache.org by "Antoine Pitrou (Jira)" <ji...@apache.org> on 2019/09/18 11:07:00 UTC

[jira] [Updated] (ARROW-5915) [C++] [Python] Set up testing for backwards compatibility of the parquet reader

     [ https://issues.apache.org/jira/browse/ARROW-5915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Antoine Pitrou updated ARROW-5915:
----------------------------------
    Fix Version/s: 1.0.0

> [C++] [Python] Set up testing for backwards compatibility of the parquet reader
> -------------------------------------------------------------------------------
>
>                 Key: ARROW-5915
>                 URL: https://issues.apache.org/jira/browse/ARROW-5915
>             Project: Apache Arrow
>          Issue Type: Test
>          Components: C++, Python
>            Reporter: Joris Van den Bossche
>            Priority: Major
>              Labels: parquet
>             Fix For: 1.0.0
>
>
> Given the recent parquet compat problems, we should have better testing for this.
> For easy testing of backwards compatibility, we could add some files (with different types) written with older versions, and ensure they are read correctly with the current version.
> Similarly as what Kartothek is doing: https://github.com/JDASoftwareGroup/kartothek/tree/master/reference-data/arrow-compat
> An easy way would be to do that in pyarrow and add them to /pyarrow/tests/data/parquet (we already have some files from 0.7 there). 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)