You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "Ildar (JIRA)" <ji...@apache.org> on 2019/01/21 10:46:00 UTC
[jira] [Commented] (PARQUET-1494) [C++] Can't access parquet
statistics on binary columns
[ https://issues.apache.org/jira/browse/PARQUET-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16747842#comment-16747842 ]
Ildar commented on PARQUET-1494:
--------------------------------
I've created a pull request for this issue: https://github.com/apache/arrow/pull/3441
> [C++] Can't access parquet statistics on binary columns
> -------------------------------------------------------
>
> Key: PARQUET-1494
> URL: https://issues.apache.org/jira/browse/PARQUET-1494
> Project: Parquet
> Issue Type: Bug
> Components: parquet-cpp
> Reporter: Ildar
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hi,
> I'm trying to use per-column statistics (min/max values) to filter out row groups while reading parquet file. But I don't see statistics built for binary columns. I noticed that {{ApplicationVersion::HasCorrectStatistics()}} discards statistics that have sort order {{UNSIGNED and haven't been created by parquet-cpp}}. As I understand there used to be some issues in {{parquet-mr}} before. But do they still persist?
> For example, I have parquet file created with {{parquet-mr}} version 1.10, it seems to have correct min/max values for binary columns. And {{parquet-cpp}} works fine for me if I remove this code from {{HasCorrectStatistics()}} func:
>
> {code:java}
> if (SortOrder::SIGNED != sort_order && !max_equals_min) {
> return false;
> }{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)