[jira] [Commented] (ARROW-11793) [C++][Parquet] dictinct_count broken (always 0) in column chunk statistics

["David Li (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-11793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17524486#comment-17524486 ] 

David Li commented on ARROW-11793:
----------------------------------

From a quick grep it seems it's just never set by the writer in the first place. Possibly a Parquet file not written by Arrow C++ would set this.

> [C++][Parquet] dictinct_count broken (always 0) in column chunk statistics
> --------------------------------------------------------------------------
>
>                 Key: ARROW-11793
>                 URL: https://issues.apache.org/jira/browse/ARROW-11793
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++, Parquet, Python
>    Affects Versions: 3.0.0
>            Reporter: ARF
>            Priority: Minor
>              Labels: parquet-statistics
>
> The {{distinct_count}} attribute of the column chunk metadata statistics is broken: It always shows 0. This seems to be the case for all types of columns. Checked with int64 as well as dictionary encoded string columns:
> {code:python}
> import pyarrow as pa
> import pyarrow.parquet as pq
> table = pa.Table.from_pydict({
>     'foo': pa.array(['ABC', 'DEF']).dictionary_encode()
> })
> pq.write_table(table, 'test_row_group_statistics.parquet', version='2.0', data_page_version='2.0')
> pq_file = pq.ParquetFile('test_row_group_statistics.parquet')
> print(pq_file.metadata.row_group(0).column(0).statistics)
> {code}
> Output:
> {code:none}
> <pyarrow._parquet.Statistics object at 0x0000020A1699D770>
>   has_min_max: True
>   min: ABC
>   max: DEF
>   null_count: 0
>   distinct_count: 0
>   num_values: 2
>   physical_type: BYTE_ARRAY
>   logical_type: String
>   converted_type (legacy): UTF8
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)