[jira] [Commented] (ARROW-4359) [Python] Column metadata is not saved or loaded in parquet

["Antoine Pitrou (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-4359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17286049#comment-17286049 ] 

Antoine Pitrou commented on ARROW-4359:
---------------------------------------

If I try [~jorisvandenbossche]'s reproducer, it seems to work now:
{code:python}
>>> path = "xxx.pq"
>>> field1 = pa.field('field1', pa.int64(), metadata=dict(a="A", b="B"))
>>> field2 = pa.field('field2', pa.int64(), nullable=False)
>>> table = pa.Table.from_arrays([pa.array([1, 2]), pa.array([3, 4])], schema=pa.schema([field1, field2]))
>>> 
>>> pq.write_table(table, path)
>>> table2 = pq.read_table(path)
>>> 
>>> table2.schema.field('field1')
pyarrow.Field<field1: int64>
>>> table2.schema.field('field2')
pyarrow.Field<field2: int64 not null>
>>> table2.schema.field('field1').metadata
{b'PARQUET:field_id': b'1', b'a': b'A', b'b': b'B'}
>>> table2.schema.field('field2').metadata
{b'PARQUET:field_id': b'2'}
{code}


> [Python] Column metadata is not saved or loaded in parquet
> ----------------------------------------------------------
>
>                 Key: ARROW-4359
>                 URL: https://issues.apache.org/jira/browse/ARROW-4359
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>            Reporter: Seb Fru
>            Priority: Major
>              Labels: parquet
>             Fix For: 4.0.0
>
>
> Hi all,
> a while ago I posted this issue: ARROW-3866
> While working with Pyarrow I encountered another potential bug related to column metadata: If I create a table containing columns with metadata everything is fine. But after I save the table to parquet and load it back as a table using pq.read_table, the column metadata is gone.
>  
> As of now I can not say yet whether the metadata is not saved correctly or not loaded correctly, as I have no idea how to verify it. Unfortunately I also don't have the time try a lot, but I wanted to let you know anyway. 
>  
> {code}
> field0 = pa.field('field1', pa.int64(), metadata=dict(a="A", b="B"))
> field1 = pa.field('field2', pa.int64(), nullable=False)
> columns = [
>     pa.column(field0, pa.array([1, 2])),
>     pa.column(field1, pa.array([3, 4]))
> ]
> table = pa.Table.from_arrays(columns)
> pq.write_table(tab, path)
> tab2 = pq.read_table(path)
> tab2.column(0).field.metadata
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)