You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Jim Fulton (Jira)" <ji...@apache.org> on 2022/01/01 19:30:00 UTC
[jira] [Closed] (ARROW-15233) pyarrow.dataset.dataset loses type information when reading parquet files
[ https://issues.apache.org/jira/browse/ARROW-15233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Fulton closed ARROW-15233.
------------------------------
Resolution: Incomplete
> pyarrow.dataset.dataset loses type information when reading parquet files
> -------------------------------------------------------------------------
>
> Key: ARROW-15233
> URL: https://issues.apache.org/jira/browse/ARROW-15233
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 6.0.1
> Environment: Ubuntu 20.2, Python 3.8.10
> Reporter: Jim Fulton
> Priority: Major
>
> When reading a parquet containing time data:
>
> {{>>> import pyarrow.dataset}}
> {{>>> ds = pyarrow.dataset.dataset('foo.parquet', format='parquet')}}
> {{>>> ds.schema[1].type}}
> {{DataType(time32[ms])}}
> I get {{DataType}} rather than {{{}Time32Type{}}}, which means I can't query time units.
> I assume this is an issue for other non-basic types.
> I can write code to scrape the type's string representation. Is there a better way?
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)