[jira] [Closed] (ARROW-2679) pyarrow table from/to dataframe is type-lossy

["Uwe L. Korn (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe L. Korn closed ARROW-2679.
------------------------------
    Resolution: Duplicate

This is intended behaviour (to get the best performance for the user). As not everyone wants this behaviour, we should introduce a flag, see also https://issues.apache.org/jira/browse/ARROW-2646

> pyarrow table from/to dataframe is type-lossy
> ---------------------------------------------
>
>                 Key: ARROW-2679
>                 URL: https://issues.apache.org/jira/browse/ARROW-2679
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>    Affects Versions: 0.9.0
>            Reporter: Rob Ambalu
>            Priority: Major
>
> (Note: Edited after realizing this is unrelated to parquet )
>  
> While streaming a dataframe -> pyarrow -> back to dataframe I noticed that my date column had its type information switch from "object" ( which would have loaded it as a date I would imagine ) to "datetime":
>  
> {code:java}
> from datetime import date
> import pandas as pd
> import pyarrow as pa
> df = pd.DataFrame( { 'a' : [ date( 2017, 1, 1), date( 2017, 2, 1 ) ] })
> table = pa.Table.from_pandas( df )
> df2 = table.to_pandas()
> >>> df['a'].dtype
> dtype('O')
> >>> df2['a'].dtype
> dtype('<M8[ns]')
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)