[GitHub] [incubator-superset] robdiciuccio commented on issue #8225: Pandas casting int64 to float64, misrepresenting value

[GitBox <gi...@apache.org>]

robdiciuccio commented on issue #8225: Pandas casting int64 to float64, misrepresenting value
URL: https://github.com/apache/incubator-superset/issues/8225#issuecomment-554224178
 
 
   Support for PyArrow serialization of Pandas Int64 dtypes is currently merged to master in both repos, but not yet released on PyPi:
   
   https://github.com/pandas-dev/pandas/commit/34fff1f336d3b083dd09f5036c2bb9b80edfb619
   https://github.com/apache/arrow/commit/7f4165c4757be6e53bdf80d5df3119bbe1ae6c54
   
   Also requires converting the pandas Dataframe to an arrow Table prior to serialization:
   ```
   table = pa.Table.from_pandas(cdf.raw_df)
   data = (
      pa.default_serialization_context()
      .serialize(table)
      .to_buffer()
      .to_pybytes()
   )
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org