[GitHub] [arrow] js8544 commented on issue #37509: ArrowTypeError: "object of type cannot be converted to int"

["js8544 (via GitHub)" <gi...@apache.org>]

js8544 commented on issue #37509:
URL: https://github.com/apache/arrow/issues/37509#issuecomment-1702302545

   Hi, this is because Pandas reads `event.data.timestamp` as a string, not a `Datetime`, therefore the error message.
   ```python
   >>> type(df['event_data'][0]['timestamp'])
   str
   ```
   
   You can however bypass Pandas by using only pyarrow:
   ```python
   import pyarrow as pa
   import pyarrow.parquet as pq
   
   SCHEMA = pa.schema([
       ('uuid', pa.string()),
       ('timestamp', pa.timestamp('s', tz='UTC')),
       ('version', pa.float64()),
       ('event_data', pa.struct([
           ('amount', pa.float64()),
           ('user', pa.int64()),
           ('timestamp', pa.timestamp('s', tz='UTC')),
       ])),
   ])
   
   table = pa.json.read_json('test.jsonl')
   print(table)
   pq.write_table(table, 'test.parquet')
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org