[jira] [Created] (ARROW-16651) [Python] Casting Table to new schema ignores nullability of fields

["Joris Van den Bossche (Jira)" <ji...@apache.org>]

Joris Van den Bossche created ARROW-16651:
---------------------------------------------

             Summary: [Python] Casting Table to new schema ignores nullability of fields
                 Key: ARROW-16651
                 URL: https://issues.apache.org/jira/browse/ARROW-16651
             Project: Apache Arrow
          Issue Type: Bug
          Components: Python
            Reporter: Joris Van den Bossche


Similar to ARROW-15478, but not for nested fields but just for casting a full Table (in theory that could be the same code, but currently the Table.cast logic is implemented in cython). 

So currently when casting a Table to a new schema, the nullability of the fields in the schema is ignored (and as a result you get an "invalid" schema indicating a field is non-nullable that actually can have nulls):

{code}
>>> table = pa.table({'a': [None, 1]})
>>> table
pyarrow.Table
a: int64
----
a: [[null,1]]

>>> new_schema = pa.schema([pa.field("a", "int64", nullable=False)])
>>> table.cast(new_schema)
pyarrow.Table
a: int64 not null
----
a: [[null,1]]
{code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)