[jira] [Assigned] (ARROW-16651) [Python] Casting Table to new schema ignores nullability of fields

["Kshiteej K (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-16651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kshiteej K reassigned ARROW-16651:
----------------------------------

    Assignee: Kshiteej K

> [Python] Casting Table to new schema ignores nullability of fields
> ------------------------------------------------------------------
>
>                 Key: ARROW-16651
>                 URL: https://issues.apache.org/jira/browse/ARROW-16651
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>            Reporter: Joris Van den Bossche
>            Assignee: Kshiteej K
>            Priority: Major
>              Labels: good-first-issue, good-second-issue, kernel
>
> Similar to ARROW-15478, but not for nested fields but just for casting a full Table (in theory that could be the same code, but currently the Table.cast logic is implemented in cython). 
> So currently when casting a Table to a new schema, the nullability of the fields in the schema is ignored (and as a result you get an "invalid" schema indicating a field is non-nullable that actually can have nulls):
> {code}
> >>> table = pa.table({'a': [None, 1]})
> >>> table
> pyarrow.Table
> a: int64
> ----
> a: [[null,1]]
> >>> new_schema = pa.schema([pa.field("a", "int64", nullable=False)])
> >>> table.cast(new_schema)
> pyarrow.Table
> a: int64 not null
> ----
> a: [[null,1]]
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)