[GitHub] [arrow-datafusion] matthewgapp commented on issue #7636: CREATE TABLE DDL does not save correct schema, resulting in mismatched plan vs execution (record batch) schema

["matthewgapp (via GitHub)" <gi...@apache.org>]

matthewgapp commented on issue #7636:
URL: https://github.com/apache/arrow-datafusion/issues/7636#issuecomment-1734577809

   > I wonder if the nullable information isn't being updated correctly for join output 🤔
   > 
   > The code that handles setting nullability is here:
   > 
   > https://github.com/apache/arrow-datafusion/blob/b1d134e9ff37724459cb5090a6346a85152a1db7/datafusion/expr/src/logical_plan/builder.rs#L1068
   > 
   > Maybe somehow the plans in question aren't using that function or there is a bug in that function
   
   I don't think this is the case because the schema is incorrect at create table time (before joins). Joins just seem to be the place where DF complains. 
   
   If you inspect the schema of the created table, the row_num column (created by the ROW_NUMBER() window function) is nullable: true when it should be nullable false, per the [field implementation on the row number execution plan](https://github.com/apache/arrow-datafusion/blob/78d9613e81557ca5e5db8b75e5c7dec47ccee0a1/datafusion/physical-expr/src/window/row_number.rs#L54). 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org