You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "marc-pydantic (via GitHub)" <gi...@apache.org> on 2025/10/27 13:13:09 UTC

[I] `FIRST_VALUE` and `LAST_VALUE` internally do not handle NULL values in the `is_set` column [datafusion]

marc-pydantic opened a new issue, #18300:
URL: https://github.com/apache/datafusion/issues/18300

   When the `FIRST_VALUE` or `LAST_VALUE` SQL function implementation's `merge_batch` functions are called, the second column (`is_set`) is used for filtering but it is not checked for `NULL` values. The current implementation (although this seems to be largely undocumented) assumes `false` for every NULL value.
   
   Admittedly this should never happen during normal operation, but while breaking other things, I have encountered this masking a data corruption bug at least once.
   
   I'm proposing a check for NULLnes on these columns, as it is relatively cheap and help catch internal errors earlier.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] `FIRST_VALUE` and `LAST_VALUE` internally do not handle NULL values in the `is_set` column [datafusion]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey closed issue #18300: `FIRST_VALUE` and `LAST_VALUE` internally do not handle NULL values in the `is_set` column
URL: https://github.com/apache/datafusion/issues/18300


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org