[jira] [Work logged] (HIVE-24908) Adding Respect/Ignore nulls as a UDAF parameter is ambiguous

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-24908?focusedWorklogId=568962&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-568962 ]

ASF GitHub Bot logged work on HIVE-24908:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 19/Mar/21 14:29
            Start Date: 19/Mar/21 14:29
    Worklog Time Spent: 10m 
      Work Description: kasakrisz opened a new pull request #2091:
URL: https://github.com/apache/hive/pull/2091


   ### What changes were proposed in this pull request?
   1. Add `respectNulls` boolean property to `GenericUDAFParameterInfo`.
   2. Map this property to Windowing specification/definitions objects.
   3. Introduce property to `WindowFunctionDescription` which show whether the function supports handling null treatment settings. 
   * If the `supportNullTreatment` property is not set explicitly the UDAF does not support null treatment and the compiler throws `SemanticException` if specified.
   * If the `supportNullTreatment` property is set to true but the function call does not specify null treatment the default null treatment is used: `RESPECT NULLS`
   * The specified null treatment is used othervise.
   
   ### Why are the changes needed?
   This patch is a follow-up of #2060. See jira for details.
   
   ### Does this PR introduce _any_ user-facing change?
   Yes. When calling `lead` or `lag` functions by specifying `RESPECT/IGNORE NULLS`
   * and no default value parameter is given `RESPECT/IGNORE NULLS` are not passed as default value parameter: `NULL` is returned if rows has no corresponding following/preceding rows 
   * and default value parameter is given `RESPECT/IGNORE NULLS` are not passed as default value parameter: no Exception is thrown
   
   ### How was this patch tested?
   ```
   mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestNegativeLlapLocalCliDriver -Dqfile=nulltreatment.q -pl itests/qtest -Pitests
   mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=windowing_navfn.q -pl itests/qtest -Pitests
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 568962)
    Remaining Estimate: 0h
            Time Spent: 10m

> Adding Respect/Ignore nulls as a UDAF parameter is ambiguous
> ------------------------------------------------------------
>
>                 Key: HIVE-24908
>                 URL: https://issues.apache.org/jira/browse/HIVE-24908
>             Project: Hive
>          Issue Type: Bug
>          Components: UDF
>            Reporter: Krisztian Kasa
>            Assignee: Krisztian Kasa
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Both function calls translated to the same UDAF call:
> {code}
> SELECT lead(a, 2, true) ...
> SELECT lead(a, 2) IGNORE NULLS ...
> {code}
> IGNORE NULLS is passed as an extra constant boolean parameter to the UDAF
> https://github.com/apache/hive/blob/eed78dfdcb6dfc2de400397a60de12e6f62b96e2/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java#L743
> However the semantics of the two function calls has different semantics:
> * *lead(a, 2, true)* - 'true' is the default value: "The value of DEFAULT is returned as the result if there is no row corresponding to the OFFSET number of rows before R within P (for the lag function) or after R within P (for the lead function)"
> * *lead(a, 2) IGNORE NULLS* - For each row in the current window find the 2nd not-NULL value starting directly after the current row. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)