[jira] [Commented] (IMPALA-11549) Incorrent return type mismatch in Hive generic UDFs

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-11549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17696977#comment-17696977 ] 

ASF subversion and git services commented on IMPALA-11549:
----------------------------------------------------------

Commit 67bb870aa302b3509fa4a0f8d846efedc04e1514 in impala's branch refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=67bb870aa ]

IMPALA-11911: Fix NULL argument handling in Hive GenericUDFs

Before this patch if an argument of a GenericUDF was NULL, then Impala
passed it as null instead of a DeferredObject. This was incorrect, as
a DeferredObject is expected with a get() function that returns null.
See the Jira for more details and GenericUDF examples in Hive.

TestGenericUdf's NULL handling was further broken in IMPALA-11549,
leading to throwing null pointer exceptions when the UDF's result is
NULL. This test bug was not detected, because Hive udf tests were
running with default abort_java_udf_on_exception=false, which means
that exceptions from Hive UDFs only led to warnings and returning NULL,
which was the expected result in all affected test queries.

This patch fixes the behavior in HiveUdfExecutorGeneric and improves
FE/EE tests to catch null handling related issues. Most Hive UDF tests
are run with abort_java_udf_on_exception=true after this patch to treat
exceptions in UDFs as errors. The ones where the test checks that NULL
is returned if an exception is thrown while abort_java_udf_on_exception
is false are moved to new .test files.
TestGenericUdf is also fixed (and simplified) to handle NULL return
values correctly.

Change-Id: I53238612f4037572abb6d2cc913dd74ee830a9c9
Reviewed-on: http://gerrit.cloudera.org:8080/19499
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Incorrent return type mismatch in Hive generic UDFs
> ---------------------------------------------------
>
>                 Key: IMPALA-11549
>                 URL: https://issues.apache.org/jira/browse/IMPALA-11549
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Frontend
>    Affects Versions: Impala 4.2.0
>            Reporter: Csaba Ringhofer
>            Priority: Major
>             Fix For: Impala 4.3.0
>
>
> The valid looking function creation returned an error:
> CREATE FUNCTION IF NOT EXISTS st_bin(bigint, binary) returns bigint LOCATION ' ...' SYMBOL='org.apache.hadoop.hive.ql.udf.esri.ST_Bin';
> ERROR: CatalogException: Error retrieving class class org.apache.hadoop.hive.ql.udf.esri.ST_Bin: Function expected return type bigint but was created with BIGINT
> I think that issue is that we use =! to check for mismatch:
> https://github.com/apache/impala/blob/fec7a79c50de9ef5db2c652ece111aaa303d57c7/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java#L149
> Comparison of ObjectInspectors should be done in a more complex way, see
> https://github.com/apache/hive/blob/d06957f254e026e719f30027d161264be43386b0/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java#L1325



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org