You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Balazs Hevele (Code Review)" <ge...@cloudera.org> on 2026/02/11 14:03:07 UTC
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Balazs Hevele has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23963
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The constantness flags for arguments are also sent to the frontend,
where appropriate constant ObjectInspector's will be created for
them, so any constant optimization for the UDF on Hive's side will be
enabled.
TODO:
-remove allocateInputObjects in HiveUdfExecutor
-use the one from InputHandler
-handle is_arg_const properly in HiveJavaFunctionFactoryImpl::create
-handle todo in HiveGenericJavaFunction.java::checkValidFunction
-handle todo in HiveGenericJavaFunction.java::getInspectors
-can an argument be const + writable?
-handle THiveUdfExecutorCtorParams::is_constant_arg being null
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactory.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
M fe/src/test/java/org/apache/impala/hive/executor/TestHiveJavaFunctionFactory.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgUdf.java
M java/test-hive-udfs/src/main/java/org/apache/impala/GenericBufferAlteringUdf.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
19 files changed, 611 insertions(+), 149 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/1
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 1
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#11).
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The "isConstant" flags for arguments are also sent to the frontend,
where ConstantObjectInspectors will be created for them, so any
constant optimization in the UDF on Hive's side will be enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
Benchmark:
Checked with the following:
set num_nodes=1; set mt_dop=1;
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_geomfromtext(
"polygon ((0 0, 0 500000, 500000 500000, 500000 0, 0 0))"
)
);
Before change: 3.15s (MaterializeTupleTime: 2s601ms)
After change: 1.54s (MaterializeTupleTime: 879.397ms)
Note that st_intersects is optimized on the Hive side when one argument
is a constant, contributing to most of the gain in performance.
The skipping of re-evaluation and copying is relatively insignificant,
comparing only that doesn't yield any measurable difference.
Testing:
-added a test UDF GenericAlltypeArgConstCheckUdf that prints
const information about arguments
-added const arg check cases to generic-java-udf.test
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgConstCheckUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
15 files changed, 615 insertions(+), 171 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/11
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 11
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 15: Code-Review+1
Thanks, Balazs! If nobody has any objections, I can +2 it.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 15
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Mar 2026 09:05:55 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 16: Code-Review+2
This seems good to me. Thanks!
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 16
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Mar 2026 16:27:28 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#6).
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The constantness flags for arguments are also sent to the frontend,
where appropriate constant ObjectInspector's will be created for
them, so any constant optimization for the UDF on Hive's side will be
enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgUdf.java
M java/test-hive-udfs/src/main/java/org/apache/impala/GenericBufferAlteringUdf.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
18 files changed, 604 insertions(+), 206 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/6
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 6
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Balazs Hevele has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 13:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/23963/12/common/thrift/Frontend.thrift
File common/thrift/Frontend.thrift:
http://gerrit.cloudera.org:8080/#/c/23963/12/common/thrift/Frontend.thrift@71
PS12, Line 71: Hive
> nit: Hive UDF executor?
Done
http://gerrit.cloudera.org:8080/#/c/23963/12/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:
http://gerrit.cloudera.org:8080/#/c/23963/12/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@1
PS12, Line 1: // Licensed to the Apache Software Foundation (ASF) under one
> nit: indent
Done
http://gerrit.cloudera.org:8080/#/c/23963/12/fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
File fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java:
http://gerrit.cloudera.org:8080/#/c/23963/12/fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java@26
PS12, Line 26: import org.apache.hadoop.io.Writable;
> nit: lot of unused imports
Done
http://gerrit.cloudera.org:8080/#/c/23963/12/fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java@170
PS12, Line 170: case LONG_WRITABLE:
> This is pretty similar to HiveUdfExecutorLegacy.java:77. Could you please e
Done
http://gerrit.cloudera.org:8080/#/c/23963/12/fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java@226
PS12, Line 226:
> We usually don't print out the stacktrace.
Done
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 13
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Feb 2026 11:30:37 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 13:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/21783/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 13
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Wed, 25 Feb 2026 09:27:52 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/21633/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 2
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 11 Feb 2026 15:00:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 11:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/23963/11//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/23963/11//COMMIT_MSG@40
PS11, Line 40: Testing:
Can you look through the existing tests to check if all types are covered with non-const args?
My concern is that many tests use const args which won't be sent to JVM per row after this patch. Generic UDFs are covered after this patch for all types, but legacy udfs with writable/Java type arguments may not be covered.
http://gerrit.cloudera.org:8080/#/c/23963/10/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
File testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test:
http://gerrit.cloudera.org:8080/#/c/23963/10/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@383
PS10, Line 383: select alltypeargs_constcheck(true, tinyint_col, smallint_col, int_col, bigint_col,
: float_col, double_col, string_col, cast("const string" as binary))
: from functional.alltypestiny;
Can you cover all types with both const and non-const?
My idea would be to set one half of arguments const in one query, then the other half in the other query.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 11
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 19 Feb 2026 18:27:00 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 8:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/21689/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 8
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 18 Feb 2026 11:02:12 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Peter Rozsa, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#13).
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The "isConstant" flags for arguments are also sent to the frontend,
where ConstantObjectInspectors will be created for them, so any
constant optimization in the UDF on Hive's side will be enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
Benchmark:
Checked with the following:
set num_nodes=1; set mt_dop=1;
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_geomfromtext(
"polygon ((0 0, 0 500000, 500000 500000, 500000 0, 0 0))"
)
);
Before change: 3.15s (MaterializeTupleTime: 2s601ms)
After change: 1.54s (MaterializeTupleTime: 879.397ms)
In some cases with geospatial, even bigger performance gain:
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_buffer(st_geomfromtext("point (250000 250000)"), 250000)
);
Before change: MaterializeTupleTime: 19s266ms
After change: MaterializeTupleTime: 1s587ms
Note that st_intersects is optimized on the Hive side when one argument
is a constant, contributing to most of the gain in performance.
The skipping of re-evaluation and copying is relatively insignificant,
comparing only that doesn't yield any measurable difference.
Testing:
-added a test UDF GenericAlltypeArgConstCheckUdf that prints
const information about arguments
-added const arg check cases to generic-java-udf.test
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgConstCheckUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
16 files changed, 646 insertions(+), 226 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/13
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 13
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 18:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/13189/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 18
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Wed, 04 Mar 2026 13:27:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Balazs Hevele has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 18:
Good catch. I broke HiveUdfExecutor.evaluateForTesting in Patch 13 by not passing the input object array to HiveUdfInputHandler.fillArgArray in HiveUdfExecutorLegacy.evaluateDerived.
Fixed it in Patch 18, and also cleaned up evalulateDerived by removing unused arguments.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 18
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Wed, 04 Mar 2026 09:00:36 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The "isConstant" flags for arguments are also sent to the frontend,
where ConstantObjectInspectors will be created for them, so any
constant optimization in the UDF on Hive's side will be enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
Benchmark:
Checked with the following:
set num_nodes=1; set mt_dop=1;
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_geomfromtext(
"polygon ((0 0, 0 500000, 500000 500000, 500000 0, 0 0))"
)
);
Before change: 3.15s (MaterializeTupleTime: 2s601ms)
After change: 1.54s (MaterializeTupleTime: 879.397ms)
In some cases with geospatial, even bigger performance gain:
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_buffer(st_geomfromtext("point (250000 250000)"), 250000)
);
Before change: MaterializeTupleTime: 19s266ms
After change: MaterializeTupleTime: 1s587ms
Note that st_intersects is optimized on the Hive side when one argument
is a constant, contributing to most of the gain in performance.
The skipping of re-evaluation and copying is relatively insignificant,
comparing only that doesn't yield any measurable difference.
Testing:
-added a test UDF GenericAlltypeArgConstCheckUdf that prints
const information about arguments
-added const arg check cases to generic-java-udf.test
-added legacy udf call tests for several types to java-udf.test
to make sure it still works for all types
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Reviewed-on: http://gerrit.cloudera.org:8080/23963
Reviewed-by: Csaba Ringhofer <cs...@cloudera.com>
Tested-by: Michael Smith <mi...@cloudera.com>
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgConstCheckUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
16 files changed, 665 insertions(+), 232 deletions(-)
Approvals:
Csaba Ringhofer: Looks good to me, approved
Michael Smith: Verified
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 19
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#3).
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The constantness flags for arguments are also sent to the frontend,
where appropriate constant ObjectInspector's will be created for
them, so any constant optimization for the UDF on Hive's side will be
enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
TODO:
-handle is_arg_const properly in HiveJavaFunctionFactoryImpl::create
-handle todo in HiveGenericJavaFunction.java::checkValidFunction
-handle todo in HiveGenericJavaFunction.java::getInspectors
-can an argument be const + writable?
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactory.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
M fe/src/test/java/org/apache/impala/hive/executor/TestHiveJavaFunctionFactory.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgUdf.java
M java/test-hive-udfs/src/main/java/org/apache/impala/GenericBufferAlteringUdf.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
21 files changed, 618 insertions(+), 213 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/3
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 3
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 12:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/21721/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 12
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Feb 2026 14:18:22 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#2).
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The constantness flags for arguments are also sent to the frontend,
where appropriate constant ObjectInspector's will be created for
them, so any constant optimization for the UDF on Hive's side will be
enabled.
TODO:
-remove allocateInputObjects in HiveUdfExecutor
-use the one from InputHandler
-handle is_arg_const properly in HiveJavaFunctionFactoryImpl::create
-handle todo in HiveGenericJavaFunction.java::checkValidFunction
-handle todo in HiveGenericJavaFunction.java::getInspectors
-can an argument be const + writable?
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactory.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
M fe/src/test/java/org/apache/impala/hive/executor/TestHiveJavaFunctionFactory.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgUdf.java
M java/test-hive-udfs/src/main/java/org/apache/impala/GenericBufferAlteringUdf.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
19 files changed, 610 insertions(+), 148 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/2
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 2
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 17: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 17
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Mar 2026 16:27:43 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 18: Verified+1
Failure is https://issues.apache.org/jira/browse/IMPALA-11572, rest looks fine.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 18
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Wed, 04 Mar 2026 18:02:32 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 12:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/23963/12/common/thrift/Frontend.thrift
File common/thrift/Frontend.thrift:
http://gerrit.cloudera.org:8080/#/c/23963/12/common/thrift/Frontend.thrift@71
PS12, Line 71: Hive
nit: Hive UDF executor?
http://gerrit.cloudera.org:8080/#/c/23963/12/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:
http://gerrit.cloudera.org:8080/#/c/23963/12/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@1
PS12, Line 1: // Licensed to the Apache Software Foundation (ASF) under one
nit: indent
http://gerrit.cloudera.org:8080/#/c/23963/12/fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
File fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java:
http://gerrit.cloudera.org:8080/#/c/23963/12/fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java@26
PS12, Line 26: import org.apache.hadoop.hive.ql.exec.UDF;
nit: lot of unused imports
http://gerrit.cloudera.org:8080/#/c/23963/12/fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java@170
PS12, Line 170: public void fillConstArgArray(Object[] inputArgs)
This is pretty similar to HiveUdfExecutorLegacy.java:77. Could you please extract it as a common method?
http://gerrit.cloudera.org:8080/#/c/23963/12/fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java@226
PS12, Line 226: e.printStackTrace(System.err);
We usually don't print out the stacktrace.
http://gerrit.cloudera.org:8080/#/c/23963/12/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
File testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test:
http://gerrit.cloudera.org:8080/#/c/23963/12/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@421
PS12, Line 421: ====
Could you please add a test where const and non-const arguments are mixed, like (const, non-const, const, non-const )or a similar random pattern?
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 12
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Tue, 24 Feb 2026 12:39:32 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#9).
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The "isConstant" flags for arguments are also sent to the frontend,
where ConstantObjectInspectors will be created for them, so any
constant optimization in the UDF on Hive's side will be enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
Benchmark:
Checked with the following:
set num_nodes=1; set mt_dop=1;
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_geomfromtext(
"polygon ((0 0, 0 500000, 500000 500000, 500000 0, 0 0))"
)
);
Before change: 3.15s
After change: 1.54s
Note that st_intersects is optimized on the Hive side when one argument
is a constant, contributing to most of the gain in performance.
The skipping of re-evaluation and copying is relatively insignificant,
comparing only that doesn't yield any measurable difference.
Testing:
-added a test UDF GenericAlltypeArgConstCheckUdf that prints
const information about arguments
-added const arg check cases to generic-java-udf.test
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgConstCheckUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
15 files changed, 615 insertions(+), 171 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/9
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 9
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Balazs Hevele has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 14:
(1 comment)
Sorry, forgot about one comment.
http://gerrit.cloudera.org:8080/#/c/23963/12/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
File testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test:
http://gerrit.cloudera.org:8080/#/c/23963/12/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@421
PS12, Line 421: 'Args: const true; non-const 0; const 2; non-const 0; const 3; non-const 0.0; const 4.0; non-const 0; const const string; '
> Could you please add a test where const and non-const arguments are mixed,
Done
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 14
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Thu, 26 Feb 2026 09:48:58 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#4).
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The constantness flags for arguments are also sent to the frontend,
where appropriate constant ObjectInspector's will be created for
them, so any constant optimization for the UDF on Hive's side will be
enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
TODO:
-handle is_arg_const properly in HiveJavaFunctionFactoryImpl::create
-handle todo in HiveGenericJavaFunction.java::checkValidFunction
-handle todo in HiveGenericJavaFunction.java::getInspectors
-can an argument be const + writable?
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactory.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
M fe/src/test/java/org/apache/impala/hive/executor/TestHiveJavaFunctionFactory.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgUdf.java
M java/test-hive-udfs/src/main/java/org/apache/impala/GenericBufferAlteringUdf.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
21 files changed, 618 insertions(+), 213 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/4
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 4
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#12).
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The "isConstant" flags for arguments are also sent to the frontend,
where ConstantObjectInspectors will be created for them, so any
constant optimization in the UDF on Hive's side will be enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
Benchmark:
Checked with the following:
set num_nodes=1; set mt_dop=1;
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_geomfromtext(
"polygon ((0 0, 0 500000, 500000 500000, 500000 0, 0 0))"
)
);
Before change: 3.15s (MaterializeTupleTime: 2s601ms)
After change: 1.54s (MaterializeTupleTime: 879.397ms)
In some cases with geospatial, even bigger performance gain:
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_buffer(st_geomfromtext("point (250000 250000)"), 250000)
);
Before change: MaterializeTupleTime: 19s266ms
After change: MaterializeTupleTime: 1s587ms
Note that st_intersects is optimized on the Hive side when one argument
is a constant, contributing to most of the gain in performance.
The skipping of re-evaluation and copying is relatively insignificant,
comparing only that doesn't yield any measurable difference.
Testing:
-added a test UDF GenericAlltypeArgConstCheckUdf that prints
const information about arguments
-added const arg check cases to generic-java-udf.test
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgConstCheckUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
16 files changed, 658 insertions(+), 176 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/12
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 12
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/21641/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 3
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 12 Feb 2026 08:34:33 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 1:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/21632/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 1
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 11 Feb 2026 14:23:37 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 10:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/21690/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 10
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 18 Feb 2026 13:50:17 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#5).
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The constantness flags for arguments are also sent to the frontend,
where appropriate constant ObjectInspector's will be created for
them, so any constant optimization for the UDF on Hive's side will be
enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactory.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/JavaUdfDataType.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/TestGenericUdf.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgUdf.java
M java/test-hive-udfs/src/main/java/org/apache/impala/GenericBufferAlteringUdf.java
M java/test-hive-udfs/src/main/java/org/apache/impala/TestGenericUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
20 files changed, 608 insertions(+), 206 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/5
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 5
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/21672/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 5
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 17 Feb 2026 13:22:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Peter Rozsa, Csaba Ringhofer, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#16).
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The "isConstant" flags for arguments are also sent to the frontend,
where ConstantObjectInspectors will be created for them, so any
constant optimization in the UDF on Hive's side will be enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
Benchmark:
Checked with the following:
set num_nodes=1; set mt_dop=1;
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_geomfromtext(
"polygon ((0 0, 0 500000, 500000 500000, 500000 0, 0 0))"
)
);
Before change: 3.15s (MaterializeTupleTime: 2s601ms)
After change: 1.54s (MaterializeTupleTime: 879.397ms)
In some cases with geospatial, even bigger performance gain:
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_buffer(st_geomfromtext("point (250000 250000)"), 250000)
);
Before change: MaterializeTupleTime: 19s266ms
After change: MaterializeTupleTime: 1s587ms
Note that st_intersects is optimized on the Hive side when one argument
is a constant, contributing to most of the gain in performance.
The skipping of re-evaluation and copying is relatively insignificant,
comparing only that doesn't yield any measurable difference.
Testing:
-added a test UDF GenericAlltypeArgConstCheckUdf that prints
const information about arguments
-added const arg check cases to generic-java-udf.test
-added legacy udf call tests for several types to java-udf.test
to make sure it still works for all types
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgConstCheckUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
16 files changed, 662 insertions(+), 226 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/16
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 16
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Balazs Hevele has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 12:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/23963/11//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/23963/11//COMMIT_MSG@23
PS11, Line 23:
> As discussed offline, can you also add an example with a more complex predi
Done
http://gerrit.cloudera.org:8080/#/c/23963/11//COMMIT_MSG@40
PS11, Line 40: );
> Can you look through the existing tests to check if all types are covered w
Good idea,
there are some tests here and there, but sometimes the value is only used indirectly (e.g. in a where clause),
so I added a query to java-udf.test that directly outputs some values through an (identity) UDF call
http://gerrit.cloudera.org:8080/#/c/23963/10/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
File testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test:
http://gerrit.cloudera.org:8080/#/c/23963/10/testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test@383
PS10, Line 383: select alltypeargs_constcheck(true, 1, 2, 3, 4,
: float_col, double_col, string_col, cast(string_col as binary))
: from functional.alltypestiny;
> Can you cover all types with both const and non-const?
Done
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 12
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Feb 2026 13:50:41 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 15:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/23963/15/be/src/exprs/expr-value.h
File be/src/exprs/expr-value.h:
http://gerrit.cloudera.org:8080/#/c/23963/15/be/src/exprs/expr-value.h@211
PS15, Line 211: void* setToAnyVal(impala_udf::AnyVal* v, const ColumnType& type) {
nit: capitalization doesn't match the adjacent functions. Seems like pattern here would be SetToAnyVal.
http://gerrit.cloudera.org:8080/#/c/23963/15/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:
http://gerrit.cloudera.org:8080/#/c/23963/15/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@170
PS15, Line 170: PrimitiveCategory cat = getPrimitiveCategory(t);
Could we move this after the isConst check?
http://gerrit.cloudera.org:8080/#/c/23963/15/testdata/workloads/functional-query/queries/QueryTest/java-udf.test
File testdata/workloads/functional-query/queries/QueryTest/java-udf.test:
http://gerrit.cloudera.org:8080/#/c/23963/15/testdata/workloads/functional-query/queries/QueryTest/java-udf.test@381
PS15, Line 381: select identity(bool_col), identity(tinyint_col),
Do we have a test with mixed values somewhere? It'd be nice to double check that's covered. If there is, just mention it under Testing in the commit message.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 15
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Mar 2026 20:22:17 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 16:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/21843/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 16
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Mar 2026 08:38:49 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Balazs Hevele has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 16:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/23963/15/be/src/exprs/expr-value.h
File be/src/exprs/expr-value.h:
http://gerrit.cloudera.org:8080/#/c/23963/15/be/src/exprs/expr-value.h@211
PS15, Line 211: void* SetToAnyVal(impala_udf::AnyVal* v, const ColumnType& type) {
> nit: capitalization doesn't match the adjacent functions. Seems like patter
Done
http://gerrit.cloudera.org:8080/#/c/23963/15/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
File fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java:
http://gerrit.cloudera.org:8080/#/c/23963/15/fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java@170
PS15, Line 170: if (isConst) {
> Could we move this after the isConst check?
Done
http://gerrit.cloudera.org:8080/#/c/23963/15/testdata/workloads/functional-query/queries/QueryTest/java-udf.test
File testdata/workloads/functional-query/queries/QueryTest/java-udf.test:
http://gerrit.cloudera.org:8080/#/c/23963/15/testdata/workloads/functional-query/queries/QueryTest/java-udf.test@381
PS15, Line 381: select identity(bool_col), identity(tinyint_col),
> Do we have a test with mixed values somewhere? It'd be nice to double check
There's a bunch of different cases added to generic-java-udf.test.
In this file (java-udf.test) with legacy hive udf function calls, I only added this to make sure legacy still works for different types (const optimization only covers generic hive udfs).
I added a little info about this to the commit message.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 16
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Mar 2026 08:12:22 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 17:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/13182/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 17
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Mar 2026 16:27:44 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 17: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/13182/
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 17
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Wed, 04 Mar 2026 00:19:15 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 18: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 18
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Wed, 04 Mar 2026 13:27:02 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 18: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/13189/
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 18
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Wed, 04 Mar 2026 17:44:13 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Peter Rozsa, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#14).
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The "isConstant" flags for arguments are also sent to the frontend,
where ConstantObjectInspectors will be created for them, so any
constant optimization in the UDF on Hive's side will be enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
Benchmark:
Checked with the following:
set num_nodes=1; set mt_dop=1;
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_geomfromtext(
"polygon ((0 0, 0 500000, 500000 500000, 500000 0, 0 0))"
)
);
Before change: 3.15s (MaterializeTupleTime: 2s601ms)
After change: 1.54s (MaterializeTupleTime: 879.397ms)
In some cases with geospatial, even bigger performance gain:
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_buffer(st_geomfromtext("point (250000 250000)"), 250000)
);
Before change: MaterializeTupleTime: 19s266ms
After change: MaterializeTupleTime: 1s587ms
Note that st_intersects is optimized on the Hive side when one argument
is a constant, contributing to most of the gain in performance.
The skipping of re-evaluation and copying is relatively insignificant,
comparing only that doesn't yield any measurable difference.
Testing:
-added a test UDF GenericAlltypeArgConstCheckUdf that prints
const information about arguments
-added const arg check cases to generic-java-udf.test
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgConstCheckUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
16 files changed, 662 insertions(+), 226 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/14
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 14
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 12:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/13120/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 12
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 23 Feb 2026 08:45:08 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 11:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/23963/11//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/23963/11//COMMIT_MSG@23
PS11, Line 23: Checked with the following:
As discussed offline, can you also add an example with a more complex predicate?
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 11
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Feb 2026 12:20:56 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 12: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 12
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 23 Feb 2026 13:30:23 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 17:
> Patch Set 17: Verified-1
>
> Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/13182/
22:01:30 [WARNING] Corrupted channel by directly writing to native stream in forked JVM 1. See FAQ web page and the dump file /home/ubuntu/Impala/logs/fe_tests/2026-03-03T17-45-44_062-jvmRun1.dumpstream
22:01:30 [INFO] BUILD FAILURE
22:01:30 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0:test (default-test) on project impala-frontend:
22:01:30 [ERROR]
22:01:30 [ERROR] Please refer to /home/ubuntu/Impala/logs/fe_tests for the individual test results.
22:01:30 [ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
22:01:30 [ERROR] The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
22:01:30 [ERROR] Command was /bin/sh -c cd '/home/ubuntu/Impala/fe' && '/usr/lib/jvm/java-17-openjdk-amd64/bin/java' '-jar' '/home/ubuntu/Impala/fe/target/surefire/surefirebooter-20260303174544188_3.jar' '/home/ubuntu/Impala/fe/target/surefire' '2026-03-03T17-45-44_062-jvmRun1' 'surefire-20260303174544188_1tmp' 'surefire_0-20260303174544188_2tmp'
22:01:30 [ERROR] Error occurred in starting fork, check output in log
22:01:30 [ERROR] Process Exit Code: 134
22:01:30 [ERROR] Crashed tests:
22:01:30 [ERROR] org.apache.impala.hive.executor.UdfExecutorTest
22:01:30 [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
Seems like it could be caused by this change. The logs tarball likely has the dumpstream.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 17
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Wed, 04 Mar 2026 02:28:42 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 18:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/21857/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 18
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Wed, 04 Mar 2026 09:25:55 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 16: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 16
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Mar 2026 15:15:03 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#8).
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The "isConstant" flags for arguments are also sent to the frontend,
where ConstantObjectInspectors will be created for them, so any
constant optimization in the UDF on Hive's side will be enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
Testing:
-added a test UDF GenericAlltypeArgConstCheckUdf that prints
const information about arguments
-added const arg check cases to generic-java-udf.test
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgConstCheckUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
15 files changed, 615 insertions(+), 171 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/8
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 8
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 12:
lgtm! as it is based on a POC by me, I probably shouldn't add +1/+2
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 12
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 20 Feb 2026 14:50:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#10).
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The "isConstant" flags for arguments are also sent to the frontend,
where ConstantObjectInspectors will be created for them, so any
constant optimization in the UDF on Hive's side will be enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
Benchmark:
Checked with the following:
set num_nodes=1; set mt_dop=1;
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_geomfromtext(
"polygon ((0 0, 0 500000, 500000 500000, 500000 0, 0 0))"
)
);
Before change: 3.15s
After change: 1.54s
Note that st_intersects is optimized on the Hive side when one argument
is a constant, contributing to most of the gain in performance.
The skipping of re-evaluation and copying is relatively insignificant,
comparing only that doesn't yield any measurable difference.
Testing:
-added a test UDF GenericAlltypeArgConstCheckUdf that prints
const information about arguments
-added const arg check cases to generic-java-udf.test
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgConstCheckUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
15 files changed, 615 insertions(+), 171 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/10
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 10
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Peter Rozsa, Csaba Ringhofer, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#18).
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The "isConstant" flags for arguments are also sent to the frontend,
where ConstantObjectInspectors will be created for them, so any
constant optimization in the UDF on Hive's side will be enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
Benchmark:
Checked with the following:
set num_nodes=1; set mt_dop=1;
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_geomfromtext(
"polygon ((0 0, 0 500000, 500000 500000, 500000 0, 0 0))"
)
);
Before change: 3.15s (MaterializeTupleTime: 2s601ms)
After change: 1.54s (MaterializeTupleTime: 879.397ms)
In some cases with geospatial, even bigger performance gain:
select count(*) from tpch.lineitem where st_intersects(
st_point(l_partkey, l_suppkey),
st_buffer(st_geomfromtext("point (250000 250000)"), 250000)
);
Before change: MaterializeTupleTime: 19s266ms
After change: MaterializeTupleTime: 1s587ms
Note that st_intersects is optimized on the Hive side when one argument
is a constant, contributing to most of the gain in performance.
The skipping of re-evaluation and copying is relatively insignificant,
comparing only that doesn't yield any measurable difference.
Testing:
-added a test UDF GenericAlltypeArgConstCheckUdf that prints
const information about arguments
-added const arg check cases to generic-java-udf.test
-added legacy udf call tests for several types to java-udf.test
to make sure it still works for all types
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgConstCheckUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
16 files changed, 665 insertions(+), 232 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/18
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 18
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 16: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 16
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Mar 2026 13:52:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 14:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/21797/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 14
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Comment-Date: Thu, 26 Feb 2026 10:12:32 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 7:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/21688/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 7
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 18 Feb 2026 09:42:16 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 6:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/21673/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 6
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 17 Feb 2026 13:29:02 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/21666/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 4
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 17 Feb 2026 07:45:00 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Balazs Hevele has posted comments on this change. ( http://gerrit.cloudera.org:8080/23963 )
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Patch Set 12:
Starting a verify to check potential issues
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 12
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 23 Feb 2026 08:45:44 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23963
to look at the new patch set (#7).
Change subject: WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
WIP IMPALA-14575: Add constant handling for Hive GenericUDFs
Based on PoC by Csaba Ringhofer.
If an argument is constant, it is now only evaluated and copied
to the input buffer once, in HiveUdfCall::OpenEvaluator.
This means, we save the re-evaluation and re-copying of the value
for each evaluation.
The constantness flags for arguments are also sent to the frontend,
where appropriate constant ObjectInspector's will be created for
them, so any constant optimization for the UDF on Hive's side will be
enabled.
Moved input handling for Hive UDF calls to a separate class
HiveUdfInputHandler.
Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
---
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/hive/executor/HiveGenericJavaFunction.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveJavaFunctionFactoryImpl.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutor.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorGeneric.java
M fe/src/main/java/org/apache/impala/hive/executor/HiveUdfExecutorLegacy.java
A fe/src/main/java/org/apache/impala/hive/executor/HiveUdfInputHandler.java
M fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java
M fe/src/test/java/org/apache/impala/hive/executor/UdfExecutorTest.java
A java/test-hive-udfs/src/main/java/org/apache/impala/GenericAlltypeArgConstCheckUdf.java
M testdata/workloads/functional-query/queries/QueryTest/generic-java-udf.test
M testdata/workloads/functional-query/queries/QueryTest/load-generic-java-udfs.test
15 files changed, 617 insertions(+), 171 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/23963/7
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 7
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-14575: Add constant handling for Hive GenericUDFs
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has removed a vote on this change.
Change subject: IMPALA-14575: Add constant handling for Hive GenericUDFs
......................................................................
Removed Verified-1 by Impala Public Jenkins <im...@cloudera.com>
--
To view, visit http://gerrit.cloudera.org:8080/23963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I4a6ca8c0bab499dffed88bb9786753da559af4c5
Gerrit-Change-Number: 23963
Gerrit-PatchSet: 18
Gerrit-Owner: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>