You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by jo...@apache.org on 2020/03/26 03:57:48 UTC

[impala] branch master updated: IMPALA-9548: UdfExecutorTest failures after HIVE-22893

This is an automated email from the ASF dual-hosted git repository.

joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git


The following commit(s) were added to refs/heads/master by this push:
     new 46a6598  IMPALA-9548: UdfExecutorTest failures after HIVE-22893
46a6598 is described below

commit 46a659871fa349f6aa828d7ea39fe19f64a3ab27
Author: Sahil Takiar <ta...@gmail.com>
AuthorDate: Mon Mar 23 19:15:58 2020 -0700

    IMPALA-9548: UdfExecutorTest failures after HIVE-22893
    
    HIVE-22893: "Enhance data size estimation for fields computed by UDFs"
    modified o.a.h.hive.ql.udf.UDFSubstr and added a dependency on a few new
    Hive classes located in the hive-exec jar. These classes include:
    
    o.a.h.hive.ql.plan.ColStatistics
    o.a.h.hive.ql.stats.estimator.StatEstimator
    o.a.h.hive.ql.stats.estimator.StatEstimatorProvider
    
    The test UdfExecutorTest#HiveStringsTest loads the class UDFSubstr and
    thus needs to load the aforementioned stats classes as well.
    
    shaded-deps/pom.xml selectively pulls in certain classes from the
    hive-exec jar, and excludes all others. This patch simply addes the
    necessary stats classes to load UDFSubstr. Thus, fixing
    UdfExecutorTest.
    
    Testing:
    * Ran core tests with CDP_BUILD_NUMBER=2244454 and USE_CDP_HIVE=true,
      validated that UdfExecutorTest now passes
    
    Change-Id: I5b3d5df08c2d48d21293d5a5308eb453f40184bf
    Reviewed-on: http://gerrit.cloudera.org:8080/15544
    Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
    Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
 shaded-deps/pom.xml | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/shaded-deps/pom.xml b/shaded-deps/pom.xml
index eefd73b..851403d 100644
--- a/shaded-deps/pom.xml
+++ b/shaded-deps/pom.xml
@@ -92,6 +92,9 @@ the same dependencies
                 <include>org/apache/hive/service/rpc/thrift/**</include>
                 <include>org/apache/hadoop/hive/serde/**</include>
                 <include>org/apache/hadoop/hive/serde2/**</include>
+                <!-- IMPALA-9548: Add UDF stats estimator -->
+                <include>org/apache/hadoop/hive/ql/plan/ColStatistics.class</include>
+                <include>org/apache/hadoop/hive/ql/stats/estimator/**</include>
                 <include>org/apache/hive/service/rpc/thrift/**</include>
                 <include>org/apache/hive/common/HiveVersionAnnotation.class</include>
                 <include>org/apache/orc/**</include>