You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2020/12/08 07:21:00 UTC
[jira] [Assigned] (SPARK-33704) Support latest version of
initialize() in HiveGenericUDTF
[ https://issues.apache.org/jira/browse/SPARK-33704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-33704:
------------------------------------
Assignee: (was: Apache Spark)
> Support latest version of initialize() in HiveGenericUDTF
> ---------------------------------------------------------
>
> Key: SPARK-33704
> URL: https://issues.apache.org/jira/browse/SPARK-33704
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.0, 2.3.4, 2.4.3
> Reporter: chenliang
> Priority: Major
>
> For HiveGenericUDTF , there are two initialization methods:
> {code:java}
> public StructObjectInspector initialize(StructObjectInspector argOIs)
> throws UDFArgumentException {
> List<? extends StructField> inputFields = argOIs.getAllStructFieldRefs();
> ObjectInspector[] udtfInputOIs = new ObjectInspector[inputFields.size()];
> for (int i = 0; i < inputFields.size(); i++) {
> udtfInputOIs[i] = inputFields.get(i).getFieldObjectInspector();
> }
> return initialize(udtfInputOIs);
> }
> @Deprecated
> public StructObjectInspector initialize(ObjectInspector[] argOIs)
> throws UDFArgumentException {
> throw new IllegalStateException("Should not be called directly");
> }
> {code}
> As https://issues.apache.org/jira/browse/HIVE-5737 mentioned, hive provided StructObjectInspector for UDTFs rather than ObjectInspect[], but Spark SQL still only support deprecated function.
> An exception will be reported before fix:
> Error in query: No handler for UDF/UDAF/UDTF 'FeatureParseUDTF1': java.lang.IllegalStateException: Should not be called directly
> Please make sure your function overrides public StructObjectInspector initialize(ObjectInspector[] args).; line 1 pos 7
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org