You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Bruce Robbins (Jira)" <ji...@apache.org> on 2023/07/18 17:57:00 UTC
[jira] [Commented] (SPARK-44477) CheckAnalysis uses error subclass as an error class
[ https://issues.apache.org/jira/browse/SPARK-44477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744314#comment-17744314 ]
Bruce Robbins commented on SPARK-44477:
---------------------------------------
PR here: https://github.com/apache/spark/pull/42064
> CheckAnalysis uses error subclass as an error class
> ---------------------------------------------------
>
> Key: SPARK-44477
> URL: https://issues.apache.org/jira/browse/SPARK-44477
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.5.0
> Reporter: Bruce Robbins
> Priority: Minor
>
> {{CheckAnalysis}} treats {{TYPE_CHECK_FAILURE_WITH_HINT}} as an error class, but it is instead an error subclass of {{{}DATATYPE_MISMATCH{}}}.
> {noformat}
> spark-sql (default)> select bitmap_count(12);
> [INTERNAL_ERROR] Cannot find main error class 'TYPE_CHECK_FAILURE_WITH_HINT'
> org.apache.spark.SparkException: [INTERNAL_ERROR] Cannot find main error class 'TYPE_CHECK_FAILURE_WITH_HINT'
> at org.apache.spark.SparkException$.internalError(SparkException.scala:83)
> at org.apache.spark.SparkException$.internalError(SparkException.scala:87)
> at org.apache.spark.ErrorClassesJsonReader.$anonfun$getMessageTemplate$1(ErrorClassesJSONReader.scala:68)
> at scala.collection.immutable.HashMap$HashMap1.getOrElse0(HashMap.scala:361)
> at scala.collection.immutable.HashMap$HashTrieMap.getOrElse0(HashMap.scala:594)
> at scala.collection.immutable.HashMap$HashTrieMap.getOrElse0(HashMap.scala:589)
> at scala.collection.immutable.HashMap.getOrElse(HashMap.scala:73)
> {noformat}
> This issue only occurs when an expression uses {{TypeCheckResult.TypeCheckFailure}} to indicate input type check failure. {{TypeCheckResult.TypeCheckFailure}} appears to be deprecated in favor of {{{}TypeCheckResult.DataTypeMismatch{}}}, but recently two expressions were added that use {{{}TypeCheckResult.TypeCheckFailure{}}}: {{BitmapCount}} and {{{}BitmapOrAgg{}}}.
> {{BitmapCount}} and {{BitmapOrAgg}} should probably be fixed to use {{{}TypeCheckResult.DataTypeMismatch{}}}. Regardless, the code in {{CheckAnalysis}} that handles {{TypeCheckResult.TypeCheckFailure}} should be corrected (or removed).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org