You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "dinesh (Jira)" <ji...@apache.org> on 2020/09/24 18:41:00 UTC
[jira] [Updated] (SPARK-32988) ExternalCatalog vs
ExternalCatalogWithListener: backward compatibility problem
[ https://issues.apache.org/jira/browse/SPARK-32988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dinesh updated SPARK-32988:
---------------------------
Summary: ExternalCatalog vs ExternalCatalogWithListener: backward compatibility problem (was: ExternalCatalog backward compatibility problems due to ExternalCatalog )
> ExternalCatalog vs ExternalCatalogWithListener: backward compatibility problem
> -------------------------------------------------------------------------------
>
> Key: SPARK-32988
> URL: https://issues.apache.org/jira/browse/SPARK-32988
> Project: Spark
> Issue Type: Bug
> Components: Java API
> Affects Versions: 2.3.2, 2.4.0
> Reporter: dinesh
> Priority: Minor
>
> With Spark 2.3
> spark.sharedState.externalCatalog used to give an object of ExternalCatalog which was actually an instance of HiveExternalCatalog
> With Spark 2.4
> spark.sharedState.externalCatalog gives an object of ExternalCatalogWithListener which has HiveExternalCatalog as unwrapped delegate.
>
> We use HiveExternalCatalog to get HiveClient to perform Hive Operations.
>
>
> This change in method Signature is giving us backward compatibility problems such that code compiled with Spark 2.3 can not work with 2.4 due to the change. This results in Runtime exceptions including ClassCastException, NoSuchMethodError
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org