You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "Zhichao Zhang (JIRA)" <ji...@apache.org> on 2019/07/26 06:48:00 UTC
[jira] [Commented] (CARBONDATA-3469) CarbonData with 2.3.2 can not
run on CDH spark 2.4
[ https://issues.apache.org/jira/browse/CARBONDATA-3469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16893380#comment-16893380 ]
Zhichao Zhang commented on CARBONDATA-3469:
--------------------------------------------
[~imperio] I think it can't work with spark 2.4, there maybe some interfaces of spark changed, community will integrate with spark 2.4 in the next time.
> CarbonData with 2.3.2 can not run on CDH spark 2.4
> --------------------------------------------------
>
> Key: CARBONDATA-3469
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3469
> Project: CarbonData
> Issue Type: Bug
> Components: spark-integration
> Affects Versions: 1.5.3
> Reporter: wxmimperio
> Priority: Major
>
> *{color:#333333}spark2-shell --jars [apache-carbondata-1.5.3-bin-spark2.3.2-hadoop2.7.2.jar|https://dist.apache.org/repos/dist/release/carbondata/1.5.3/apache-carbondata-1.5.3-bin-spark2.3.2-hadoop2.7.2.jar]{color}*
>
> {code:java}
> java.lang.NoSuchMethodError: org.apache.spark.sql.internal.SharedState.externalCatalog()Lorg/apache/spark/sql/catalyst/catalog/ExternalCatalog;{code}
> {code:java}
> scala> carbon.sql(
> | s"""
> | | CREATE TABLE IF NOT EXISTS test_table(
> | | id string,
> | | name string,
> | | city string,
> | | age Int)
> | | STORED AS carbondata
> | """.stripMargin)
> java.lang.NoSuchMethodError: org.apache.spark.sql.internal.SharedState.externalCatalog()Lorg/apache/spark/sql/catalyst/catalog/ExternalCatalog;
> at org.apache.spark.sql.hive.CarbonSessionStateBuilder.externalCatalog(CarbonSessionState.scala:227)
> at org.apache.spark.sql.hive.CarbonSessionStateBuilder.catalog$lzycompute(CarbonSessionState.scala:214)
> at org.apache.spark.sql.hive.CarbonSessionStateBuilder.catalog(CarbonSessionState.scala:212)
> at org.apache.spark.sql.hive.CarbonSessionStateBuilder.catalog(CarbonSessionState.scala:191)
> at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$1.apply(BaseSessionStateBuilder.scala:291)
> at org.apache.spark.sql.internal.BaseSessionStateBuilder$$anonfun$build$1.apply(BaseSessionStateBuilder.scala:291)
> at org.apache.spark.sql.internal.SessionState.catalog$lzycompute(SessionState.scala:77)
> at org.apache.spark.sql.internal.SessionState.catalog(SessionState.scala:77)
> at org.apache.spark.sql.CarbonEnv$.getInstance(CarbonEnv.scala:135)
> at org.apache.spark.sql.CarbonSession$.updateSessionInfoToCurrentThread(CarbonSession.scala:326)
> at org.apache.spark.sql.parser.CarbonSparkSqlParser.parsePlan(CarbonSparkSqlParser.scala:47)
> at org.apache.spark.sql.CarbonSession.withProfiler(CarbonSession.scala:125)
> at org.apache.spark.sql.CarbonSession.sql(CarbonSession.scala:88)
> ... 59 elided
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)