Re: Support for spark 2.1.1 (HDP spark versions)

[Liang Chen <ch...@gmail.com>]

Hi

Did you successfully compile with "mvn -DskipTests -Pspark-2.1
-Dspark.version=2.1.1 clean package" ?
I used the community spark 2.1.1, it is working fine.

Regards
Liang


satish.sidnakoppa wrote
> <http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/file/t389/versions.png> 
> 
> The carbon supported spark versions and Hortonworks - HDP supported
> versions
> are different.
> I am unable to integrate carbondata and HDP .
> 
> In brief , how can I build carbondata jar to on spark 2.1.1
> Because I am working on HDP 2.6.2 that has spark 2.1.1 as MAJOR version
> whereas carbondata default support is 2.1.0
> 
> Due to this when I run carbon session creation.I get below error
> 
> val cc =
> SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession(hdfs
> path)
>             and got error:
>             java.lang.NoClassDefFoundError:
> org/apache/spark/sql/catalyst/CatalystConf 
> 
> 
> 
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/





--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: Support for spark 2.1.1 (HDP spark versions)

["satish.sidnakoppa" <sa...@gmail.com>]

I tried to build using
mvn -DskipTests -Pspark-2.1 -Dspark.version=2.1.1 clean package

Build failing for spark 2.1.1


[INFO] Apache CarbonData :: Parent ........................ SUCCESS [  3.981
s]
.
.
.
[INFO] Apache CarbonData :: Bloom Index DataMap ........... SUCCESS [ 13.166
s]
[INFO] Apache CarbonData :: Spark2 ........................ FAILURE [ 27.384
s]

ERROR] Failed to execute goal
org.scala-tools:maven-scala-plugin:2.15.2:compile (default) on project
carbondata-spark2: wrap: org.apache.commons.exec.ExecuteException: Process
exited with an error: 1(Exit value: 1) -> [Help 1]


But the build is failing




-----
regards
Satish
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/