You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by zhujin <fo...@126.com> on 2016/10/13 03:04:12 UTC
Disscusion shall CI support run carbondata based on multi version
spark?
One issue:
I modified the spark.version in pom.xml,using spark1.6.2, then compliation failed.
Root cause:
There was a "unused import statement" warinng in CarbonOptimizer class before, we imported AggregationExpression like the following :
import org.apache.spark.sql.catalyst.expressions.aggregate._
import org.apache.spark.sql.catalyst.expressions._
But in spark1.6.2, AggregateExpressions is moved to subpackage "aggregate" from "expressions"(that is for spark1.5.2).
So if we didn't known this change, we removed this import "import org.apache.spark.sql.catalyst.expressions.aggregate._", it will cause compliation failure when using spark1.6.2
Question:
So, maybe the CI should verify carbondata based on different version spark, then it will be more helful to check the correctness of the commits, shall we?
Re: Disscusion shall CI support run carbondata based on multi version spark?
Posted by Vimal Das Kammath <vi...@gmail.com>.
Yes, I Agree. CI should be configured to build Carbon on different spark
versions.
On Fri, Oct 14, 2016 at 7:56 AM, Liang Chen <ch...@gmail.com> wrote:
>
> Yes, need to solve it , the CI should support different spark version.
>
> Regards
> Liang
>
>
> zhujin wrote
> > One issue:
> > I modified the spark.version in pom.xml,using spark1.6.2, then
> compliation
> > failed.
> >
> >
> > Root cause:
> > There was a "unused import statement" warinng in CarbonOptimizer class
> > before, we imported AggregationExpression like the following :
> > import org.apache.spark.sql.catalyst.expressions.aggregate._
> > import org.apache.spark.sql.catalyst.expressions._
> > But in spark1.6.2, AggregateExpressions is moved to subpackage
> "aggregate"
> > from "expressions"(that is for spark1.5.2).
> > So if we didn't known this change, we removed this import "import
> > org.apache.spark.sql.catalyst.expressions.aggregate._", it will cause
> > compliation failure when using spark1.6.2
> >
> >
> > Question:
> > So, maybe the CI should verify carbondata based on different version
> > spark, then it will be more helful to check the correctness of the
> > commits, shall we?
>
>
>
>
>
> --
> View this message in context: http://apache-carbondata-
> mailing-list-archive.1130556.n5.nabble.com/Disscusion-
> shall-CI-support-run-carbondata-based-on-multi-
> version-spark-tp1836p1890.html
> Sent from the Apache CarbonData Mailing List archive mailing list archive
> at Nabble.com.
>
Re: Disscusion shall CI support run carbondata based on multi
version spark?
Posted by Liang Chen <ch...@gmail.com>.
Yes, need to solve it , the CI should support different spark version.
Regards
Liang
zhujin wrote
> One issue:
> I modified the spark.version in pom.xml,using spark1.6.2, then compliation
> failed.
>
>
> Root cause:
> There was a "unused import statement" warinng in CarbonOptimizer class
> before, we imported AggregationExpression like the following :
> import org.apache.spark.sql.catalyst.expressions.aggregate._
> import org.apache.spark.sql.catalyst.expressions._
> But in spark1.6.2, AggregateExpressions is moved to subpackage "aggregate"
> from "expressions"(that is for spark1.5.2).
> So if we didn't known this change, we removed this import "import
> org.apache.spark.sql.catalyst.expressions.aggregate._", it will cause
> compliation failure when using spark1.6.2
>
>
> Question:
> So, maybe the CI should verify carbondata based on different version
> spark, then it will be more helful to check the correctness of the
> commits, shall we?
--
View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Disscusion-shall-CI-support-run-carbondata-based-on-multi-version-spark-tp1836p1890.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com.