You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@kylin.apache.org by xx...@apache.org on 2021/04/12 03:22:16 UTC

[kylin] branch kylin-on-parquet-v2 updated (dfd012f -> 6153889)

This is an automated email from the ASF dual-hosted git repository.

xxyu pushed a change to branch kylin-on-parquet-v2
in repository https://gitbox.apache.org/repos/asf/kylin.git.


    from dfd012f  KYLIN-4945 repartition encoded dataset to avoid data skew caused by single column
     add 6153889  KYLIN-4967 Forbid to set 'spark.sql.adaptive.enabled' to true when building cube with Spark 2.X

No new revisions were added by this update.

Summary of changes:
 .../org/apache/spark/utils/SparkVersionUtils.scala | 74 ++++++++++++++++++++++
 .../engine/spark/application/SparkApplication.java | 12 ++++
 .../kylin/engine/spark/job/NSparkExecutable.java   | 13 ++++
 .../spark/builder/CubeDictionaryBuilder.scala      | 16 ++---
 .../engine/spark/builder/CubeTableEncoder.scala    |  5 ++
 .../kylin/engine/spark/utils/Repartitioner.java    |  4 --
 .../engine/spark/builder/TestGlobalDictBuild.scala |  3 -
 7 files changed, 108 insertions(+), 19 deletions(-)
 create mode 100644 kylin-spark-project/kylin-spark-common/src/main/scala/org/apache/spark/utils/SparkVersionUtils.scala