You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Shaofeng SHI (JIRA)" <ji...@apache.org> on 2017/02/04 07:44:51 UTC
[jira] [Updated] (KYLIN-2421) Add spark engine to Integration Test
[ https://issues.apache.org/jira/browse/KYLIN-2421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shaofeng SHI updated KYLIN-2421:
--------------------------------
Description:
Let Jenkins to test Spark cubing daily. Will change the engine type for "ci_inner_join_cube" to "4" (spark).
After this change, you need do a couple changes manually in your HDP 2.4 sandbox to finish the Integration Test:
1. manually install spark-1.6.3-bin-hadoop2.6 a local folder like /usr/local/spark
2. manually upload the spark-assembly jar to HDFS as /kylin/spark/spark-assembly-1.6.3-hadoop2.6.0.jar
3. manually create local temp folder for hbase client (if it doesn't exist):
mkdir -p /hadoop/hbase/local/jars
chmod 777 /hadoop/hbase/local/jars
4. Add SPARK_HOME property when run mvn:
mvn verify -fae -Dhdp.version=2.4.0.0-169 -DSPARK_HOME=/usr/local/spark -P sandbox
was:Let Jenkins to test Spark cubing daily.
> Add spark engine to Integration Test
> ------------------------------------
>
> Key: KYLIN-2421
> URL: https://issues.apache.org/jira/browse/KYLIN-2421
> Project: Kylin
> Issue Type: Sub-task
> Components: Job Engine
> Affects Versions: v2.0.0
> Reporter: Shaofeng SHI
> Assignee: Shaofeng SHI
> Fix For: v2.0.0
>
>
> Let Jenkins to test Spark cubing daily. Will change the engine type for "ci_inner_join_cube" to "4" (spark).
> After this change, you need do a couple changes manually in your HDP 2.4 sandbox to finish the Integration Test:
> 1. manually install spark-1.6.3-bin-hadoop2.6 a local folder like /usr/local/spark
> 2. manually upload the spark-assembly jar to HDFS as /kylin/spark/spark-assembly-1.6.3-hadoop2.6.0.jar
> 3. manually create local temp folder for hbase client (if it doesn't exist):
> mkdir -p /hadoop/hbase/local/jars
> chmod 777 /hadoop/hbase/local/jars
> 4. Add SPARK_HOME property when run mvn:
> mvn verify -fae -Dhdp.version=2.4.0.0-169 -DSPARK_HOME=/usr/local/spark -P sandbox
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)