Re: kylin stay running in step1 when Create Intermediate Flat Hive Table

[Li Yang <li...@apache.org>]

Suggest check yarn and see if hive MR is really submitted and running. It
maybe hanging inside the MR and you need to find out why.

On Thu, Jun 30, 2016 at 11:54 AM, wangfeng <wa...@gmail.com>
wrote:

> kylin能够正常启动，sample.sh
> 脚本也能够正常运行，同时在hive中能够查到所建立的三张表如下图所示，下图是运行了sample.sh后的hive中的default
> database中的tables，可以看见生成了官方文档上所描述的三张表：
>
> <http://apache-kylin.74782.x6.nabble.com/file/n5163/hive.png>
>
> 但是在运行build kylin_sales_cube 的时候，却一直在Create Intermediate Flat Hive Table
> 这里运行，并且log 无信息，如下所示：
>
> <http://apache-kylin.74782.x6.nabble.com/file/n5163/runing.png>
> <http://apache-kylin.74782.x6.nabble.com/file/n5163/output.png>
>
>
> 按照文档，hive的元数据库我使用mysql替换了derby数据库，同时也将mysql-connector-java-x.x.xx.jar文件拷贝到$Hive_Home/lib目录中，
>
> 运行sample.sh 后,remote mysql 中生成了如下的信息：
>
> <http://apache-kylin.74782.x6.nabble.com/file/n5163/mysql.png>
>
> 检查kylin.log 文件部分信息，没有Error错误，内容如下：
>
> SET hive.support.concurrency=false;
> SET mapreduce.job.split.metainfo.maxsize=-1;
> INSERT OVERWRITE TABLE
> kylin_intermediate_kylin_sales_cube_desc_20120101000000_20160608000000
> SELECT
> KYLIN_SALES.PART_DT
> ,KYLIN_SALES.LEAF_CATEG_ID
> ,KYLIN_SALES.LSTG_SITE_ID
> ,KYLIN_CATEGORY_GROUPINGS.META_CATEG_NAME
> ,KYLIN_CATEGORY_GROUPINGS.CATEG_LVL2_NAME
> ,KYLIN_CATEGORY_GROUPINGS.CATEG_LVL3_NAME
> ,KYLIN_SALES.LSTG_FORMAT_NAME
> ,KYLIN_SALES.PRICE
> ,KYLIN_SALES.SELLER_ID
> FROM DEFAULT.KYLIN_SALES as KYLIN_SALES
> INNER JOIN DEFAULT.KYLIN_CAL_DT as KYLIN_CAL_DT
> ON KYLIN_SALES.PART_DT = KYLIN_CAL_DT.CAL_DT
> INNER JOIN DEFAULT.KYLIN_CATEGORY_GROUPINGS as KYLIN_CATEGORY_GROUPINGS
> ON KYLIN_SALES.LEAF_CATEG_ID = KYLIN_CATEGORY_GROUPINGS.LEAF_CATEG_ID AND
> KYLIN_SALES.LSTG_SITE_ID = KYLIN_CATEGORY_GROUPINGS.SITE_ID
> WHERE (KYLIN_SALES.PART_DT >= '2012-01-01' AND KYLIN_SALES.PART_DT <
> '2016-06-08')
> ;
>
> "
> 2016-06-30 11:25:14,255 INFO  [pool-6-thread-1]
> threadpool.DefaultScheduler:106 : Job Fetcher: 0 running, 1 actual running,
> 1 ready, 24 others
> 2016-06-30 11:25:40,090 DEBUG [http-bio-7070-exec-7]
> service.AdminService:90
> : Get Kylin Runtime Config
> 2016-06-30 11:25:40,151 DEBUG [http-bio-7070-exec-6]
> controller.UserController:64 : authentication.getPrincipal() is
> org.springframework.security.core.userdetails.User@3b40b2f: Username:
> ADMIN;
> Password: [PROTECTED]; Enabled: true; AccountNonExpired: true;
> credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities:
> ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER
> 2016-06-30 11:25:40,227 DEBUG [http-bio-7070-exec-6]
> controller.ProjectController:97 : authentication.getPrincipal() is
> org.springframework.security.core.userdetails.User@3b40b2f: Username:
> ADMIN;
> Password: [PROTECTED]; Enabled: true; AccountNonExpired: true;
> credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities:
> ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER
> 2016-06-30 11:26:14,241 INFO  [pool-6-thread-1]
> threadpool.DefaultScheduler:106 : Job Fetcher: 1 running, 1 actual running,
> 0 ready, 24 others
> 2016-06-30 11:26:46,633 DEBUG [http-bio-7070-exec-2]
> service.AdminService:90
> : Get Kylin Runtime Config
> 2016-06-30 11:26:46,708 DEBUG [http-bio-7070-exec-6]
> controller.UserController:64 : authentication.getPrincipal() is
> org.springframework.security.core.userdetails.User@3b40b2f: Username:
> ADMIN;
> Password: [PROTECTED]; Enabled: true; AccountNonExpired: true;
> credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities:
> ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER
> 2016-06-30 11:26:46,786 DEBUG [http-bio-7070-exec-6]
> controller.ProjectController:97 : authentication.getPrincipal() is
> org.springframework.security.core.userdetails.User@3b40b2f: Username:
> ADMIN;
> Password: [PROTECTED]; Enabled: true; AccountNonExpired: true;
> credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities:
> ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER
> 2016-06-30 11:27:14,238 INFO  [pool-6-thread-1]
> threadpool.DefaultScheduler:106 : Job Fetcher: 1 running, 1 actual running,
> 0 ready, 24 others
>
> 下面贴出可能涉及到的配置文件信息：
>
> hive-sive.xml 配置如下：
>
> <property>
>   <name>javax.jdo.option.ConnectionURL</name>
>
>
> <value>jdbc:mysql://remote_ip:3306/hive?createDatabaseIfNotExist=true</value>
> </property>
>
> <property>
>   <name>javax.jdo.option.ConnectionDriverName</name>
>   <value>com.mysql.jdbc.Driver</value>
> </property>
>
> <property>
>   <name>javax.jdo.option.ConnectionUserName</name>
>   <value>hive</value>
> </property>
>
> <property>
>   <name>javax.jdo.option.ConnectionPassword</name>
>   <value>hive</value>
> </property>
> hbase-site.xml
>
> <configuration>
>      <property>
>           <name>hbase.rootdir</name>
>           <value>hdfs://localhost:9000/hbase</value>
>     </property>
>      <property>
>         <name>hbase.zookeeper.property.clientPort</name>
>         <value>2182 </value>
>     </property>
> </configuration>
> kylin.properties
>
> kylin.owner=wdcloud@kylin.apache.org
> kylin.rest.servers=localhost:7070
> kylin.hdfs.working.dir=/home/hadoop/kylin/kylin_hdfs_working_dir
> kylin.job.remote.cli.working.dir=/home/../kylin/kylin_job_working_dir
> kylin.job.jar=/home/../kylin/lib/kylin-job-1.5.0-SNAPSHOT.jar
>
> kylin.coprocessor.local.jar=/home/../kylin/lib/kylin-coprocessor-1.5.0-SNAPSHOT.jar
> and 我运行了find-hive-dependency.sh & kylin]$ find-hbase-dependency.sh
> 都没有问题，却一直停留在运行的第一步。
>
> --
> View this message in context:
> http://apache-kylin.74782.x6.nabble.com/kylin-stay-running-in-step1-when-Create-Intermediate-Flat-Hive-Table-tp5163.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>