You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by 88265920 <88...@qq.com> on 2017/06/09 16:59:50 UTC
kylin2.0.0版本 streaming build 一直在pending 状态
想尝试kylin 的 streaming 构建，但是build的时候一直在pending状态，下面是任务信息
Job Name test - 50_55 - BUILD - GMT+08:00 2017-06-09 14:57:32
Job ID 67cf1818-9ac2-4303-91b1-e21d4125fba1
Status PENDING
Duration 0.00 mins
MapReduce Waiting 0.00 mins
#1 Step Name: Save data from Kafka
Duration: 0 seconds Waiting: 0 seconds
#2 Step Name: Extract Fact Table Distinct Columns
Duration: 0 seconds Waiting: 0 seconds
#3 Step Name: Build Dimension Dictionary
Duration: 0 seconds Waiting: 0 seconds
#4 Step Name: Save Cuboid Statistics
Duration: 0 seconds Waiting: 0 seconds
#5 Step Name: Create HTable
Duration: 0 seconds Waiting: 0 seconds
#6 Step Name: Build Cube with Spark
Duration: 0 seconds Waiting: 0 seconds
#7 Step Name: Convert Cuboid Data to HFile
Duration: 0 seconds Waiting: 0 seconds
#8 Step Name: Load HFile to HBase Table
Duration: 0 seconds Waiting: 0 seconds
#9 Step Name: Update Cube Info
Duration: 0 seconds Waiting: 0 seconds
#10 Step Name: Kafka Intermediate File Cleanup
Duration: 0 seconds Waiting: 0 seconds
请问这个是什么原因呢？
--
View this message in context: http://apache-kylin.74782.x6.nabble.com/kylin2-0-0-streaming-build-pending-tp8202.html
Sent from the Apache Kylin mailing list archive at Nabble.com.
Re: kylin2.0.0版本 streaming build 一直在pending 状态
Posted by ShaoFeng Shi <sh...@apache.org>.
Hi, Kylin uses a ZK lock to ensure only one Job engine instance is running.
The lock path "/kylin/job_engine/lock" is a path on Zookeeper, not a local
or HDFS folder I think. Can you double confirm your probem was solved by
making a local dir? Thanks.
2017-06-10 16:06 GMT+08:00 88265920 <88...@qq.com>:
> public class ZookeeperJobLock implements JobLock {
> private Logger logger = LoggerFactory.getLogger(
> ZookeeperJobLock.class);
>
> private static final String ZOOKEEPER_LOCK_PATH =
> "/kylin/job_engine/lock";
>
> // 一直停留在这行代码
> hasLock = sharedLock.acquire(3, TimeUnit.SECONDS);
>
> 问题原因是 zookeeper创建分布式锁的时候，系统目录不存在，代码停留在锁的那一行了，导致整个任务调度线程没有启动
>
> 解决：
> sudo mkdir /kylin/
> sudo mkdir /kylin/job_engine/
> sudo mkdir /kylin/job_engine/lock
> sudo chown kangjiao:kangjiao /kylin/job_engine/lock/
>
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/kylin2-0-0-streaming-build-pending-tp8202p8206.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>
--
Best regards,
Shaofeng Shi 史少锋
Re: kylin2.0.0版本 streaming build 一直在pending 状态
Posted by 88265920 <88...@qq.com>.
public class ZookeeperJobLock implements JobLock {
private Logger logger = LoggerFactory.getLogger(ZookeeperJobLock.class);
private static final String ZOOKEEPER_LOCK_PATH =
"/kylin/job_engine/lock";
// 一直停留在这行代码
hasLock = sharedLock.acquire(3, TimeUnit.SECONDS);
问题原因是 zookeeper创建分布式锁的时候，系统目录不存在，代码停留在锁的那一行了，导致整个任务调度线程没有启动
解决：
sudo mkdir /kylin/
sudo mkdir /kylin/job_engine/
sudo mkdir /kylin/job_engine/lock
sudo chown kangjiao:kangjiao /kylin/job_engine/lock/
--
View this message in context: http://apache-kylin.74782.x6.nabble.com/kylin2-0-0-streaming-build-pending-tp8202p8206.html
Sent from the Apache Kylin mailing list archive at Nabble.com.