You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "liyang (JIRA)" <ji...@apache.org> on 2016/10/07 13:44:20 UTC
[jira] [Comment Edited] (KYLIN-2006) Make job engine distributed
and HA
[ https://issues.apache.org/jira/browse/KYLIN-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15555114#comment-15555114 ]
liyang edited comment on KYLIN-2006 at 10/7/16 1:44 PM:
--------------------------------------------------------
I have a few suggestions/questions on the patch.
- Instead of adding 3 new methods to {{JobLock}} interface, consider extend a sub-interface. Then in {{JobService.lockCube()}}, we can use {{jobLock instanceof XXX}} to replace the hard coding {{kylinConfig.getSchedulerType() == 1}}. I think that is more elegant.
- Consider move {{JobController.jobLock}} and the content of {{JobController.afterPropertiesSet()}} into {{JobService}}. The locking and scheduling fit better into service. Then in {{JobService.lockCube()}}, the {{jobLock}} can be reused.
- The change in {{ExecutableManager}} does not sound right. In the resume method, why {{if (task.getStatus() == ExecutableState.ERROR || task.getStatus() == ExecutableState.RUNNING)}}, then update the task status to READY? Changing a running task to status READY does not sound right. Might cause a step be executed twice?
- In {{SucceedTestExecutable}}, the wait time was changed from 1 second to 1 minute. This will cause {{ExecutableManagerTest}} to run 5+ minutes, which is a big waste of time for people who run `mvn test` frequently. Please make the wait time configurable, or create another class.
was (Author: liyang.gmt8@gmail.com):
I have a few suggestions/questions on the patch.
- Instead of adding 3 new methods to {{JobLock}} interface, consider extend a sub-interface. Then in {{JobService.lockCube()}}, we can use {{jobLock instanceof XXX}} to replace the hard coding{{kylinConfig.getSchedulerType() == 1}}. I think that is more elegant.
- Consider move {{JobController.jobLock}} and the content of {{JobController.afterPropertiesSet()}} into {{JobService}}. The locking and scheduling fit better into service. Then in {{JobService.lockCube()}}, the {{jobLock}} can be reused.
- The change in {{ExecutableManager}} does not sound right. In the resume method, why {{if (task.getStatus() == ExecutableState.ERROR || task.getStatus() == ExecutableState.RUNNING)}}, then update the task status to READY? Changing a running task to status READY does not sound right. Might cause a step be executed twice?
- In {{SucceedTestExecutable}}, the wait time was changed from 1 second to 1 minute. This will cause {{ExecutableManagerTest}} to run 5+ minutes, which is a big waste of time for people who run `mvn test` frequently. Please make the wait time configurable, or create another class.
> Make job engine distributed and HA
> ----------------------------------
>
> Key: KYLIN-2006
> URL: https://issues.apache.org/jira/browse/KYLIN-2006
> Project: Kylin
> Issue Type: New Feature
> Components: Job Engine
> Reporter: kangkaisen
> Assignee: kangkaisen
> Attachments: KYLIN-2006.patch
>
>
> Currently, the Kylin job build server is single-point。
> In order to make Kylin job build server more extensible, available and reliable, we should support distributed job build server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)