[jira] [Commented] (KYLIN-2857) MR configuration should be overwritten by user specified parameters when resuming MR jobs

["liyang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KYLIN-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160332#comment-16160332 ] 

liyang commented on KYLIN-2857:
-------------------------------

A few comments on the patch.
* How about calling {{Configuration.addResource(new Path(confValue))}} instead of parsing XML by ourselves?
* The Configuration object returned from {{HadoopUtil.getCurrentConfiguration()}} is a shared object and must not be modified for one job.
* Will be most ideal if new jobs can launch with the {{getJobConf()}} returned configuration, to make sure resume and launch share exactly the same config. Requires refactoring of the {{MRUtil.runMRJob()}} however...

> MR configuration should be overwritten by user specified parameters when resuming MR jobs
> -----------------------------------------------------------------------------------------
>
>                 Key: KYLIN-2857
>                 URL: https://issues.apache.org/jira/browse/KYLIN-2857
>             Project: Kylin
>          Issue Type: Improvement
>          Components: Job Engine
>            Reporter: zhengdong
>            Assignee: Dong Li
>         Attachments: KYLIN-2857-MR-configuration-should-be-overwritten-by.patch
>
>
> For now, MR job conf could be overwritten by kylin_job_conf.xml file, and also further overwritten at project/cube level. 
> Nonetheless, when resuming a MR job after a job server restarting or other kind of interruption, we only use the default conf to trace the job status. 
> That's usually not a problem until user hope to overwrite job status tracing related parameters (job history server address e.g.) to support multi history servers.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)