You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Peter Cseh (JIRA)" <ji...@apache.org> on 2017/02/02 18:09:51 UTC
[jira] [Updated] (OOZIE-2687) OYA: Figure out how to handle
MR-specific properties
[ https://issues.apache.org/jira/browse/OOZIE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Peter Cseh updated OOZIE-2687:
------------------------------
Description:
It was possible to manipulate the MapperLauncher's environment through properties like:
# mapreduce.map.memory.mb
# mapreduce.map.cpu.vcores
# mapred.child.env
# mapred.child.java.opts
# mapred.job.queue.name - ability to set launcher queue
E.g. We were using mapred.child.env to pass SPARK_HOME to the LauncherMapper and make PySpark work.
Fixing OOZIE-2596 added a hack. We should decide how we support or break compatibility and how we allow the manipulation of the Launcher environment.
Verify if the new launcher section in global applies to actions in sub-workflows as well. It did not use to work before and was only fixed in OOZIE-2030. It would be good to have that testcase (TestSubWorkflowActionExecutor. testParentGlobalConf) updated with the new launcher section as well.
was:
It was possible to manipulate the MapperLauncher's environment through properties like:
# mapreduce.map.memory.mb
# mapreduce.map.cpu.vcores
# mapred.child.env
# mapred.child.java.opts
# mapred.job.queue.name - ability to set launcher queue
E.g. We were using mapred.child.env to pass SPARK_HOME to the LauncherMapper and make PySpark work.
Fixing OOZIE-2596 added a hack. We should decide how we support or break compatibility and how we allow the manipulation of the Launcher environment.
> OYA: Figure out how to handle MR-specific properties
> ----------------------------------------------------
>
> Key: OOZIE-2687
> URL: https://issues.apache.org/jira/browse/OOZIE-2687
> Project: Oozie
> Issue Type: Sub-task
> Reporter: Peter Cseh
> Assignee: Peter Bacsko
>
> It was possible to manipulate the MapperLauncher's environment through properties like:
> # mapreduce.map.memory.mb
> # mapreduce.map.cpu.vcores
> # mapred.child.env
> # mapred.child.java.opts
> # mapred.job.queue.name - ability to set launcher queue
> E.g. We were using mapred.child.env to pass SPARK_HOME to the LauncherMapper and make PySpark work.
> Fixing OOZIE-2596 added a hack. We should decide how we support or break compatibility and how we allow the manipulation of the Launcher environment.
> Verify if the new launcher section in global applies to actions in sub-workflows as well. It did not use to work before and was only fixed in OOZIE-2030. It would be good to have that testcase (TestSubWorkflowActionExecutor. testParentGlobalConf) updated with the new launcher section as well.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)