You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Matyas Orhidi (JIRA)" <ji...@apache.org> on 2017/03/10 12:59:04 UTC
[jira] [Commented] (OOZIE-2818) Can't overwrite
oozie.action.max.output.data on a per-workflow basis
[ https://issues.apache.org/jira/browse/OOZIE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905032#comment-15905032 ]
Matyas Orhidi commented on OOZIE-2818:
--------------------------------------
This did not work either:
{code}
<workflow-app name="LargeOutputWorkflow" xmlns="uri:oozie:workflow:0.5">
<global>
<configuration>
<property>
<name>oozie.action.max.output.data</name>
<value>20000</value>
</property>
<property>
<name>oozie.launcher.oozie.action.max.output.data</name>
<value>20000</value>
</property>
</configuration>
</global>
<start to="shell-e207"/>
<kill name="Kill">
<message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<action name="shell-e207">
<shell xmlns="uri:oozie:shell-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<exec>/user/admin/large_output.sh</exec>
<argument>10</argument>
<file>/user/admin/large_output.sh#large_output.sh</file>
<capture-output/>
</shell>
<ok to="End"/>
<error to="Kill"/>
</action>
<end name="End"/>
</workflow-app>
{code}
> Can't overwrite oozie.action.max.output.data on a per-workflow basis
> --------------------------------------------------------------------
>
> Key: OOZIE-2818
> URL: https://issues.apache.org/jira/browse/OOZIE-2818
> Project: Oozie
> Issue Type: Bug
> Reporter: Peter Cseh
>
> It's not possible to raise the limit of the output on a per-action basis.
> With the workflow:
> {quote}
> <workflow-app name="LargeOutputWorkflow" xmlns="uri:oozie:workflow:0.5">
> <start to="shell-e207"/>
> <kill name="Kill">
> <message>Action failed, error message\[$\{wf:errorMessage(wf:lastErrorNode())}]</message>
> </kill>
> <action name="shell-e207">
> <shell xmlns="uri:oozie:shell-action:0.1">
> <job-tracker>$\{jobTracker}</job-tracker>
> <name-node>$\{nameNode}</name-node>
> <configuration>
> <property>
> <name>oozie.action.max.output.data</name>
> <value>100000</value>
> </property>
> <property>
> <name>oozie.launcher.oozie.action.max.output.data</name>
> <value>100000</value>
> </property>
> </configuration>
> <exec>/user/admin/large_output.sh</exec>
> <argument>10</argument>
> <file>/user/admin/large_output.sh#large_output.sh</file>
> <capture-output/>
> </shell>
> <ok to="End"/>
> <error to="Kill"/>
> </action>
> <end name="End"/>
> </workflow-app>
> {quote}
> In the localized files in the job.xml the value of this property gets overwritten:
> {quote}
> # cat /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_000002/job.xml |grep max.outp
> <property><name>oozie.launcher.oozie.action.max.output.data</name><value>100000</value><source>programatically</source><source>job.xml</source></property>
> <property><name>oozie.action.max.output.data</name><value>2048</value><source>programatically</source><source>job.xml</source></property>
> # cat /yarn/nm/usercache/admin/appcache/application_1489145190857_0001/container_1489145190857_0001_01_000002/action.xml |grep max.outp
> <property><name>oozie.launcher.oozie.action.max.output.data</name><value>100000</value><source>programatically</source></property>
> <property><name>oozie.action.max.output.data</name><value>100000</value><source>programatically</source></property>
> {quote}
> I could not find a reason for this behavior, so I think it's a bug.
> The value is read from [ConfigurationService|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L175] and set in [createLauncherConf|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L986] in JavaActionExecutor. There could be a check to see if it's already set.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)