You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Tim Chan <ti...@chan.net> on 2012/07/07 03:15:03 UTC
setting CONF_OOZIE_ACTION_MAX_OUTPUT_DATA
I'm trying to pass some output from one action to another using capture-output.
I'm getting this error:
Failing Oozie Launcher, Output data size [50,999] exceeds maximum [2,048]
I believe setting CONF_OOZIE_ACTION_MAX_OUTPUT_DATA to a larger value
will solve this. How do I set this?
I've tried:
<action name="my-shell-action">
<shell xmlns="uri:oozie:shell-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<job-xml>job.xml</job-xml>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
<property>
<name>oozie.action.max.output.data</name>
<value>100000</value>
</property>
</configuration>
<exec>python</exec>
...
<capture-output/>
</shell>
<ok to="process-cross-ref-tsv-and-hdfs-put"/>
<error to="fail"/>
</action>
I've tried setting it in the job.xml.
I've tried setting in the coordinator.xml.
I've also tried setting it in the coordinator.properties.
How do I set a new value for this?
Re: setting CONF_OOZIE_ACTION_MAX_OUTPUT_DATA
Posted by Virag Kothari <vi...@yahoo-inc.com>.
Hi Tim,
Yes, you are right. Its mainly used for passing control data.
Also, you can override this property at system level (oozie-site.xml)
Thanks,
Virag
On 7/6/12 6:32 PM, "Tim Chan" <ti...@chan.net> wrote:
> I'm trying to pass the output for this script to the next action.
> I suppose this is not a wise route to pursue. Thank you for letting me
> know the property can not be overridden by the user.
>
> On Fri, Jul 6, 2012 at 6:27 PM, Mona Chitnis <ch...@yahoo-inc.com> wrote:
>> Hi Tim,
>>
>> The property 'oozie.action.max.output.data' cannot be overridden by user.
>> So you are hitting this issue while running your python script? Most
>> probably this is due to a long file path that you are passing. You can
>> store such file paths in a variable and then pass to your script/pig job
>> (if applicable)
>>
>> --
>> Mona Chitnis
>>
>>
>>
>>
>> On 7/6/12 6:15 PM, "Tim Chan" <ti...@chan.net> wrote:
>>
>>> I'm trying to pass some output from one action to another using
>>> capture-output.
>>>
>>> I'm getting this error:
>>>
>>> Failing Oozie Launcher, Output data size [50,999] exceeds maximum [2,048]
>>>
>>> I believe setting CONF_OOZIE_ACTION_MAX_OUTPUT_DATA to a larger value
>>> will solve this. How do I set this?
>>>
>>> I've tried:
>>>
>>> <action name="my-shell-action">
>>> <shell xmlns="uri:oozie:shell-action:0.1">
>>> <job-tracker>${jobTracker}</job-tracker>
>>> <name-node>${nameNode}</name-node>
>>> <job-xml>job.xml</job-xml>
>>> <configuration>
>>> <property>
>>> <name>mapred.job.queue.name</name>
>>> <value>${queueName}</value>
>>> </property>
>>> <property>
>>> <name>oozie.action.max.output.data</name>
>>> <value>100000</value>
>>> </property>
>>> </configuration>
>>>
>>> <exec>python</exec>
>>> ...
>>>
>>>
>>> <capture-output/>
>>> </shell>
>>>
>>> <ok to="process-cross-ref-tsv-and-hdfs-put"/>
>>> <error to="fail"/>
>>> </action>
>>>
>>> I've tried setting it in the job.xml.
>>> I've tried setting in the coordinator.xml.
>>> I've also tried setting it in the coordinator.properties.
>>>
>>> How do I set a new value for this?
>>
>
>
Re: setting CONF_OOZIE_ACTION_MAX_OUTPUT_DATA
Posted by Tim Chan <ti...@chan.net>.
I'm trying to pass the output for this script to the next action.
I suppose this is not a wise route to pursue. Thank you for letting me
know the property can not be overridden by the user.
On Fri, Jul 6, 2012 at 6:27 PM, Mona Chitnis <ch...@yahoo-inc.com> wrote:
> Hi Tim,
>
> The property 'oozie.action.max.output.data' cannot be overridden by user.
> So you are hitting this issue while running your python script? Most
> probably this is due to a long file path that you are passing. You can
> store such file paths in a variable and then pass to your script/pig job
> (if applicable)
>
> --
> Mona Chitnis
>
>
>
>
> On 7/6/12 6:15 PM, "Tim Chan" <ti...@chan.net> wrote:
>
>>I'm trying to pass some output from one action to another using
>>capture-output.
>>
>>I'm getting this error:
>>
>>Failing Oozie Launcher, Output data size [50,999] exceeds maximum [2,048]
>>
>>I believe setting CONF_OOZIE_ACTION_MAX_OUTPUT_DATA to a larger value
>>will solve this. How do I set this?
>>
>>I've tried:
>>
>><action name="my-shell-action">
>> <shell xmlns="uri:oozie:shell-action:0.1">
>> <job-tracker>${jobTracker}</job-tracker>
>> <name-node>${nameNode}</name-node>
>> <job-xml>job.xml</job-xml>
>> <configuration>
>> <property>
>> <name>mapred.job.queue.name</name>
>> <value>${queueName}</value>
>> </property>
>> <property>
>> <name>oozie.action.max.output.data</name>
>> <value>100000</value>
>> </property>
>> </configuration>
>>
>> <exec>python</exec>
>> ...
>>
>>
>> <capture-output/>
>> </shell>
>>
>> <ok to="process-cross-ref-tsv-and-hdfs-put"/>
>> <error to="fail"/>
>> </action>
>>
>>I've tried setting it in the job.xml.
>>I've tried setting in the coordinator.xml.
>>I've also tried setting it in the coordinator.properties.
>>
>>How do I set a new value for this?
>
--
Tim Chan // tim@chan.net // 213.784.2523
Re: setting CONF_OOZIE_ACTION_MAX_OUTPUT_DATA
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
Hi Tim,
The property 'oozie.action.max.output.data' cannot be overridden by user.
So you are hitting this issue while running your python script? Most
probably this is due to a long file path that you are passing. You can
store such file paths in a variable and then pass to your script/pig job
(if applicable)
--
Mona Chitnis
On 7/6/12 6:15 PM, "Tim Chan" <ti...@chan.net> wrote:
>I'm trying to pass some output from one action to another using
>capture-output.
>
>I'm getting this error:
>
>Failing Oozie Launcher, Output data size [50,999] exceeds maximum [2,048]
>
>I believe setting CONF_OOZIE_ACTION_MAX_OUTPUT_DATA to a larger value
>will solve this. How do I set this?
>
>I've tried:
>
><action name="my-shell-action">
> <shell xmlns="uri:oozie:shell-action:0.1">
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <job-xml>job.xml</job-xml>
> <configuration>
> <property>
> <name>mapred.job.queue.name</name>
> <value>${queueName}</value>
> </property>
> <property>
> <name>oozie.action.max.output.data</name>
> <value>100000</value>
> </property>
> </configuration>
>
> <exec>python</exec>
> ...
>
>
> <capture-output/>
> </shell>
>
> <ok to="process-cross-ref-tsv-and-hdfs-put"/>
> <error to="fail"/>
> </action>
>
>I've tried setting it in the job.xml.
>I've tried setting in the coordinator.xml.
>I've also tried setting it in the coordinator.properties.
>
>How do I set a new value for this?