You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Ken Ernst <ke...@mobileposse.com> on 2011/11/02 19:48:09 UTC
Re: Oozie decision action - Switch / Case predicate
Hi Mohammad,
Anything further on this issue?
Thanks!
Ken
On 10/26/11 12:45 AM, "Mohammad Islam" <mi...@yahoo.com> wrote:
>Hi Ken,
>You are right it is comparing dictionary wise. I reproduce your problem.
>For example, if you specify param02=02 (instead of param02=2), you will
>see the false as expected.
>
>I thought EL function automatically convert it to number.
>
>Let me dig further tomorrow.
>
>Regards,
>Mohammad
>
>
>
>________________________________
>From: Ken Ernst <ke...@mobileposse.com>
>To: "oozie-users@incubator.apache.org" <oo...@incubator.apache.org>
>Sent: Tuesday, October 25, 2011 2:06 PM
>Subject: Re: Oozie decision action - Switch / Case predicate
>
>Any help on this as to wether I am doing something wrong or this is a bug
>would be greatly appreciated.
>
>Thanks.
>
>
>Ken
>
>
>
>
>On 10/24/11 12:50 PM, "Ken Ernst" <ke...@mobileposse.com> wrote:
>
>>OK, so I retract that this is working properly...
>>
>>It looks like that the predicate is a byte by byte string compare?
>>
>>Here is my job definition... (all from the oozie console)
>>-----------------------------------------------------------
>><workflow-app xmlns='uri:oozie:workflow:0.1' name='consolidate-files-wf'>
>> <start to='check-return' />
>> <decision name="check-return"> <switch>
>> <case to="end">${param02 ge param01}</case>
>> <case to="end">${param03 ge param01}</case>
>> <case to="end">${param04 ge param01}</case>
>> <case to="end">${param05 ge param01}</case>
>> <case to="end">${param06 ge param01}</case>
>> <default to="end"/> </switch>
>> </decision>
>> <kill name="fail"> <message>Directory consolidation for ${tableName}
>>failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
>> </kill>
>> <end name='end' />
>></workflow-app>
>>-----------------------------------------------------------
>>Here is the job configuration (irrelevant properties removed)
>>-----------------------------------------------------------
>><configuration>
>>
>> <property>
>> <name>param01</name>
>> <value>15</value>
>> </property>
>> <property> <name>param02</name>
>> <value>2</value>
>> </property>
>>
>> <property>
>> <name>param03</name>
>> <value>4</value>
>> </property>
>>
>> <property>
>> <name>param04</name> <value>04</value>
>> </property>
>>
>> <property> <name>param05</name>
>> <value>11</value>
>> </property>
>> <property>
>> <name>param06</name>
>> <value>16</value>
>> </property>
>>
>></configuration>
>>---------------------------------------
>>And the relevant piece of the job log with annotations... ---
>>------------------------------------
>>ACTION[0000384-111021075506415-oozie-oozi-W@check-return] Start, name
>>[check-return] type [switch] configuration
>>
>><switch xmlns="uri:oozie:workflow:0.1">
>> <case to="end">true</case> --- 2 ge 15 (should have been false)
>> <case to="end">true</case> --- 4 ge 15 (should have been false)
>> <case to="end">false</case>--- 04 ge 15 (OK)
>> <case to="end">false</case>--- 11 ge 15 (OK)
>> <case to="end">true</case> --- 16 ge 15 (OK)
>> <default to="end" />
>></switch>
>>
>>2011-10-24 11:38:43,962 WARN ActionStartCommand:528 - USER[
>>
>>--------------------------------------
>>
>>
>>
>>
>>
>>
>>
>>
>>On 10/24/11 10:43 AM, "Ken Ernst" <ke...@mobileposse.com> wrote:
>>
>>>Tried to reproduce this issue and could not - must have had a typo
>>>somewhere... This is a workflow calling a sub-workflow...
>>>
>>>Parameters do work on the right hand side of the predicate.
>>>
>>>
>>>Ken Ernst
>>>
>>>
>>>On 10/22/11 12:28 PM, "Alejandro Abdelnur" <tu...@cloudera.com> wrote:
>>>
>>>
>>>>Hi Ken,
>>>>
>>>>Would you check, in the oozie webconsole, for your decision node
>>>>action,
>>>>the
>>>>configuration after the run? It should have all values of EL functions
>>>>resolved.
>>>>
>>>>Thanks.
>>>>
>>>>Alejandro
>>>>
>>>>On Sat, Oct 22, 2011 at 6:05 AM, Ken Ernst <ke...@mobileposse.com>
>>>>wrote:
>>>>
>>>>> Hello,
>>>>> I want to parameterize the right had side of the predicate in a case
>>>>> statement but it does not seem to work. It always returns false. Does
>>>>>anyone
>>>>> know of why this would not work? For example I have the following
>>>>>decision
>>>>> control node where the the right hand side of the case predicate is
>>>>>12,
>>>>> however I would like to make this a job property (I.e.
>>>>>fileThresholdCount).
>>>>>
>>>>> This works (returns true when the output of my java action
>>>>> 'dir.numberOfFiles' is greater than 12).
>>>>> -----------------------------------------------------------
>>>>> <decision name="check-return">
>>>>> <switch>
>>>>> <case
>>>>>
>>>>>to="hive-consolidate-dir">${wf:actionData('hfsNumFiles')['dir.numberOf
>>>>>F
>>>>>i
>>>>>l
>>>>>es']
>>>>> ge 12}</case>
>>>>> <default to="end"/>
>>>>> </switch>
>>>>> </decision>
>>>>> -----------------------------------------------------------
>>>>>
>>>>> These do not seem to work, always returns false.
>>>>>
>>>>> -----------------------------------------------------------
>>>>> <decision name="check-return">
>>>>> <switch>
>>>>> <case
>>>>>
>>>>>to="hive-consolidate-dir">${wf:actionData('hfsNumFiles')['dir.numberOf
>>>>>F
>>>>>i
>>>>>l
>>>>>es']
>>>>> ge wf:conf('fileThresholdCount')}</case>
>>>>> <default to="end"/>
>>>>> </switch>
>>>>> </decision>
>>>>>
>>>>> <decision name="check-return">
>>>>> <switch>
>>>>> <case
>>>>>
>>>>>to="hive-consolidate-dir">${wf:actionData('hfsNumFiles')['dir.numberOf
>>>>>F
>>>>>i
>>>>>l
>>>>>es']
>>>>> ge fileThresholdCount}</case>
>>>>> <default to="end"/>
>>>>> </switch>
>>>>> </decision>
>>>>> -----------------------------------------------------------
>>>>>
>>>>> Thanks!
>>>>>
>>>>> Ken Ernst
>>>>>
>>>>>
>>>>>
>>>
Re: Oozie decision action - Switch / Case predicate
Posted by "Angelo K. Huang" <an...@apache.org>.
I guess it is using string comparison not numeric comparison it should be.
Could you try using 2.0 or 4.0 or 15.0 to see if JSP EL function recognizes
them as numbers?
Thanks,
Angelo
On Wed, Nov 2, 2011 at 11:48 AM, Ken Ernst <ke...@mobileposse.com> wrote:
> Hi Mohammad,
>
> Anything further on this issue?
>
>
> Thanks!
> Ken
>
>
>
> On 10/26/11 12:45 AM, "Mohammad Islam" <mi...@yahoo.com> wrote:
>
> >Hi Ken,
> >You are right it is comparing dictionary wise. I reproduce your problem.
> >For example, if you specify param02=02 (instead of param02=2), you will
> >see the false as expected.
> >
> >I thought EL function automatically convert it to number.
> >
> >Let me dig further tomorrow.
> >
> >Regards,
> >Mohammad
> >
> >
> >
> >________________________________
> >From: Ken Ernst <ke...@mobileposse.com>
> >To: "oozie-users@incubator.apache.org" <oo...@incubator.apache.org>
> >Sent: Tuesday, October 25, 2011 2:06 PM
> >Subject: Re: Oozie decision action - Switch / Case predicate
> >
> >Any help on this as to wether I am doing something wrong or this is a bug
> >would be greatly appreciated.
> >
> >Thanks.
> >
> >
> >Ken
> >
> >
> >
> >
> >On 10/24/11 12:50 PM, "Ken Ernst" <ke...@mobileposse.com> wrote:
> >
> >>OK, so I retract that this is working properly...
> >>
> >>It looks like that the predicate is a byte by byte string compare?
> >>
> >>Here is my job definition... (all from the oozie console)
> >>-----------------------------------------------------------
> >><workflow-app xmlns='uri:oozie:workflow:0.1' name='consolidate-files-wf'>
> >> <start to='check-return' />
> >> <decision name="check-return"> <switch>
> >> <case to="end">${param02 ge param01}</case>
> >> <case to="end">${param03 ge param01}</case>
> >> <case to="end">${param04 ge param01}</case>
> >> <case to="end">${param05 ge param01}</case>
> >> <case to="end">${param06 ge param01}</case>
> >> <default to="end"/> </switch>
> >> </decision>
> >> <kill name="fail"> <message>Directory consolidation for ${tableName}
> >>failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
> >> </kill>
> >> <end name='end' />
> >></workflow-app>
> >>-----------------------------------------------------------
> >>Here is the job configuration (irrelevant properties removed)
> >>-----------------------------------------------------------
> >><configuration>
> >>
> >> <property>
> >> <name>param01</name>
> >> <value>15</value>
> >> </property>
> >> <property> <name>param02</name>
> >> <value>2</value>
> >> </property>
> >>
> >> <property>
> >> <name>param03</name>
> >> <value>4</value>
> >> </property>
> >>
> >> <property>
> >> <name>param04</name> <value>04</value>
> >> </property>
> >>
> >> <property> <name>param05</name>
> >> <value>11</value>
> >> </property>
> >> <property>
> >> <name>param06</name>
> >> <value>16</value>
> >> </property>
> >>
> >></configuration>
> >>---------------------------------------
> >>And the relevant piece of the job log with annotations... ---
> >>------------------------------------
> >>ACTION[0000384-111021075506415-oozie-oozi-W@check-return] Start, name
> >>[check-return] type [switch] configuration
> >>
> >><switch xmlns="uri:oozie:workflow:0.1">
> >> <case to="end">true</case> --- 2 ge 15 (should have been false)
> >> <case to="end">true</case> --- 4 ge 15 (should have been false)
> >> <case to="end">false</case>--- 04 ge 15 (OK)
> >> <case to="end">false</case>--- 11 ge 15 (OK)
> >> <case to="end">true</case> --- 16 ge 15 (OK)
> >> <default to="end" />
> >></switch>
> >>
> >>2011-10-24 11:38:43,962 WARN ActionStartCommand:528 - USER[
> >>
> >>--------------------------------------
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>On 10/24/11 10:43 AM, "Ken Ernst" <ke...@mobileposse.com> wrote:
> >>
> >>>Tried to reproduce this issue and could not - must have had a typo
> >>>somewhere... This is a workflow calling a sub-workflow...
> >>>
> >>>Parameters do work on the right hand side of the predicate.
> >>>
> >>>
> >>>Ken Ernst
> >>>
> >>>
> >>>On 10/22/11 12:28 PM, "Alejandro Abdelnur" <tu...@cloudera.com> wrote:
> >>>
> >>>
> >>>>Hi Ken,
> >>>>
> >>>>Would you check, in the oozie webconsole, for your decision node
> >>>>action,
> >>>>the
> >>>>configuration after the run? It should have all values of EL functions
> >>>>resolved.
> >>>>
> >>>>Thanks.
> >>>>
> >>>>Alejandro
> >>>>
> >>>>On Sat, Oct 22, 2011 at 6:05 AM, Ken Ernst <ke...@mobileposse.com>
> >>>>wrote:
> >>>>
> >>>>> Hello,
> >>>>> I want to parameterize the right had side of the predicate in a case
> >>>>> statement but it does not seem to work. It always returns false. Does
> >>>>>anyone
> >>>>> know of why this would not work? For example I have the following
> >>>>>decision
> >>>>> control node where the the right hand side of the case predicate is
> >>>>>12,
> >>>>> however I would like to make this a job property (I.e.
> >>>>>fileThresholdCount).
> >>>>>
> >>>>> This works (returns true when the output of my java action
> >>>>> 'dir.numberOfFiles' is greater than 12).
> >>>>> -----------------------------------------------------------
> >>>>> <decision name="check-return">
> >>>>> <switch>
> >>>>> <case
> >>>>>
> >>>>>to="hive-consolidate-dir">${wf:actionData('hfsNumFiles')['dir.numberOf
> >>>>>F
> >>>>>i
> >>>>>l
> >>>>>es']
> >>>>> ge 12}</case>
> >>>>> <default to="end"/>
> >>>>> </switch>
> >>>>> </decision>
> >>>>> -----------------------------------------------------------
> >>>>>
> >>>>> These do not seem to work, always returns false.
> >>>>>
> >>>>> -----------------------------------------------------------
> >>>>> <decision name="check-return">
> >>>>> <switch>
> >>>>> <case
> >>>>>
> >>>>>to="hive-consolidate-dir">${wf:actionData('hfsNumFiles')['dir.numberOf
> >>>>>F
> >>>>>i
> >>>>>l
> >>>>>es']
> >>>>> ge wf:conf('fileThresholdCount')}</case>
> >>>>> <default to="end"/>
> >>>>> </switch>
> >>>>> </decision>
> >>>>>
> >>>>> <decision name="check-return">
> >>>>> <switch>
> >>>>> <case
> >>>>>
> >>>>>to="hive-consolidate-dir">${wf:actionData('hfsNumFiles')['dir.numberOf
> >>>>>F
> >>>>>i
> >>>>>l
> >>>>>es']
> >>>>> ge fileThresholdCount}</case>
> >>>>> <default to="end"/>
> >>>>> </switch>
> >>>>> </decision>
> >>>>> -----------------------------------------------------------
> >>>>>
> >>>>> Thanks!
> >>>>>
> >>>>> Ken Ernst
> >>>>>
> >>>>>
> >>>>>
> >>>
>
>
Re: Oozie decision action - Switch / Case predicate
Posted by Mohammad Islam <mi...@yahoo.com>.
Sorry for not updating you.
Actually our QE (Michelle) also recreate the problem.
We are checking if one fix (coming in the next release) will resolve the issue.
Regards,
Mohammad
________________________________
From: Ken Ernst <ke...@mobileposse.com>
To: "oozie-users@incubator.apache.org" <oo...@incubator.apache.org>; Mohammad Islam <mi...@yahoo.com>
Sent: Wednesday, November 2, 2011 11:48 AM
Subject: Re: Oozie decision action - Switch / Case predicate
Hi Mohammad,
Anything further on this issue?
Thanks!
Ken
On 10/26/11 12:45 AM, "Mohammad Islam" <mi...@yahoo.com> wrote:
>Hi Ken,
>You are right it is comparing dictionary wise. I reproduce your problem.
>For example, if you specify param02=02 (instead of param02=2), you will
>see the false as expected.
>
>I thought EL function automatically convert it to number.
>
>Let me dig further tomorrow.
>
>Regards,
>Mohammad
>
>
>
>________________________________
>From: Ken Ernst <ke...@mobileposse.com>
>To: "oozie-users@incubator.apache.org" <oo...@incubator.apache.org>
>Sent: Tuesday, October 25, 2011 2:06 PM
>Subject: Re: Oozie decision action - Switch / Case predicate
>
>Any help on this as to wether I am doing something wrong or this is a bug
>would be greatly appreciated.
>
>Thanks.
>
>
>Ken
>
>
>
>
>On 10/24/11 12:50 PM, "Ken Ernst" <ke...@mobileposse.com> wrote:
>
>>OK, so I retract that this is working properly...
>>
>>It looks like that the predicate is a byte by byte string compare?
>>
>>Here is my job definition... (all from the oozie console)
>>-----------------------------------------------------------
>><workflow-app xmlns='uri:oozie:workflow:0.1' name='consolidate-files-wf'>
>> <start to='check-return' />
>> <decision name="check-return"> <switch>
>> <case to="end">${param02 ge param01}</case>
>> <case to="end">${param03 ge param01}</case>
>> <case to="end">${param04 ge param01}</case>
>> <case to="end">${param05 ge param01}</case>
>> <case to="end">${param06 ge param01}</case>
>> <default to="end"/> </switch>
>> </decision>
>> <kill name="fail"> <message>Directory consolidation for ${tableName}
>>failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
>> </kill>
>> <end name='end' />
>></workflow-app>
>>-----------------------------------------------------------
>>Here is the job configuration (irrelevant properties removed)
>>-----------------------------------------------------------
>><configuration>
>>
>> <property>
>> <name>param01</name>
>> <value>15</value>
>> </property>
>> <property> <name>param02</name>
>> <value>2</value>
>> </property>
>>
>> <property>
>> <name>param03</name>
>> <value>4</value>
>> </property>
>>
>> <property>
>> <name>param04</name> <value>04</value>
>> </property>
>>
>> <property> <name>param05</name>
>> <value>11</value>
>> </property>
>> <property>
>> <name>param06</name>
>> <value>16</value>
>> </property>
>>
>></configuration>
>>---------------------------------------
>>And the relevant piece of the job log with annotations... ---
>>------------------------------------
>>ACTION[0000384-111021075506415-oozie-oozi-W@check-return] Start, name
>>[check-return] type [switch] configuration
>>
>><switch xmlns="uri:oozie:workflow:0.1">
>> <case to="end">true</case> --- 2 ge 15 (should have been false)
>> <case to="end">true</case> --- 4 ge 15 (should have been false)
>> <case to="end">false</case>--- 04 ge 15 (OK)
>> <case to="end">false</case>--- 11 ge 15 (OK)
>> <case to="end">true</case> --- 16 ge 15 (OK)
>> <default to="end" />
>></switch>
>>
>>2011-10-24 11:38:43,962 WARN ActionStartCommand:528 - USER[
>>
>>--------------------------------------
>>
>>
>>
>>
>>
>>
>>
>>
>>On 10/24/11 10:43 AM, "Ken Ernst" <ke...@mobileposse.com> wrote:
>>
>>>Tried to reproduce this issue and could not - must have had a typo
>>>somewhere... This is a workflow calling a sub-workflow...
>>>
>>>Parameters do work on the right hand side of the predicate.
>>>
>>>
>>>Ken Ernst
>>>
>>>
>>>On 10/22/11 12:28 PM, "Alejandro Abdelnur" <tu...@cloudera.com> wrote:
>>>
>>>
>>>>Hi Ken,
>>>>
>>>>Would you check, in the oozie webconsole, for your decision node
>>>>action,
>>>>the
>>>>configuration after the run? It should have all values of EL functions
>>>>resolved.
>>>>
>>>>Thanks.
>>>>
>>>>Alejandro
>>>>
>>>>On Sat, Oct 22, 2011 at 6:05 AM, Ken Ernst <ke...@mobileposse.com>
>>>>wrote:
>>>>
>>>>> Hello,
>>>>> I want to parameterize the right had side of the predicate in a case
>>>>> statement but it does not seem to work. It always returns false. Does
>>>>>anyone
>>>>> know of why this would not work? For example I have the following
>>>>>decision
>>>>> control node where the the right hand side of the case predicate is
>>>>>12,
>>>>> however I would like to make this a job property (I.e.
>>>>>fileThresholdCount).
>>>>>
>>>>> This works (returns true when the output of my java action
>>>>> 'dir.numberOfFiles' is greater than 12).
>>>>> -----------------------------------------------------------
>>>>> <decision name="check-return">
>>>>> <switch>
>>>>> <case
>>>>>
>>>>>to="hive-consolidate-dir">${wf:actionData('hfsNumFiles')['dir.numberOf
>>>>>F
>>>>>i
>>>>>l
>>>>>es']
>>>>> ge 12}</case>
>>>>> <default to="end"/>
>>>>> </switch>
>>>>> </decision>
>>>>> -----------------------------------------------------------
>>>>>
>>>>> These do not seem to work, always returns false.
>>>>>
>>>>> -----------------------------------------------------------
>>>>> <decision name="check-return">
>>>>> <switch>
>>>>> <case
>>>>>
>>>>>to="hive-consolidate-dir">${wf:actionData('hfsNumFiles')['dir.numberOf
>>>>>F
>>>>>i
>>>>>l
>>>>>es']
>>>>> ge wf:conf('fileThresholdCount')}</case>
>>>>> <default to="end"/>
>>>>> </switch>
>>>>> </decision>
>>>>>
>>>>> <decision name="check-return">
>>>>> <switch>
>>>>> <case
>>>>>
>>>>>to="hive-consolidate-dir">${wf:actionData('hfsNumFiles')['dir.numberOf
>>>>>F
>>>>>i
>>>>>l
>>>>>es']
>>>>> ge fileThresholdCount}</case>
>>>>> <default to="end"/>
>>>>> </switch>
>>>>> </decision>
>>>>> -----------------------------------------------------------
>>>>>
>>>>> Thanks!
>>>>>
>>>>> Ken Ernst
>>>>>
>>>>>
>>>>>
>>>