You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Vincent Peplinski <vi...@gmail.com> on 2015/06/24 21:41:06 UTC
HDFS EL Function - fs:dirSize(String path)
Hello,
I'm trying to use the fs:dirSize() HDFS EL function in one of my workflows
action nodes but I'm having some difficulty and I hope someone can help.
When I call the function using a literal string such as:
<param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=20150624')}</param>
The function returns the size of the file as expected. However; when I try
the following:
<param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=${pDate}')}</param>
The workflow fails with an EL_ERROR - Illegal character in path index 26
/user/peppy/testing/p_dt=${pDate}.
How can I pass a variable to the function that will be properly expanded?
Thank you in advance.
V.
Re: HDFS EL Function - fs:dirSize(String path)
Posted by Laurent H <la...@gmail.com>.
Yes, I think too because it's a true/false condition, but in the other
example, it's a fs check...
--
Laurent HATIER - Consultant Big Data & Business Intelligence chez CapGemini
fr.linkedin.com/pub/laurent-hatier/25/36b/a86/
<http://fr.linkedin.com/pub/laurent-h/25/36b/a86/>
2015-06-29 10:21 GMT+02:00 Oussama Chougna <ou...@hotmail.com>:
> I'm 100% sure that the suggested should work:
> I have this in my workflow and it works perfect:
> <decision name="check-preconditions-directories"> <switch>
> <!-- Check if the warehouse dir for all the data exists. If not, create
> it with the right permissions. --> <case
> to="create-warehouse-dirs">${!fs:exists(wf:conf('hiveMainTableLocation'))
> || !fs:exists(wf:conf('hive_temp_tables_root'))}</case> <!--
> default: just proceed. --> <default to="preconditions-join"/>
> </switch> </decision>
> in my bundle.xml (I'm using a bundle):
>
> <property> <name>hiveMainTableLocation</name>
> <value>${hive_tables_root}/OrderItems</value>
> </property>
> In my job.properties:
> hive_temp_tables_root=${nameNode}/user/hive/warehouse/pzt/tmp/hive
>
>
> This works 100%.
> Oozie 4.1.0-cdh5.4.2
>
> Cheers,
> Oussama Chougna
>
> > Date: Mon, 29 Jun 2015 10:15:15 +0200
> > Subject: Re: HDFS EL Function - fs:dirSize(String path)
> > From: laurent.hatier@gmail.com
> > To: user@oozie.apache.org
> >
> > I think that is not possible because oozie can replace variable once, not
> > twice (fsDir + your variable) I think you have to create your own Oozie
> > function, or maybe use an shell action to do some stuf (fs dir of your
> > folder in parameter) and then use an capture-output to get the size of
> your
> > folder :)
> >
> > --
> > Laurent HATIER - Consultant Big Data & Business Intelligence chez
> CapGemini
> > fr.linkedin.com/pub/laurent-hatier/25/36b/a86/
> > <http://fr.linkedin.com/pub/laurent-h/25/36b/a86/>
> >
> > 2015-06-25 22:33 GMT+02:00 Vincent Peplinski <vi...@gmail.com>:
> >
> > > Neither of those suggestions resolved the issue...
> > >
> > > On Thu, Jun 25, 2015 at 3:02 PM, Idris Ali <ps...@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > Can you try changing the single quotes only around pDate to double
> quotes
> > > > "". Seems to be xml wrapping issue.
> > > >
> > > > Thanks,
> > > > Idris
> > > >
> > > > On Thursday 25 June 2015, Vincent Peplinski <vi...@gmail.com>
> > > > wrote:
> > > >
> > > > > Thanks for the suggestion Oussama, unfortunately it did not
> resolve my
> > > > > issue :(
> > > > >
> > > > > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:
> > > > > conf('pDate')')}</param>
> > > > >
> > > > > Resulted in the following:
> > > > >
> > > > > Error Code : EL_ERROR
> > > > > Error Message : Encountered "pDate", expected one of [".", ">",
> > > "gt",
> > > > > "<", "lt", "==", "eq", "<=", "le", ">=", "ge", "!=", "ne", ")",
> ",",
> > > "[",
> > > > > "+", "-", "*", "/", "div", "%", "mod", "and", "&&", "or", "||",
> "?"]
> > > > >
> > > > > On Thu, Jun 25, 2015 at 3:13 AM, Oussama Chougna <
> > > > oussama1983@hotmail.com
> > > > > <javascript:;>>
> > > > > wrote:
> > > > >
> > > > > > You should try this:
> > > > > > If you are using a coordinator:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:conf('pDate')')}</param>
> > > > > > If you only using a workflow:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=wf:conf('pDate')')}</param>
> > > > > >
> > > > > > Nested EL does not work.
> > > > > >
> > > > > >
> > > > > > Good luck,
> > > > > > Cheers.
> > > > > > Oussama Chougna
> > > > > >
> > > > > >
> > > > > >
> > > > > > > Date: Wed, 24 Jun 2015 20:49:32 +0000
> > > > > > > From: purushah@yahoo-inc.com.INVALID
> > > > > > > To: user@oozie.apache.org <javascript:;>
> > > > > > > Subject: Re: HDFS EL Function - fs:dirSize(String path)
> > > > > > >
> > > > > > >
> > > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=pDate')}</param>
> > > > > .
> > > > > > pDate without ${}.
> > > > > > > From: Vincent Peplinski <vinpeplinski@gmail.com
> > > <javascript:;>>
> > > > > > > To: user@oozie.apache.org <javascript:;>
> > > > > > > Sent: Wednesday, June 24, 2015 12:41 PM
> > > > > > > Subject: HDFS EL Function - fs:dirSize(String path)
> > > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > I'm trying to use the fs:dirSize() HDFS EL function in one of
> my
> > > > > > workflows
> > > > > > > action nodes but I'm having some difficulty and I hope someone
> can
> > > > > help.
> > > > > > >
> > > > > > > When I call the function using a literal string such as:
> > > > > > >
> > > > > > >
> > > > >
> > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=20150624')}</param>
> > > > > > >
> > > > > > >
> > > > > > > The function returns the size of the file as expected. However;
> > > when
> > > > I
> > > > > > try
> > > > > > > the following:
> > > > > > >
> > > > > > >
> > > > >
> > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=${pDate}')}</param>
> > > > > > >
> > > > > > > The workflow fails with an EL_ERROR - Illegal character in path
> > > index
> > > > > 26
> > > > > > > /user/peppy/testing/p_dt=${pDate}.
> > > > > > >
> > > > > > >
> > > > > > > How can I pass a variable to the function that will be properly
> > > > > expanded?
> > > > > > >
> > > > > > > Thank you in advance.
> > > > > > >
> > > > > > > V.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
>
>
RE: HDFS EL Function - fs:dirSize(String path)
Posted by Oussama Chougna <ou...@hotmail.com>.
I'm 100% sure that the suggested should work:
I have this in my workflow and it works perfect:
<decision name="check-preconditions-directories"> <switch> <!-- Check if the warehouse dir for all the data exists. If not, create it with the right permissions. --> <case to="create-warehouse-dirs">${!fs:exists(wf:conf('hiveMainTableLocation')) || !fs:exists(wf:conf('hive_temp_tables_root'))}</case> <!-- default: just proceed. --> <default to="preconditions-join"/> </switch> </decision>
in my bundle.xml (I'm using a bundle):
<property> <name>hiveMainTableLocation</name> <value>${hive_tables_root}/OrderItems</value> </property>
In my job.properties:
hive_temp_tables_root=${nameNode}/user/hive/warehouse/pzt/tmp/hive
This works 100%.
Oozie 4.1.0-cdh5.4.2
Cheers,
Oussama Chougna
> Date: Mon, 29 Jun 2015 10:15:15 +0200
> Subject: Re: HDFS EL Function - fs:dirSize(String path)
> From: laurent.hatier@gmail.com
> To: user@oozie.apache.org
>
> I think that is not possible because oozie can replace variable once, not
> twice (fsDir + your variable) I think you have to create your own Oozie
> function, or maybe use an shell action to do some stuf (fs dir of your
> folder in parameter) and then use an capture-output to get the size of your
> folder :)
>
> --
> Laurent HATIER - Consultant Big Data & Business Intelligence chez CapGemini
> fr.linkedin.com/pub/laurent-hatier/25/36b/a86/
> <http://fr.linkedin.com/pub/laurent-h/25/36b/a86/>
>
> 2015-06-25 22:33 GMT+02:00 Vincent Peplinski <vi...@gmail.com>:
>
> > Neither of those suggestions resolved the issue...
> >
> > On Thu, Jun 25, 2015 at 3:02 PM, Idris Ali <ps...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > Can you try changing the single quotes only around pDate to double quotes
> > > "". Seems to be xml wrapping issue.
> > >
> > > Thanks,
> > > Idris
> > >
> > > On Thursday 25 June 2015, Vincent Peplinski <vi...@gmail.com>
> > > wrote:
> > >
> > > > Thanks for the suggestion Oussama, unfortunately it did not resolve my
> > > > issue :(
> > > >
> > > > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:
> > > > conf('pDate')')}</param>
> > > >
> > > > Resulted in the following:
> > > >
> > > > Error Code : EL_ERROR
> > > > Error Message : Encountered "pDate", expected one of [".", ">",
> > "gt",
> > > > "<", "lt", "==", "eq", "<=", "le", ">=", "ge", "!=", "ne", ")", ",",
> > "[",
> > > > "+", "-", "*", "/", "div", "%", "mod", "and", "&&", "or", "||", "?"]
> > > >
> > > > On Thu, Jun 25, 2015 at 3:13 AM, Oussama Chougna <
> > > oussama1983@hotmail.com
> > > > <javascript:;>>
> > > > wrote:
> > > >
> > > > > You should try this:
> > > > > If you are using a coordinator:
> > > > >
> > > > >
> > > >
> > >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:conf('pDate')')}</param>
> > > > > If you only using a workflow:
> > > > >
> > > > >
> > > >
> > >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=wf:conf('pDate')')}</param>
> > > > >
> > > > > Nested EL does not work.
> > > > >
> > > > >
> > > > > Good luck,
> > > > > Cheers.
> > > > > Oussama Chougna
> > > > >
> > > > >
> > > > >
> > > > > > Date: Wed, 24 Jun 2015 20:49:32 +0000
> > > > > > From: purushah@yahoo-inc.com.INVALID
> > > > > > To: user@oozie.apache.org <javascript:;>
> > > > > > Subject: Re: HDFS EL Function - fs:dirSize(String path)
> > > > > >
> > > > > >
> > > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=pDate')}</param>
> > > > .
> > > > > pDate without ${}.
> > > > > > From: Vincent Peplinski <vinpeplinski@gmail.com
> > <javascript:;>>
> > > > > > To: user@oozie.apache.org <javascript:;>
> > > > > > Sent: Wednesday, June 24, 2015 12:41 PM
> > > > > > Subject: HDFS EL Function - fs:dirSize(String path)
> > > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I'm trying to use the fs:dirSize() HDFS EL function in one of my
> > > > > workflows
> > > > > > action nodes but I'm having some difficulty and I hope someone can
> > > > help.
> > > > > >
> > > > > > When I call the function using a literal string such as:
> > > > > >
> > > > > >
> > > >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=20150624')}</param>
> > > > > >
> > > > > >
> > > > > > The function returns the size of the file as expected. However;
> > when
> > > I
> > > > > try
> > > > > > the following:
> > > > > >
> > > > > >
> > > >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=${pDate}')}</param>
> > > > > >
> > > > > > The workflow fails with an EL_ERROR - Illegal character in path
> > index
> > > > 26
> > > > > > /user/peppy/testing/p_dt=${pDate}.
> > > > > >
> > > > > >
> > > > > > How can I pass a variable to the function that will be properly
> > > > expanded?
> > > > > >
> > > > > > Thank you in advance.
> > > > > >
> > > > > > V.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> >
Re: HDFS EL Function - fs:dirSize(String path)
Posted by Laurent H <la...@gmail.com>.
I think that is not possible because oozie can replace variable once, not
twice (fsDir + your variable) I think you have to create your own Oozie
function, or maybe use an shell action to do some stuf (fs dir of your
folder in parameter) and then use an capture-output to get the size of your
folder :)
--
Laurent HATIER - Consultant Big Data & Business Intelligence chez CapGemini
fr.linkedin.com/pub/laurent-hatier/25/36b/a86/
<http://fr.linkedin.com/pub/laurent-h/25/36b/a86/>
2015-06-25 22:33 GMT+02:00 Vincent Peplinski <vi...@gmail.com>:
> Neither of those suggestions resolved the issue...
>
> On Thu, Jun 25, 2015 at 3:02 PM, Idris Ali <ps...@gmail.com> wrote:
>
> > Hi,
> >
> > Can you try changing the single quotes only around pDate to double quotes
> > "". Seems to be xml wrapping issue.
> >
> > Thanks,
> > Idris
> >
> > On Thursday 25 June 2015, Vincent Peplinski <vi...@gmail.com>
> > wrote:
> >
> > > Thanks for the suggestion Oussama, unfortunately it did not resolve my
> > > issue :(
> > >
> > > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:
> > > conf('pDate')')}</param>
> > >
> > > Resulted in the following:
> > >
> > > Error Code : EL_ERROR
> > > Error Message : Encountered "pDate", expected one of [".", ">",
> "gt",
> > > "<", "lt", "==", "eq", "<=", "le", ">=", "ge", "!=", "ne", ")", ",",
> "[",
> > > "+", "-", "*", "/", "div", "%", "mod", "and", "&&", "or", "||", "?"]
> > >
> > > On Thu, Jun 25, 2015 at 3:13 AM, Oussama Chougna <
> > oussama1983@hotmail.com
> > > <javascript:;>>
> > > wrote:
> > >
> > > > You should try this:
> > > > If you are using a coordinator:
> > > >
> > > >
> > >
> >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:conf('pDate')')}</param>
> > > > If you only using a workflow:
> > > >
> > > >
> > >
> >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=wf:conf('pDate')')}</param>
> > > >
> > > > Nested EL does not work.
> > > >
> > > >
> > > > Good luck,
> > > > Cheers.
> > > > Oussama Chougna
> > > >
> > > >
> > > >
> > > > > Date: Wed, 24 Jun 2015 20:49:32 +0000
> > > > > From: purushah@yahoo-inc.com.INVALID
> > > > > To: user@oozie.apache.org <javascript:;>
> > > > > Subject: Re: HDFS EL Function - fs:dirSize(String path)
> > > > >
> > > > >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=pDate')}</param>
> > > .
> > > > pDate without ${}.
> > > > > From: Vincent Peplinski <vinpeplinski@gmail.com
> <javascript:;>>
> > > > > To: user@oozie.apache.org <javascript:;>
> > > > > Sent: Wednesday, June 24, 2015 12:41 PM
> > > > > Subject: HDFS EL Function - fs:dirSize(String path)
> > > > >
> > > > > Hello,
> > > > >
> > > > > I'm trying to use the fs:dirSize() HDFS EL function in one of my
> > > > workflows
> > > > > action nodes but I'm having some difficulty and I hope someone can
> > > help.
> > > > >
> > > > > When I call the function using a literal string such as:
> > > > >
> > > > >
> > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=20150624')}</param>
> > > > >
> > > > >
> > > > > The function returns the size of the file as expected. However;
> when
> > I
> > > > try
> > > > > the following:
> > > > >
> > > > >
> > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=${pDate}')}</param>
> > > > >
> > > > > The workflow fails with an EL_ERROR - Illegal character in path
> index
> > > 26
> > > > > /user/peppy/testing/p_dt=${pDate}.
> > > > >
> > > > >
> > > > > How can I pass a variable to the function that will be properly
> > > expanded?
> > > > >
> > > > > Thank you in advance.
> > > > >
> > > > > V.
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> >
>
Re: HDFS EL Function - fs:dirSize(String path)
Posted by Vincent Peplinski <vi...@gmail.com>.
Neither of those suggestions resolved the issue...
On Thu, Jun 25, 2015 at 3:02 PM, Idris Ali <ps...@gmail.com> wrote:
> Hi,
>
> Can you try changing the single quotes only around pDate to double quotes
> "". Seems to be xml wrapping issue.
>
> Thanks,
> Idris
>
> On Thursday 25 June 2015, Vincent Peplinski <vi...@gmail.com>
> wrote:
>
> > Thanks for the suggestion Oussama, unfortunately it did not resolve my
> > issue :(
> >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:
> > conf('pDate')')}</param>
> >
> > Resulted in the following:
> >
> > Error Code : EL_ERROR
> > Error Message : Encountered "pDate", expected one of [".", ">", "gt",
> > "<", "lt", "==", "eq", "<=", "le", ">=", "ge", "!=", "ne", ")", ",", "[",
> > "+", "-", "*", "/", "div", "%", "mod", "and", "&&", "or", "||", "?"]
> >
> > On Thu, Jun 25, 2015 at 3:13 AM, Oussama Chougna <
> oussama1983@hotmail.com
> > <javascript:;>>
> > wrote:
> >
> > > You should try this:
> > > If you are using a coordinator:
> > >
> > >
> >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:conf('pDate')')}</param>
> > > If you only using a workflow:
> > >
> > >
> >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=wf:conf('pDate')')}</param>
> > >
> > > Nested EL does not work.
> > >
> > >
> > > Good luck,
> > > Cheers.
> > > Oussama Chougna
> > >
> > >
> > >
> > > > Date: Wed, 24 Jun 2015 20:49:32 +0000
> > > > From: purushah@yahoo-inc.com.INVALID
> > > > To: user@oozie.apache.org <javascript:;>
> > > > Subject: Re: HDFS EL Function - fs:dirSize(String path)
> > > >
> > > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=pDate')}</param>
> > .
> > > pDate without ${}.
> > > > From: Vincent Peplinski <vinpeplinski@gmail.com <javascript:;>>
> > > > To: user@oozie.apache.org <javascript:;>
> > > > Sent: Wednesday, June 24, 2015 12:41 PM
> > > > Subject: HDFS EL Function - fs:dirSize(String path)
> > > >
> > > > Hello,
> > > >
> > > > I'm trying to use the fs:dirSize() HDFS EL function in one of my
> > > workflows
> > > > action nodes but I'm having some difficulty and I hope someone can
> > help.
> > > >
> > > > When I call the function using a literal string such as:
> > > >
> > > >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=20150624')}</param>
> > > >
> > > >
> > > > The function returns the size of the file as expected. However; when
> I
> > > try
> > > > the following:
> > > >
> > > >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=${pDate}')}</param>
> > > >
> > > > The workflow fails with an EL_ERROR - Illegal character in path index
> > 26
> > > > /user/peppy/testing/p_dt=${pDate}.
> > > >
> > > >
> > > > How can I pass a variable to the function that will be properly
> > expanded?
> > > >
> > > > Thank you in advance.
> > > >
> > > > V.
> > > >
> > > >
> > > >
> > >
> > >
> >
>
Re: HDFS EL Function - fs:dirSize(String path)
Posted by Idris Ali <ps...@gmail.com>.
Hi,
Can you try changing the single quotes only around pDate to double quotes
"". Seems to be xml wrapping issue.
Thanks,
Idris
On Thursday 25 June 2015, Vincent Peplinski <vi...@gmail.com> wrote:
> Thanks for the suggestion Oussama, unfortunately it did not resolve my
> issue :(
>
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:
> conf('pDate')')}</param>
>
> Resulted in the following:
>
> Error Code : EL_ERROR
> Error Message : Encountered "pDate", expected one of [".", ">", "gt",
> "<", "lt", "==", "eq", "<=", "le", ">=", "ge", "!=", "ne", ")", ",", "[",
> "+", "-", "*", "/", "div", "%", "mod", "and", "&&", "or", "||", "?"]
>
> On Thu, Jun 25, 2015 at 3:13 AM, Oussama Chougna <oussama1983@hotmail.com
> <javascript:;>>
> wrote:
>
> > You should try this:
> > If you are using a coordinator:
> >
> >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:conf('pDate')')}</param>
> > If you only using a workflow:
> >
> >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=wf:conf('pDate')')}</param>
> >
> > Nested EL does not work.
> >
> >
> > Good luck,
> > Cheers.
> > Oussama Chougna
> >
> >
> >
> > > Date: Wed, 24 Jun 2015 20:49:32 +0000
> > > From: purushah@yahoo-inc.com.INVALID
> > > To: user@oozie.apache.org <javascript:;>
> > > Subject: Re: HDFS EL Function - fs:dirSize(String path)
> > >
> > > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=pDate')}</param>
> .
> > pDate without ${}.
> > > From: Vincent Peplinski <vinpeplinski@gmail.com <javascript:;>>
> > > To: user@oozie.apache.org <javascript:;>
> > > Sent: Wednesday, June 24, 2015 12:41 PM
> > > Subject: HDFS EL Function - fs:dirSize(String path)
> > >
> > > Hello,
> > >
> > > I'm trying to use the fs:dirSize() HDFS EL function in one of my
> > workflows
> > > action nodes but I'm having some difficulty and I hope someone can
> help.
> > >
> > > When I call the function using a literal string such as:
> > >
> > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=20150624')}</param>
> > >
> > >
> > > The function returns the size of the file as expected. However; when I
> > try
> > > the following:
> > >
> > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=${pDate}')}</param>
> > >
> > > The workflow fails with an EL_ERROR - Illegal character in path index
> 26
> > > /user/peppy/testing/p_dt=${pDate}.
> > >
> > >
> > > How can I pass a variable to the function that will be properly
> expanded?
> > >
> > > Thank you in advance.
> > >
> > > V.
> > >
> > >
> > >
> >
> >
>
RE: HDFS EL Function - fs:dirSize(String path)
Posted by Oussama Chougna <ou...@hotmail.com>.
Hi Vincent,
It is not correct how you fixed it. You cannot use coord:conf() in the workflow definition, you should use wf:conf('pDate').Did you try that?
Cheers,
Oussama Chougna
Date: Thu, 25 Jun 2015 12:57:42 -0400
Subject: Re: HDFS EL Function - fs:dirSize(String path)
From: vinpeplinski@gmail.com
To: user@oozie.apache.org
I'm creating a coordinator application. See the attachment for workflow and coordinator xml files.
On Thu, Jun 25, 2015 at 9:37 AM, Oussama Chougna <ou...@hotmail.com> wrote:
Vincent,
Are you creating a workflow ony oozie application or a coordinator application?
Could you post some of your xml definition files?
Thanks.
Oussama Chougna
> Date: Thu, 25 Jun 2015 08:27:35 -0400
> Subject: Re: HDFS EL Function - fs:dirSize(String path)
> From: vinpeplinski@gmail.com
> To: user@oozie.apache.org
>
> Thanks for the suggestion Oussama, unfortunately it did not resolve my
> issue :(
>
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:
> conf('pDate')')}</param>
>
> Resulted in the following:
>
> Error Code : EL_ERROR
> Error Message : Encountered "pDate", expected one of [".", ">", "gt",
> "<", "lt", "==", "eq", "<=", "le", ">=", "ge", "!=", "ne", ")", ",", "[",
> "+", "-", "*", "/", "div", "%", "mod", "and", "&&", "or", "||", "?"]
>
> On Thu, Jun 25, 2015 at 3:13 AM, Oussama Chougna <ou...@hotmail.com>
> wrote:
>
> > You should try this:
> > If you are using a coordinator:
> >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:conf('pDate')')}</param>
> > If you only using a workflow:
> >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=wf:conf('pDate')')}</param>
> >
> > Nested EL does not work.
> >
> >
> > Good luck,
> > Cheers.
> > Oussama Chougna
> >
> >
> >
> > > Date: Wed, 24 Jun 2015 20:49:32 +0000
> > > From: purushah@yahoo-inc.com.INVALID
> > > To: user@oozie.apache.org
> > > Subject: Re: HDFS EL Function - fs:dirSize(String path)
> > >
> > > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=pDate')}</param> .
> > pDate without ${}.
> > > From: Vincent Peplinski <vi...@gmail.com>
> > > To: user@oozie.apache.org
> > > Sent: Wednesday, June 24, 2015 12:41 PM
> > > Subject: HDFS EL Function - fs:dirSize(String path)
> > >
> > > Hello,
> > >
> > > I'm trying to use the fs:dirSize() HDFS EL function in one of my
> > workflows
> > > action nodes but I'm having some difficulty and I hope someone can help.
> > >
> > > When I call the function using a literal string such as:
> > >
> > > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=20150624')}</param>
> > >
> > >
> > > The function returns the size of the file as expected. However; when I
> > try
> > > the following:
> > >
> > > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=${pDate}')}</param>
> > >
> > > The workflow fails with an EL_ERROR - Illegal character in path index 26
> > > /user/peppy/testing/p_dt=${pDate}.
> > >
> > >
> > > How can I pass a variable to the function that will be properly expanded?
> > >
> > > Thank you in advance.
> > >
> > > V.
> > >
> > >
> > >
> >
> >
Re: HDFS EL Function - fs:dirSize(String path)
Posted by Vincent Peplinski <vi...@gmail.com>.
I'm creating a coordinator application. See the attachment for workflow and
coordinator xml files.
On Thu, Jun 25, 2015 at 9:37 AM, Oussama Chougna <ou...@hotmail.com>
wrote:
> Vincent,
> Are you creating a workflow ony oozie application or a coordinator
> application?
> Could you post some of your xml definition files?
> Thanks.
>
> Oussama Chougna
>
> > Date: Thu, 25 Jun 2015 08:27:35 -0400
> > Subject: Re: HDFS EL Function - fs:dirSize(String path)
> > From: vinpeplinski@gmail.com
> > To: user@oozie.apache.org
> >
> > Thanks for the suggestion Oussama, unfortunately it did not resolve my
> > issue :(
> >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:
> > conf('pDate')')}</param>
> >
> > Resulted in the following:
> >
> > Error Code : EL_ERROR
> > Error Message : Encountered "pDate", expected one of [".", ">", "gt",
> > "<", "lt", "==", "eq", "<=", "le", ">=", "ge", "!=", "ne", ")", ",", "[",
> > "+", "-", "*", "/", "div", "%", "mod", "and", "&&", "or", "||", "?"]
> >
> > On Thu, Jun 25, 2015 at 3:13 AM, Oussama Chougna <
> oussama1983@hotmail.com>
> > wrote:
> >
> > > You should try this:
> > > If you are using a coordinator:
> > >
> > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:conf('pDate')')}</param>
> > > If you only using a workflow:
> > >
> > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=wf:conf('pDate')')}</param>
> > >
> > > Nested EL does not work.
> > >
> > >
> > > Good luck,
> > > Cheers.
> > > Oussama Chougna
> > >
> > >
> > >
> > > > Date: Wed, 24 Jun 2015 20:49:32 +0000
> > > > From: purushah@yahoo-inc.com.INVALID
> > > > To: user@oozie.apache.org
> > > > Subject: Re: HDFS EL Function - fs:dirSize(String path)
> > > >
> > > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=pDate')}</param> .
> > > pDate without ${}.
> > > > From: Vincent Peplinski <vi...@gmail.com>
> > > > To: user@oozie.apache.org
> > > > Sent: Wednesday, June 24, 2015 12:41 PM
> > > > Subject: HDFS EL Function - fs:dirSize(String path)
> > > >
> > > > Hello,
> > > >
> > > > I'm trying to use the fs:dirSize() HDFS EL function in one of my
> > > workflows
> > > > action nodes but I'm having some difficulty and I hope someone can
> help.
> > > >
> > > > When I call the function using a literal string such as:
> > > >
> > > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=20150624')}</param>
> > > >
> > > >
> > > > The function returns the size of the file as expected. However; when
> I
> > > try
> > > > the following:
> > > >
> > > >
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=${pDate}')}</param>
> > > >
> > > > The workflow fails with an EL_ERROR - Illegal character in path
> index 26
> > > > /user/peppy/testing/p_dt=${pDate}.
> > > >
> > > >
> > > > How can I pass a variable to the function that will be properly
> expanded?
> > > >
> > > > Thank you in advance.
> > > >
> > > > V.
> > > >
> > > >
> > > >
> > >
> > >
>
>
RE: HDFS EL Function - fs:dirSize(String path)
Posted by Oussama Chougna <ou...@hotmail.com>.
Vincent,
Are you creating a workflow ony oozie application or a coordinator application?
Could you post some of your xml definition files?
Thanks.
Oussama Chougna
> Date: Thu, 25 Jun 2015 08:27:35 -0400
> Subject: Re: HDFS EL Function - fs:dirSize(String path)
> From: vinpeplinski@gmail.com
> To: user@oozie.apache.org
>
> Thanks for the suggestion Oussama, unfortunately it did not resolve my
> issue :(
>
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:
> conf('pDate')')}</param>
>
> Resulted in the following:
>
> Error Code : EL_ERROR
> Error Message : Encountered "pDate", expected one of [".", ">", "gt",
> "<", "lt", "==", "eq", "<=", "le", ">=", "ge", "!=", "ne", ")", ",", "[",
> "+", "-", "*", "/", "div", "%", "mod", "and", "&&", "or", "||", "?"]
>
> On Thu, Jun 25, 2015 at 3:13 AM, Oussama Chougna <ou...@hotmail.com>
> wrote:
>
> > You should try this:
> > If you are using a coordinator:
> >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:conf('pDate')')}</param>
> > If you only using a workflow:
> >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=wf:conf('pDate')')}</param>
> >
> > Nested EL does not work.
> >
> >
> > Good luck,
> > Cheers.
> > Oussama Chougna
> >
> >
> >
> > > Date: Wed, 24 Jun 2015 20:49:32 +0000
> > > From: purushah@yahoo-inc.com.INVALID
> > > To: user@oozie.apache.org
> > > Subject: Re: HDFS EL Function - fs:dirSize(String path)
> > >
> > > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=pDate')}</param> .
> > pDate without ${}.
> > > From: Vincent Peplinski <vi...@gmail.com>
> > > To: user@oozie.apache.org
> > > Sent: Wednesday, June 24, 2015 12:41 PM
> > > Subject: HDFS EL Function - fs:dirSize(String path)
> > >
> > > Hello,
> > >
> > > I'm trying to use the fs:dirSize() HDFS EL function in one of my
> > workflows
> > > action nodes but I'm having some difficulty and I hope someone can help.
> > >
> > > When I call the function using a literal string such as:
> > >
> > > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=20150624')}</param>
> > >
> > >
> > > The function returns the size of the file as expected. However; when I
> > try
> > > the following:
> > >
> > > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=${pDate}')}</param>
> > >
> > > The workflow fails with an EL_ERROR - Illegal character in path index 26
> > > /user/peppy/testing/p_dt=${pDate}.
> > >
> > >
> > > How can I pass a variable to the function that will be properly expanded?
> > >
> > > Thank you in advance.
> > >
> > > V.
> > >
> > >
> > >
> >
> >
Re: HDFS EL Function - fs:dirSize(String path)
Posted by Vincent Peplinski <vi...@gmail.com>.
Thanks for the suggestion Oussama, unfortunately it did not resolve my
issue :(
<param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:
conf('pDate')')}</param>
Resulted in the following:
Error Code : EL_ERROR
Error Message : Encountered "pDate", expected one of [".", ">", "gt",
"<", "lt", "==", "eq", "<=", "le", ">=", "ge", "!=", "ne", ")", ",", "[",
"+", "-", "*", "/", "div", "%", "mod", "and", "&&", "or", "||", "?"]
On Thu, Jun 25, 2015 at 3:13 AM, Oussama Chougna <ou...@hotmail.com>
wrote:
> You should try this:
> If you are using a coordinator:
>
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:conf('pDate')')}</param>
> If you only using a workflow:
>
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=wf:conf('pDate')')}</param>
>
> Nested EL does not work.
>
>
> Good luck,
> Cheers.
> Oussama Chougna
>
>
>
> > Date: Wed, 24 Jun 2015 20:49:32 +0000
> > From: purushah@yahoo-inc.com.INVALID
> > To: user@oozie.apache.org
> > Subject: Re: HDFS EL Function - fs:dirSize(String path)
> >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=pDate')}</param> .
> pDate without ${}.
> > From: Vincent Peplinski <vi...@gmail.com>
> > To: user@oozie.apache.org
> > Sent: Wednesday, June 24, 2015 12:41 PM
> > Subject: HDFS EL Function - fs:dirSize(String path)
> >
> > Hello,
> >
> > I'm trying to use the fs:dirSize() HDFS EL function in one of my
> workflows
> > action nodes but I'm having some difficulty and I hope someone can help.
> >
> > When I call the function using a literal string such as:
> >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=20150624')}</param>
> >
> >
> > The function returns the size of the file as expected. However; when I
> try
> > the following:
> >
> > <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=${pDate}')}</param>
> >
> > The workflow fails with an EL_ERROR - Illegal character in path index 26
> > /user/peppy/testing/p_dt=${pDate}.
> >
> >
> > How can I pass a variable to the function that will be properly expanded?
> >
> > Thank you in advance.
> >
> > V.
> >
> >
> >
>
>
RE: HDFS EL Function - fs:dirSize(String path)
Posted by Oussama Chougna <ou...@hotmail.com>.
You should try this:
If you are using a coordinator:
<param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=coord:conf('pDate')')}</param>
If you only using a workflow:
<param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=wf:conf('pDate')')}</param>
Nested EL does not work.
Good luck,
Cheers.
Oussama Chougna
> Date: Wed, 24 Jun 2015 20:49:32 +0000
> From: purushah@yahoo-inc.com.INVALID
> To: user@oozie.apache.org
> Subject: Re: HDFS EL Function - fs:dirSize(String path)
>
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=pDate')}</param> . pDate without ${}.
> From: Vincent Peplinski <vi...@gmail.com>
> To: user@oozie.apache.org
> Sent: Wednesday, June 24, 2015 12:41 PM
> Subject: HDFS EL Function - fs:dirSize(String path)
>
> Hello,
>
> I'm trying to use the fs:dirSize() HDFS EL function in one of my workflows
> action nodes but I'm having some difficulty and I hope someone can help.
>
> When I call the function using a literal string such as:
>
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=20150624')}</param>
>
>
> The function returns the size of the file as expected. However; when I try
> the following:
>
> <param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=${pDate}')}</param>
>
> The workflow fails with an EL_ERROR - Illegal character in path index 26
> /user/peppy/testing/p_dt=${pDate}.
>
>
> How can I pass a variable to the function that will be properly expanded?
>
> Thank you in advance.
>
> V.
>
>
>
Re: HDFS EL Function - fs:dirSize(String path)
Posted by Purshotam Shah <pu...@yahoo-inc.com.INVALID>.
<param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=pDate')}</param> . pDate without ${}.
From: Vincent Peplinski <vi...@gmail.com>
To: user@oozie.apache.org
Sent: Wednesday, June 24, 2015 12:41 PM
Subject: HDFS EL Function - fs:dirSize(String path)
Hello,
I'm trying to use the fs:dirSize() HDFS EL function in one of my workflows
action nodes but I'm having some difficulty and I hope someone can help.
When I call the function using a literal string such as:
<param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=20150624')}</param>
The function returns the size of the file as expected. However; when I try
the following:
<param>byt_wrt=${fs:dirSize('/user/peppy/testing/p_dt=${pDate}')}</param>
The workflow fails with an EL_ERROR - Illegal character in path index 26
/user/peppy/testing/p_dt=${pDate}.
How can I pass a variable to the function that will be properly expanded?
Thank you in advance.
V.