You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Gautam Singaraju <ga...@gmail.com> on 2012/05/30 23:37:37 UTC
Oozie newbie question: V 3.2 on AWS
I might be overlooking something very minor.
But, I just installed RC 1 for Oozie version 3.2 and ran the example
program; it worked great. I wrote another and placed the jar file in the
lib folder. However, the jar file is not being picked up. Instead a default
MR seems to be running. Any suggestions?
Thanks!
---
Gautam
Re: Oozie newbie question: V 3.2 on AWS
Posted by Gautam Singaraju <ga...@gmail.com>.
Thanks Mona. I should have sent this earlier. My job.config is:
<workflow-app xmlns="uri:oozie:workflow:0.2" name="wordCount">
<start to="wc"/>
<action name="wc">
<map-reduce>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<prepare>
<delete path="${outputDir}"/>
</prepare>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
<property>
<name>mapred.mapper.class</name>
<value>org.wcMapper</value>
</property>
<property>
<name>mapred.reducer.class</name>
<value>org.wcReducer</value>
</property>
<property>
<name>mapred.reduce.tasks</name>
<value>21</value>
</property>
<property>
<name>mapred.mapper.new-api</name>
<value>true</value>
</property>
<property>
<name>mapred.reducer.new-api</name>
<value>true</value>
</property>
<property>
<name>mapred.input.dir</name>
<value>${inputDir}</value>
</property>
<property>
<name>mapred.output.dir</name>
<value>${outputDir}</value>
</property>
</configuration>
</map-reduce>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Map/Reduce failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
Thank you,
---
Gautam
On Wed, May 30, 2012 at 3:51 PM, Mona Chitnis <ch...@yahoo-inc.com> wrote:
> Hi,
>
> I must have misunderstood your use-case then. Does you jar specify your
> custom mapper and reducer class which you wish to use instead of the
> default MR? In that case, you need to specify those classes through your
> workflow.xml configuration.
>
> <property>
> <name>mapred.mapper.class</name>
> <value>org.apache.oozie.example.SampleMapper</value>
> </property>
> <property>
> <name>mapred.reducer.class</name>
> <value>org.apache.oozie.example.SampleReducer</value>
> </property>
>
>
> --
> Mona Chitnis
>
>
>
>
> On 5/30/12 3:09 PM, "Gautam Singaraju" <ga...@gmail.com> wrote:
>
> >Hi Mona,
> >From the jobtracker's job.xml UI, I can see that the jar file, mapper, and
> >reducers have been correctly specified.
> >---
> >Gautam
> >
> >
> >On Wed, May 30, 2012 at 2:59 PM, Gautam Singaraju <
> >gautam.singaraju@gmail.com> wrote:
> >
> >> Hi Mona,
> >>
> >> Thank you for the suggestion. I have not set oozie.use.system.libpath.
> >>
> >> ---
> >> Gautam
> >>
> >>
> >>
> >> On Wed, May 30, 2012 at 2:42 PM, Mona Chitnis
> >><ch...@yahoo-inc.com>wrote:
> >>
> >>> Gautam,
> >>>
> >>> Have you set the property "oozie.use.system.libpath" to true anywhere?
> >>> This will cause the server to use its system libraries instead.
> >>>
> >>> --
> >>> Mona Chitnis,
> >>> Oozie dev
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On 5/30/12 2:37 PM, "Gautam Singaraju" <ga...@gmail.com>
> >>> wrote:
> >>>
> >>> >I might be overlooking something very minor.
> >>> >But, I just installed RC 1 for Oozie version 3.2 and ran the example
> >>> >program; it worked great. I wrote another and placed the jar file in
> >>>the
> >>> >lib folder. However, the jar file is not being picked up. Instead a
> >>> >default
> >>> >MR seems to be running. Any suggestions?
> >>> >Thanks!
> >>> >---
> >>> >Gautam
> >>>
> >>>
> >>
>
>
Re: Oozie newbie question: V 3.2 on AWS
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
Hi,
I must have misunderstood your use-case then. Does you jar specify your
custom mapper and reducer class which you wish to use instead of the
default MR? In that case, you need to specify those classes through your
workflow.xml configuration.
<property>
<name>mapred.mapper.class</name>
<value>org.apache.oozie.example.SampleMapper</value>
</property>
<property>
<name>mapred.reducer.class</name>
<value>org.apache.oozie.example.SampleReducer</value>
</property>
--
Mona Chitnis
On 5/30/12 3:09 PM, "Gautam Singaraju" <ga...@gmail.com> wrote:
>Hi Mona,
>From the jobtracker's job.xml UI, I can see that the jar file, mapper, and
>reducers have been correctly specified.
>---
>Gautam
>
>
>On Wed, May 30, 2012 at 2:59 PM, Gautam Singaraju <
>gautam.singaraju@gmail.com> wrote:
>
>> Hi Mona,
>>
>> Thank you for the suggestion. I have not set oozie.use.system.libpath.
>>
>> ---
>> Gautam
>>
>>
>>
>> On Wed, May 30, 2012 at 2:42 PM, Mona Chitnis
>><ch...@yahoo-inc.com>wrote:
>>
>>> Gautam,
>>>
>>> Have you set the property "oozie.use.system.libpath" to true anywhere?
>>> This will cause the server to use its system libraries instead.
>>>
>>> --
>>> Mona Chitnis,
>>> Oozie dev
>>>
>>>
>>>
>>>
>>>
>>> On 5/30/12 2:37 PM, "Gautam Singaraju" <ga...@gmail.com>
>>> wrote:
>>>
>>> >I might be overlooking something very minor.
>>> >But, I just installed RC 1 for Oozie version 3.2 and ran the example
>>> >program; it worked great. I wrote another and placed the jar file in
>>>the
>>> >lib folder. However, the jar file is not being picked up. Instead a
>>> >default
>>> >MR seems to be running. Any suggestions?
>>> >Thanks!
>>> >---
>>> >Gautam
>>>
>>>
>>
Re: Oozie newbie question: V 3.2 on AWS
Posted by Gautam Singaraju <ga...@gmail.com>.
Hi Mona,
>From the jobtracker's job.xml UI, I can see that the jar file, mapper, and
reducers have been correctly specified.
---
Gautam
On Wed, May 30, 2012 at 2:59 PM, Gautam Singaraju <
gautam.singaraju@gmail.com> wrote:
> Hi Mona,
>
> Thank you for the suggestion. I have not set oozie.use.system.libpath.
>
> ---
> Gautam
>
>
>
> On Wed, May 30, 2012 at 2:42 PM, Mona Chitnis <ch...@yahoo-inc.com>wrote:
>
>> Gautam,
>>
>> Have you set the property "oozie.use.system.libpath" to true anywhere?
>> This will cause the server to use its system libraries instead.
>>
>> --
>> Mona Chitnis,
>> Oozie dev
>>
>>
>>
>>
>>
>> On 5/30/12 2:37 PM, "Gautam Singaraju" <ga...@gmail.com>
>> wrote:
>>
>> >I might be overlooking something very minor.
>> >But, I just installed RC 1 for Oozie version 3.2 and ran the example
>> >program; it worked great. I wrote another and placed the jar file in the
>> >lib folder. However, the jar file is not being picked up. Instead a
>> >default
>> >MR seems to be running. Any suggestions?
>> >Thanks!
>> >---
>> >Gautam
>>
>>
>
Re: Oozie newbie question: V 3.2 on AWS
Posted by Gautam Singaraju <ga...@gmail.com>.
Hi Mona,
Thank you for the suggestion. I have not set oozie.use.system.libpath.
---
Gautam
On Wed, May 30, 2012 at 2:42 PM, Mona Chitnis <ch...@yahoo-inc.com> wrote:
> Gautam,
>
> Have you set the property "oozie.use.system.libpath" to true anywhere?
> This will cause the server to use its system libraries instead.
>
> --
> Mona Chitnis,
> Oozie dev
>
>
>
>
>
> On 5/30/12 2:37 PM, "Gautam Singaraju" <ga...@gmail.com> wrote:
>
> >I might be overlooking something very minor.
> >But, I just installed RC 1 for Oozie version 3.2 and ran the example
> >program; it worked great. I wrote another and placed the jar file in the
> >lib folder. However, the jar file is not being picked up. Instead a
> >default
> >MR seems to be running. Any suggestions?
> >Thanks!
> >---
> >Gautam
>
>
Re: Oozie newbie question: V 3.2 on AWS
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
Gautam,
Have you set the property "oozie.use.system.libpath" to true anywhere?
This will cause the server to use its system libraries instead.
--
Mona Chitnis,
Oozie dev
On 5/30/12 2:37 PM, "Gautam Singaraju" <ga...@gmail.com> wrote:
>I might be overlooking something very minor.
>But, I just installed RC 1 for Oozie version 3.2 and ran the example
>program; it worked great. I wrote another and placed the jar file in the
>lib folder. However, the jar file is not being picked up. Instead a
>default
>MR seems to be running. Any suggestions?
>Thanks!
>---
>Gautam