You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Chris Stillwell <ch...@farecompare.com> on 2012/05/30 21:08:49 UTC
Custom input format class ignored
I am trying to run an existing mapreduce job as an oozie workflow job. The
job uses a custom input format class which I specified in the workflow.xml
as below. However, when running under oozie the job fails with an
exception and the diagnostice message indicates that the input has not been
processed by our custom input format class. Are there any other properties
that would need to be set in the workflow.xml to ensure that our custom
class is actually used and not ignored.
<property>
<name>mapred.input.format.class</name>
<value>com.farecompare.atpcore.storageenginemodule.impl.hbase.util.ZipFileInputFormat</value>
</property>
Thanks
--
*Chris Stillwell*
*Email: *chris.stillwell@farecompare.com
*FareCompare* <http://www.farecompare.com/signup>
<http://maps.google.com/maps?q=18111+Preston+Road%2CSuite+800%2CDallas%2CTX+75252&hl=en>18111
Preston Rd
Suite 800
Dallas, TX 75252
*Phone:* (972) 755-1058 ext. xxx
*Web:* www.farecompare.com
*Find deals from your airport <http://www.farecompare.com/twitter> | Connect
with FareCompare on Facebook <http://www.facebook.com/farecompare>*
Re: Custom input format class ignored
Posted by Chris Stillwell <ch...@farecompare.com>.
It turns out I had gotten a hold of a bad example. The correct property
name to use is <name>mapreduce.inputformat.class</name> with no dot between
input and format. It now works.
On Wed, May 30, 2012 at 6:09 PM, Mona Chitnis <ch...@yahoo-inc.com> wrote:
> Hi Chris,
>
> Can you provide the exception trace you are seeing? You might need to set
> additional properties such as the mapper and reducer class which handle
> this type of input.
>
>
> --
> Mona Chitnis
> Oozie dev
>
>
>
>
> On 5/30/12 12:08 PM, "Chris Stillwell" <ch...@farecompare.com>
> wrote:
>
> >I am trying to run an existing mapreduce job as an oozie workflow job.
> >The
> >job uses a custom input format class which I specified in the workflow.xml
> >as below. However, when running under oozie the job fails with an
> >exception and the diagnostice message indicates that the input has not
> >been
> >processed by our custom input format class. Are there any other
> >properties
> >that would need to be set in the workflow.xml to ensure that our custom
> >class is actually used and not ignored.
> >
> > <property>
> > <name>mapred.input.format.class</name>
> >
> >
> ><value>com.farecompare.atpcore.storageenginemodule.impl.hbase.util.ZipFile
> >InputFormat</value>
> > </property>
> >
> >Thanks
> >--
> >*Chris Stillwell*
> >
> >*Email: *chris.stillwell@farecompare.com
> >*FareCompare* <http://www.farecompare.com/signup>
> ><
> http://maps.google.com/maps?q=18111+Preston+Road%2CSuite+800%2CDallas%2CT
> >X+75252&hl=en>18111
> >Preston Rd
> >Suite 800
> >Dallas, TX 75252
> >*Phone:* (972) 755-1058 ext. xxx
> >*Web:* www.farecompare.com
> >*Find deals from your airport <http://www.farecompare.com/twitter> |
> >Connect
> >with FareCompare on Facebook <http://www.facebook.com/farecompare>*
>
>
--
*Chris Stillwell*
*Email: *chris.stillwell@farecompare.com
*FareCompare* <http://www.farecompare.com/signup>
<http://maps.google.com/maps?q=18111+Preston+Road%2CSuite+800%2CDallas%2CTX+75252&hl=en>18111
Preston Rd
Suite 800
Dallas, TX 75252
*Phone:* (972) 755-1058 ext. xxx
*Web:* www.farecompare.com
*Find deals from your airport <http://www.farecompare.com/twitter> | Connect
with FareCompare on Facebook <http://www.facebook.com/farecompare>*
Re: Custom input format class ignored
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
Hi Chris,
Can you provide the exception trace you are seeing? You might need to set
additional properties such as the mapper and reducer class which handle
this type of input.
--
Mona Chitnis
Oozie dev
On 5/30/12 12:08 PM, "Chris Stillwell" <ch...@farecompare.com>
wrote:
>I am trying to run an existing mapreduce job as an oozie workflow job.
>The
>job uses a custom input format class which I specified in the workflow.xml
>as below. However, when running under oozie the job fails with an
>exception and the diagnostice message indicates that the input has not
>been
>processed by our custom input format class. Are there any other
>properties
>that would need to be set in the workflow.xml to ensure that our custom
>class is actually used and not ignored.
>
> <property>
> <name>mapred.input.format.class</name>
>
>
><value>com.farecompare.atpcore.storageenginemodule.impl.hbase.util.ZipFile
>InputFormat</value>
> </property>
>
>Thanks
>--
>*Chris Stillwell*
>
>*Email: *chris.stillwell@farecompare.com
>*FareCompare* <http://www.farecompare.com/signup>
><http://maps.google.com/maps?q=18111+Preston+Road%2CSuite+800%2CDallas%2CT
>X+75252&hl=en>18111
>Preston Rd
>Suite 800
>Dallas, TX 75252
>*Phone:* (972) 755-1058 ext. xxx
>*Web:* www.farecompare.com
>*Find deals from your airport <http://www.farecompare.com/twitter> |
>Connect
>with FareCompare on Facebook <http://www.facebook.com/farecompare>*