You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Jeff Zhang <zj...@gmail.com> on 2011/03/23 02:19:20 UTC
Is there any way to add jar when invoking hadoop command
Hi all,
When I use command "hadoop fs -text" I need to add extra jar to CLASSPATH,
because there's custom type in my sequence file. One way is that copying jar
to $HADOOP_HOME/lib
But in my case, I am not administrator, so I do not have the permission to
copy files under $HADOOP_HOME/lib
Is there any other ways to add extra jar to CLASSPATH ?
--
Best Regards
Jeff Zhang
Re: Is there any way to add jar when invoking hadoop command
Posted by Jeff Zhang <zj...@gmail.com>.
Thanks, it works
On Wed, Mar 23, 2011 at 10:04 AM, James Seigel <ja...@tynt.com> wrote:
> Ahhhhh lib jars should do...let me see if I can dig up an example.
>
>
> hadoop fs -libjars ./adhoc/xxxxx.jar -text
> /xxxx/xxx/xxx/xxxx/20110101/part-r-00001.evt
>
>
> cheers
> James.
>
>
>
> On 2011-03-22, at 8:01 PM, Jeff Zhang wrote:
>
> > Actually I don't use the jar for mapreduce job. I only need it to display
> > sequence file.
> >
> >
> >
> > On Wed, Mar 23, 2011 at 9:41 AM, James Seigel <ja...@tynt.com> wrote:
> >
> >> Hello, some quick advice for you....
> >>
> >>
> >> which portion of your job needs the jar? if answer = mapper or reducer,
> >> then add it to the -libjars flag.
> >>
> >> If it is in the job initiation..bundle it in your job jar for fun.
> >>
> >> Cheers
> >> James.
> >> On 2011-03-22, at 7:35 PM, Jeff Zhang wrote:
> >>
> >>> Another work around I can think of is that have my own copy of hadoop,
> >> and
> >>> copy extra jars to my hadoop. But it result into more maintenance
> effort
> >>>
> >>> On Wed, Mar 23, 2011 at 9:19 AM, Jeff Zhang <zj...@gmail.com> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> When I use command "hadoop fs -text" I need to add extra jar to
> >> CLASSPATH,
> >>>> because there's custom type in my sequence file. One way is that
> copying
> >> jar
> >>>> to $HADOOP_HOME/lib
> >>>> But in my case, I am not administrator, so I do not have the
> permission
> >> to
> >>>> copy files under $HADOOP_HOME/lib
> >>>>
> >>>> Is there any other ways to add extra jar to CLASSPATH ?
> >>>>
> >>>>
> >>>> --
> >>>> Best Regards
> >>>>
> >>>> Jeff Zhang
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Best Regards
> >>>
> >>> Jeff Zhang
> >>
> >>
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
>
>
--
Best Regards
Jeff Zhang
Re: Is there any way to add jar when invoking hadoop command
Posted by James Seigel <ja...@tynt.com>.
Ahhhhh lib jars should do...let me see if I can dig up an example.
hadoop fs -libjars ./adhoc/xxxxx.jar -text /xxxx/xxx/xxx/xxxx/20110101/part-r-00001.evt
cheers
James.
On 2011-03-22, at 8:01 PM, Jeff Zhang wrote:
> Actually I don't use the jar for mapreduce job. I only need it to display
> sequence file.
>
>
>
> On Wed, Mar 23, 2011 at 9:41 AM, James Seigel <ja...@tynt.com> wrote:
>
>> Hello, some quick advice for you....
>>
>>
>> which portion of your job needs the jar? if answer = mapper or reducer,
>> then add it to the -libjars flag.
>>
>> If it is in the job initiation..bundle it in your job jar for fun.
>>
>> Cheers
>> James.
>> On 2011-03-22, at 7:35 PM, Jeff Zhang wrote:
>>
>>> Another work around I can think of is that have my own copy of hadoop,
>> and
>>> copy extra jars to my hadoop. But it result into more maintenance effort
>>>
>>> On Wed, Mar 23, 2011 at 9:19 AM, Jeff Zhang <zj...@gmail.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> When I use command "hadoop fs -text" I need to add extra jar to
>> CLASSPATH,
>>>> because there's custom type in my sequence file. One way is that copying
>> jar
>>>> to $HADOOP_HOME/lib
>>>> But in my case, I am not administrator, so I do not have the permission
>> to
>>>> copy files under $HADOOP_HOME/lib
>>>>
>>>> Is there any other ways to add extra jar to CLASSPATH ?
>>>>
>>>>
>>>> --
>>>> Best Regards
>>>>
>>>> Jeff Zhang
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
Re: Is there any way to add jar when invoking hadoop command
Posted by Jeff Zhang <zj...@gmail.com>.
Actually I don't use the jar for mapreduce job. I only need it to display
sequence file.
On Wed, Mar 23, 2011 at 9:41 AM, James Seigel <ja...@tynt.com> wrote:
> Hello, some quick advice for you....
>
>
> which portion of your job needs the jar? if answer = mapper or reducer,
> then add it to the -libjars flag.
>
> If it is in the job initiation..bundle it in your job jar for fun.
>
> Cheers
> James.
> On 2011-03-22, at 7:35 PM, Jeff Zhang wrote:
>
> > Another work around I can think of is that have my own copy of hadoop,
> and
> > copy extra jars to my hadoop. But it result into more maintenance effort
> >
> > On Wed, Mar 23, 2011 at 9:19 AM, Jeff Zhang <zj...@gmail.com> wrote:
> >
> >> Hi all,
> >>
> >> When I use command "hadoop fs -text" I need to add extra jar to
> CLASSPATH,
> >> because there's custom type in my sequence file. One way is that copying
> jar
> >> to $HADOOP_HOME/lib
> >> But in my case, I am not administrator, so I do not have the permission
> to
> >> copy files under $HADOOP_HOME/lib
> >>
> >> Is there any other ways to add extra jar to CLASSPATH ?
> >>
> >>
> >> --
> >> Best Regards
> >>
> >> Jeff Zhang
> >>
> >
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
>
>
--
Best Regards
Jeff Zhang
Re: Is there any way to add jar when invoking hadoop command
Posted by James Seigel <ja...@tynt.com>.
Hello, some quick advice for you....
which portion of your job needs the jar? if answer = mapper or reducer, then add it to the -libjars flag.
If it is in the job initiation..bundle it in your job jar for fun.
Cheers
James.
On 2011-03-22, at 7:35 PM, Jeff Zhang wrote:
> Another work around I can think of is that have my own copy of hadoop, and
> copy extra jars to my hadoop. But it result into more maintenance effort
>
> On Wed, Mar 23, 2011 at 9:19 AM, Jeff Zhang <zj...@gmail.com> wrote:
>
>> Hi all,
>>
>> When I use command "hadoop fs -text" I need to add extra jar to CLASSPATH,
>> because there's custom type in my sequence file. One way is that copying jar
>> to $HADOOP_HOME/lib
>> But in my case, I am not administrator, so I do not have the permission to
>> copy files under $HADOOP_HOME/lib
>>
>> Is there any other ways to add extra jar to CLASSPATH ?
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
Re: Is there any way to add jar when invoking hadoop command
Posted by Jeff Zhang <zj...@gmail.com>.
Another work around I can think of is that have my own copy of hadoop, and
copy extra jars to my hadoop. But it result into more maintenance effort
On Wed, Mar 23, 2011 at 9:19 AM, Jeff Zhang <zj...@gmail.com> wrote:
> Hi all,
>
> When I use command "hadoop fs -text" I need to add extra jar to CLASSPATH,
> because there's custom type in my sequence file. One way is that copying jar
> to $HADOOP_HOME/lib
> But in my case, I am not administrator, so I do not have the permission to
> copy files under $HADOOP_HOME/lib
>
> Is there any other ways to add extra jar to CLASSPATH ?
>
>
> --
> Best Regards
>
> Jeff Zhang
>
--
Best Regards
Jeff Zhang
Re: Is there any way to add jar when invoking hadoop command
Posted by Jeff Zhang <zj...@gmail.com>.
Another work around I can think of is that have my own copy of hadoop, and
copy extra jars to my hadoop. But it result into more maintenance effort
On Wed, Mar 23, 2011 at 9:19 AM, Jeff Zhang <zj...@gmail.com> wrote:
> Hi all,
>
> When I use command "hadoop fs -text" I need to add extra jar to CLASSPATH,
> because there's custom type in my sequence file. One way is that copying jar
> to $HADOOP_HOME/lib
> But in my case, I am not administrator, so I do not have the permission to
> copy files under $HADOOP_HOME/lib
>
> Is there any other ways to add extra jar to CLASSPATH ?
>
>
> --
> Best Regards
>
> Jeff Zhang
>
--
Best Regards
Jeff Zhang