You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Tim Robertson <ti...@gmail.com> on 2014/11/05 10:27:35 UTC
hbase-server required on CP for running MR jobs?
Hey folks,
I'm upgrading an application from CDH4.3 to CDH5.2 so jumping from 0.94 to
0.98 and wanted to just ask for confirmation on the dependencies now hbase
has split into hbase-client and hbase-server etc.
If I am submitting MR jobs (to Yarn) that use things like
TableMapReduceUtil it seems I need to have hbase-server jar on the CP to
compile. This just seems... bizarre. I'd have expected hbase-client or
some hbase-mapreduce library to have this so as not to bring in the
dependencies of the servers.
Can someone please confirm that I'm not overlooking something obvious? I
suspect I must be.
Thanks,
Tim
Re: hbase-server required on CP for running MR jobs?
Posted by Nick Dimiduk <nd...@gmail.com>.
There's also a ticket to separate out MR:
https://issues.apache.org/jira/plugins/servlet/mobile#issue/HBASE-11843
On Wednesday, November 5, 2014, Ted Yu <yu...@gmail.com> wrote:
> See this JIRA:
> https://issues.apache.org/jira/browse/HBASE-11549
>
> Cheers
>
> On Nov 5, 2014, at 4:58 AM, Tim Robertson <timrobertson100@gmail.com
> <javascript:;>> wrote:
>
> > Thanks for confirming Ted
> >
> > I'll use hbase-server and then exclude most of the transient
> dependencies,
> > and the hadoop-core (MR1 stuff). I can't find a Jira for this so will
> > write one up.
> >
> > Cheers,
> > Tim
> >
> >
> >
> >
> >
> >
> >
> > On Wed, Nov 5, 2014 at 10:59 AM, Ted Yu <yuzhihong@gmail.com
> <javascript:;>> wrote:
> >
> >> Your observation is right.
> >> There isn't hbase-mapreduce module yet. For now, you need to include
> >> hbase-server module.
> >>
> >> Cheers
> >>
> >> On Nov 5, 2014, at 1:27 AM, Tim Robertson <timrobertson100@gmail.com
> <javascript:;>>
> >> wrote:
> >>
> >>> Hey folks,
> >>>
> >>> I'm upgrading an application from CDH4.3 to CDH5.2 so jumping from 0.94
> >> to
> >>> 0.98 and wanted to just ask for confirmation on the dependencies now
> >> hbase
> >>> has split into hbase-client and hbase-server etc.
> >>>
> >>> If I am submitting MR jobs (to Yarn) that use things like
> >>> TableMapReduceUtil it seems I need to have hbase-server jar on the CP
> to
> >>> compile. This just seems... bizarre. I'd have expected hbase-client
> or
> >>> some hbase-mapreduce library to have this so as not to bring in the
> >>> dependencies of the servers.
> >>>
> >>> Can someone please confirm that I'm not overlooking something
> obvious? I
> >>> suspect I must be.
> >>>
> >>> Thanks,
> >>> Tim
> >>
>
Re: hbase-server required on CP for running MR jobs?
Posted by Ted Yu <yu...@gmail.com>.
See this JIRA:
https://issues.apache.org/jira/browse/HBASE-11549
Cheers
On Nov 5, 2014, at 4:58 AM, Tim Robertson <ti...@gmail.com> wrote:
> Thanks for confirming Ted
>
> I'll use hbase-server and then exclude most of the transient dependencies,
> and the hadoop-core (MR1 stuff). I can't find a Jira for this so will
> write one up.
>
> Cheers,
> Tim
>
>
>
>
>
>
>
> On Wed, Nov 5, 2014 at 10:59 AM, Ted Yu <yu...@gmail.com> wrote:
>
>> Your observation is right.
>> There isn't hbase-mapreduce module yet. For now, you need to include
>> hbase-server module.
>>
>> Cheers
>>
>> On Nov 5, 2014, at 1:27 AM, Tim Robertson <ti...@gmail.com>
>> wrote:
>>
>>> Hey folks,
>>>
>>> I'm upgrading an application from CDH4.3 to CDH5.2 so jumping from 0.94
>> to
>>> 0.98 and wanted to just ask for confirmation on the dependencies now
>> hbase
>>> has split into hbase-client and hbase-server etc.
>>>
>>> If I am submitting MR jobs (to Yarn) that use things like
>>> TableMapReduceUtil it seems I need to have hbase-server jar on the CP to
>>> compile. This just seems... bizarre. I'd have expected hbase-client or
>>> some hbase-mapreduce library to have this so as not to bring in the
>>> dependencies of the servers.
>>>
>>> Can someone please confirm that I'm not overlooking something obvious? I
>>> suspect I must be.
>>>
>>> Thanks,
>>> Tim
>>
Re: hbase-server required on CP for running MR jobs?
Posted by Tim Robertson <ti...@gmail.com>.
Thanks for confirming Ted
I'll use hbase-server and then exclude most of the transient dependencies,
and the hadoop-core (MR1 stuff). I can't find a Jira for this so will
write one up.
Cheers,
Tim
On Wed, Nov 5, 2014 at 10:59 AM, Ted Yu <yu...@gmail.com> wrote:
> Your observation is right.
> There isn't hbase-mapreduce module yet. For now, you need to include
> hbase-server module.
>
> Cheers
>
> On Nov 5, 2014, at 1:27 AM, Tim Robertson <ti...@gmail.com>
> wrote:
>
> > Hey folks,
> >
> > I'm upgrading an application from CDH4.3 to CDH5.2 so jumping from 0.94
> to
> > 0.98 and wanted to just ask for confirmation on the dependencies now
> hbase
> > has split into hbase-client and hbase-server etc.
> >
> > If I am submitting MR jobs (to Yarn) that use things like
> > TableMapReduceUtil it seems I need to have hbase-server jar on the CP to
> > compile. This just seems... bizarre. I'd have expected hbase-client or
> > some hbase-mapreduce library to have this so as not to bring in the
> > dependencies of the servers.
> >
> > Can someone please confirm that I'm not overlooking something obvious? I
> > suspect I must be.
> >
> > Thanks,
> > Tim
>
Re: hbase-server required on CP for running MR jobs?
Posted by Ted Yu <yu...@gmail.com>.
Your observation is right.
There isn't hbase-mapreduce module yet. For now, you need to include hbase-server module.
Cheers
On Nov 5, 2014, at 1:27 AM, Tim Robertson <ti...@gmail.com> wrote:
> Hey folks,
>
> I'm upgrading an application from CDH4.3 to CDH5.2 so jumping from 0.94 to
> 0.98 and wanted to just ask for confirmation on the dependencies now hbase
> has split into hbase-client and hbase-server etc.
>
> If I am submitting MR jobs (to Yarn) that use things like
> TableMapReduceUtil it seems I need to have hbase-server jar on the CP to
> compile. This just seems... bizarre. I'd have expected hbase-client or
> some hbase-mapreduce library to have this so as not to bring in the
> dependencies of the servers.
>
> Can someone please confirm that I'm not overlooking something obvious? I
> suspect I must be.
>
> Thanks,
> Tim