You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Scott Kuehn <sc...@opower.com> on 2014/04/24 18:17:00 UTC
Options for limiting bandwidth in ExportSnapshot
Any recommendations on how to limit the bandwidth consumption of an
ExportSnapshot job? I'd like to control bandwidth consumption so important
network workloads, like shuffling and balancing, are reserved some
bandwidth while a lower-priority ExportSnapshot job is running.
Note that distcp jobs provide the ability to limit bandwidth by exposing
the num-mappers and bandwidth options. I'm looking to achieve something
similar with ExportSnapshot.
ExportSnapshot ref (bottom of page):
http://hbase.apache.org/book/ops.snapshots.html
Re: Options for limiting bandwidth in ExportSnapshot
Posted by Scott Kuehn <sc...@opower.com>.
Ted -- thanks for the patch! We're using hadoop 2.0.0-cdh4.5.0.
On Sat, Apr 26, 2014 at 9:34 AM, Ted Yu <yu...@gmail.com> wrote:
> Scott:
> Can you tell me which hadoop release you're using ?
>
> Cheers
>
>
> On Fri, Apr 25, 2014 at 10:36 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> Here is the JIRA: HBASE-11083 ExportSnapshot should provide capability to
>> limit bandwidth consumption
>>
>> I uploaded a patch.
>>
>> Cheers
>>
>>
>> On Thu, Apr 24, 2014 at 9:22 AM, Matteo Bertozzi <theo.bertozzi@gmail.com
>> > wrote:
>>
>>> at the moment, you can only control the number of mappers (patches are
>>> welcome)
>>>
>>>
>>> Matteo
>>>
>>>
>>>
>>> On Thu, Apr 24, 2014 at 9:17 AM, Scott Kuehn <sc...@opower.com>
>>> wrote:
>>>
>>> > Any recommendations on how to limit the bandwidth consumption of an
>>> > ExportSnapshot job? I'd like to control bandwidth consumption so
>>> important
>>> > network workloads, like shuffling and balancing, are reserved some
>>> > bandwidth while a lower-priority ExportSnapshot job is running.
>>> >
>>> > Note that distcp jobs provide the ability to limit bandwidth by
>>> exposing
>>> > the num-mappers and bandwidth options. I'm looking to achieve something
>>> > similar with ExportSnapshot.
>>> >
>>> > ExportSnapshot ref (bottom of page):
>>> > http://hbase.apache.org/book/ops.snapshots.html
>>> >
>>>
>>
>>
>
Re: Options for limiting bandwidth in ExportSnapshot
Posted by Ted Yu <yu...@gmail.com>.
Scott:
Can you tell me which hadoop release you're using ?
Cheers
On Fri, Apr 25, 2014 at 10:36 PM, Ted Yu <yu...@gmail.com> wrote:
> Here is the JIRA: HBASE-11083 ExportSnapshot should provide capability to
> limit bandwidth consumption
>
> I uploaded a patch.
>
> Cheers
>
>
> On Thu, Apr 24, 2014 at 9:22 AM, Matteo Bertozzi <th...@gmail.com>wrote:
>
>> at the moment, you can only control the number of mappers (patches are
>> welcome)
>>
>>
>> Matteo
>>
>>
>>
>> On Thu, Apr 24, 2014 at 9:17 AM, Scott Kuehn <sc...@opower.com>
>> wrote:
>>
>> > Any recommendations on how to limit the bandwidth consumption of an
>> > ExportSnapshot job? I'd like to control bandwidth consumption so
>> important
>> > network workloads, like shuffling and balancing, are reserved some
>> > bandwidth while a lower-priority ExportSnapshot job is running.
>> >
>> > Note that distcp jobs provide the ability to limit bandwidth by exposing
>> > the num-mappers and bandwidth options. I'm looking to achieve something
>> > similar with ExportSnapshot.
>> >
>> > ExportSnapshot ref (bottom of page):
>> > http://hbase.apache.org/book/ops.snapshots.html
>> >
>>
>
>
Re: Options for limiting bandwidth in ExportSnapshot
Posted by Ted Yu <yu...@gmail.com>.
Here is the JIRA: HBASE-11083 ExportSnapshot should provide capability to
limit bandwidth consumption
I uploaded a patch.
Cheers
On Thu, Apr 24, 2014 at 9:22 AM, Matteo Bertozzi <th...@gmail.com>wrote:
> at the moment, you can only control the number of mappers (patches are
> welcome)
>
>
> Matteo
>
>
>
> On Thu, Apr 24, 2014 at 9:17 AM, Scott Kuehn <sc...@opower.com>
> wrote:
>
> > Any recommendations on how to limit the bandwidth consumption of an
> > ExportSnapshot job? I'd like to control bandwidth consumption so
> important
> > network workloads, like shuffling and balancing, are reserved some
> > bandwidth while a lower-priority ExportSnapshot job is running.
> >
> > Note that distcp jobs provide the ability to limit bandwidth by exposing
> > the num-mappers and bandwidth options. I'm looking to achieve something
> > similar with ExportSnapshot.
> >
> > ExportSnapshot ref (bottom of page):
> > http://hbase.apache.org/book/ops.snapshots.html
> >
>
Re: Options for limiting bandwidth in ExportSnapshot
Posted by Matteo Bertozzi <th...@gmail.com>.
at the moment, you can only control the number of mappers (patches are
welcome)
Matteo
On Thu, Apr 24, 2014 at 9:17 AM, Scott Kuehn <sc...@opower.com> wrote:
> Any recommendations on how to limit the bandwidth consumption of an
> ExportSnapshot job? I'd like to control bandwidth consumption so important
> network workloads, like shuffling and balancing, are reserved some
> bandwidth while a lower-priority ExportSnapshot job is running.
>
> Note that distcp jobs provide the ability to limit bandwidth by exposing
> the num-mappers and bandwidth options. I'm looking to achieve something
> similar with ExportSnapshot.
>
> ExportSnapshot ref (bottom of page):
> http://hbase.apache.org/book/ops.snapshots.html
>