You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hbase.apache.org by Ted Yu <yu...@gmail.com> on 2011/01/22 17:00:55 UTC

parallelizing HBaseAdmin.flush()

In 0.90, HBaseAdmin.flush() uses a loop to go over List<Pair<HRegionInfo,
HServerAddress>>

Should executor service be used for higher parallelism ?

Thanks

Re: parallelizing HBaseAdmin.flush()

Posted by Ryan Rawson <ry...@gmail.com>.

dont forget that this wont parallelize the flushes or compactions,
since they happen region-server side and there are built in limits
there to keep io down.

this will accelerate sending all the command messages though.

-ryan

On Mon, Jan 24, 2011 at 11:18 AM, Ted Yu <yu...@gmail.com> wrote:
> https://issues.apache.org/jira/browse/HBASE-3471 is created
>
> On Mon, Jan 24, 2011 at 10:56 AM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>
>> I'd guess so and the same could be done for splits and compactions
>> since it's (almost) the same code.
>>
>> J-D
>>
>> On Sat, Jan 22, 2011 at 8:00 AM, Ted Yu <yu...@gmail.com> wrote:
>> > In 0.90, HBaseAdmin.flush() uses a loop to go over List<Pair<HRegionInfo,
>> > HServerAddress>>
>> >
>> > Should executor service be used for higher parallelism ?
>> >
>> > Thanks
>> >
>>
>

Re: parallelizing HBaseAdmin.flush()

Posted by Ted Yu <yu...@gmail.com>.

https://issues.apache.org/jira/browse/HBASE-3471 is created

On Mon, Jan 24, 2011 at 10:56 AM, Jean-Daniel Cryans <jd...@apache.org>wrote:

> I'd guess so and the same could be done for splits and compactions
> since it's (almost) the same code.
>
> J-D
>
> On Sat, Jan 22, 2011 at 8:00 AM, Ted Yu <yu...@gmail.com> wrote:
> > In 0.90, HBaseAdmin.flush() uses a loop to go over List<Pair<HRegionInfo,
> > HServerAddress>>
> >
> > Should executor service be used for higher parallelism ?
> >
> > Thanks
> >
>

Re: parallelizing HBaseAdmin.flush()

Posted by Jean-Daniel Cryans <jd...@apache.org>.

I'd guess so and the same could be done for splits and compactions
since it's (almost) the same code.

J-D

On Sat, Jan 22, 2011 at 8:00 AM, Ted Yu <yu...@gmail.com> wrote:
> In 0.90, HBaseAdmin.flush() uses a loop to go over List<Pair<HRegionInfo,
> HServerAddress>>
>
> Should executor service be used for higher parallelism ?
>
> Thanks
>