Put while move or split

[Jean-Marc Spaggiari <je...@spaggiari.org>]

Hi,

When a region is moving or splitting, is it still accessible? Or the
region is offline until the operation ends?

Thanks,

JM

Re: Put while move or split

[Jean-Marc Spaggiari <je...@spaggiari.org>]

Thanks all for your comments.

I was talking about the 3 minutes period because by default, ZK will
have a 3 minute timeout. So if a server crash, it can take up to 3
minutes to Zookeeper to detect that. And in this meantime, the client
might not be able to access the impacted regions.

So what I was wondering is, what's the right wait to handle that.
Should we always catch all the failures of all the put/get and others
and have a backup plan? Or is sometime hbase taking care of that.

So what I have understand here is that HBase will take care of the
put/get when regions are "simply" moving and temporary unavailable
(compaction, etc.) but the client should take care of bigger issues
like when the region is not available for a longer period for any
reason.

JM


2012/12/3, Harsh J <ha...@cloudera.com>:
> Hey Mike,
>
> A move is logical in terms of a region and region servers (i.e. only
> an ID and a command is passed around), but yes the location of blocks
> could impact its performance a bit (until a further compaction), but
> would not prevent availability.
>
> On Mon, Dec 3, 2012 at 3:53 PM, Michel Segel <mi...@hotmail.com>
> wrote:
>> He could have meant that the region was unavailable for a longer
>> duration.
>>
>> My take is that you could have limited bandwidth, and a very large region
>> on the move.
>> While I don't imagine that it could take that long to move a region, you
>> never know how things are tuned and you could in theory hit a timeout.
>>
>> I also have to wonder if HBase relies on the underlying HDFS to handle the
>> move.
>>
>> Thoughts?
>>
>>
>> Sent from a remote device. Please excuse any typos...
>>
>> Mike Segel
>>
>> On Dec 3, 2012, at 3:45 AM, ramkrishna vasudevan
>> <ra...@gmail.com> wrote:
>>
>>> Hi Jean
>>>
>>> Incase of Puts and Scans if the split is in progress it will retry.
>>> Suppose before the client gets to know about the split, still if the put
>>> is directed with the parent region row key, internally HBase directs it
>>> to
>>> the correct daughter region.
>>> But i am not clear in what you mean by Region server down for 3 mins?
>>> You
>>> mean during split your RS went down?
>>>
>>> Regards
>>> Ram
>>>
>>>
>>> On Mon, Dec 3, 2012 at 8:23 AM, Jean-Marc Spaggiari
>>> <jean-marc@spaggiari.org
>>>> wrote:
>>>
>>>> Ok. so if on the client side I'm "simply" using Puts, scans and gets
>>>> fron the HTableInterface, I don't have to worry about that since it
>>>> will retry.
>>>>
>>>> But I still should consider catching exceptions when regions are not
>>>> available because the region server is down for less than 3 minutes,
>>>> right?
>>>>
>>>> Thanks,
>>>>
>>>> JM
>>>>
>>>> 2012/12/2, Nicolas Liochon <nk...@gmail.com>:
>>>>> It's not accessible, but it's more or less transparent (latency impact
>>>>> aside) for the end user: the hbase client will retry the operation.
>>>>>
>>>>> On Sun, Dec 2, 2012 at 11:10 PM, Jean-Marc Spaggiari <
>>>>> jean-marc@spaggiari.org> wrote:
>>>>>
>>>>>> ?
>>>>
>
>
>
> --
> Harsh J
>

Re: Put while move or split

[Harsh J <ha...@cloudera.com>]

Hey Mike,

A move is logical in terms of a region and region servers (i.e. only
an ID and a command is passed around), but yes the location of blocks
could impact its performance a bit (until a further compaction), but
would not prevent availability.

On Mon, Dec 3, 2012 at 3:53 PM, Michel Segel <mi...@hotmail.com> wrote:
> He could have meant that the region was unavailable for a longer duration.
>
> My take is that you could have limited bandwidth, and a very large region on the move.
> While I don't imagine that it could take that long to move a region, you never know how things are tuned and you could in theory hit a timeout.
>
> I also have to wonder if HBase relies on the underlying HDFS to handle the move.
>
> Thoughts?
>
>
> Sent from a remote device. Please excuse any typos...
>
> Mike Segel
>
> On Dec 3, 2012, at 3:45 AM, ramkrishna vasudevan <ra...@gmail.com> wrote:
>
>> Hi Jean
>>
>> Incase of Puts and Scans if the split is in progress it will retry.
>> Suppose before the client gets to know about the split, still if the put
>> is directed with the parent region row key, internally HBase directs it to
>> the correct daughter region.
>> But i am not clear in what you mean by Region server down for 3 mins?  You
>> mean during split your RS went down?
>>
>> Regards
>> Ram
>>
>>
>> On Mon, Dec 3, 2012 at 8:23 AM, Jean-Marc Spaggiari <jean-marc@spaggiari.org
>>> wrote:
>>
>>> Ok. so if on the client side I'm "simply" using Puts, scans and gets
>>> fron the HTableInterface, I don't have to worry about that since it
>>> will retry.
>>>
>>> But I still should consider catching exceptions when regions are not
>>> available because the region server is down for less than 3 minutes,
>>> right?
>>>
>>> Thanks,
>>>
>>> JM
>>>
>>> 2012/12/2, Nicolas Liochon <nk...@gmail.com>:
>>>> It's not accessible, but it's more or less transparent (latency impact
>>>> aside) for the end user: the hbase client will retry the operation.
>>>>
>>>> On Sun, Dec 2, 2012 at 11:10 PM, Jean-Marc Spaggiari <
>>>> jean-marc@spaggiari.org> wrote:
>>>>
>>>>> ?
>>>



-- 
Harsh J

Re: Put while move or split

[Michel Segel <mi...@hotmail.com>]

He could have meant that the region was unavailable for a longer duration.

My take is that you could have limited bandwidth, and a very large region on the move.
While I don't imagine that it could take that long to move a region, you never know how things are tuned and you could in theory hit a timeout.

I also have to wonder if HBase relies on the underlying HDFS to handle the move.

Thoughts?


Sent from a remote device. Please excuse any typos...

Mike Segel

On Dec 3, 2012, at 3:45 AM, ramkrishna vasudevan <ra...@gmail.com> wrote:

> Hi Jean
> 
> Incase of Puts and Scans if the split is in progress it will retry.
> Suppose before the client gets to know about the split, still if the put
> is directed with the parent region row key, internally HBase directs it to
> the correct daughter region.
> But i am not clear in what you mean by Region server down for 3 mins?  You
> mean during split your RS went down?
> 
> Regards
> Ram
> 
> 
> On Mon, Dec 3, 2012 at 8:23 AM, Jean-Marc Spaggiari <jean-marc@spaggiari.org
>> wrote:
> 
>> Ok. so if on the client side I'm "simply" using Puts, scans and gets
>> fron the HTableInterface, I don't have to worry about that since it
>> will retry.
>> 
>> But I still should consider catching exceptions when regions are not
>> available because the region server is down for less than 3 minutes,
>> right?
>> 
>> Thanks,
>> 
>> JM
>> 
>> 2012/12/2, Nicolas Liochon <nk...@gmail.com>:
>>> It's not accessible, but it's more or less transparent (latency impact
>>> aside) for the end user: the hbase client will retry the operation.
>>> 
>>> On Sun, Dec 2, 2012 at 11:10 PM, Jean-Marc Spaggiari <
>>> jean-marc@spaggiari.org> wrote:
>>> 
>>>> ?
>> 

Re: Put while move or split

[ramkrishna vasudevan <ra...@gmail.com>]

Hi Jean

Incase of Puts and Scans if the split is in progress it will retry.
 Suppose before the client gets to know about the split, still if the put
is directed with the parent region row key, internally HBase directs it to
the correct daughter region.
But i am not clear in what you mean by Region server down for 3 mins?  You
mean during split your RS went down?

Regards
Ram


On Mon, Dec 3, 2012 at 8:23 AM, Jean-Marc Spaggiari <jean-marc@spaggiari.org
> wrote:

> Ok. so if on the client side I'm "simply" using Puts, scans and gets
> fron the HTableInterface, I don't have to worry about that since it
> will retry.
>
> But I still should consider catching exceptions when regions are not
> available because the region server is down for less than 3 minutes,
> right?
>
> Thanks,
>
> JM
>
> 2012/12/2, Nicolas Liochon <nk...@gmail.com>:
> > It's not accessible, but it's more or less transparent (latency impact
> > aside) for the end user: the hbase client will retry the operation.
> >
> > On Sun, Dec 2, 2012 at 11:10 PM, Jean-Marc Spaggiari <
> > jean-marc@spaggiari.org> wrote:
> >
> >> ?
> >
>

Re: Put while move or split

[Nicolas Liochon <nk...@gmail.com>]

If a server crashes, its regions will be moved to other servers. So, if a
query fails because the region server does not respond, the hbase client
will retry as well. The principle is to make the failure transparent to the
application using the hbase client.
Le 3 déc. 2012 03:53, "Jean-Marc Spaggiari" <je...@spaggiari.org> a
écrit :

> Ok. so if on the client side I'm "simply" using Puts, scans and gets
> fron the HTableInterface, I don't have to worry about that since it
> will retry.
>
> But I still should consider catching exceptions when regions are not
> available because the region server is down for less than 3 minutes,
> right?
>
> Thanks,
>
> JM
>
> 2012/12/2, Nicolas Liochon <nk...@gmail.com>:
> > It's not accessible, but it's more or less transparent (latency impact
> > aside) for the end user: the hbase client will retry the operation.
> >
> > On Sun, Dec 2, 2012 at 11:10 PM, Jean-Marc Spaggiari <
> > jean-marc@spaggiari.org> wrote:
> >
> >> ?
> >
>

Re: Put while move or split

[Jean-Marc Spaggiari <je...@spaggiari.org>]

Ok. so if on the client side I'm "simply" using Puts, scans and gets
fron the HTableInterface, I don't have to worry about that since it
will retry.

But I still should consider catching exceptions when regions are not
available because the region server is down for less than 3 minutes,
right?

Thanks,

JM

2012/12/2, Nicolas Liochon <nk...@gmail.com>:
> It's not accessible, but it's more or less transparent (latency impact
> aside) for the end user: the hbase client will retry the operation.
>
> On Sun, Dec 2, 2012 at 11:10 PM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
>> ?
>

Re: Put while move or split

[Nicolas Liochon <nk...@gmail.com>]

It's not accessible, but it's more or less transparent (latency impact
aside) for the end user: the hbase client will retry the operation.

On Sun, Dec 2, 2012 at 11:10 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> ?