You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Jerry Lam <ch...@gmail.com> on 2012/08/07 17:22:28 UTC
CheckAndAppend Feature
Hi HBase community:
I checked the HTable API, it has checkAndPut and checkAndDelete but I'm
looking for checkAndAppend. Is there a way to simulate similarly?
For instance, I want to check the last 32 bytes of a value (let assume that
it has 128 bytes in total) in a column before appending atomically some
values into it.
Thanks!
Jerry
Re: CheckAndAppend Feature
Posted by syed kather <in...@gmail.com>.
Team ,
Sorry my mail box not yet got updated when i am replying. Now only i
came to know that . after other people reply
Thanks and Regards,
S SYED ABDUL KATHER
On Tue, Aug 7, 2012 at 9:39 PM, syed kather <in...@gmail.com> wrote:
> Hi Jerry
>
> I also had similar use case . i had not found anything similar to that
> append :( .
>
> Thanks and Regards,
> S SYED ABDUL KATHER
>
>
>
> On Tue, Aug 7, 2012 at 8:52 PM, Jerry Lam <ch...@gmail.com> wrote:
>
>> Hi HBase community:
>>
>> I checked the HTable API, it has checkAndPut and checkAndDelete but I'm
>> looking for checkAndAppend. Is there a way to simulate similarly?
>> For instance, I want to check the last 32 bytes of a value (let assume
>> that
>> it has 128 bytes in total) in a column before appending atomically some
>> values into it.
>>
>> Thanks!
>>
>> Jerry
>>
>
>
Re: CheckAndAppend Feature
Posted by syed kather <in...@gmail.com>.
Hi Jerry
I also had similar use case . i had not found anything similar to that
append :( .
Thanks and Regards,
S SYED ABDUL KATHER
On Tue, Aug 7, 2012 at 8:52 PM, Jerry Lam <ch...@gmail.com> wrote:
> Hi HBase community:
>
> I checked the HTable API, it has checkAndPut and checkAndDelete but I'm
> looking for checkAndAppend. Is there a way to simulate similarly?
> For instance, I want to check the last 32 bytes of a value (let assume that
> it has 128 bytes in total) in a column before appending atomically some
> values into it.
>
> Thanks!
>
> Jerry
>
Re: CheckAndAppend Feature
Posted by lars hofhansl <lh...@yahoo.com>.
Yep... It's up to implementer of the RegionObserver to avoid deadlocks (lock ordering, two phase locking if needed, etc).
As long as only a single lock is taken (and released) there can be no deadlock.
More rope for a Coprocessor implementer to hang him/her self :)
Since Coprocessor are there to extend HBase without the need to subclass - say - HRegionServer or HRegion this is OK (IMHO).
-- Lars
----- Original Message -----
From: Michael Segel <mi...@hotmail.com>
To: user@hbase.apache.org; lars hofhansl <lh...@yahoo.com>
Cc:
Sent: Tuesday, August 7, 2012 6:25 PM
Subject: Re: CheckAndAppend Feature
Hi Lars,
To be honest its just an initial gut feel.
I haven't thought the issue through and I wonder if there are any downsides like dead locks or if some code goes awry how to recover.
I understand that its on the server side, and that's why I'm a little concerned.
On Aug 7, 2012, at 8:05 PM, lars hofhansl <lh...@yahoo.com> wrote:
> Hi Mike,
>
> do you have a specific downside in mind?The built-in checkAndSet type operations do the same.
> Coprocessors are an extension mechanism for HBase, not to be compared with (say) server side stored procedures in relational databases.
>
> (the canonical example is that nothing stops a RegionObserver implementation from calling System.exit(), taking the RegionServer with it).
>
>
> -- Lars
>
>
>
> ----- Original Message -----
> From: Michael Segel <mi...@hotmail.com>
> To: user@hbase.apache.org; lars hofhansl <lh...@yahoo.com>
> Cc:
> Sent: Tuesday, August 7, 2012 5:54 PM
> Subject: Re: CheckAndAppend Feature
>
> While this may be a trivial fix, have you considered possible down sides to the implementation?
>
> I'm not sure its a bad idea, but one that could have some potential issues when put into practice.
>
> -Mike
>
> On Aug 7, 2012, at 7:30 PM, lars hofhansl <lh...@yahoo.com> wrote:
>
>> I filed HBASE-6522. It is a trivial change to make locks and leases available to coprocessors.
>> So checkAndSet type operations can then be implemented via coprocessor endpoints: lock row, check, fail or update, unlock row.
>>
>> Since the patch is so simple I'll commit that soon (to 0.94.2 and 0.96)
>>
>>
>> -- Lars
>>
>> ________________________________
>> From: lars hofhansl <lh...@yahoo.com>
>> To: "user@hbase.apache.org" <us...@hbase.apache.org>
>> Sent: Tuesday, August 7, 2012 8:55 AM
>> Subject: Re: CheckAndAppend Feature
>>
>> There is no such functionality currently, and there is no good way to simulate that.
>>
>> Currently that cannot even be done with a coprocessor endpoint, because region coprocessors have no way to create a region lock (just checked the code).
>> (That is something we have to change I think - I will create an issue once the Jira system is back from the walk in the park).
>>
>> -- Lars
>>
>>
>> ----- Original Message -----
>> From: Jerry Lam <ch...@gmail.com>
>> To: user <us...@hbase.apache.org>
>> Cc:
>> Sent: Tuesday, August 7, 2012 8:22 AM
>> Subject: CheckAndAppend Feature
>>
>> Hi HBase community:
>>
>> I checked the HTable API, it has checkAndPut and checkAndDelete but I'm
>> looking for checkAndAppend. Is there a way to simulate similarly?
>> For instance, I want to check the last 32 bytes of a value (let assume that
>> it has 128 bytes in total) in a column before appending atomically some
>> values into it.
>>
>> Thanks!
>>
>> Jerry
>>
>
Re: CheckAndAppend Feature
Posted by Michael Segel <mi...@hotmail.com>.
Hi Lars,
To be honest its just an initial gut feel.
I haven't thought the issue through and I wonder if there are any downsides like dead locks or if some code goes awry how to recover.
I understand that its on the server side, and that's why I'm a little concerned.
On Aug 7, 2012, at 8:05 PM, lars hofhansl <lh...@yahoo.com> wrote:
> Hi Mike,
>
> do you have a specific downside in mind?The built-in checkAndSet type operations do the same.
> Coprocessors are an extension mechanism for HBase, not to be compared with (say) server side stored procedures in relational databases.
>
> (the canonical example is that nothing stops a RegionObserver implementation from calling System.exit(), taking the RegionServer with it).
>
>
> -- Lars
>
>
>
> ----- Original Message -----
> From: Michael Segel <mi...@hotmail.com>
> To: user@hbase.apache.org; lars hofhansl <lh...@yahoo.com>
> Cc:
> Sent: Tuesday, August 7, 2012 5:54 PM
> Subject: Re: CheckAndAppend Feature
>
> While this may be a trivial fix, have you considered possible down sides to the implementation?
>
> I'm not sure its a bad idea, but one that could have some potential issues when put into practice.
>
> -Mike
>
> On Aug 7, 2012, at 7:30 PM, lars hofhansl <lh...@yahoo.com> wrote:
>
>> I filed HBASE-6522. It is a trivial change to make locks and leases available to coprocessors.
>> So checkAndSet type operations can then be implemented via coprocessor endpoints: lock row, check, fail or update, unlock row.
>>
>> Since the patch is so simple I'll commit that soon (to 0.94.2 and 0.96)
>>
>>
>> -- Lars
>>
>> ________________________________
>> From: lars hofhansl <lh...@yahoo.com>
>> To: "user@hbase.apache.org" <us...@hbase.apache.org>
>> Sent: Tuesday, August 7, 2012 8:55 AM
>> Subject: Re: CheckAndAppend Feature
>>
>> There is no such functionality currently, and there is no good way to simulate that.
>>
>> Currently that cannot even be done with a coprocessor endpoint, because region coprocessors have no way to create a region lock (just checked the code).
>> (That is something we have to change I think - I will create an issue once the Jira system is back from the walk in the park).
>>
>> -- Lars
>>
>>
>> ----- Original Message -----
>> From: Jerry Lam <ch...@gmail.com>
>> To: user <us...@hbase.apache.org>
>> Cc:
>> Sent: Tuesday, August 7, 2012 8:22 AM
>> Subject: CheckAndAppend Feature
>>
>> Hi HBase community:
>>
>> I checked the HTable API, it has checkAndPut and checkAndDelete but I'm
>> looking for checkAndAppend. Is there a way to simulate similarly?
>> For instance, I want to check the last 32 bytes of a value (let assume that
>> it has 128 bytes in total) in a column before appending atomically some
>> values into it.
>>
>> Thanks!
>>
>> Jerry
>>
>
Re: CheckAndAppend Feature
Posted by lars hofhansl <lh...@yahoo.com>.
Hi Mike,
do you have a specific downside in mind?The built-in checkAndSet type operations do the same.
Coprocessors are an extension mechanism for HBase, not to be compared with (say) server side stored procedures in relational databases.
(the canonical example is that nothing stops a RegionObserver implementation from calling System.exit(), taking the RegionServer with it).
-- Lars
----- Original Message -----
From: Michael Segel <mi...@hotmail.com>
To: user@hbase.apache.org; lars hofhansl <lh...@yahoo.com>
Cc:
Sent: Tuesday, August 7, 2012 5:54 PM
Subject: Re: CheckAndAppend Feature
While this may be a trivial fix, have you considered possible down sides to the implementation?
I'm not sure its a bad idea, but one that could have some potential issues when put into practice.
-Mike
On Aug 7, 2012, at 7:30 PM, lars hofhansl <lh...@yahoo.com> wrote:
> I filed HBASE-6522. It is a trivial change to make locks and leases available to coprocessors.
> So checkAndSet type operations can then be implemented via coprocessor endpoints: lock row, check, fail or update, unlock row.
>
> Since the patch is so simple I'll commit that soon (to 0.94.2 and 0.96)
>
>
> -- Lars
>
> ________________________________
> From: lars hofhansl <lh...@yahoo.com>
> To: "user@hbase.apache.org" <us...@hbase.apache.org>
> Sent: Tuesday, August 7, 2012 8:55 AM
> Subject: Re: CheckAndAppend Feature
>
> There is no such functionality currently, and there is no good way to simulate that.
>
> Currently that cannot even be done with a coprocessor endpoint, because region coprocessors have no way to create a region lock (just checked the code).
> (That is something we have to change I think - I will create an issue once the Jira system is back from the walk in the park).
>
> -- Lars
>
>
> ----- Original Message -----
> From: Jerry Lam <ch...@gmail.com>
> To: user <us...@hbase.apache.org>
> Cc:
> Sent: Tuesday, August 7, 2012 8:22 AM
> Subject: CheckAndAppend Feature
>
> Hi HBase community:
>
> I checked the HTable API, it has checkAndPut and checkAndDelete but I'm
> looking for checkAndAppend. Is there a way to simulate similarly?
> For instance, I want to check the last 32 bytes of a value (let assume that
> it has 128 bytes in total) in a column before appending atomically some
> values into it.
>
> Thanks!
>
> Jerry
>
Re: CheckAndAppend Feature
Posted by Michael Segel <mi...@hotmail.com>.
While this may be a trivial fix, have you considered possible down sides to the implementation?
I'm not sure its a bad idea, but one that could have some potential issues when put into practice.
-Mike
On Aug 7, 2012, at 7:30 PM, lars hofhansl <lh...@yahoo.com> wrote:
> I filed HBASE-6522. It is a trivial change to make locks and leases available to coprocessors.
> So checkAndSet type operations can then be implemented via coprocessor endpoints: lock row, check, fail or update, unlock row.
>
> Since the patch is so simple I'll commit that soon (to 0.94.2 and 0.96)
>
>
> -- Lars
>
> ________________________________
> From: lars hofhansl <lh...@yahoo.com>
> To: "user@hbase.apache.org" <us...@hbase.apache.org>
> Sent: Tuesday, August 7, 2012 8:55 AM
> Subject: Re: CheckAndAppend Feature
>
> There is no such functionality currently, and there is no good way to simulate that.
>
> Currently that cannot even be done with a coprocessor endpoint, because region coprocessors have no way to create a region lock (just checked the code).
> (That is something we have to change I think - I will create an issue once the Jira system is back from the walk in the park).
>
> -- Lars
>
>
> ----- Original Message -----
> From: Jerry Lam <ch...@gmail.com>
> To: user <us...@hbase.apache.org>
> Cc:
> Sent: Tuesday, August 7, 2012 8:22 AM
> Subject: CheckAndAppend Feature
>
> Hi HBase community:
>
> I checked the HTable API, it has checkAndPut and checkAndDelete but I'm
> looking for checkAndAppend. Is there a way to simulate similarly?
> For instance, I want to check the last 32 bytes of a value (let assume that
> it has 128 bytes in total) in a column before appending atomically some
> values into it.
>
> Thanks!
>
> Jerry
>
Re: CheckAndAppend Feature
Posted by Jerry Lam <ch...@gmail.com>.
Hi Lars:
This helps a lot! Thanks!
Best Regards,
Jerry
Sent from my iPad (sorry for spelling mistakes)
On 2012-08-07, at 20:30, lars hofhansl <lh...@yahoo.com> wrote:
> I filed HBASE-6522. It is a trivial change to make locks and leases available to coprocessors.
> So checkAndSet type operations can then be implemented via coprocessor endpoints: lock row, check, fail or update, unlock row.
>
> Since the patch is so simple I'll commit that soon (to 0.94.2 and 0.96)
>
>
> -- Lars
>
> ________________________________
> From: lars hofhansl <lh...@yahoo.com>
> To: "user@hbase.apache.org" <us...@hbase.apache.org>
> Sent: Tuesday, August 7, 2012 8:55 AM
> Subject: Re: CheckAndAppend Feature
>
> There is no such functionality currently, and there is no good way to simulate that.
>
> Currently that cannot even be done with a coprocessor endpoint, because region coprocessors have no way to create a region lock (just checked the code).
> (That is something we have to change I think - I will create an issue once the Jira system is back from the walk in the park).
>
> -- Lars
>
>
> ----- Original Message -----
> From: Jerry Lam <ch...@gmail.com>
> To: user <us...@hbase.apache.org>
> Cc:
> Sent: Tuesday, August 7, 2012 8:22 AM
> Subject: CheckAndAppend Feature
>
> Hi HBase community:
>
> I checked the HTable API, it has checkAndPut and checkAndDelete but I'm
> looking for checkAndAppend. Is there a way to simulate similarly?
> For instance, I want to check the last 32 bytes of a value (let assume that
> it has 128 bytes in total) in a column before appending atomically some
> values into it.
>
> Thanks!
>
> Jerry
Re: CheckAndAppend Feature
Posted by lars hofhansl <lh...@yahoo.com>.
I filed HBASE-6522. It is a trivial change to make locks and leases available to coprocessors.
So checkAndSet type operations can then be implemented via coprocessor endpoints: lock row, check, fail or update, unlock row.
Since the patch is so simple I'll commit that soon (to 0.94.2 and 0.96)
-- Lars
________________________________
From: lars hofhansl <lh...@yahoo.com>
To: "user@hbase.apache.org" <us...@hbase.apache.org>
Sent: Tuesday, August 7, 2012 8:55 AM
Subject: Re: CheckAndAppend Feature
There is no such functionality currently, and there is no good way to simulate that.
Currently that cannot even be done with a coprocessor endpoint, because region coprocessors have no way to create a region lock (just checked the code).
(That is something we have to change I think - I will create an issue once the Jira system is back from the walk in the park).
-- Lars
----- Original Message -----
From: Jerry Lam <ch...@gmail.com>
To: user <us...@hbase.apache.org>
Cc:
Sent: Tuesday, August 7, 2012 8:22 AM
Subject: CheckAndAppend Feature
Hi HBase community:
I checked the HTable API, it has checkAndPut and checkAndDelete but I'm
looking for checkAndAppend. Is there a way to simulate similarly?
For instance, I want to check the last 32 bytes of a value (let assume that
it has 128 bytes in total) in a column before appending atomically some
values into it.
Thanks!
Jerry
Re: CheckAndAppend Feature
Posted by Alex Baranau <al...@gmail.com>.
Hi Jerry,
Out of curiosity, what is your use-case? How do you want to use this?
Also, I guess, feel free to file a jira issue for this functionality (I
believe there's no such yet) .
Alex Baranau
------
Sematext :: http://sematext.com/ :: Hadoop - HBase - ElasticSearch - Solr
On Tue, Aug 7, 2012 at 11:55 AM, lars hofhansl <lh...@yahoo.com> wrote:
> There is no such functionality currently, and there is no good way to
> simulate that.
>
> Currently that cannot even be done with a coprocessor endpoint, because
> region coprocessors have no way to create a region lock (just checked the
> code).
> (That is something we have to change I think - I will create an issue once
> the Jira system is back from the walk in the park).
>
> -- Lars
>
>
> ----- Original Message -----
> From: Jerry Lam <ch...@gmail.com>
> To: user <us...@hbase.apache.org>
> Cc:
> Sent: Tuesday, August 7, 2012 8:22 AM
> Subject: CheckAndAppend Feature
>
> Hi HBase community:
>
> I checked the HTable API, it has checkAndPut and checkAndDelete but I'm
> looking for checkAndAppend. Is there a way to simulate similarly?
> For instance, I want to check the last 32 bytes of a value (let assume that
> it has 128 bytes in total) in a column before appending atomically some
> values into it.
>
> Thanks!
>
> Jerry
>
>
--
Alex Baranau
------
Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch -
Solr
Re: CheckAndAppend Feature
Posted by lars hofhansl <lh...@yahoo.com>.
There is no such functionality currently, and there is no good way to simulate that.
Currently that cannot even be done with a coprocessor endpoint, because region coprocessors have no way to create a region lock (just checked the code).
(That is something we have to change I think - I will create an issue once the Jira system is back from the walk in the park).
-- Lars
----- Original Message -----
From: Jerry Lam <ch...@gmail.com>
To: user <us...@hbase.apache.org>
Cc:
Sent: Tuesday, August 7, 2012 8:22 AM
Subject: CheckAndAppend Feature
Hi HBase community:
I checked the HTable API, it has checkAndPut and checkAndDelete but I'm
looking for checkAndAppend. Is there a way to simulate similarly?
For instance, I want to check the last 32 bytes of a value (let assume that
it has 128 bytes in total) in a column before appending atomically some
values into it.
Thanks!
Jerry