You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by "Hiller, Dean (Contractor)" <de...@broadridge.com> on 2011/01/27 22:28:17 UTC
is there a pluggable conflict resolver in hbase
Tthere is this really really cool feature in gemfire for conflict
resolution(and I hope hbase has something similar here as it seems so
clean). You can implement an interface that gets called just before the
write to storage that gives you the old value and the new value(and is
single threaded for that row so multiple updates on that row are in same
thread).
Perfect example
Key="myCar" value=contains people A, B, C version 4
A put done from node 1 and a put done from node 2 at the same time
1. Put people A, B, C, D version 4
2. Put people A, B, C, E version 4
So the conflict resolver can put 1 fine but then on 2, it can see that
version is 5 on oldValue and can merge A, b, C, E with A, B, C, D
resulting in the full list A, B, C, D, E. this is a dang nice feature.
Does hbase have something similar that one can implement that gets
called on each node out there to do conflict resolution?
Thanks,
Dean
This message and any attachments are intended only for the use of the addressee and
may contain information that is privileged and confidential. If the reader of the
message is not the intended recipient or an authorized representative of the
intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication in
error, please notify us immediately by e-mail and delete the message and any
attachments from your system.
Re: is there a pluggable conflict resolver in hbase
Posted by Lars George <la...@gmail.com>.
Hi Dean,
Yes you can do that. See a (slightly outdated) oost about them here
http://hbaseblog.com/2010/11/30/hbase-coprocessors/
Think of Coprocessors at their simplest level as triggers before and
after basically every event that can happen on the server side. You
can use this as you intended to merge values or even replace them
completely.
Lars
On Fri, Jan 28, 2011 at 7:34 PM, Jean-Daniel Cryans <jd...@apache.org> wrote:
>> Cool, so the coprocessor will feed the value in the database to me and the value that is coming in just before it is written?
>>
>> With bytes, I am using serialized json so the example still applies perfectly where I could merge the results in the coprocessor and the coprocessor writes the final result. Would that work? If so, I will definitely check it out.
>
> It is my understanding that yes, it would work like that at a high level.
>
> J-D
>
Re: is there a pluggable conflict resolver in hbase
Posted by Jean-Daniel Cryans <jd...@apache.org>.
> Cool, so the coprocessor will feed the value in the database to me and the value that is coming in just before it is written?
>
> With bytes, I am using serialized json so the example still applies perfectly where I could merge the results in the coprocessor and the coprocessor writes the final result. Would that work? If so, I will definitely check it out.
It is my understanding that yes, it would work like that at a high level.
J-D
RE: is there a pluggable conflict resolver in hbase
Posted by "Hiller, Dean (Contractor)" <de...@broadridge.com>.
Cool, so the coprocessor will feed the value in the database to me and the value that is coming in just before it is written?
With bytes, I am using serialized json so the example still applies perfectly where I could merge the results in the coprocessor and the coprocessor writes the final result. Would that work? If so, I will definitely check it out.
Thanks,
Dean
-----Original Message-----
From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-Daniel Cryans
Sent: Thursday, January 27, 2011 4:27 PM
To: user@hbase.apache.org
Subject: Re: is there a pluggable conflict resolver in hbase
For HBase it's all byte[], so there's currently nothing as involved as
what you describe. What happens is that the second Put will overshadow
the first one, but both are accessible.
In the future I guess you could implement a coprocessor that does that
kind of check, but it's only available in trunk.
The closest you could do at the moment would be to do a Get first and
then do http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#checkAndPut(byte[],
byte[], byte[], byte[], org.apache.hadoop.hbase.client.Put)
J-D
On Thu, Jan 27, 2011 at 1:28 PM, Hiller, Dean (Contractor)
<de...@broadridge.com> wrote:
> Tthere is this really really cool feature in gemfire for conflict
> resolution(and I hope hbase has something similar here as it seems so
> clean). You can implement an interface that gets called just before the
> write to storage that gives you the old value and the new value(and is
> single threaded for that row so multiple updates on that row are in same
> thread).
>
>
>
> Perfect example
>
> Key="myCar" value=contains people A, B, C version 4
>
>
>
> A put done from node 1 and a put done from node 2 at the same time
>
> 1. Put people A, B, C, D version 4
>
> 2. Put people A, B, C, E version 4
>
>
>
> So the conflict resolver can put 1 fine but then on 2, it can see that
> version is 5 on oldValue and can merge A, b, C, E with A, B, C, D
> resulting in the full list A, B, C, D, E. this is a dang nice feature.
>
>
>
> Does hbase have something similar that one can implement that gets
> called on each node out there to do conflict resolution?
>
>
>
> Thanks,
>
> Dean
>
>
>
>
>
>
> This message and any attachments are intended only for the use of the addressee and
> may contain information that is privileged and confidential. If the reader of the
> message is not the intended recipient or an authorized representative of the
> intended recipient, you are hereby notified that any dissemination of this
> communication is strictly prohibited. If you have received this communication in
> error, please notify us immediately by e-mail and delete the message and any
> attachments from your system.
>
>
This message and any attachments are intended only for the use of the addressee and
may contain information that is privileged and confidential. If the reader of the
message is not the intended recipient or an authorized representative of the
intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication in
error, please notify us immediately by e-mail and delete the message and any
attachments from your system.
Re: is there a pluggable conflict resolver in hbase
Posted by Jean-Daniel Cryans <jd...@apache.org>.
For HBase it's all byte[], so there's currently nothing as involved as
what you describe. What happens is that the second Put will overshadow
the first one, but both are accessible.
In the future I guess you could implement a coprocessor that does that
kind of check, but it's only available in trunk.
The closest you could do at the moment would be to do a Get first and
then do http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#checkAndPut(byte[],
byte[], byte[], byte[], org.apache.hadoop.hbase.client.Put)
J-D
On Thu, Jan 27, 2011 at 1:28 PM, Hiller, Dean (Contractor)
<de...@broadridge.com> wrote:
> Tthere is this really really cool feature in gemfire for conflict
> resolution(and I hope hbase has something similar here as it seems so
> clean). You can implement an interface that gets called just before the
> write to storage that gives you the old value and the new value(and is
> single threaded for that row so multiple updates on that row are in same
> thread).
>
>
>
> Perfect example
>
> Key="myCar" value=contains people A, B, C version 4
>
>
>
> A put done from node 1 and a put done from node 2 at the same time
>
> 1. Put people A, B, C, D version 4
>
> 2. Put people A, B, C, E version 4
>
>
>
> So the conflict resolver can put 1 fine but then on 2, it can see that
> version is 5 on oldValue and can merge A, b, C, E with A, B, C, D
> resulting in the full list A, B, C, D, E. this is a dang nice feature.
>
>
>
> Does hbase have something similar that one can implement that gets
> called on each node out there to do conflict resolution?
>
>
>
> Thanks,
>
> Dean
>
>
>
>
>
>
> This message and any attachments are intended only for the use of the addressee and
> may contain information that is privileged and confidential. If the reader of the
> message is not the intended recipient or an authorized representative of the
> intended recipient, you are hereby notified that any dissemination of this
> communication is strictly prohibited. If you have received this communication in
> error, please notify us immediately by e-mail and delete the message and any
> attachments from your system.
>
>