You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hbase.apache.org by Mingtao Zhang <ma...@gmail.com> on 2014/07/25 19:49:39 UTC

Consistency Model

Hi,

I have the following row key and value (column family/column qualifier
ignored):

hash(sitename) + sessionid
  value: duration

I need to calculate the duration in a distributed environment correctly. As
far as I know, I need to lock the specific row during read-write (in my
case my read is only happen before the write) to ensure consistency.

Previously from the HBase Definitive Guide or 0.94.15, I could see (though
deprecated)

RowLock lockRow(byte[] row) throws IOException;
void unlockRow(RowLock rl) throws IOException;

But it's seems gone with the latest API.

https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTableInterface.html

Is there anyway I could achieve the consistency with latest HBase?

Best Regards,
Mingtao

Re: Consistency Model

Posted by Mingtao Zhang <ma...@gmail.com>.

The later case.

Thank you :).

Mingtao Sent from iPhone

> On Jul 25, 2014, at 8:09 PM, Stack <st...@duboce.net> wrote:
> 
> What do you mean by  'duplicate 1 piece of data in multiple places'?  Your
> client wants to put duplicate data in two places and have it consistent or
> are you referring to hbase's keeping up multiple instances of data under
> wraps.  If the former, please clarify, if the latter -- hbase takes care of
> it.
> Yours,
> St.Ack
> 
> 
> On Fri, Jul 25, 2014 at 3:58 PM, Mingtao Zhang <ma...@gmail.com>
> wrote:
> 
>> Oh ... I might have used a bad word; I mean 'duplicate 1 piece of data in
>> multiple places'.
>> 
>> Mingtao
>> 
>> 
>>> On Fri, Jul 25, 2014 at 6:54 PM, Stack <st...@duboce.net> wrote:
>>> 
>>> On Fri, Jul 25, 2014 at 3:47 PM, Mingtao Zhang <ma...@gmail.com>
>>> wrote:
>>> 
>>>> I think from client view I am good to go.
>>>> 
>>>> Will HBase ensure consistency within it self? In redundant/clustered
>>>> environment?
>>> 
>>> 
>>> Generally, if we are not consistent, its a bug but what you mean by
>>> 'redundant' in the above?
>>> 
>>> St.Ack
>>

Re: Consistency Model

Posted by Stack <st...@duboce.net>.

What do you mean by  'duplicate 1 piece of data in multiple places'?  Your
client wants to put duplicate data in two places and have it consistent or
are you referring to hbase's keeping up multiple instances of data under
wraps.  If the former, please clarify, if the latter -- hbase takes care of
it.
Yours,
St.Ack

On Fri, Jul 25, 2014 at 3:58 PM, Mingtao Zhang <ma...@gmail.com>
wrote:

> Oh ... I might have used a bad word; I mean 'duplicate 1 piece of data in
> multiple places'.
>
> Mingtao
>
>
> On Fri, Jul 25, 2014 at 6:54 PM, Stack <st...@duboce.net> wrote:
>
> > On Fri, Jul 25, 2014 at 3:47 PM, Mingtao Zhang <ma...@gmail.com>
> > wrote:
> >
> > > I think from client view I am good to go.
> > >
> > > Will HBase ensure consistency within it self? In redundant/clustered
> > > environment?
> > >
> >
> >
> > Generally, if we are not consistent, its a bug but what you mean by
> > 'redundant' in the above?
> >
> > St.Ack
> >
>

Re: Consistency Model

Posted by Mingtao Zhang <ma...@gmail.com>.

Oh ... I might have used a bad word; I mean 'duplicate 1 piece of data in
multiple places'.

Mingtao

On Fri, Jul 25, 2014 at 6:54 PM, Stack <st...@duboce.net> wrote:

> On Fri, Jul 25, 2014 at 3:47 PM, Mingtao Zhang <ma...@gmail.com>
> wrote:
>
> > I think from client view I am good to go.
> >
> > Will HBase ensure consistency within it self? In redundant/clustered
> > environment?
> >
>
>
> Generally, if we are not consistent, its a bug but what you mean by
> 'redundant' in the above?
>
> St.Ack
>

Re: Consistency Model

Posted by Stack <st...@duboce.net>.

On Fri, Jul 25, 2014 at 3:47 PM, Mingtao Zhang <ma...@gmail.com>
wrote:

> I think from client view I am good to go.
>
> Will HBase ensure consistency within it self? In redundant/clustered
> environment?
>

Generally, if we are not consistent, its a bug but what you mean by
'redundant' in the above?

St.Ack

Re: Consistency Model

Posted by Mingtao Zhang <ma...@gmail.com>.

I think from client view I am good to go.

Will HBase ensure consistency within it self? In redundant/clustered environment?

Mingtao Sent from iPhone

> On Jul 25, 2014, at 2:56 PM, Stack <st...@duboce.net> wrote:
> 
> On Fri, Jul 25, 2014 at 11:31 AM, Mingtao Zhang <ma...@gmail.com>
> wrote:
> 
>> Like looping until it's success?
> 
> Yes.  That is how it is usually done.
> St.Ack

Re: Consistency Model

Posted by Stack <st...@duboce.net>.

On Fri, Jul 25, 2014 at 11:31 AM, Mingtao Zhang <ma...@gmail.com>
wrote:

> Like looping until it's success?
>

Yes.  That is how it is usually done.
St.Ack

Re: Consistency Model

Posted by Mingtao Zhang <ma...@gmail.com>.

Like looping until it's success?

Mingtao Sent from iPhone

> On Jul 25, 2014, at 2:21 PM, Stack <st...@duboce.net> wrote:
> 
> Will the CAS methods work for you?
> 
> e.g.
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTableInterface.html#checkAndPut(byte[],
> byte[], byte[], byte[], org.apache.hadoop.hbase.client.Put)
> 
> St.Ack
> 
> 
> On Fri, Jul 25, 2014 at 10:49 AM, Mingtao Zhang <ma...@gmail.com>
> wrote:
> 
>> Hi,
>> 
>> I have the following row key and value (column family/column qualifier
>> ignored):
>> 
>> hash(sitename) + sessionid
>>  value: duration
>> 
>> I need to calculate the duration in a distributed environment correctly. As
>> far as I know, I need to lock the specific row during read-write (in my
>> case my read is only happen before the write) to ensure consistency.
>> 
>> Previously from the HBase Definitive Guide or 0.94.15, I could see (though
>> deprecated)
>> 
>> RowLock lockRow(byte[] row) throws IOException;
>> void unlockRow(RowLock rl) throws IOException;
>> 
>> But it's seems gone with the latest API.
>> 
>> 
>> https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTableInterface.html
>> 
>> Is there anyway I could achieve the consistency with latest HBase?
>> 
>> Best Regards,
>> Mingtao
>>

Re: Consistency Model

Posted by Stack <st...@duboce.net>.

Will the CAS methods work for you?

e.g.
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTableInterface.html#checkAndPut(byte[],
byte[], byte[], byte[], org.apache.hadoop.hbase.client.Put)

St.Ack


On Fri, Jul 25, 2014 at 10:49 AM, Mingtao Zhang <ma...@gmail.com>
wrote:

> Hi,
>
> I have the following row key and value (column family/column qualifier
> ignored):
>
> hash(sitename) + sessionid
>   value: duration
>
> I need to calculate the duration in a distributed environment correctly. As
> far as I know, I need to lock the specific row during read-write (in my
> case my read is only happen before the write) to ensure consistency.
>
> Previously from the HBase Definitive Guide or 0.94.15, I could see (though
> deprecated)
>
> RowLock lockRow(byte[] row) throws IOException;
> void unlockRow(RowLock rl) throws IOException;
>
> But it's seems gone with the latest API.
>
>
> https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTableInterface.html
>
> Is there anyway I could achieve the consistency with latest HBase?
>
> Best Regards,
> Mingtao
>