You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Rural Hunter <ru...@gmail.com> on 2016/01/22 10:51:17 UTC
data write/read consistency issue
Hi,
I have a hbase cluster with 7 servers at version 0.98.13-hadoop2,
dfs.replication=2.
In a write session, we update some data. Then in a new read session
immediately, we read the data using Get class and found it sometimes
returns the old version of the data(before the update).
We have to add a retry-loop in the read session to read the correct value.
Is this a normal behavior of hbase cluster?
Re: data write/read consistency issue
Posted by Rural Hunter <ru...@gmail.com>.
It is default to 'true' according to the javadoc right? We didn't change
this setting.
2016-01-26 3:25 GMT+08:00 Vladimir Rodionov <vl...@gmail.com>:
> Correction:
>
> not always visible. To make sure that they are visible - call the table
> flush or setAutoFlush(true) (not recommended)
>
> -Vlad
>
> On Mon, Jan 25, 2016 at 11:21 AM, Vladimir Rodionov <
> vladrodionov@gmail.com>
> wrote:
>
> > Writes are not visible until client calls flush on a table.
> >
> > -Vlad
> >
> > On Mon, Jan 25, 2016 at 9:18 AM, Stack <st...@duboce.net> wrote:
> >
> >> Try some more basic and see if you can reproduce the failure to
> >> read-your-own-writes. Do a put, flush of the writes, then a get. Past
> too
> >> your addTagColumn code. Thanks.
> >> St.Ack
> >>
> >> On Mon, Jan 25, 2016 at 12:33 AM, Rural Hunter <ru...@gmail.com>
> >> wrote:
> >>
> >> > No. The code logic is like this:
> >> > The main method:
> >> >
> >> > String rowKey="...";
> >> > addTagColumn(rowKey, "tag_"+id);
> >> > List tags=getTagColumns(rowKey);
> >> > //here I had to add re-try logic to ensure the tags list contains the
> id
> >> > just added.
> >> >
> >> > addTagColumn method just does a simple HTable.put.
> >> > getTagColumns uses a ColumnPrefixFilter to get all the columns
> starting
> >> > with "tag_".
> >> >
> >> > The other possiblity is that some other processes removed the new id
> >> > between the call of add and get. But we checked other code logic and
> it
> >> > seems not likely. or there is a problem with the ColumnPrefixFilter?
> >> >
> >> > 2016-01-23 3:42 GMT+08:00 Stack <st...@duboce.net>:
> >> >
> >> > > On Fri, Jan 22, 2016 at 1:51 AM, Rural Hunter <
> ruralhunter@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Hi,
> >> > > >
> >> > > > I have a hbase cluster with 7 servers at version 0.98.13-hadoop2,
> >> > > > dfs.replication=2.
> >> > > > In a write session, we update some data. Then in a new read
> session
> >> > > > immediately, we read the data using Get class and found it
> sometimes
> >> > > > returns the old version of the data(before the update).
> >> > > > We have to add a retry-loop in the read session to read the
> correct
> >> > > value.
> >> > > > Is this a normal behavior of hbase cluster?
> >> > > >
> >> > >
> >> > >
> >> > > No. Tell us more. For sure the Get and Write are not concurrent
> with
> >> > > perhaps the Get happening before the update?
> >> > > St.Ack
> >> > >
> >> >
> >>
> >
> >
>
Re: data write/read consistency issue
Posted by Vladimir Rodionov <vl...@gmail.com>.
Correction:
not always visible. To make sure that they are visible - call the table
flush or setAutoFlush(true) (not recommended)
-Vlad
On Mon, Jan 25, 2016 at 11:21 AM, Vladimir Rodionov <vl...@gmail.com>
wrote:
> Writes are not visible until client calls flush on a table.
>
> -Vlad
>
> On Mon, Jan 25, 2016 at 9:18 AM, Stack <st...@duboce.net> wrote:
>
>> Try some more basic and see if you can reproduce the failure to
>> read-your-own-writes. Do a put, flush of the writes, then a get. Past too
>> your addTagColumn code. Thanks.
>> St.Ack
>>
>> On Mon, Jan 25, 2016 at 12:33 AM, Rural Hunter <ru...@gmail.com>
>> wrote:
>>
>> > No. The code logic is like this:
>> > The main method:
>> >
>> > String rowKey="...";
>> > addTagColumn(rowKey, "tag_"+id);
>> > List tags=getTagColumns(rowKey);
>> > //here I had to add re-try logic to ensure the tags list contains the id
>> > just added.
>> >
>> > addTagColumn method just does a simple HTable.put.
>> > getTagColumns uses a ColumnPrefixFilter to get all the columns starting
>> > with "tag_".
>> >
>> > The other possiblity is that some other processes removed the new id
>> > between the call of add and get. But we checked other code logic and it
>> > seems not likely. or there is a problem with the ColumnPrefixFilter?
>> >
>> > 2016-01-23 3:42 GMT+08:00 Stack <st...@duboce.net>:
>> >
>> > > On Fri, Jan 22, 2016 at 1:51 AM, Rural Hunter <ru...@gmail.com>
>> > > wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > I have a hbase cluster with 7 servers at version 0.98.13-hadoop2,
>> > > > dfs.replication=2.
>> > > > In a write session, we update some data. Then in a new read session
>> > > > immediately, we read the data using Get class and found it sometimes
>> > > > returns the old version of the data(before the update).
>> > > > We have to add a retry-loop in the read session to read the correct
>> > > value.
>> > > > Is this a normal behavior of hbase cluster?
>> > > >
>> > >
>> > >
>> > > No. Tell us more. For sure the Get and Write are not concurrent with
>> > > perhaps the Get happening before the update?
>> > > St.Ack
>> > >
>> >
>>
>
>
Re: data write/read consistency issue
Posted by Vladimir Rodionov <vl...@gmail.com>.
Writes are not visible until client calls flush on a table.
-Vlad
On Mon, Jan 25, 2016 at 9:18 AM, Stack <st...@duboce.net> wrote:
> Try some more basic and see if you can reproduce the failure to
> read-your-own-writes. Do a put, flush of the writes, then a get. Past too
> your addTagColumn code. Thanks.
> St.Ack
>
> On Mon, Jan 25, 2016 at 12:33 AM, Rural Hunter <ru...@gmail.com>
> wrote:
>
> > No. The code logic is like this:
> > The main method:
> >
> > String rowKey="...";
> > addTagColumn(rowKey, "tag_"+id);
> > List tags=getTagColumns(rowKey);
> > //here I had to add re-try logic to ensure the tags list contains the id
> > just added.
> >
> > addTagColumn method just does a simple HTable.put.
> > getTagColumns uses a ColumnPrefixFilter to get all the columns starting
> > with "tag_".
> >
> > The other possiblity is that some other processes removed the new id
> > between the call of add and get. But we checked other code logic and it
> > seems not likely. or there is a problem with the ColumnPrefixFilter?
> >
> > 2016-01-23 3:42 GMT+08:00 Stack <st...@duboce.net>:
> >
> > > On Fri, Jan 22, 2016 at 1:51 AM, Rural Hunter <ru...@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I have a hbase cluster with 7 servers at version 0.98.13-hadoop2,
> > > > dfs.replication=2.
> > > > In a write session, we update some data. Then in a new read session
> > > > immediately, we read the data using Get class and found it sometimes
> > > > returns the old version of the data(before the update).
> > > > We have to add a retry-loop in the read session to read the correct
> > > value.
> > > > Is this a normal behavior of hbase cluster?
> > > >
> > >
> > >
> > > No. Tell us more. For sure the Get and Write are not concurrent with
> > > perhaps the Get happening before the update?
> > > St.Ack
> > >
> >
>
Re: data write/read consistency issue
Posted by Rural Hunter <ru...@gmail.com>.
OK, we will try that. It only happens seldom. So there is no consistent way
to reproduce it. The addTagColumn is like this:
HTableInterface table = null;
try
{
table = getHtable(TB_TAG);
Put put = new Put(Bytes.toBytes(rowid));
put.add(CF_TAG_LIST, Bytes.toBytes("tag_" + tag),
Bytes.toBytes(System.currentTimeMillis()));
table.put(put);
}
finally
{
closeHTable(table);
}
2016-01-26 1:18 GMT+08:00 Stack <st...@duboce.net>:
> Try some more basic and see if you can reproduce the failure to
> read-your-own-writes. Do a put, flush of the writes, then a get. Past too
> your addTagColumn code. Thanks.
> St.Ack
>
> On Mon, Jan 25, 2016 at 12:33 AM, Rural Hunter <ru...@gmail.com>
> wrote:
>
> > No. The code logic is like this:
> > The main method:
> >
> > String rowKey="...";
> > addTagColumn(rowKey, "tag_"+id);
> > List tags=getTagColumns(rowKey);
> > //here I had to add re-try logic to ensure the tags list contains the id
> > just added.
> >
> > addTagColumn method just does a simple HTable.put.
> > getTagColumns uses a ColumnPrefixFilter to get all the columns starting
> > with "tag_".
> >
> > The other possiblity is that some other processes removed the new id
> > between the call of add and get. But we checked other code logic and it
> > seems not likely. or there is a problem with the ColumnPrefixFilter?
> >
> > 2016-01-23 3:42 GMT+08:00 Stack <st...@duboce.net>:
> >
> > > On Fri, Jan 22, 2016 at 1:51 AM, Rural Hunter <ru...@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I have a hbase cluster with 7 servers at version 0.98.13-hadoop2,
> > > > dfs.replication=2.
> > > > In a write session, we update some data. Then in a new read session
> > > > immediately, we read the data using Get class and found it sometimes
> > > > returns the old version of the data(before the update).
> > > > We have to add a retry-loop in the read session to read the correct
> > > value.
> > > > Is this a normal behavior of hbase cluster?
> > > >
> > >
> > >
> > > No. Tell us more. For sure the Get and Write are not concurrent with
> > > perhaps the Get happening before the update?
> > > St.Ack
> > >
> >
>
Re: data write/read consistency issue
Posted by Stack <st...@duboce.net>.
Try some more basic and see if you can reproduce the failure to
read-your-own-writes. Do a put, flush of the writes, then a get. Past too
your addTagColumn code. Thanks.
St.Ack
On Mon, Jan 25, 2016 at 12:33 AM, Rural Hunter <ru...@gmail.com>
wrote:
> No. The code logic is like this:
> The main method:
>
> String rowKey="...";
> addTagColumn(rowKey, "tag_"+id);
> List tags=getTagColumns(rowKey);
> //here I had to add re-try logic to ensure the tags list contains the id
> just added.
>
> addTagColumn method just does a simple HTable.put.
> getTagColumns uses a ColumnPrefixFilter to get all the columns starting
> with "tag_".
>
> The other possiblity is that some other processes removed the new id
> between the call of add and get. But we checked other code logic and it
> seems not likely. or there is a problem with the ColumnPrefixFilter?
>
> 2016-01-23 3:42 GMT+08:00 Stack <st...@duboce.net>:
>
> > On Fri, Jan 22, 2016 at 1:51 AM, Rural Hunter <ru...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I have a hbase cluster with 7 servers at version 0.98.13-hadoop2,
> > > dfs.replication=2.
> > > In a write session, we update some data. Then in a new read session
> > > immediately, we read the data using Get class and found it sometimes
> > > returns the old version of the data(before the update).
> > > We have to add a retry-loop in the read session to read the correct
> > value.
> > > Is this a normal behavior of hbase cluster?
> > >
> >
> >
> > No. Tell us more. For sure the Get and Write are not concurrent with
> > perhaps the Get happening before the update?
> > St.Ack
> >
>
Re: data write/read consistency issue
Posted by Rural Hunter <ru...@gmail.com>.
No. The code logic is like this:
The main method:
String rowKey="...";
addTagColumn(rowKey, "tag_"+id);
List tags=getTagColumns(rowKey);
//here I had to add re-try logic to ensure the tags list contains the id
just added.
addTagColumn method just does a simple HTable.put.
getTagColumns uses a ColumnPrefixFilter to get all the columns starting
with "tag_".
The other possiblity is that some other processes removed the new id
between the call of add and get. But we checked other code logic and it
seems not likely. or there is a problem with the ColumnPrefixFilter?
2016-01-23 3:42 GMT+08:00 Stack <st...@duboce.net>:
> On Fri, Jan 22, 2016 at 1:51 AM, Rural Hunter <ru...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I have a hbase cluster with 7 servers at version 0.98.13-hadoop2,
> > dfs.replication=2.
> > In a write session, we update some data. Then in a new read session
> > immediately, we read the data using Get class and found it sometimes
> > returns the old version of the data(before the update).
> > We have to add a retry-loop in the read session to read the correct
> value.
> > Is this a normal behavior of hbase cluster?
> >
>
>
> No. Tell us more. For sure the Get and Write are not concurrent with
> perhaps the Get happening before the update?
> St.Ack
>
Re: data write/read consistency issue
Posted by Stack <st...@duboce.net>.
On Fri, Jan 22, 2016 at 1:51 AM, Rural Hunter <ru...@gmail.com> wrote:
> Hi,
>
> I have a hbase cluster with 7 servers at version 0.98.13-hadoop2,
> dfs.replication=2.
> In a write session, we update some data. Then in a new read session
> immediately, we read the data using Get class and found it sometimes
> returns the old version of the data(before the update).
> We have to add a retry-loop in the read session to read the correct value.
> Is this a normal behavior of hbase cluster?
>
No. Tell us more. For sure the Get and Write are not concurrent with
perhaps the Get happening before the update?
St.Ack