Thrift timestamps exclusing in HBase 0.20.2

[Peter Falk <pe...@bugsoft.nu>]

Hi,

We have recently switched to HBase 0.20.2 (with a new thrift version from
trunk) and noticed a strange behaviour when retriving rows with given
timestamp. When trying to retrieve a row with e.g. timestamp 123, then only
versions with timestamp less than 123 will be returned (e.g. 122 or less).
Should the timestamp specification be inclusive, i.e. rows with 123 or less
should be returned? I have verified this using both a ruby and a python
thrift client.

Trying the same from the HBase shell works as before, i.e. versions with
timestamp 123 inclusive is returned. Seems very strange that the behaviour
differs between thrift and the shell. Is this a bug, or should perhaps
thrift be configured differently? Should an older thrift version be used?

TIA,
Peter

Re: Thrift timestamps exclusing in HBase 0.20.2

[Lars Francke <la...@gmail.com>]

> I was using the getRowTs method, not the getVerTs but they seem to have the same
> behavior. Would be nice if the documentation at least was changed.

I've missed this! In this case I think we should change the behavior
as getRowTs clearly is intended to only get one row from one specific
timestamp and this method doesn't do that. I've opened a ticket for
this[1] and will try to get it done before 0.20.4 is released.

Thanks for the report!

Lars

[1] https://issues.apache.org/jira/browse/HBASE-2166

Re: Thrift timestamps exclusing in HBase 0.20.2

[Peter Falk <pe...@bugsoft.nu>]

Thanks Lars for the informative and surprising answer. I do not have the
date or version number of the old thrift available currently. Seems
reasonable that thrift should not affect this behavior, as You say. I was
using the getRowTs method, not the getVerTs but they seem to have the same
behavior. Would be nice if the documentation at least was changed.

Seems like the best solution for us is to simply increase timestamp by one,
to get the old behavior back. However, it does not feel like a nice
solution. Can't help but feeling like the old definition is more natural.

Thanks again,
Peter

On Mon, Jan 25, 2010 at 14:16, Lars Francke <la...@gmail.com> wrote:

> > We have recently switched to HBase 0.20.2 (with a new thrift version from
> trunk)
>
> Which version were you using before that?
>
> > Trying the same from the HBase shell works as before, i.e. versions with
> > timestamp 123 inclusive is returned. Seems very strange that the
> behaviour
> > differs between thrift and the shell. Is this a bug, or should perhaps
> > thrift be configured differently? Should an older thrift version be used?
>
> The Thrift version _shouldn't_ affect this at all. As long as you
> receive any reply with a valid row you can be fairly certain that it
> is not Thrift's fault. I can't say much about the shell though, sorry.
>
> I assume you were using getVerTs? The timestamp you specify there is
> _exclusive_. The ThriftServer uses the setTimeRange method from the
> Get object[1] which documents this behavior. The Hbase.thrift (and
> thus the generated Thrift documentation) state that "only versions
> less than or equal to the specified timestamp will be returned" which
> is not true. So this is a bug. The only question is if we should
> correct the documentation or alter the behavior of this function to
> comply with the current documentation.
>
> I'd prefer just changing the documentation for multiple reasons: This
> API is used by some people and I'd guess that they depend on the
> current behavior and this behavior mirrors the Java API which is
> always a good thing for people familiar with both APIs.
>
> Cheers,
> Lars
>
> [1] <
> http://hadoop.apache.org/hbase/docs/r0.20.2/api/org/apache/hadoop/hbase/client/Get.html#setTimeRange(long
> ,
> long)>
>

Re: Thrift timestamps exclusing in HBase 0.20.2

[Lars Francke <la...@gmail.com>]

> We have recently switched to HBase 0.20.2 (with a new thrift version from trunk)

Which version were you using before that?

> Trying the same from the HBase shell works as before, i.e. versions with
> timestamp 123 inclusive is returned. Seems very strange that the behaviour
> differs between thrift and the shell. Is this a bug, or should perhaps
> thrift be configured differently? Should an older thrift version be used?

The Thrift version _shouldn't_ affect this at all. As long as you
receive any reply with a valid row you can be fairly certain that it
is not Thrift's fault. I can't say much about the shell though, sorry.

I assume you were using getVerTs? The timestamp you specify there is
_exclusive_. The ThriftServer uses the setTimeRange method from the
Get object[1] which documents this behavior. The Hbase.thrift (and
thus the generated Thrift documentation) state that "only versions
less than or equal to the specified timestamp will be returned" which
is not true. So this is a bug. The only question is if we should
correct the documentation or alter the behavior of this function to
comply with the current documentation.

I'd prefer just changing the documentation for multiple reasons: This
API is used by some people and I'd guess that they depend on the
current behavior and this behavior mirrors the Java API which is
always a good thing for people familiar with both APIs.

Cheers,
Lars

[1] <http://hadoop.apache.org/hbase/docs/r0.20.2/api/org/apache/hadoop/hbase/client/Get.html#setTimeRange(long,
long)>