You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Asaf Mesika <as...@gmail.com> on 2012/07/09 15:19:10 UTC
Composing your own timestamp
Hi,
We've been tinkering around ideas of implementing secondary index.
One of the ideas is based on concatenating three meaningful fields into a long: int, short (2 bytes), short. This long will be used as timestamp when issuing a put to the secondary index table.
This means an puts, timestamp wise, will not occur chronologically. since this is not a real timestamp.
Will this create any issues?
Thanks!
Asaf
Re: Composing your own timestamp
Posted by Mohammad Tariq <do...@gmail.com>.
Hi Asaf,
Apologies for being so dumb. I should have read the question properly.
Regards,
Mohammad Tariq
On Tue, Jul 10, 2012 at 9:01 AM, Asaf Mesika <as...@gmail.com> wrote:
> The int, short, short part goes to the time stamp.
>
> Thanks!
>
> Sent from my iPad
>
> On 10 ביול 2012, at 01:08, Mohammad Tariq <do...@gmail.com> wrote:
>
>> Hello Asaf,
>> If the 'int' parts of your rowkeys are close to each other, you may
>> face hotspotting.
>>
>> Best
>> -Tariq
>>
>> On Monday, July 9, 2012, Asaf Mesika <as...@gmail.com> wrote:
>>> No.
>>> My index is composed of several fields. Some goes to the RowKey, some to
>> the column name, and some - and hence the question - to timestamp.
>>> Those that goes to the timestamp are of types Integer, Short and short
>> which together form 8 bytes - the size of the Timestamp in hbase.
>>>
>>> So my question was if that's ok? I mean, it seems that the normal usage
>> for the timestamp is to have a timestamp (ms since epoch) and to have
>> values there which always increasing - meaning: you'll never see value 7
>> enters *before* value 3, for example.
>>>
>>> On Jul 9, 2012, at 18:54 PM, Sonal Goyal wrote:
>>>
>>>> Sorry I did not understand your question. Are you planning to use the
>>>> concatenated long as the rowkey to your secondary table?
>>>>
>>>> Best Regards,
>>>> Sonal
>>>> Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
>>>> Nube Technologies <http://www.nubetech.co>
>>>>
>>>> <http://in.linkedin.com/in/sonalgoyal>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Jul 9, 2012 at 6:49 PM, Asaf Mesika <as...@gmail.com>
>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> We've been tinkering around ideas of implementing secondary index.
>>>>> One of the ideas is based on concatenating three meaningful fields into
>> a
>>>>> long: int, short (2 bytes), short. This long will be used as timestamp
>> when
>>>>> issuing a put to the secondary index table.
>>>>> This means an puts, timestamp wise, will not occur chronologically.
>> since
>>>>> this is not a real timestamp.
>>>>>
>>>>> Will this create any issues?
>>>>>
>>>>>
>>>>> Thanks!
>>>>>
>>>>> Asaf
>>>>>
>>>>>
>>>
>>>
>>
>> --
>> Regards,
>> Mohammad Tariq
Re: Composing your own timestamp
Posted by Asaf Mesika <as...@gmail.com>.
The int, short, short part goes to the time stamp.
Thanks!
Sent from my iPad
On 10 ביול 2012, at 01:08, Mohammad Tariq <do...@gmail.com> wrote:
> Hello Asaf,
> If the 'int' parts of your rowkeys are close to each other, you may
> face hotspotting.
>
> Best
> -Tariq
>
> On Monday, July 9, 2012, Asaf Mesika <as...@gmail.com> wrote:
>> No.
>> My index is composed of several fields. Some goes to the RowKey, some to
> the column name, and some - and hence the question - to timestamp.
>> Those that goes to the timestamp are of types Integer, Short and short
> which together form 8 bytes - the size of the Timestamp in hbase.
>>
>> So my question was if that's ok? I mean, it seems that the normal usage
> for the timestamp is to have a timestamp (ms since epoch) and to have
> values there which always increasing - meaning: you'll never see value 7
> enters *before* value 3, for example.
>>
>> On Jul 9, 2012, at 18:54 PM, Sonal Goyal wrote:
>>
>>> Sorry I did not understand your question. Are you planning to use the
>>> concatenated long as the rowkey to your secondary table?
>>>
>>> Best Regards,
>>> Sonal
>>> Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
>>> Nube Technologies <http://www.nubetech.co>
>>>
>>> <http://in.linkedin.com/in/sonalgoyal>
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Jul 9, 2012 at 6:49 PM, Asaf Mesika <as...@gmail.com>
> wrote:
>>>
>>>> Hi,
>>>>
>>>> We've been tinkering around ideas of implementing secondary index.
>>>> One of the ideas is based on concatenating three meaningful fields into
> a
>>>> long: int, short (2 bytes), short. This long will be used as timestamp
> when
>>>> issuing a put to the secondary index table.
>>>> This means an puts, timestamp wise, will not occur chronologically.
> since
>>>> this is not a real timestamp.
>>>>
>>>> Will this create any issues?
>>>>
>>>>
>>>> Thanks!
>>>>
>>>> Asaf
>>>>
>>>>
>>
>>
>
> --
> Regards,
> Mohammad Tariq
Re: Composing your own timestamp
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Asaf,
If the 'int' parts of your rowkeys are close to each other, you may
face hotspotting.
Best
-Tariq
On Monday, July 9, 2012, Asaf Mesika <as...@gmail.com> wrote:
> No.
> My index is composed of several fields. Some goes to the RowKey, some to
the column name, and some - and hence the question - to timestamp.
> Those that goes to the timestamp are of types Integer, Short and short
which together form 8 bytes - the size of the Timestamp in hbase.
>
> So my question was if that's ok? I mean, it seems that the normal usage
for the timestamp is to have a timestamp (ms since epoch) and to have
values there which always increasing - meaning: you'll never see value 7
enters *before* value 3, for example.
>
> On Jul 9, 2012, at 18:54 PM, Sonal Goyal wrote:
>
>> Sorry I did not understand your question. Are you planning to use the
>> concatenated long as the rowkey to your secondary table?
>>
>> Best Regards,
>> Sonal
>> Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
>> Nube Technologies <http://www.nubetech.co>
>>
>> <http://in.linkedin.com/in/sonalgoyal>
>>
>>
>>
>>
>>
>> On Mon, Jul 9, 2012 at 6:49 PM, Asaf Mesika <as...@gmail.com>
wrote:
>>
>>> Hi,
>>>
>>> We've been tinkering around ideas of implementing secondary index.
>>> One of the ideas is based on concatenating three meaningful fields into
a
>>> long: int, short (2 bytes), short. This long will be used as timestamp
when
>>> issuing a put to the secondary index table.
>>> This means an puts, timestamp wise, will not occur chronologically.
since
>>> this is not a real timestamp.
>>>
>>> Will this create any issues?
>>>
>>>
>>> Thanks!
>>>
>>> Asaf
>>>
>>>
>
>
--
Regards,
Mohammad Tariq
Re: Composing your own timestamp
Posted by Asaf Mesika <as...@gmail.com>.
No.
My index is composed of several fields. Some goes to the RowKey, some to the column name, and some - and hence the question - to timestamp.
Those that goes to the timestamp are of types Integer, Short and short which together form 8 bytes - the size of the Timestamp in hbase.
So my question was if that's ok? I mean, it seems that the normal usage for the timestamp is to have a timestamp (ms since epoch) and to have values there which always increasing - meaning: you'll never see value 7 enters *before* value 3, for example.
On Jul 9, 2012, at 18:54 PM, Sonal Goyal wrote:
> Sorry I did not understand your question. Are you planning to use the
> concatenated long as the rowkey to your secondary table?
>
> Best Regards,
> Sonal
> Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
> Nube Technologies <http://www.nubetech.co>
>
> <http://in.linkedin.com/in/sonalgoyal>
>
>
>
>
>
> On Mon, Jul 9, 2012 at 6:49 PM, Asaf Mesika <as...@gmail.com> wrote:
>
>> Hi,
>>
>> We've been tinkering around ideas of implementing secondary index.
>> One of the ideas is based on concatenating three meaningful fields into a
>> long: int, short (2 bytes), short. This long will be used as timestamp when
>> issuing a put to the secondary index table.
>> This means an puts, timestamp wise, will not occur chronologically. since
>> this is not a real timestamp.
>>
>> Will this create any issues?
>>
>>
>> Thanks!
>>
>> Asaf
>>
>>
Re: Composing your own timestamp
Posted by Sonal Goyal <so...@gmail.com>.
Sorry I did not understand your question. Are you planning to use the
concatenated long as the rowkey to your secondary table?
Best Regards,
Sonal
Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
Nube Technologies <http://www.nubetech.co>
<http://in.linkedin.com/in/sonalgoyal>
On Mon, Jul 9, 2012 at 6:49 PM, Asaf Mesika <as...@gmail.com> wrote:
> Hi,
>
> We've been tinkering around ideas of implementing secondary index.
> One of the ideas is based on concatenating three meaningful fields into a
> long: int, short (2 bytes), short. This long will be used as timestamp when
> issuing a put to the secondary index table.
> This means an puts, timestamp wise, will not occur chronologically. since
> this is not a real timestamp.
>
> Will this create any issues?
>
>
> Thanks!
>
> Asaf
>
>