You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by Venkat <rk...@gmail.com> on 2013/04/22 20:29:42 UTC

timestamp order

hello accumulo,

what is a good way to get rows in chronological order ?   i am trying to
get to a use case of "give me the last 5 inserted for this particular cf,
cq ..."  timestampfilter works well when I know the times but i dont know
the exact times of insertion.

Re: timestamp order

Posted by Eric Newton <er...@gmail.com>.
That approach will tend to hot-spot on ingest: the latest updates will
always go to the first tablet.  Maybe that's ok, but you might want some
scheme to spread those updates to other tablets.

-Eric



On Tue, Apr 23, 2013 at 8:12 PM, Venkat <rk...@gmail.com> wrote:

> Thanks Keith, This is a great suggestion. Trying out the reverse of that
> .. (999999999999 - timestamp)+machineID
>
> Thanks !
>
>
> On Tue, Apr 23, 2013 at 7:59 AM, Keith Turner <ke...@deenlo.com> wrote:
>
>>
>>
>>
>> On Mon, Apr 22, 2013 at 8:59 PM, Venkat <rk...@gmail.com> wrote:
>>
>>> It looks like versioningiterator gives me the past versions for a cell
>>>  .... but I need to look across rows ..
>>>
>>> I am inserting time series data for set of machines into accumulo and i
>>> am using a combo of machineid + timestamp as the row ID and metric as the
>>> cq.  So for me to get the last time a metric has been inserted for an
>>> asset, i need to look across row IDs and get the last 5 rows that got
>>> inserted with that metric ...
>>>
>>
>> You could consider sorting your data such that the most recent timestamps
>> come first.   If you insert (machineid + (999999999999 - timestamp)).   To
>> get the 5 most recent row, start scanning at (machineid + 999999999999) and
>> stop when you have 5 rows.   I picked a completely arbitrary number of 9's
>> for the example, not sure what time resolution its appropriate for.
>>
>>
>>>
>>>
>>>
>>> On Mon, Apr 22, 2013 at 2:48 PM, Billie Rinaldi <
>>> billie.rinaldi@gmail.com> wrote:
>>>
>>>> You could use a VersioningIterator with maxVersions set to 5.
>>>>
>>>> Billie
>>>>
>>>>
>>>>
>>>> On Mon, Apr 22, 2013 at 11:29 AM, Venkat <rk...@gmail.com> wrote:
>>>>
>>>>> hello accumulo,
>>>>>
>>>>> what is a good way to get rows in chronological order ?   i am trying
>>>>> to get to a use case of "give me the last 5 inserted for this particular
>>>>> cf, cq ..."  timestampfilter works well when I know the times but i dont
>>>>> know the exact times of insertion.
>>>>>
>>>>
>>>>
>>>
>>
>

Re: timestamp order

Posted by Venkat <rk...@gmail.com>.
Thanks Keith, This is a great suggestion. Trying out the reverse of
that .. (999999999999
- timestamp)+machineID

Thanks !


On Tue, Apr 23, 2013 at 7:59 AM, Keith Turner <ke...@deenlo.com> wrote:

>
>
>
> On Mon, Apr 22, 2013 at 8:59 PM, Venkat <rk...@gmail.com> wrote:
>
>> It looks like versioningiterator gives me the past versions for a cell
>>  .... but I need to look across rows ..
>>
>> I am inserting time series data for set of machines into accumulo and i
>> am using a combo of machineid + timestamp as the row ID and metric as the
>> cq.  So for me to get the last time a metric has been inserted for an
>> asset, i need to look across row IDs and get the last 5 rows that got
>> inserted with that metric ...
>>
>
> You could consider sorting your data such that the most recent timestamps
> come first.   If you insert (machineid + (999999999999 - timestamp)).   To
> get the 5 most recent row, start scanning at (machineid + 999999999999) and
> stop when you have 5 rows.   I picked a completely arbitrary number of 9's
> for the example, not sure what time resolution its appropriate for.
>
>
>>
>>
>>
>> On Mon, Apr 22, 2013 at 2:48 PM, Billie Rinaldi <billie.rinaldi@gmail.com
>> > wrote:
>>
>>> You could use a VersioningIterator with maxVersions set to 5.
>>>
>>> Billie
>>>
>>>
>>>
>>> On Mon, Apr 22, 2013 at 11:29 AM, Venkat <rk...@gmail.com> wrote:
>>>
>>>> hello accumulo,
>>>>
>>>> what is a good way to get rows in chronological order ?   i am trying
>>>> to get to a use case of "give me the last 5 inserted for this particular
>>>> cf, cq ..."  timestampfilter works well when I know the times but i dont
>>>> know the exact times of insertion.
>>>>
>>>
>>>
>>
>

Re: timestamp order

Posted by Keith Turner <ke...@deenlo.com>.
On Mon, Apr 22, 2013 at 8:59 PM, Venkat <rk...@gmail.com> wrote:

> It looks like versioningiterator gives me the past versions for a cell
>  .... but I need to look across rows ..
>
> I am inserting time series data for set of machines into accumulo and i am
> using a combo of machineid + timestamp as the row ID and metric as the cq.
>  So for me to get the last time a metric has been inserted for an asset, i
> need to look across row IDs and get the last 5 rows that got inserted with
> that metric ...
>

You could consider sorting your data such that the most recent timestamps
come first.   If you insert (machineid + (999999999999 - timestamp)).   To
get the 5 most recent row, start scanning at (machineid + 999999999999) and
stop when you have 5 rows.   I picked a completely arbitrary number of 9's
for the example, not sure what time resolution its appropriate for.


>
>
>
> On Mon, Apr 22, 2013 at 2:48 PM, Billie Rinaldi <bi...@gmail.com>wrote:
>
>> You could use a VersioningIterator with maxVersions set to 5.
>>
>> Billie
>>
>>
>>
>> On Mon, Apr 22, 2013 at 11:29 AM, Venkat <rk...@gmail.com> wrote:
>>
>>> hello accumulo,
>>>
>>> what is a good way to get rows in chronological order ?   i am trying to
>>> get to a use case of "give me the last 5 inserted for this particular cf,
>>> cq ..."  timestampfilter works well when I know the times but i dont know
>>> the exact times of insertion.
>>>
>>
>>
>

Re: timestamp order

Posted by Venkat <rk...@gmail.com>.
It looks like versioningiterator gives me the past versions for a cell
 .... but I need to look across rows ..

I am inserting time series data for set of machines into accumulo and i am
using a combo of machineid + timestamp as the row ID and metric as the cq.
 So for me to get the last time a metric has been inserted for an asset, i
need to look across row IDs and get the last 5 rows that got inserted with
that metric ...




On Mon, Apr 22, 2013 at 2:48 PM, Billie Rinaldi <bi...@gmail.com>wrote:

> You could use a VersioningIterator with maxVersions set to 5.
>
> Billie
>
>
>
> On Mon, Apr 22, 2013 at 11:29 AM, Venkat <rk...@gmail.com> wrote:
>
>> hello accumulo,
>>
>> what is a good way to get rows in chronological order ?   i am trying to
>> get to a use case of "give me the last 5 inserted for this particular cf,
>> cq ..."  timestampfilter works well when I know the times but i dont know
>> the exact times of insertion.
>>
>
>

Re: timestamp order

Posted by Billie Rinaldi <bi...@gmail.com>.
You could use a VersioningIterator with maxVersions set to 5.

Billie


On Mon, Apr 22, 2013 at 11:29 AM, Venkat <rk...@gmail.com> wrote:

> hello accumulo,
>
> what is a good way to get rows in chronological order ?   i am trying to
> get to a use case of "give me the last 5 inserted for this particular cf,
> cq ..."  timestampfilter works well when I know the times but i dont know
> the exact times of insertion.
>