Keys with identical timestamps

[Dave Hardcastle <ha...@gmail.com>]

Hi,

Could someone clarify whether the following statement from the manual - "If
two inserts are made into Accumulo with the same rowID, column, and
timestamp, then the behavior is non-deterministic" - applies even if the
versioning iterator is off? Is the non-determinism the fact that the order
is undetermined if two identical inserts are made and all versions are kept?

I have an application where the key corresponds to an object and a time
range, and the value is properties of the object over that time range. The
time range is stored in the column qualifier, but I also put the end of the
time range as the timestamp of the key. I frequently get data late, and so
create a key and insert that, but that key may already exist in the table.
When multiple identical versions get put in, the values are aggregated
using a combiner. This seems to be working fine. But maybe I shouldn't be
assuming that Accumulo won't silently drop one of the two keys?

Thanks,

Dave.

Re: Keys with identical timestamps

[Dave Hardcastle <ha...@gmail.com>]

Adam, Billie,

That's what I thought - thanks for confirming, and for the quick response.

Dave.

On 9 February 2015 at 21:37, Billie Rinaldi <bi...@gmail.com>
wrote:

> On Mon, Feb 9, 2015 at 1:04 PM, Adam Fuchs <af...@apache.org> wrote:
>
>> Hi Dave,
>>
>> As long as your combiner is associative and commutative both of the
>> values should be represented in the combined result. The
>> non-determinism is really around ordering, which generally doesn't
>> matter for a combiner.
>>
>
> Yes.  However, do not attempt to insert identical keys in a single
> mutation.  Only one will be kept, whether versioning is enabled or not.
>
> Billie
>
>
>>
>> Adam
>>
>> On Mon, Feb 9, 2015 at 3:49 PM, Dave Hardcastle
>> <ha...@gmail.com> wrote:
>> > Hi,
>> >
>> > Could someone clarify whether the following statement from the manual -
>> "If
>> > two inserts are made into Accumulo with the same rowID, column, and
>> > timestamp, then the behavior is non-deterministic" - applies even if the
>> > versioning iterator is off? Is the non-determinism the fact that the
>> order
>> > is undetermined if two identical inserts are made and all versions are
>> kept?
>> >
>> > I have an application where the key corresponds to an object and a time
>> > range, and the value is properties of the object over that time range.
>> The
>> > time range is stored in the column qualifier, but I also put the end of
>> the
>> > time range as the timestamp of the key. I frequently get data late, and
>> so
>> > create a key and insert that, but that key may already exist in the
>> table.
>> > When multiple identical versions get put in, the values are aggregated
>> using
>> > a combiner. This seems to be working fine. But maybe I shouldn't be
>> assuming
>> > that Accumulo won't silently drop one of the two keys?
>> >
>> > Thanks,
>> >
>> > Dave.
>>
>
>

Re: Keys with identical timestamps

[Billie Rinaldi <bi...@gmail.com>]

On Mon, Feb 9, 2015 at 1:04 PM, Adam Fuchs <af...@apache.org> wrote:

> Hi Dave,
>
> As long as your combiner is associative and commutative both of the
> values should be represented in the combined result. The
> non-determinism is really around ordering, which generally doesn't
> matter for a combiner.
>

Yes.  However, do not attempt to insert identical keys in a single
mutation.  Only one will be kept, whether versioning is enabled or not.

Billie


>
> Adam
>
> On Mon, Feb 9, 2015 at 3:49 PM, Dave Hardcastle
> <ha...@gmail.com> wrote:
> > Hi,
> >
> > Could someone clarify whether the following statement from the manual -
> "If
> > two inserts are made into Accumulo with the same rowID, column, and
> > timestamp, then the behavior is non-deterministic" - applies even if the
> > versioning iterator is off? Is the non-determinism the fact that the
> order
> > is undetermined if two identical inserts are made and all versions are
> kept?
> >
> > I have an application where the key corresponds to an object and a time
> > range, and the value is properties of the object over that time range.
> The
> > time range is stored in the column qualifier, but I also put the end of
> the
> > time range as the timestamp of the key. I frequently get data late, and
> so
> > create a key and insert that, but that key may already exist in the
> table.
> > When multiple identical versions get put in, the values are aggregated
> using
> > a combiner. This seems to be working fine. But maybe I shouldn't be
> assuming
> > that Accumulo won't silently drop one of the two keys?
> >
> > Thanks,
> >
> > Dave.
>

Re: Keys with identical timestamps

[Adam Fuchs <af...@apache.org>]

Hi Dave,

As long as your combiner is associative and commutative both of the
values should be represented in the combined result. The
non-determinism is really around ordering, which generally doesn't
matter for a combiner.

Adam

On Mon, Feb 9, 2015 at 3:49 PM, Dave Hardcastle
<ha...@gmail.com> wrote:
> Hi,
>
> Could someone clarify whether the following statement from the manual - "If
> two inserts are made into Accumulo with the same rowID, column, and
> timestamp, then the behavior is non-deterministic" - applies even if the
> versioning iterator is off? Is the non-determinism the fact that the order
> is undetermined if two identical inserts are made and all versions are kept?
>
> I have an application where the key corresponds to an object and a time
> range, and the value is properties of the object over that time range. The
> time range is stored in the column qualifier, but I also put the end of the
> time range as the timestamp of the key. I frequently get data late, and so
> create a key and insert that, but that key may already exist in the table.
> When multiple identical versions get put in, the values are aggregated using
> a combiner. This seems to be working fine. But maybe I shouldn't be assuming
> that Accumulo won't silently drop one of the two keys?
>
> Thanks,
>
> Dave.