about HBase versions

[Zhenyu Zhong <zh...@gmail.com>]

Hi,

I would like to use nice feature of HBase --versions to store a timeseries
data for a rowkey.
However, I get duplicates for the same rowkey and the same timestamp if I
use Put and run mapreduce job multiple times.

For example.
Put put = new Put(rowkey.getBytes());
put.add("f1:c1".getBytes(), ts, value.getBytes());

I use TableOutputFormat as the output of the MapReduce job.
If I run the MapReduce job twice, I would get 2 records with the same rowkey
and same timestamp.

May I ask whether the Put just adds a row no matter that there is already a
row with the same key and timestamp in the table?

Best,
zhenyu

Re: about HBase versions

[Ted Yu <yu...@gmail.com>]

Can you provide a bit more detail on your use case ?
If row key and timestamp are different across writes, there would be 1
version.

Cheers

On Wed, Jan 28, 2015 at 7:25 AM, sirajudeen.v <si...@tcs.com> wrote:

> Is the HBase version is reliable solution to use in critical data. In case
> I
> am not going to store duplicate data (same row key and time stamp).
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/about-HBase-versions-tp4024747p4067937.html
> Sent from the HBase User mailing list archive at Nabble.com.
>

Re: about HBase versions

["sirajudeen.v" <si...@tcs.com>]

Is the HBase version is reliable solution to use in critical data. In case I
am not going to store duplicate data (same row key and time stamp).



--
View this message in context: http://apache-hbase.679495.n3.nabble.com/about-HBase-versions-tp4024747p4067937.html
Sent from the HBase User mailing list archive at Nabble.com.

Re: about HBase versions

[Zhenyu Zhong <zh...@gmail.com>]

Ryan,

Thanks a lot.

I also tried the hbase shell.
By using put command, I was able to overwrite the latest row which has the
same rowkey and timestamp.
Is the implementation in hbase shell different from TableOutputFormat?

Best,
zhenyu

On Tue, Nov 24, 2009 at 5:38 PM, Ryan Rawson <ry...@gmail.com> wrote:

> Right now the implementation will return an indeterminate version when
> there are duplicates with the same timestamp. If they happen to have
> the same value, you are ok.
>
> I think there are a few other gotchas with regards to compaction.
> Each timestamp only counts as 1 version, thus you may end up with more
> data than you intended depending on how many duplicates you have.
>
> On Tue, Nov 24, 2009 at 2:35 PM, Zhenyu Zhong <zh...@gmail.com>
> wrote:
> > Hi,
> >
> > I would like to use nice feature of HBase --versions to store a
> timeseries
> > data for a rowkey.
> > However, I get duplicates for the same rowkey and the same timestamp if I
> > use Put and run mapreduce job multiple times.
> >
> > For example.
> > Put put = new Put(rowkey.getBytes());
> > put.add("f1:c1".getBytes(), ts, value.getBytes());
> >
> > I use TableOutputFormat as the output of the MapReduce job.
> > If I run the MapReduce job twice, I would get 2 records with the same
> rowkey
> > and same timestamp.
> >
> > May I ask whether the Put just adds a row no matter that there is already
> a
> > row with the same key and timestamp in the table?
> >
> > Best,
> > zhenyu
> >
>

Re: about HBase versions

[Ryan Rawson <ry...@gmail.com>]

Right now the implementation will return an indeterminate version when
there are duplicates with the same timestamp. If they happen to have
the same value, you are ok.

I think there are a few other gotchas with regards to compaction.
Each timestamp only counts as 1 version, thus you may end up with more
data than you intended depending on how many duplicates you have.

On Tue, Nov 24, 2009 at 2:35 PM, Zhenyu Zhong <zh...@gmail.com> wrote:
> Hi,
>
> I would like to use nice feature of HBase --versions to store a timeseries
> data for a rowkey.
> However, I get duplicates for the same rowkey and the same timestamp if I
> use Put and run mapreduce job multiple times.
>
> For example.
> Put put = new Put(rowkey.getBytes());
> put.add("f1:c1".getBytes(), ts, value.getBytes());
>
> I use TableOutputFormat as the output of the MapReduce job.
> If I run the MapReduce job twice, I would get 2 records with the same rowkey
> and same timestamp.
>
> May I ask whether the Put just adds a row no matter that there is already a
> row with the same key and timestamp in the table?
>
> Best,
> zhenyu
>

Re: about HBase versions

[kennyut <ke...@gmail.com>]

I am also working with a very similar project. 

I tried two version for time-series data:

1. use "id-timestamp" as rowkey:
    It's good for loading new inputs and also good for search. But bad for
the storage and slow when doing aggregation. 

2. use "id" as rowkey and use array string for time and value pairs:
    for example:
    id1,  cf1:time (t1,t2,t3,...,tn),  cf1:value (v1,v2,v3,...,vn)

    It's save lots of storage room and faster/easier when doing aggregation.
But still not sure about data loading/updating. (my data updates every 15
minutes) 

The method you mentioned which puts all time and values in the same cell, if
they have the same rowkey. But does hbase keeps unlimited versions? 

Just my $0.02. Thanks


Instead of putting values in the same cell for the same key. I 



--
View this message in context: http://apache-hbase.679495.n3.nabble.com/about-HBase-versions-tp4024747p4067939.html
Sent from the HBase User mailing list archive at Nabble.com.