You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Jian Lu <jl...@local.com> on 2010/11/01 23:25:02 UTC
HBase Secondary Index
Hi All,
I'm learning how to add a secondary index to my HTable by following the instructions from: http://rajeev1982.blogspot.com/2009/06/secondary-indexes-in-hbase.html
Could someone please tell me where I can download the HBase contrib package for the hbase-0.20.0-transactional.jar?
Also, could someone please give me some advices on how good the transactional package work with Hbase 0.89?
Thank you!
Jack.
RE: HBase Secondary Index
Posted by Jian Lu <jl...@local.com>.
Thanks Steve, Mike! I will try using two tables and manage my own indexing on the second table.
Jack.
-----Original Message-----
From: Steven Noels [mailto:stevenn@outerthought.org]
Sent: Tuesday, November 02, 2010 9:09 AM
To: user
Subject: Re: HBase Secondary Index
On Tue, Nov 2, 2010 at 12:36 AM, Michael Segel <mi...@hotmail.com>wrote:
>
> Jack...
>
> When we looked at the secondary index, it started off ok, but we started
> having issues.
> There were some serious problems that needed to be addressed and the
> contributor wasn't keeping things up to date. (Such is life)
>
> So while the contrib/transactional didn't work for us, there are other
> alternatives for creating indexes.
> (Here's the cue for someone in the Lily camp to step up.)
>
Feeding your secondary index:
http://www.lilyproject.org/lily/about/playground/hbaserowlog.html
Managing your index:
http://www.lilyproject.org/lily/about/playground/hbaseindexes.html
Both are components we use in Lily, more specifically to maintain SOLR
indexes and back/forward link tables.
Cheers,
Steven.
--
Steven Noels
http://outerthought.org/
Open Source Content Applications
Makers of Kauri, Daisy CMS and Lily
Re: HBase Secondary Index
Posted by Steven Noels <st...@outerthought.org>.
On Tue, Nov 2, 2010 at 12:36 AM, Michael Segel <mi...@hotmail.com>wrote:
>
> Jack...
>
> When we looked at the secondary index, it started off ok, but we started
> having issues.
> There were some serious problems that needed to be addressed and the
> contributor wasn't keeping things up to date. (Such is life)
>
> So while the contrib/transactional didn't work for us, there are other
> alternatives for creating indexes.
> (Here's the cue for someone in the Lily camp to step up.)
>
Feeding your secondary index:
http://www.lilyproject.org/lily/about/playground/hbaserowlog.html
Managing your index:
http://www.lilyproject.org/lily/about/playground/hbaseindexes.html
Both are components we use in Lily, more specifically to maintain SOLR
indexes and back/forward link tables.
Cheers,
Steven.
--
Steven Noels
http://outerthought.org/
Open Source Content Applications
Makers of Kauri, Daisy CMS and Lily
RE: HBase Secondary Index
Posted by Michael Segel <mi...@hotmail.com>.
Jack,
If your tables are narrow, then this will work well. But if you're storing a lot of data per row (wide rows), then you may want to store the data in one table and then in the second table contains a reference to the first table's row key. An example is if you're holding an image or 'blob' in HBase.
> From: jlu@local.com
> To: user@hbase.apache.org
> Date: Mon, 1 Nov 2010 17:04:40 -0700
> Subject: RE: HBase Secondary Index
>
> Thanks again Mike!
>
> Two tables to store the same data and the only difference is just the row key? Table 1 uses milliseconds timestamp as row key, Table 2 uses column values (unique across time series) as row key? So, when searching by timestamp, I use table 1. When searching by column values, I go to table 2? Is this correct?
>
> I will have about 140 million new records / day.
>
>
> Please advise! Thanks!
>
> Jack.
>
>
>
> -----Original Message-----
> From: Michael Segel [mailto:michael_segel@hotmail.com]
> Sent: Monday, November 01, 2010 4:36 PM
> To: user@hbase.apache.org
> Subject: RE: HBase Secondary Index
>
>
> Jack...
>
> When we looked at the secondary index, it started off ok, but we started having issues.
> There were some serious problems that needed to be addressed and the contributor wasn't keeping things up to date. (Such is life)
>
> So while the contrib/transactional didn't work for us, there are other alternatives for creating indexes.
> (Here's the cue for someone in the Lily camp to step up.)
>
> You can also roll your own too.
>
> If the data you want to capture in the time series, you may want to just write the record to two different tables...
>
> HTH
>
> -Mike
>
> > From: jlu@local.com
> > To: user@hbase.apache.org
> > Date: Mon, 1 Nov 2010 15:43:22 -0700
> > Subject: RE: HBase Secondary Index
> >
> > Hi Michael, thanks a lot!
> >
> > Is adding a secondary index to HTable a bad idea given that my data will never be updated once added to HTable?
> >
> > I am trying to find a solution from the previous post for "Time series-schema". I need to query by 1) time stamp, 2) and by column values across different time stamps.
> >
> > Thanks for the advices!
> >
> > Jack.
> >
> >
> >
> > -----Original Message-----
> > From: Michael Segel [mailto:michael_segel@hotmail.com]
> > Sent: Monday, November 01, 2010 3:36 PM
> > To: user@hbase.apache.org
> > Subject: RE: HBase Secondary Index
> >
> >
> > Jack,
> >
> > Its not.
> > Long story short, the transactional jar is out in git hub.
> > As to 0.89 support, wasn't there the last time I checked.
> >
> >
> >
> > > From: jlu@local.com
> > > To: user@hbase.apache.org
> > > Date: Mon, 1 Nov 2010 15:25:02 -0700
> > > Subject: HBase Secondary Index
> > >
> > > Hi All,
> > >
> > > I'm learning how to add a secondary index to my HTable by following the instructions from: http://rajeev1982.blogspot.com/2009/06/secondary-indexes-in-hbase.html
> > >
> > > Could someone please tell me where I can download the HBase contrib package for the hbase-0.20.0-transactional.jar?
> > >
> > > Also, could someone please give me some advices on how good the transactional package work with Hbase 0.89?
> > >
> > >
> > > Thank you!
> > > Jack.
> >
>
RE: HBase Secondary Index
Posted by Jian Lu <jl...@local.com>.
Thanks again Mike!
Two tables to store the same data and the only difference is just the row key? Table 1 uses milliseconds timestamp as row key, Table 2 uses column values (unique across time series) as row key? So, when searching by timestamp, I use table 1. When searching by column values, I go to table 2? Is this correct?
I will have about 140 million new records / day.
Please advise! Thanks!
Jack.
-----Original Message-----
From: Michael Segel [mailto:michael_segel@hotmail.com]
Sent: Monday, November 01, 2010 4:36 PM
To: user@hbase.apache.org
Subject: RE: HBase Secondary Index
Jack...
When we looked at the secondary index, it started off ok, but we started having issues.
There were some serious problems that needed to be addressed and the contributor wasn't keeping things up to date. (Such is life)
So while the contrib/transactional didn't work for us, there are other alternatives for creating indexes.
(Here's the cue for someone in the Lily camp to step up.)
You can also roll your own too.
If the data you want to capture in the time series, you may want to just write the record to two different tables...
HTH
-Mike
> From: jlu@local.com
> To: user@hbase.apache.org
> Date: Mon, 1 Nov 2010 15:43:22 -0700
> Subject: RE: HBase Secondary Index
>
> Hi Michael, thanks a lot!
>
> Is adding a secondary index to HTable a bad idea given that my data will never be updated once added to HTable?
>
> I am trying to find a solution from the previous post for "Time series-schema". I need to query by 1) time stamp, 2) and by column values across different time stamps.
>
> Thanks for the advices!
>
> Jack.
>
>
>
> -----Original Message-----
> From: Michael Segel [mailto:michael_segel@hotmail.com]
> Sent: Monday, November 01, 2010 3:36 PM
> To: user@hbase.apache.org
> Subject: RE: HBase Secondary Index
>
>
> Jack,
>
> Its not.
> Long story short, the transactional jar is out in git hub.
> As to 0.89 support, wasn't there the last time I checked.
>
>
>
> > From: jlu@local.com
> > To: user@hbase.apache.org
> > Date: Mon, 1 Nov 2010 15:25:02 -0700
> > Subject: HBase Secondary Index
> >
> > Hi All,
> >
> > I'm learning how to add a secondary index to my HTable by following the instructions from: http://rajeev1982.blogspot.com/2009/06/secondary-indexes-in-hbase.html
> >
> > Could someone please tell me where I can download the HBase contrib package for the hbase-0.20.0-transactional.jar?
> >
> > Also, could someone please give me some advices on how good the transactional package work with Hbase 0.89?
> >
> >
> > Thank you!
> > Jack.
>
RE: HBase Secondary Index
Posted by Michael Segel <mi...@hotmail.com>.
Jack...
When we looked at the secondary index, it started off ok, but we started having issues.
There were some serious problems that needed to be addressed and the contributor wasn't keeping things up to date. (Such is life)
So while the contrib/transactional didn't work for us, there are other alternatives for creating indexes.
(Here's the cue for someone in the Lily camp to step up.)
You can also roll your own too.
If the data you want to capture in the time series, you may want to just write the record to two different tables...
HTH
-Mike
> From: jlu@local.com
> To: user@hbase.apache.org
> Date: Mon, 1 Nov 2010 15:43:22 -0700
> Subject: RE: HBase Secondary Index
>
> Hi Michael, thanks a lot!
>
> Is adding a secondary index to HTable a bad idea given that my data will never be updated once added to HTable?
>
> I am trying to find a solution from the previous post for "Time series-schema". I need to query by 1) time stamp, 2) and by column values across different time stamps.
>
> Thanks for the advices!
>
> Jack.
>
>
>
> -----Original Message-----
> From: Michael Segel [mailto:michael_segel@hotmail.com]
> Sent: Monday, November 01, 2010 3:36 PM
> To: user@hbase.apache.org
> Subject: RE: HBase Secondary Index
>
>
> Jack,
>
> Its not.
> Long story short, the transactional jar is out in git hub.
> As to 0.89 support, wasn't there the last time I checked.
>
>
>
> > From: jlu@local.com
> > To: user@hbase.apache.org
> > Date: Mon, 1 Nov 2010 15:25:02 -0700
> > Subject: HBase Secondary Index
> >
> > Hi All,
> >
> > I'm learning how to add a secondary index to my HTable by following the instructions from: http://rajeev1982.blogspot.com/2009/06/secondary-indexes-in-hbase.html
> >
> > Could someone please tell me where I can download the HBase contrib package for the hbase-0.20.0-transactional.jar?
> >
> > Also, could someone please give me some advices on how good the transactional package work with Hbase 0.89?
> >
> >
> > Thank you!
> > Jack.
>
RE: HBase Secondary Index
Posted by Jian Lu <jl...@local.com>.
Hi Michael, thanks a lot!
Is adding a secondary index to HTable a bad idea given that my data will never be updated once added to HTable?
I am trying to find a solution from the previous post for "Time series-schema". I need to query by 1) time stamp, 2) and by column values across different time stamps.
Thanks for the advices!
Jack.
-----Original Message-----
From: Michael Segel [mailto:michael_segel@hotmail.com]
Sent: Monday, November 01, 2010 3:36 PM
To: user@hbase.apache.org
Subject: RE: HBase Secondary Index
Jack,
Its not.
Long story short, the transactional jar is out in git hub.
As to 0.89 support, wasn't there the last time I checked.
> From: jlu@local.com
> To: user@hbase.apache.org
> Date: Mon, 1 Nov 2010 15:25:02 -0700
> Subject: HBase Secondary Index
>
> Hi All,
>
> I'm learning how to add a secondary index to my HTable by following the instructions from: http://rajeev1982.blogspot.com/2009/06/secondary-indexes-in-hbase.html
>
> Could someone please tell me where I can download the HBase contrib package for the hbase-0.20.0-transactional.jar?
>
> Also, could someone please give me some advices on how good the transactional package work with Hbase 0.89?
>
>
> Thank you!
> Jack.
RE: HBase Secondary Index
Posted by Michael Segel <mi...@hotmail.com>.
Jack,
Its not.
Long story short, the transactional jar is out in git hub.
As to 0.89 support, wasn't there the last time I checked.
> From: jlu@local.com
> To: user@hbase.apache.org
> Date: Mon, 1 Nov 2010 15:25:02 -0700
> Subject: HBase Secondary Index
>
> Hi All,
>
> I'm learning how to add a secondary index to my HTable by following the instructions from: http://rajeev1982.blogspot.com/2009/06/secondary-indexes-in-hbase.html
>
> Could someone please tell me where I can download the HBase contrib package for the hbase-0.20.0-transactional.jar?
>
> Also, could someone please give me some advices on how good the transactional package work with Hbase 0.89?
>
>
> Thank you!
> Jack.