You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by ilay raja <il...@gmail.com> on 2010/07/11 06:32:04 UTC
Hbase search
HI..
Am experimenting with HBase for a search application. Have done bulk import
of input data into hbase table using mapreduce..
Input format: <key> <value>
I used SampleUploader.java for populating the table. (from examples)
key is stored as row-key and value as column.
Would like to know how to perform search on this table for a specific column
value...
I do not find any mapreduce examples that does search on the table. Please
let me know any pointers or how do we do this??
Should I use HBase client APIs to perform search on the table... from
DemoClient.java
Regards,
IlayaRaja
--
-----------------------------
Product Engineer,
Rediff, Mumbai.
+91 97691 67921
------------------------------
Re: Hbase search
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Maybe you are looking for something like this:
http://www.javarants.com/2010/06/30/havrobase-a-searchable-evolvable-entity-store-on-top-of-hbase-and-solr/
Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/
----- Original Message ----
> From: Imran M Yousuf <im...@gmail.com>
> To: user@hbase.apache.org
> Sent: Sun, July 11, 2010 1:08:01 AM
> Subject: Re: Hbase search
>
> Hi Ilay,
>
> Not sure whether this would be helpful but my personal opinion is to
> use tool that suites best for my requirement. IMHO HBase's strength is
> in duplicability, scalability and data structure by design and
> implementation, whereas for search my personal choice is Solr/Lucene
> (w/o Katta) and some excellent people are trying to combine their
> strengths, e.g., http://github.com/akkumar/hbasene
>
> Another option that could be relevant to the cause is
> http://lilycms.org/lily/index.html; a CMS using Lucene for search and
> HBase to store contents, basically searchable contents!
>
> Hope that helps.
>
> Thank you,
>
> Imran
>
> On Sun, Jul 11, 2010 at 10:50 AM, Todd Lipcon <to...@cloudera.com> wrote:
> > Hi Ilay,
> >
> > HBase doesn't have built-in "search " functionality. Could you explain a
> > little more what you're trying to achieve? It may be that our "Filter"
> > system can do what you're looking for. Also, what latency do you expect for
> > these queries?
> >
> > Probably the best bet is to change the MapReduce job to also create an
index
> > table at the same time as it's importing the data. Then a very efficient
> > get/scan can find the data for you.
> >
> > Thanks
> > -Todd
> >
> > P.S. please use the user@ list for this, and don't cross-post to dev@. dev
> > should be for discussion of HBase development itself, not development on
>top
> > of hbase.
> >
> > On Sat, Jul 10, 2010 at 9:32 PM, ilay raja <il...@gmail.com> wrote:
> >
> >> HI..
> >>
> >> Am experimenting with HBase for a search application. Have done bulk
import
> >> of input data into hbase table using mapreduce..
> >>
> >> Input format: <key> <value>
> >> I used SampleUploader.java for populating the table. (from examples)
> >> key is stored as row-key and value as column.
> >>
> >> Would like to know how to perform search on this table for a specific
> >> column
> >> value...
> >> I do not find any mapreduce examples that does search on the table.
Please
> >> let me know any pointers or how do we do this??
> >>
> >> Should I use HBase client APIs to perform search on the table... from
> >> DemoClient.java
> >>
> >> Regards,
> >> IlayaRaja
> >>
> >>
> >> --
> >> -----------------------------
> >> Product Engineer,
> >> Rediff, Mumbai.
> >>
> >> +91 97691 67921
> >> ------------------------------
> >>
> >
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>
>
>
> --
> Imran M Yousuf
> Entrepreneur & Software Engineer
> Smart IT Engineering
> Dhaka, Bangladesh
> Email: imran@smartitengineering.com
> Blog: http://imyousuf-tech.blogs.smartitengineering.com/
> Mobile: +880-1711402557
>
Re: Hbase search
Posted by Imran M Yousuf <im...@gmail.com>.
Hi Ilay,
Not sure whether this would be helpful but my personal opinion is to
use tool that suites best for my requirement. IMHO HBase's strength is
in duplicability, scalability and data structure by design and
implementation, whereas for search my personal choice is Solr/Lucene
(w/o Katta) and some excellent people are trying to combine their
strengths, e.g., http://github.com/akkumar/hbasene
Another option that could be relevant to the cause is
http://lilycms.org/lily/index.html; a CMS using Lucene for search and
HBase to store contents, basically searchable contents!
Hope that helps.
Thank you,
Imran
On Sun, Jul 11, 2010 at 10:50 AM, Todd Lipcon <to...@cloudera.com> wrote:
> Hi Ilay,
>
> HBase doesn't have built-in "search " functionality. Could you explain a
> little more what you're trying to achieve? It may be that our "Filter"
> system can do what you're looking for. Also, what latency do you expect for
> these queries?
>
> Probably the best bet is to change the MapReduce job to also create an index
> table at the same time as it's importing the data. Then a very efficient
> get/scan can find the data for you.
>
> Thanks
> -Todd
>
> P.S. please use the user@ list for this, and don't cross-post to dev@. dev
> should be for discussion of HBase development itself, not development on top
> of hbase.
>
> On Sat, Jul 10, 2010 at 9:32 PM, ilay raja <il...@gmail.com> wrote:
>
>> HI..
>>
>> Am experimenting with HBase for a search application. Have done bulk import
>> of input data into hbase table using mapreduce..
>>
>> Input format: <key> <value>
>> I used SampleUploader.java for populating the table. (from examples)
>> key is stored as row-key and value as column.
>>
>> Would like to know how to perform search on this table for a specific
>> column
>> value...
>> I do not find any mapreduce examples that does search on the table. Please
>> let me know any pointers or how do we do this??
>>
>> Should I use HBase client APIs to perform search on the table... from
>> DemoClient.java
>>
>> Regards,
>> IlayaRaja
>>
>>
>> --
>> -----------------------------
>> Product Engineer,
>> Rediff, Mumbai.
>>
>> +91 97691 67921
>> ------------------------------
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
--
Imran M Yousuf
Entrepreneur & Software Engineer
Smart IT Engineering
Dhaka, Bangladesh
Email: imran@smartitengineering.com
Blog: http://imyousuf-tech.blogs.smartitengineering.com/
Mobile: +880-1711402557
RE: Hbase search
Posted by Michael Segel <mi...@hotmail.com>.
ilay,
What do you plan on doing with the search.
Sure you can use a filtered scan to feed a m/r job, but what do you want to do with the data it returns?
> Date: Sun, 11 Jul 2010 10:35:01 +0530
> Subject: Re: Hbase search
> From: ilay.msp@gmail.com
> To: user@hbase.apache.org
>
> Hi Todd,
>
> Say, for example, I want HBase to be scanned and return all the rows that
> matches my input value (query) on a particular column. Assume that the
> column stores "string" and the input query is also a string.
>
> I would want to perform this search in a mapreduce program, where the map
> functions will parallely query the same hbase table.
>
> Thanks,
> Ilay
>
> On Sun, Jul 11, 2010 at 10:20 AM, Todd Lipcon <to...@cloudera.com> wrote:
>
> > Hi Ilay,
> >
> > HBase doesn't have built-in "search " functionality. Could you explain a
> > little more what you're trying to achieve? It may be that our "Filter"
> > system can do what you're looking for. Also, what latency do you expect for
> > these queries?
> >
> > Probably the best bet is to change the MapReduce job to also create an
> > index
> > table at the same time as it's importing the data. Then a very efficient
> > get/scan can find the data for you.
> >
> > Thanks
> > -Todd
> >
> > P.S. please use the user@ list for this, and don't cross-post to dev@. dev
> > should be for discussion of HBase development itself, not development on
> > top
> > of hbase.
> >
> > On Sat, Jul 10, 2010 at 9:32 PM, ilay raja <il...@gmail.com> wrote:
> >
> > > HI..
> > >
> > > Am experimenting with HBase for a search application. Have done bulk
> > import
> > > of input data into hbase table using mapreduce..
> > >
> > > Input format: <key> <value>
> > > I used SampleUploader.java for populating the table. (from examples)
> > > key is stored as row-key and value as column.
> > >
> > > Would like to know how to perform search on this table for a specific
> > > column
> > > value...
> > > I do not find any mapreduce examples that does search on the table.
> > Please
> > > let me know any pointers or how do we do this??
> > >
> > > Should I use HBase client APIs to perform search on the table... from
> > > DemoClient.java
> > >
> > > Regards,
> > > IlayaRaja
> > >
> > >
> > > --
> > > -----------------------------
> > > Product Engineer,
> > > Rediff, Mumbai.
> > >
> > > +91 97691 67921
> > > ------------------------------
> > >
> >
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>
>
>
> --
> -----------------------------
> Product Engineer,
> Rediff, Mumbai.
>
> +91 97691 67921
> ------------------------------
_________________________________________________________________
The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail.
http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4
Re: Hbase search
Posted by ilay raja <il...@gmail.com>.
Hi Todd,
Say, for example, I want HBase to be scanned and return all the rows that
matches my input value (query) on a particular column. Assume that the
column stores "string" and the input query is also a string.
I would want to perform this search in a mapreduce program, where the map
functions will parallely query the same hbase table.
Thanks,
Ilay
On Sun, Jul 11, 2010 at 10:20 AM, Todd Lipcon <to...@cloudera.com> wrote:
> Hi Ilay,
>
> HBase doesn't have built-in "search " functionality. Could you explain a
> little more what you're trying to achieve? It may be that our "Filter"
> system can do what you're looking for. Also, what latency do you expect for
> these queries?
>
> Probably the best bet is to change the MapReduce job to also create an
> index
> table at the same time as it's importing the data. Then a very efficient
> get/scan can find the data for you.
>
> Thanks
> -Todd
>
> P.S. please use the user@ list for this, and don't cross-post to dev@. dev
> should be for discussion of HBase development itself, not development on
> top
> of hbase.
>
> On Sat, Jul 10, 2010 at 9:32 PM, ilay raja <il...@gmail.com> wrote:
>
> > HI..
> >
> > Am experimenting with HBase for a search application. Have done bulk
> import
> > of input data into hbase table using mapreduce..
> >
> > Input format: <key> <value>
> > I used SampleUploader.java for populating the table. (from examples)
> > key is stored as row-key and value as column.
> >
> > Would like to know how to perform search on this table for a specific
> > column
> > value...
> > I do not find any mapreduce examples that does search on the table.
> Please
> > let me know any pointers or how do we do this??
> >
> > Should I use HBase client APIs to perform search on the table... from
> > DemoClient.java
> >
> > Regards,
> > IlayaRaja
> >
> >
> > --
> > -----------------------------
> > Product Engineer,
> > Rediff, Mumbai.
> >
> > +91 97691 67921
> > ------------------------------
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
--
-----------------------------
Product Engineer,
Rediff, Mumbai.
+91 97691 67921
------------------------------
Re: Hbase search
Posted by Todd Lipcon <to...@cloudera.com>.
Hi Ilay,
HBase doesn't have built-in "search " functionality. Could you explain a
little more what you're trying to achieve? It may be that our "Filter"
system can do what you're looking for. Also, what latency do you expect for
these queries?
Probably the best bet is to change the MapReduce job to also create an index
table at the same time as it's importing the data. Then a very efficient
get/scan can find the data for you.
Thanks
-Todd
P.S. please use the user@ list for this, and don't cross-post to dev@. dev
should be for discussion of HBase development itself, not development on top
of hbase.
On Sat, Jul 10, 2010 at 9:32 PM, ilay raja <il...@gmail.com> wrote:
> HI..
>
> Am experimenting with HBase for a search application. Have done bulk import
> of input data into hbase table using mapreduce..
>
> Input format: <key> <value>
> I used SampleUploader.java for populating the table. (from examples)
> key is stored as row-key and value as column.
>
> Would like to know how to perform search on this table for a specific
> column
> value...
> I do not find any mapreduce examples that does search on the table. Please
> let me know any pointers or how do we do this??
>
> Should I use HBase client APIs to perform search on the table... from
> DemoClient.java
>
> Regards,
> IlayaRaja
>
>
> --
> -----------------------------
> Product Engineer,
> Rediff, Mumbai.
>
> +91 97691 67921
> ------------------------------
>
--
Todd Lipcon
Software Engineer, Cloudera