You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Ravikumar MAV <ra...@gmail.com> on 2010/06/18 23:49:58 UTC
Suggestion on HBQL
Hello All,
I am very new to Hbase world and had the following questions in mind. Could
anyone take some time to answer them when one has some free time?
1) Is HBase meant for real time access through website (At least just for
reads)? Can real time performance be achieved (when compared to MySQL
cluster) with reasonable clustor configuration and for 100M to 200M
records?. This may be unfair comparison as its BigTable vs Relational, but
just want to get the opinion.
2) Is HBQL better way to start using in DAO layer or HBASE APIs?. HBQL is
in alpha. Is anybody using in prod?
3) Is there Connection pooling mechanism with Hbase? I see HBQL has has some
JDBC and Connection pool support. When we say connection pool, is it at the
Table level? or Client JVM level or For the whole HBase cluster?
- Ravi
Re: Suggestion on HBQL
Posted by Sarah Sproehnle <sa...@cloudera.com>.
HI Ravi,
On Fri, Jun 18, 2010 at 2:49 PM, Ravikumar MAV <ra...@gmail.com> wrote:
> Hello All,
>
> I am very new to Hbase world and had the following questions in mind. Could
> anyone take some time to answer them when one has some free time?
>
> 1) Is HBase meant for real time access through website (At least just for
> reads)? Can real time performance be achieved (when compared to MySQL
> cluster) with reasonable clustor configuration and for 100M to 200M
> records?. This may be unfair comparison as its BigTable vs Relational, but
> just want to get the opinion.
HBase and MySQL Cluster will achieve comparable throughput of single
row reads/writes with modest amounts of data (say up to 100GB). On
larger data sets, MySQL Cluster won't scale very well because it
requires that all indexes fit in ram across the cluster.
I'll have to let someone else address your other questions.
>
> 2) Is HBQL better way to start using in DAO layer or HBASE APIs?. HBQL is
> in alpha. Is anybody using in prod?
>
> 3) Is there Connection pooling mechanism with Hbase? I see HBQL has has some
> JDBC and Connection pool support. When we say connection pool, is it at the
> Table level? or Client JVM level or For the whole HBase cluster?
>
> - Ravi
>
--
Sarah Sproehnle
Educational Services
Cloudera, Inc
http://www.cloudera.com/training
Re: Suggestion on HBQL
Posted by Alex Baranau <al...@gmail.com>.
You can look at some info on "ways of integrating with HBase (instantiating
HTable) on client-side" in this rather old post:
http://blog.sematext.com/2010/01/27/hbase-digest-january-2010/
Alex Baranau
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop - HBase
Hadoop ecosystem search :: http://search-hadoop.com/
On Mon, Jun 28, 2010 at 9:13 PM, Ravikumar MAV <ra...@gmail.com> wrote:
> Thank you Alex. Your answer is very much helpful.
>
> Hello All,
>
> Anybody using HBQL. It will be very helpful if some one posts
> feedback/experience with it.
> - Can HBQL be used for production sites
> - Is it a overhead? and how is performance
> - Is there a Spring template for Hbase or hbql?
>
> Ravi
>
> On Sun, Jun 27, 2010 at 7:54 AM, Alex Baranau <alex.baranov.v@gmail.com
> >wrote:
>
> > > 3) Is there Connection pooling mechanism with HBase?
> >
> > It's a best practise to re-use HTable instance on client (one per
> thread).
> > Typically it becomes so that you have one instance per JVM (e.g. when you
> > init and then re-use HTable instance in Map or Reduce job). You'll
> probably
> > benefit from running multiple threads reading from HBase (depend on the
> > use-case) and thus should use one per thread. From the Javadoc:
> >
> > " public class HTable extends Object
> > Used to communicate with a single HBase table. This class is not thread
> > safe. Use one instance per thread. Puts, deletes, checkAndPut and
> > incrementColumnValue are done in an exclusive (and thus serial) fashion
> for
> > each row. These calls acquire a row lock which is shared with the lockRow
> > calls. Gets and Scans will not return half written data. That is, all
> > mutation operations are atomic on a row basis with respect to other
> > concurrent readers and writers. "
> >
> > You might also want to look at HTablePool class:
> >
> > " public class HTablePool extends Object
> > A simple pool of HTable instances.
> > Each HTablePool acts as a pool for all tables. To use, instantiate an
> > HTablePool and use getTable(String) to get an HTable from the pool. Once
> > you
> > are done with it, return it to the pool with putTable(HTable).
> > A pool can be created with a maxSize which defines the most HTable
> > references that will ever be retained for each table. Otherwise the
> default
> > is Integer.MAX_VALUE."
> >
> > Alex Baranau
> > ----
> > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> > Hadoop ecosystem search :: http://search-hadoop.com/
> > HBase Digest ::
> http://blog.sematext.com/2010/06/01/hbase-digest-may-2010/
> >
> > On Sat, Jun 19, 2010 at 12:49 AM, Ravikumar MAV <ra...@gmail.com>
> wrote:
> >
> > > Hello All,
> > >
> > > I am very new to Hbase world and had the following questions in mind.
> > Could
> > > anyone take some time to answer them when one has some free time?
> > >
> > > 1) Is HBase meant for real time access through website (At least just
> for
> > > reads)? Can real time performance be achieved (when compared to MySQL
> > > cluster) with reasonable clustor configuration and for 100M to 200M
> > > records?. This may be unfair comparison as its BigTable vs Relational,
> > but
> > > just want to get the opinion.
> > >
> > > 2) Is HBQL better way to start using in DAO layer or HBASE APIs?.
> HBQL
> > is
> > > in alpha. Is anybody using in prod?
> > >
> > > 3) Is there Connection pooling mechanism with Hbase? I see HBQL has has
> > > some
> > > JDBC and Connection pool support. When we say connection pool, is it at
> > the
> > > Table level? or Client JVM level or For the whole HBase cluster?
> > >
> > > - Ravi
> > >
> >
>
>
>
> --
> - Ravi MAV
>
Re: Suggestion on HBQL
Posted by Ravikumar MAV <ra...@gmail.com>.
Thank you Alex. Your answer is very much helpful.
Hello All,
Anybody using HBQL. It will be very helpful if some one posts
feedback/experience with it.
- Can HBQL be used for production sites
- Is it a overhead? and how is performance
- Is there a Spring template for Hbase or hbql?
Ravi
On Sun, Jun 27, 2010 at 7:54 AM, Alex Baranau <al...@gmail.com>wrote:
> > 3) Is there Connection pooling mechanism with HBase?
>
> It's a best practise to re-use HTable instance on client (one per thread).
> Typically it becomes so that you have one instance per JVM (e.g. when you
> init and then re-use HTable instance in Map or Reduce job). You'll probably
> benefit from running multiple threads reading from HBase (depend on the
> use-case) and thus should use one per thread. From the Javadoc:
>
> " public class HTable extends Object
> Used to communicate with a single HBase table. This class is not thread
> safe. Use one instance per thread. Puts, deletes, checkAndPut and
> incrementColumnValue are done in an exclusive (and thus serial) fashion for
> each row. These calls acquire a row lock which is shared with the lockRow
> calls. Gets and Scans will not return half written data. That is, all
> mutation operations are atomic on a row basis with respect to other
> concurrent readers and writers. "
>
> You might also want to look at HTablePool class:
>
> " public class HTablePool extends Object
> A simple pool of HTable instances.
> Each HTablePool acts as a pool for all tables. To use, instantiate an
> HTablePool and use getTable(String) to get an HTable from the pool. Once
> you
> are done with it, return it to the pool with putTable(HTable).
> A pool can be created with a maxSize which defines the most HTable
> references that will ever be retained for each table. Otherwise the default
> is Integer.MAX_VALUE."
>
> Alex Baranau
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> Hadoop ecosystem search :: http://search-hadoop.com/
> HBase Digest :: http://blog.sematext.com/2010/06/01/hbase-digest-may-2010/
>
> On Sat, Jun 19, 2010 at 12:49 AM, Ravikumar MAV <ra...@gmail.com> wrote:
>
> > Hello All,
> >
> > I am very new to Hbase world and had the following questions in mind.
> Could
> > anyone take some time to answer them when one has some free time?
> >
> > 1) Is HBase meant for real time access through website (At least just for
> > reads)? Can real time performance be achieved (when compared to MySQL
> > cluster) with reasonable clustor configuration and for 100M to 200M
> > records?. This may be unfair comparison as its BigTable vs Relational,
> but
> > just want to get the opinion.
> >
> > 2) Is HBQL better way to start using in DAO layer or HBASE APIs?. HBQL
> is
> > in alpha. Is anybody using in prod?
> >
> > 3) Is there Connection pooling mechanism with Hbase? I see HBQL has has
> > some
> > JDBC and Connection pool support. When we say connection pool, is it at
> the
> > Table level? or Client JVM level or For the whole HBase cluster?
> >
> > - Ravi
> >
>
--
- Ravi MAV
Re: Suggestion on HBQL
Posted by Alex Baranau <al...@gmail.com>.
> 3) Is there Connection pooling mechanism with HBase?
It's a best practise to re-use HTable instance on client (one per thread).
Typically it becomes so that you have one instance per JVM (e.g. when you
init and then re-use HTable instance in Map or Reduce job). You'll probably
benefit from running multiple threads reading from HBase (depend on the
use-case) and thus should use one per thread. From the Javadoc:
" public class HTable extends Object
Used to communicate with a single HBase table. This class is not thread
safe. Use one instance per thread. Puts, deletes, checkAndPut and
incrementColumnValue are done in an exclusive (and thus serial) fashion for
each row. These calls acquire a row lock which is shared with the lockRow
calls. Gets and Scans will not return half written data. That is, all
mutation operations are atomic on a row basis with respect to other
concurrent readers and writers. "
You might also want to look at HTablePool class:
" public class HTablePool extends Object
A simple pool of HTable instances.
Each HTablePool acts as a pool for all tables. To use, instantiate an
HTablePool and use getTable(String) to get an HTable from the pool. Once you
are done with it, return it to the pool with putTable(HTable).
A pool can be created with a maxSize which defines the most HTable
references that will ever be retained for each table. Otherwise the default
is Integer.MAX_VALUE."
Alex Baranau
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Hadoop ecosystem search :: http://search-hadoop.com/
HBase Digest :: http://blog.sematext.com/2010/06/01/hbase-digest-may-2010/
On Sat, Jun 19, 2010 at 12:49 AM, Ravikumar MAV <ra...@gmail.com> wrote:
> Hello All,
>
> I am very new to Hbase world and had the following questions in mind. Could
> anyone take some time to answer them when one has some free time?
>
> 1) Is HBase meant for real time access through website (At least just for
> reads)? Can real time performance be achieved (when compared to MySQL
> cluster) with reasonable clustor configuration and for 100M to 200M
> records?. This may be unfair comparison as its BigTable vs Relational, but
> just want to get the opinion.
>
> 2) Is HBQL better way to start using in DAO layer or HBASE APIs?. HBQL is
> in alpha. Is anybody using in prod?
>
> 3) Is there Connection pooling mechanism with Hbase? I see HBQL has has
> some
> JDBC and Connection pool support. When we say connection pool, is it at the
> Table level? or Client JVM level or For the whole HBase cluster?
>
> - Ravi
>