You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Arun Allamsetty <ar...@gmail.com> on 2014/07/08 00:55:29 UTC

Using HBase in standalone mode in production

Hi all,

So this question might be stupid, retarded even, but it has been bugging me
for a while and I cannot think of a better place to ask this. I am really
impressed with the way HBase works (as a key-value store). Since it stores
everything as a byte array, I find it really convenient to store serialized
objects. Also, I understand that HBase is supposed to be used when you have
too much data to be handled by a single machine, so we can scale our
application by running it in distributed mode.

But what if I want to use it because its HashMap kind of capabilities with
an added feature to track versions. Is it recommended that I use it for a
small application (in standalone mode) with maybe 100K users and storage
needs which probably won't exceed 100G.

I know it is never recommended to be used as a transactional database (I
have read that in a million places) but I would like to know more about it.

Thanks,
Arun

Re: Using HBase in standalone mode in production

Posted by lars hofhansl <la...@apache.org>.
Chiming in late here. You might be better served with a database that was designed to run on a single host.
HBase will run in standalone mode for testing purposes, but then it uses a local wrapper for the filesystem, which does not (I think) support the sync API, and hence HBase has no way to guarantee changes on disk and machine outage can corrupt your data (I faintly remember some discussion about this, so this might have changed).
You could run a single node HDFS and a single node HBase on the same machine, but then there are simpler databases to run.

MySQL was mentioned here. PostgreSQL has good blob and even key-value support. Might want to to try Redis as well; and there are many more single node type key value stores.
Many of the design choices for HBase were made with scalability in mind.


-- Lars



________________________________
 From: Arun Allamsetty <ar...@gmail.com>
To: user@hbase.apache.org 
Sent: Tuesday, July 8, 2014 12:55 AM
Subject: Using HBase in standalone mode in production
 

Hi all,

So this question might be stupid, retarded even, but it has been bugging me
for a while and I cannot think of a better place to ask this. I am really
impressed with the way HBase works (as a key-value store). Since it stores
everything as a byte array, I find it really convenient to store serialized
objects. Also, I understand that HBase is supposed to be used when you have
too much data to be handled by a single machine, so we can scale our
application by running it in distributed mode.

But what if I want to use it because its HashMap kind of capabilities with
an added feature to track versions. Is it recommended that I use it for a
small application (in standalone mode) with maybe 100K users and storage
needs which probably won't exceed 100G.

I know it is never recommended to be used as a transactional database (I
have read that in a million places) but I would like to know more about it.

Thanks,
Arun

Re: Using HBase in standalone mode in production

Posted by Ted Yu <yu...@gmail.com>.
w.r.t. Apache Slider, see http://slider.incubator.apache.org/

bq. with flexibility of growing and shrinking

The 'flex' action achieves the growing and shrinking.

When the cluster is not needed (for some period of time), you can freeze
the cluster. In case it is to be used again, you can thaw the cluster -
your data would be available.

Cheers


On Mon, Jul 7, 2014 at 6:56 PM, Artem Ervits <ar...@nyp.org> wrote:

> Though I have not looked at it myself but you can run hbase as a long
> running process on Yarn (apache slider). As far as I understand, you can
> have an instance of any size with flexibility of growing and shrinking.
>
>
> Artem Ervits
> Data Analyst
> New York Presbyterian Hospital
>
> ----- Original Message -----
> From: Arun Allamsetty [mailto:arun.allamsetty@gmail.com]
> Sent: Monday, July 07, 2014 08:37 PM
> To: user@hbase.apache.org <us...@hbase.apache.org>
> Subject: Re: Using HBase in standalone mode in production
>
> I have never tried MySQL's blob or varbinary. I guess I can look into that.
> Thanks for answering my questions.
>
> Arun
> On Jul 7, 2014 6:22 PM, "Dima Spivak" <ds...@cloudera.com> wrote:
>
> > Does MySQL's BLOB or VARBINARY satisfy your use case?
> >
> > As for converting a pseudo-distributed cluster to a distributed one,
> unless
> > I'm mistaken, you should have no problem doing so. HDFS is quite good
> with
> > scaling, whether it's from 10 machines to 20 or 1 to 10 and I don't know
> of
> > any reason that HBase would cause any problems in this regard.
> >
> > -Dima
> >
> >
> > On Mon, Jul 7, 2014 at 5:09 PM, Arun Allamsetty <
> arun.allamsetty@gmail.com
> > >
> > wrote:
> >
> > > I understand. But for example, my use case is where even if I don't
> have
> > a
> > > lot of data, what if I would rather store serialized objects. For this
> > > traditional RDBMS are not suitable. If I can forego the fail safe
> > > capabilities, then what is a good choice (if not HBase).
> > >
> > > Also, on a different note, if I have a HBase installation in pseudo
> > > distributed mode, then can I convert it into a distributed setup by
> > adding
> > > more machines without any loss in data?
> > >
> > > Thanks,
> > > Arun
> > > On Jul 7, 2014 6:02 PM, "Dima Spivak" <ds...@cloudera.com> wrote:
> > >
> > > > In general, production systems run in distributed mode because they
> > > > leverage HBase's scalability and reliability; HBase really only shows
> > its
> > > > worth when it's charged with managing terabytes of data on a
> > > fault-tolerant
> > > > file system like HDFS. You lose both of these when you run in
> > standalone
> > > > mode, so I'd be a bit worried about using such a setup for any
> > production
> > > > use.
> > > >
> > > > -Dima
> > > >
> > > >
> > > > On Mon, Jul 7, 2014 at 4:25 PM, Arun Allamsetty <
> > > arun.allamsetty@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Ted,
> > > > >
> > > > > I have. So the book says there are two types of distributed modes.
> > One
> > > is
> > > > > pseudo distributed, which is used when we want to test HBase's
> > > > distributed
> > > > > capabilities using a single machine. As far as I understood, this
> is
> > > just
> > > > > to verify the use cases and the requirements. Then we have the
> fully
> > > > > distributed mode in which HBase can be installed over multiple
> > > machines.
> > > > >
> > > > > I understand both the scenarios. But what if my application is not
> > > large
> > > > > enough to leverage the distributed mode and the pseudo distributed
> > mode
> > > > is
> > > > > pretty much for a PoC. Since the pseudo distributed mode won't be
> > able
> > > to
> > > > > provide any fault tolerance, can one use the standalone mode in
> > > > production.
> > > > >
> > > > > I hope my question is clear even if it does not make much sense.
> > > > >
> > > > > Thanks,
> > > > > Arun
> > > > > On Jul 7, 2014 5:17 PM, "Ted Yu" <yu...@gmail.com> wrote:
> > > > >
> > > > > > Have you read http://hbase.apache.org/book.html#standalone_dist
> ?
> > > > > >
> > > > > > Cheers
> > > > > >
> > > > > >
> > > > > > On Mon, Jul 7, 2014 at 3:55 PM, Arun Allamsetty <
> > > > > arun.allamsetty@gmail.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > So this question might be stupid, retarded even, but it has
> been
> > > > > bugging
> > > > > > me
> > > > > > > for a while and I cannot think of a better place to ask this. I
> > am
> > > > > really
> > > > > > > impressed with the way HBase works (as a key-value store).
> Since
> > it
> > > > > > stores
> > > > > > > everything as a byte array, I find it really convenient to
> store
> > > > > > serialized
> > > > > > > objects. Also, I understand that HBase is supposed to be used
> > when
> > > > you
> > > > > > have
> > > > > > > too much data to be handled by a single machine, so we can
> scale
> > > our
> > > > > > > application by running it in distributed mode.
> > > > > > >
> > > > > > > But what if I want to use it because its HashMap kind of
> > > capabilities
> > > > > > with
> > > > > > > an added feature to track versions. Is it recommended that I
> use
> > it
> > > > > for a
> > > > > > > small application (in standalone mode) with maybe 100K users
> and
> > > > > storage
> > > > > > > needs which probably won't exceed 100G.
> > > > > > >
> > > > > > > I know it is never recommended to be used as a transactional
> > > database
> > > > > (I
> > > > > > > have read that in a million places) but I would like to know
> more
> > > > about
> > > > > > it.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Arun
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
> ________________________________
>
> This electronic message is intended to be for the use only of the named
> recipient, and may contain information that is confidential or privileged.
> If you are not the intended recipient, you are hereby notified that any
> disclosure, copying, distribution or use of the contents of this message is
> strictly prohibited. If you have received this message in error or are not
> the named recipient, please notify us immediately by contacting the sender
> at the electronic mail address noted above, and delete and destroy all
> copies of this message. Thank you.
>
> This electronic message is intended to be for the use only of the named
> recipient, and may contain information that is confidential or privileged.
>  If you are not the intended recipient, you are hereby notified that any
> disclosure, copying, distribution or use of the contents of this message is
> strictly prohibited.  If you have received this message in error or are not
> the named recipient, please notify us immediately by contacting the sender
> at the electronic mail address noted above, and delete and destroy all
> copies of this message.  Thank you.
>

Re: Using HBase in standalone mode in production

Posted by Artem Ervits <ar...@nyp.org>.
Though I have not looked at it myself but you can run hbase as a long running process on Yarn (apache slider). As far as I understand, you can have an instance of any size with flexibility of growing and shrinking.


Artem Ervits
Data Analyst
New York Presbyterian Hospital

----- Original Message -----
From: Arun Allamsetty [mailto:arun.allamsetty@gmail.com]
Sent: Monday, July 07, 2014 08:37 PM
To: user@hbase.apache.org <us...@hbase.apache.org>
Subject: Re: Using HBase in standalone mode in production

I have never tried MySQL's blob or varbinary. I guess I can look into that.
Thanks for answering my questions.

Arun
On Jul 7, 2014 6:22 PM, "Dima Spivak" <ds...@cloudera.com> wrote:

> Does MySQL's BLOB or VARBINARY satisfy your use case?
>
> As for converting a pseudo-distributed cluster to a distributed one, unless
> I'm mistaken, you should have no problem doing so. HDFS is quite good with
> scaling, whether it's from 10 machines to 20 or 1 to 10 and I don't know of
> any reason that HBase would cause any problems in this regard.
>
> -Dima
>
>
> On Mon, Jul 7, 2014 at 5:09 PM, Arun Allamsetty <arun.allamsetty@gmail.com
> >
> wrote:
>
> > I understand. But for example, my use case is where even if I don't have
> a
> > lot of data, what if I would rather store serialized objects. For this
> > traditional RDBMS are not suitable. If I can forego the fail safe
> > capabilities, then what is a good choice (if not HBase).
> >
> > Also, on a different note, if I have a HBase installation in pseudo
> > distributed mode, then can I convert it into a distributed setup by
> adding
> > more machines without any loss in data?
> >
> > Thanks,
> > Arun
> > On Jul 7, 2014 6:02 PM, "Dima Spivak" <ds...@cloudera.com> wrote:
> >
> > > In general, production systems run in distributed mode because they
> > > leverage HBase's scalability and reliability; HBase really only shows
> its
> > > worth when it's charged with managing terabytes of data on a
> > fault-tolerant
> > > file system like HDFS. You lose both of these when you run in
> standalone
> > > mode, so I'd be a bit worried about using such a setup for any
> production
> > > use.
> > >
> > > -Dima
> > >
> > >
> > > On Mon, Jul 7, 2014 at 4:25 PM, Arun Allamsetty <
> > arun.allamsetty@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hi Ted,
> > > >
> > > > I have. So the book says there are two types of distributed modes.
> One
> > is
> > > > pseudo distributed, which is used when we want to test HBase's
> > > distributed
> > > > capabilities using a single machine. As far as I understood, this is
> > just
> > > > to verify the use cases and the requirements. Then we have the fully
> > > > distributed mode in which HBase can be installed over multiple
> > machines.
> > > >
> > > > I understand both the scenarios. But what if my application is not
> > large
> > > > enough to leverage the distributed mode and the pseudo distributed
> mode
> > > is
> > > > pretty much for a PoC. Since the pseudo distributed mode won't be
> able
> > to
> > > > provide any fault tolerance, can one use the standalone mode in
> > > production.
> > > >
> > > > I hope my question is clear even if it does not make much sense.
> > > >
> > > > Thanks,
> > > > Arun
> > > > On Jul 7, 2014 5:17 PM, "Ted Yu" <yu...@gmail.com> wrote:
> > > >
> > > > > Have you read http://hbase.apache.org/book.html#standalone_dist ?
> > > > >
> > > > > Cheers
> > > > >
> > > > >
> > > > > On Mon, Jul 7, 2014 at 3:55 PM, Arun Allamsetty <
> > > > arun.allamsetty@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > So this question might be stupid, retarded even, but it has been
> > > > bugging
> > > > > me
> > > > > > for a while and I cannot think of a better place to ask this. I
> am
> > > > really
> > > > > > impressed with the way HBase works (as a key-value store). Since
> it
> > > > > stores
> > > > > > everything as a byte array, I find it really convenient to store
> > > > > serialized
> > > > > > objects. Also, I understand that HBase is supposed to be used
> when
> > > you
> > > > > have
> > > > > > too much data to be handled by a single machine, so we can scale
> > our
> > > > > > application by running it in distributed mode.
> > > > > >
> > > > > > But what if I want to use it because its HashMap kind of
> > capabilities
> > > > > with
> > > > > > an added feature to track versions. Is it recommended that I use
> it
> > > > for a
> > > > > > small application (in standalone mode) with maybe 100K users and
> > > > storage
> > > > > > needs which probably won't exceed 100G.
> > > > > >
> > > > > > I know it is never recommended to be used as a transactional
> > database
> > > > (I
> > > > > > have read that in a million places) but I would like to know more
> > > about
> > > > > it.
> > > > > >
> > > > > > Thanks,
> > > > > > Arun
> > > > > >
> > > > >
> > > >
> > >
> >
>

________________________________

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.

Re: Using HBase in standalone mode in production

Posted by Arun Allamsetty <ar...@gmail.com>.
I have never tried MySQL's blob or varbinary. I guess I can look into that.
Thanks for answering my questions.

Arun
On Jul 7, 2014 6:22 PM, "Dima Spivak" <ds...@cloudera.com> wrote:

> Does MySQL's BLOB or VARBINARY satisfy your use case?
>
> As for converting a pseudo-distributed cluster to a distributed one, unless
> I'm mistaken, you should have no problem doing so. HDFS is quite good with
> scaling, whether it's from 10 machines to 20 or 1 to 10 and I don't know of
> any reason that HBase would cause any problems in this regard.
>
> -Dima
>
>
> On Mon, Jul 7, 2014 at 5:09 PM, Arun Allamsetty <arun.allamsetty@gmail.com
> >
> wrote:
>
> > I understand. But for example, my use case is where even if I don't have
> a
> > lot of data, what if I would rather store serialized objects. For this
> > traditional RDBMS are not suitable. If I can forego the fail safe
> > capabilities, then what is a good choice (if not HBase).
> >
> > Also, on a different note, if I have a HBase installation in pseudo
> > distributed mode, then can I convert it into a distributed setup by
> adding
> > more machines without any loss in data?
> >
> > Thanks,
> > Arun
> > On Jul 7, 2014 6:02 PM, "Dima Spivak" <ds...@cloudera.com> wrote:
> >
> > > In general, production systems run in distributed mode because they
> > > leverage HBase's scalability and reliability; HBase really only shows
> its
> > > worth when it's charged with managing terabytes of data on a
> > fault-tolerant
> > > file system like HDFS. You lose both of these when you run in
> standalone
> > > mode, so I'd be a bit worried about using such a setup for any
> production
> > > use.
> > >
> > > -Dima
> > >
> > >
> > > On Mon, Jul 7, 2014 at 4:25 PM, Arun Allamsetty <
> > arun.allamsetty@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hi Ted,
> > > >
> > > > I have. So the book says there are two types of distributed modes.
> One
> > is
> > > > pseudo distributed, which is used when we want to test HBase's
> > > distributed
> > > > capabilities using a single machine. As far as I understood, this is
> > just
> > > > to verify the use cases and the requirements. Then we have the fully
> > > > distributed mode in which HBase can be installed over multiple
> > machines.
> > > >
> > > > I understand both the scenarios. But what if my application is not
> > large
> > > > enough to leverage the distributed mode and the pseudo distributed
> mode
> > > is
> > > > pretty much for a PoC. Since the pseudo distributed mode won't be
> able
> > to
> > > > provide any fault tolerance, can one use the standalone mode in
> > > production.
> > > >
> > > > I hope my question is clear even if it does not make much sense.
> > > >
> > > > Thanks,
> > > > Arun
> > > > On Jul 7, 2014 5:17 PM, "Ted Yu" <yu...@gmail.com> wrote:
> > > >
> > > > > Have you read http://hbase.apache.org/book.html#standalone_dist ?
> > > > >
> > > > > Cheers
> > > > >
> > > > >
> > > > > On Mon, Jul 7, 2014 at 3:55 PM, Arun Allamsetty <
> > > > arun.allamsetty@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > So this question might be stupid, retarded even, but it has been
> > > > bugging
> > > > > me
> > > > > > for a while and I cannot think of a better place to ask this. I
> am
> > > > really
> > > > > > impressed with the way HBase works (as a key-value store). Since
> it
> > > > > stores
> > > > > > everything as a byte array, I find it really convenient to store
> > > > > serialized
> > > > > > objects. Also, I understand that HBase is supposed to be used
> when
> > > you
> > > > > have
> > > > > > too much data to be handled by a single machine, so we can scale
> > our
> > > > > > application by running it in distributed mode.
> > > > > >
> > > > > > But what if I want to use it because its HashMap kind of
> > capabilities
> > > > > with
> > > > > > an added feature to track versions. Is it recommended that I use
> it
> > > > for a
> > > > > > small application (in standalone mode) with maybe 100K users and
> > > > storage
> > > > > > needs which probably won't exceed 100G.
> > > > > >
> > > > > > I know it is never recommended to be used as a transactional
> > database
> > > > (I
> > > > > > have read that in a million places) but I would like to know more
> > > about
> > > > > it.
> > > > > >
> > > > > > Thanks,
> > > > > > Arun
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Using HBase in standalone mode in production

Posted by Dima Spivak <ds...@cloudera.com>.
Does MySQL's BLOB or VARBINARY satisfy your use case?

As for converting a pseudo-distributed cluster to a distributed one, unless
I'm mistaken, you should have no problem doing so. HDFS is quite good with
scaling, whether it's from 10 machines to 20 or 1 to 10 and I don't know of
any reason that HBase would cause any problems in this regard.

-Dima


On Mon, Jul 7, 2014 at 5:09 PM, Arun Allamsetty <ar...@gmail.com>
wrote:

> I understand. But for example, my use case is where even if I don't have a
> lot of data, what if I would rather store serialized objects. For this
> traditional RDBMS are not suitable. If I can forego the fail safe
> capabilities, then what is a good choice (if not HBase).
>
> Also, on a different note, if I have a HBase installation in pseudo
> distributed mode, then can I convert it into a distributed setup by adding
> more machines without any loss in data?
>
> Thanks,
> Arun
> On Jul 7, 2014 6:02 PM, "Dima Spivak" <ds...@cloudera.com> wrote:
>
> > In general, production systems run in distributed mode because they
> > leverage HBase's scalability and reliability; HBase really only shows its
> > worth when it's charged with managing terabytes of data on a
> fault-tolerant
> > file system like HDFS. You lose both of these when you run in standalone
> > mode, so I'd be a bit worried about using such a setup for any production
> > use.
> >
> > -Dima
> >
> >
> > On Mon, Jul 7, 2014 at 4:25 PM, Arun Allamsetty <
> arun.allamsetty@gmail.com
> > >
> > wrote:
> >
> > > Hi Ted,
> > >
> > > I have. So the book says there are two types of distributed modes. One
> is
> > > pseudo distributed, which is used when we want to test HBase's
> > distributed
> > > capabilities using a single machine. As far as I understood, this is
> just
> > > to verify the use cases and the requirements. Then we have the fully
> > > distributed mode in which HBase can be installed over multiple
> machines.
> > >
> > > I understand both the scenarios. But what if my application is not
> large
> > > enough to leverage the distributed mode and the pseudo distributed mode
> > is
> > > pretty much for a PoC. Since the pseudo distributed mode won't be able
> to
> > > provide any fault tolerance, can one use the standalone mode in
> > production.
> > >
> > > I hope my question is clear even if it does not make much sense.
> > >
> > > Thanks,
> > > Arun
> > > On Jul 7, 2014 5:17 PM, "Ted Yu" <yu...@gmail.com> wrote:
> > >
> > > > Have you read http://hbase.apache.org/book.html#standalone_dist ?
> > > >
> > > > Cheers
> > > >
> > > >
> > > > On Mon, Jul 7, 2014 at 3:55 PM, Arun Allamsetty <
> > > arun.allamsetty@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > So this question might be stupid, retarded even, but it has been
> > > bugging
> > > > me
> > > > > for a while and I cannot think of a better place to ask this. I am
> > > really
> > > > > impressed with the way HBase works (as a key-value store). Since it
> > > > stores
> > > > > everything as a byte array, I find it really convenient to store
> > > > serialized
> > > > > objects. Also, I understand that HBase is supposed to be used when
> > you
> > > > have
> > > > > too much data to be handled by a single machine, so we can scale
> our
> > > > > application by running it in distributed mode.
> > > > >
> > > > > But what if I want to use it because its HashMap kind of
> capabilities
> > > > with
> > > > > an added feature to track versions. Is it recommended that I use it
> > > for a
> > > > > small application (in standalone mode) with maybe 100K users and
> > > storage
> > > > > needs which probably won't exceed 100G.
> > > > >
> > > > > I know it is never recommended to be used as a transactional
> database
> > > (I
> > > > > have read that in a million places) but I would like to know more
> > about
> > > > it.
> > > > >
> > > > > Thanks,
> > > > > Arun
> > > > >
> > > >
> > >
> >
>

Re: Using HBase in standalone mode in production

Posted by Arun Allamsetty <ar...@gmail.com>.
I understand. But for example, my use case is where even if I don't have a
lot of data, what if I would rather store serialized objects. For this
traditional RDBMS are not suitable. If I can forego the fail safe
capabilities, then what is a good choice (if not HBase).

Also, on a different note, if I have a HBase installation in pseudo
distributed mode, then can I convert it into a distributed setup by adding
more machines without any loss in data?

Thanks,
Arun
On Jul 7, 2014 6:02 PM, "Dima Spivak" <ds...@cloudera.com> wrote:

> In general, production systems run in distributed mode because they
> leverage HBase's scalability and reliability; HBase really only shows its
> worth when it's charged with managing terabytes of data on a fault-tolerant
> file system like HDFS. You lose both of these when you run in standalone
> mode, so I'd be a bit worried about using such a setup for any production
> use.
>
> -Dima
>
>
> On Mon, Jul 7, 2014 at 4:25 PM, Arun Allamsetty <arun.allamsetty@gmail.com
> >
> wrote:
>
> > Hi Ted,
> >
> > I have. So the book says there are two types of distributed modes. One is
> > pseudo distributed, which is used when we want to test HBase's
> distributed
> > capabilities using a single machine. As far as I understood, this is just
> > to verify the use cases and the requirements. Then we have the fully
> > distributed mode in which HBase can be installed over multiple machines.
> >
> > I understand both the scenarios. But what if my application is not large
> > enough to leverage the distributed mode and the pseudo distributed mode
> is
> > pretty much for a PoC. Since the pseudo distributed mode won't be able to
> > provide any fault tolerance, can one use the standalone mode in
> production.
> >
> > I hope my question is clear even if it does not make much sense.
> >
> > Thanks,
> > Arun
> > On Jul 7, 2014 5:17 PM, "Ted Yu" <yu...@gmail.com> wrote:
> >
> > > Have you read http://hbase.apache.org/book.html#standalone_dist ?
> > >
> > > Cheers
> > >
> > >
> > > On Mon, Jul 7, 2014 at 3:55 PM, Arun Allamsetty <
> > arun.allamsetty@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > So this question might be stupid, retarded even, but it has been
> > bugging
> > > me
> > > > for a while and I cannot think of a better place to ask this. I am
> > really
> > > > impressed with the way HBase works (as a key-value store). Since it
> > > stores
> > > > everything as a byte array, I find it really convenient to store
> > > serialized
> > > > objects. Also, I understand that HBase is supposed to be used when
> you
> > > have
> > > > too much data to be handled by a single machine, so we can scale our
> > > > application by running it in distributed mode.
> > > >
> > > > But what if I want to use it because its HashMap kind of capabilities
> > > with
> > > > an added feature to track versions. Is it recommended that I use it
> > for a
> > > > small application (in standalone mode) with maybe 100K users and
> > storage
> > > > needs which probably won't exceed 100G.
> > > >
> > > > I know it is never recommended to be used as a transactional database
> > (I
> > > > have read that in a million places) but I would like to know more
> about
> > > it.
> > > >
> > > > Thanks,
> > > > Arun
> > > >
> > >
> >
>

Re: Using HBase in standalone mode in production

Posted by Dima Spivak <ds...@cloudera.com>.
In general, production systems run in distributed mode because they
leverage HBase's scalability and reliability; HBase really only shows its
worth when it's charged with managing terabytes of data on a fault-tolerant
file system like HDFS. You lose both of these when you run in standalone
mode, so I'd be a bit worried about using such a setup for any production
use.

-Dima


On Mon, Jul 7, 2014 at 4:25 PM, Arun Allamsetty <ar...@gmail.com>
wrote:

> Hi Ted,
>
> I have. So the book says there are two types of distributed modes. One is
> pseudo distributed, which is used when we want to test HBase's distributed
> capabilities using a single machine. As far as I understood, this is just
> to verify the use cases and the requirements. Then we have the fully
> distributed mode in which HBase can be installed over multiple machines.
>
> I understand both the scenarios. But what if my application is not large
> enough to leverage the distributed mode and the pseudo distributed mode is
> pretty much for a PoC. Since the pseudo distributed mode won't be able to
> provide any fault tolerance, can one use the standalone mode in production.
>
> I hope my question is clear even if it does not make much sense.
>
> Thanks,
> Arun
> On Jul 7, 2014 5:17 PM, "Ted Yu" <yu...@gmail.com> wrote:
>
> > Have you read http://hbase.apache.org/book.html#standalone_dist ?
> >
> > Cheers
> >
> >
> > On Mon, Jul 7, 2014 at 3:55 PM, Arun Allamsetty <
> arun.allamsetty@gmail.com
> > >
> > wrote:
> >
> > > Hi all,
> > >
> > > So this question might be stupid, retarded even, but it has been
> bugging
> > me
> > > for a while and I cannot think of a better place to ask this. I am
> really
> > > impressed with the way HBase works (as a key-value store). Since it
> > stores
> > > everything as a byte array, I find it really convenient to store
> > serialized
> > > objects. Also, I understand that HBase is supposed to be used when you
> > have
> > > too much data to be handled by a single machine, so we can scale our
> > > application by running it in distributed mode.
> > >
> > > But what if I want to use it because its HashMap kind of capabilities
> > with
> > > an added feature to track versions. Is it recommended that I use it
> for a
> > > small application (in standalone mode) with maybe 100K users and
> storage
> > > needs which probably won't exceed 100G.
> > >
> > > I know it is never recommended to be used as a transactional database
> (I
> > > have read that in a million places) but I would like to know more about
> > it.
> > >
> > > Thanks,
> > > Arun
> > >
> >
>

Re: Using HBase in standalone mode in production

Posted by Arun Allamsetty <ar...@gmail.com>.
Hi Ted,

I have. So the book says there are two types of distributed modes. One is
pseudo distributed, which is used when we want to test HBase's distributed
capabilities using a single machine. As far as I understood, this is just
to verify the use cases and the requirements. Then we have the fully
distributed mode in which HBase can be installed over multiple machines.

I understand both the scenarios. But what if my application is not large
enough to leverage the distributed mode and the pseudo distributed mode is
pretty much for a PoC. Since the pseudo distributed mode won't be able to
provide any fault tolerance, can one use the standalone mode in production.

I hope my question is clear even if it does not make much sense.

Thanks,
Arun
On Jul 7, 2014 5:17 PM, "Ted Yu" <yu...@gmail.com> wrote:

> Have you read http://hbase.apache.org/book.html#standalone_dist ?
>
> Cheers
>
>
> On Mon, Jul 7, 2014 at 3:55 PM, Arun Allamsetty <arun.allamsetty@gmail.com
> >
> wrote:
>
> > Hi all,
> >
> > So this question might be stupid, retarded even, but it has been bugging
> me
> > for a while and I cannot think of a better place to ask this. I am really
> > impressed with the way HBase works (as a key-value store). Since it
> stores
> > everything as a byte array, I find it really convenient to store
> serialized
> > objects. Also, I understand that HBase is supposed to be used when you
> have
> > too much data to be handled by a single machine, so we can scale our
> > application by running it in distributed mode.
> >
> > But what if I want to use it because its HashMap kind of capabilities
> with
> > an added feature to track versions. Is it recommended that I use it for a
> > small application (in standalone mode) with maybe 100K users and storage
> > needs which probably won't exceed 100G.
> >
> > I know it is never recommended to be used as a transactional database (I
> > have read that in a million places) but I would like to know more about
> it.
> >
> > Thanks,
> > Arun
> >
>

Re: Using HBase in standalone mode in production

Posted by Ted Yu <yu...@gmail.com>.
Have you read http://hbase.apache.org/book.html#standalone_dist ?

Cheers


On Mon, Jul 7, 2014 at 3:55 PM, Arun Allamsetty <ar...@gmail.com>
wrote:

> Hi all,
>
> So this question might be stupid, retarded even, but it has been bugging me
> for a while and I cannot think of a better place to ask this. I am really
> impressed with the way HBase works (as a key-value store). Since it stores
> everything as a byte array, I find it really convenient to store serialized
> objects. Also, I understand that HBase is supposed to be used when you have
> too much data to be handled by a single machine, so we can scale our
> application by running it in distributed mode.
>
> But what if I want to use it because its HashMap kind of capabilities with
> an added feature to track versions. Is it recommended that I use it for a
> small application (in standalone mode) with maybe 100K users and storage
> needs which probably won't exceed 100G.
>
> I know it is never recommended to be used as a transactional database (I
> have read that in a million places) but I would like to know more about it.
>
> Thanks,
> Arun
>