Phoenix scalability

[Aleksandr Zuravliov <al...@visual-meta.com>]

Hi all!

We are looking for a solution to replace MySQL as a single point of failure
in our DB layer.
Being a non-redundant component it causes us downtimes in cases of hardware
failure and during upgrades.

We are an e-commerce company maintaining around 100 mln. items in more than
10 countries. That makes ~300 Gb of meta data per country. To cope with the
volumes we use custom made sharding on MySQL therefore we need a solution
providing high write and read performance. MySQL cluster does not suit us
as it uses to transfer between nodes high volumes of data. Our
infrastructure is made of around 100 nodes. We use Hadoop, non-relational
information is stored in Hbase.

   - I would like to know if anybody here is using Phoenix for similar
   scale? What issues are you facing during regular maintenance and during the
   peak load periods?
   - What were the scaling impediments you had to overcome? Or do you scale
   up by simply adding nodes?
   - Do you have downtimes? What are the causes?
   - What about failover? Do you experience data losses?
   - Do the maintenance operations - like backups at runtime affect
   performance?
   - How is upgrading the systems organized? Is it seamless for the cluster?
   - Are you able to reclaim the freed space (to compact the files used by
   the DB)? Without shutting down a node?
   - What about performance during multiple write/read operations? Any
   locking issues?


Pretty much need to get a feeling how painful it is to handle such volumes
on Phoenix.

Thank you!

All the best,
Aleksandr

Re: Phoenix scalability

[James Taylor <ja...@apache.org>]

Hi Aleksandr,
Most of your questions are more around HBase than Phoenix, so you may get a
more detailed answer if you ask your question on the HBase mailing list
(just replace "Phoenix" with "HBase" when you ask). In order to get HBase
and Phoenix running in production at Salesforce, we've had to have
satisfactory answers to all of these questions. Perhaps one of the PMC
members of Phoenix and HBase at Salesforce or one of the other companies
running both of them in production can chime in here?

Specific to Phoenix, our upgrade contract is documented here:
https://phoenix.apache.org/upgrading.html. We support seamlessly upgrading
to patch and minor releases (at least two minor releases back) with no
downtime. The server must be upgraded to new minor releases prior to the
client.

Thanks,
James

On Fri, Jul 24, 2015 at 5:09 AM, Aleksandr Zuravliov <
aleksandr.zuravliov@visual-meta.com> wrote:

> Hi all!
>
> We are looking for a solution to replace MySQL as a single point of
> failure in our DB layer.
> Being a non-redundant component it causes us downtimes in cases of
> hardware failure and during upgrades.
>
> We are an e-commerce company maintaining around 100 mln. items in more
> than 10 countries. That makes ~300 Gb of meta data per country. To cope
> with the volumes we use custom made sharding on MySQL therefore we need a
> solution providing high write and read performance. MySQL cluster does not
> suit us as it uses to transfer between nodes high volumes of data. Our
> infrastructure is made of around 100 nodes. We use Hadoop, non-relational
> information is stored in Hbase.
>
>    - I would like to know if anybody here is using Phoenix for similar
>    scale? What issues are you facing during regular maintenance and during the
>    peak load periods?
>    - What were the scaling impediments you had to overcome? Or do you
>    scale up by simply adding nodes?
>    - Do you have downtimes? What are the causes?
>    - What about failover? Do you experience data losses?
>    - Do the maintenance operations - like backups at runtime affect
>    performance?
>    - How is upgrading the systems organized? Is it seamless for the
>    cluster?
>    - Are you able to reclaim the freed space (to compact the files used
>    by the DB)? Without shutting down a node?
>    - What about performance during multiple write/read operations? Any
>    locking issues?
>
>
> Pretty much need to get a feeling how painful it is to handle such volumes
> on Phoenix.
>
> Thank you!
>
> All the best,
> Aleksandr
>
>
>
>
>
>
>
>
>
>