Ignite performance

[luqmanahmad <lu...@gmail.com>]

Hi Igniters,

I will try to keep it short as much as I can. We have 8 caches total size is
around 6GB to 8GB in redis and the cluster is made up of 1200 nodes. All the
operations are read-only. It is a very low latency system where each request
doesn't need to be more than 2ms, in very rare cases 3-4ms. Everything is
hosted in our own datacenters and is working out of the box. We have a very
busy traffic I am talking about roughly 4.4 billion requests in an hour.

Now the challenge we have is to move everything to AWS as company rolling
all the projects to cloud and Elastic cache is reaching to its limit with
only 15% traffic and with no multi-region support which means each region
needs to have its own cache is not ideal for us. 

Company is really not very keen to move away from redis but looking at the
elastic cache limitations, agreed to look at the alternate solutions. I want
to go ahead with Ignite but I am really not sure whether Ignite can handle
that much traffic. Although I am an ignite user for a very long time and
have a firm faith on it :) but with that much low latency and hight traffic
is it really possible? All I want to know your views on whether Ignite can
handle that much traffic. How many nodes cluster would be sufficient for
that size traffic ? Max number of ignite nodes ever been deployed in a
cluster ? 

Best regards,
L



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite performance

[luqmanahmad <lu...@gmail.com>]

Hi Val,

Thanks for the quick response. I was checking at the network throughput for
the redis cache for one minute and the traffic is around 5GB, I can bet it
is not optimised, with one farm which has got 10 nodes. 

Lets say if we have an Ignite cluster of 100 nodes how you would and check
the maximum throughput across this cluster and each node as well? 

Thanks,
Luqman



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite performance

[vkulichenko <va...@gmail.com>]

Hi Luqman,

I don't see why not. It will probably require pretty big cluster, but looks
like your Redis cluster is not very small either :) Ignite is a highly
scalable system, so you can test with smaller clusters of different sizes,
check what maximum throughput they provide and then extrapolate to estimate
how many nodes you need.

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite performance

[Prasad Bhalerao <pr...@gmail.com>]

Hi luqmanahmed,

Could you please tell me why do need 1200 nodes to get 8 gb in memory?
Is it replicated cache? Do you need 1200 nodes to serve high number of
requests/sec?
If yes then does each node act as a primary for whole data? If yes, do you
use load balancer to reroute the requests?

Thanks,
Prasad

On Feb 14, 2018 11:21 PM, "luqmanahmad" <@gmail.com> wrote:

Hi Igniters,

I will try to keep it short as much as I can. We have 8 caches total size is
around 6GB to 8GB in redis and the cluster is made up of 1200 nodes. All the
operations are read-only. It is a very low latency system where each request
doesn't need to be more than 2ms, in very rare cases 3-4ms. Everything is
hosted in our own datacenters and is working out of the box. We have a very
busy traffic I am talking about roughly 4.4 billion requests in an hour.

Now the challenge we have is to move everything to AWS as company rolling
all the projects to cloud and Elastic cache is reaching to its limit with
only 15% traffic and with no multi-region support which means each region
needs to have its own cache is not ideal for us.

Company is really not very keen to move away from redis but looking at the
elastic cache limitations, agreed to look at the alternate solutions. I want
to go ahead with Ignite but I am really not sure whether Ignite can handle
that much traffic. Although I am an ignite user for a very long time and
have a firm faith on it :) but with that much low latency and hight traffic
is it really possible? All I want to know your views on whether Ignite can
handle that much traffic. How many nodes cluster would be sufficient for
that size traffic ? Max number of ignite nodes ever been deployed in a
cluster ?

Best regards,
L



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/