You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@ignite.apache.org by pragmaticbigdata <am...@gmail.com> on 2017/01/17 13:25:10 UTC

Data Rebalancing & Transactions

I have a few questions about data balancing

1. With synchronous data rebalancing, I understand that the newly added node
will not available for any cache operations till the data is completely
rebalanced in the cluster. This means that other nodes including the one
from which the data is being transferred to the new node will be available
for cache operations. Currently running transactions would not be affected.
Is my understanding correct?

2. With asynchronous data rebalancing mode, does the newly added node start
serving cache request as soon as the first cache entry is added to the node?
Does the new node even participate in cache transactions that are started
after this first cache entry was added?

3. With none rebalancing option, we are basically configuring a particular
cache to not take the advantage of any new nodes added to the cluster. Is my
understanding correct?

Thanks.



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-Rebalancing-Transactions-tp10092.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data Rebalancing & Transactions

Posted by Yakov Zhdanov <yz...@apache.org>.

Sam, you can control rebalancing with CacheConfiguration.setRebalance***()
properties. Take a look and rebalanceDelay, batchsize and prefetch count.
You can disable rebalancing by setting negative delay - this way you will
have to manually trigger rebalancing from API or Visor.

> 2. PUT will re-balance partitions among multiple cache server node or not?
Did not understand this. Cache updates are propagated to all affinity nodes
(primary and backups) including those which are currently being loaded.

--Yakov

2017-01-19 22:43 GMT+03:00 javastuff.sam@gmail.com <ja...@gmail.com>
:

> Hi Val,
>
> I agree ASYNC makes more sense, but need tuning and lighting fast network
> to
> complete GBs of data.
>
> If it is just about backups then using NONE re-balancing mode can help in
> certain usecases, by avoiding network transfers of GBs of data.
>
> Side cache - Persistent store is DB, Cache does not have required data then
> app will fetch from DB and supply to cache for further fetches. Cache
> Partition backup not needed because there is DB backup.
>
> In this case losing data from cache does not hurt alot with up-down scaling
> of cache cluster.
>
> Question -
> 1. NONE mode - shut down of a node will transfer data to other node or not?
> 2. PUT will re-balance partitions among multiple cache server node or not?
>
> Thanks,
> -Sam
>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Data-Rebalancing-Transactions-tp10092p10146.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Data Rebalancing & Transactions

Posted by "javastuff.sam@gmail.com" <ja...@gmail.com>.

Hi Val,

I agree ASYNC makes more sense, but need tuning and lighting fast network to
complete GBs of data.

If it is just about backups then using NONE re-balancing mode can help in
certain usecases, by avoiding network transfers of GBs of data.

Side cache - Persistent store is DB, Cache does not have required data then
app will fetch from DB and supply to cache for further fetches. Cache
Partition backup not needed because there is DB backup.

In this case losing data from cache does not hurt alot with up-down scaling
of cache cluster.

Question -
1. NONE mode - shut down of a node will transfer data to other node or not?
2. PUT will re-balance partitions among multiple cache server node or not?

Thanks,
-Sam

 



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-Rebalancing-Transactions-tp10092p10146.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data Rebalancing & Transactions

Posted by vkulichenko <va...@gmail.com>.

Yakov,

Makes sense.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-Rebalancing-Transactions-tp10092p10142.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data Rebalancing & Transactions

Posted by Yakov Zhdanov <yz...@apache.org>.

Val, NONE means rebalancing is disabled. I think we will remove ability to
set rebalance mode in 2.0. I think async is always fine.

--Yakov

2017-01-19 5:59 GMT+03:00 vkulichenko <va...@gmail.com>:

> Sam,
>
> NONE mode is something that should be used very accurately. It means what
> it
> means - no rebalancing is happening when topology is changing, unless you
> trigger the rebalancing manually. For example, if you have a cache with one
> backup and you lose one of the nodes, you end up having only one copy for
> some partitions. Unless there is a specific reason to do otherwise, I would
> recommend to use automatic rebalancing.
>
> -Val
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Data-Rebalancing-Transactions-tp10092p10134.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Data Rebalancing & Transactions

Posted by vkulichenko <va...@gmail.com>.

Sam,

NONE mode is something that should be used very accurately. It means what it
means - no rebalancing is happening when topology is changing, unless you
trigger the rebalancing manually. For example, if you have a cache with one
backup and you lose one of the nodes, you end up having only one copy for
some partitions. Unless there is a specific reason to do otherwise, I would
recommend to use automatic rebalancing.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-Rebalancing-Transactions-tp10092p10134.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data Rebalancing & Transactions

Posted by "javastuff.sam@gmail.com" <ja...@gmail.com>.

Hi Val,

Asynchronous re-balancing seems better option because of no blocking.
However I would like to understand more on NONE mode. What is the advantage
over other mode and how is the impact on over all cluster for serving data
in terms of GET/PUT latency? 

With NONE mode what will happen when a node start shutting down or crashes?

Thanks,
-Sam





--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-Rebalancing-Transactions-tp10092p10132.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data Rebalancing & Transactions

Posted by vkulichenko <va...@gmail.com>.

Hi,

1. Rebalancing mode only changes the behavior of public API, i.e. if you
call IgniteCache.put() for example, it will wait for rebalancing to finish.
If it's a standalone node where you do not use the API, rebalancing mode
does not have any affect.

2. Newly added node start serving cache requests as soon as it's started,
this does not depend on rebalancing mode.

3. NONE mode assumes that you will trigger rebalancing manually using
IgniteCache.rebalance() method.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-Rebalancing-Transactions-tp10092p10105.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.