You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by GitBox <gi...@apache.org> on 2021/08/23 21:14:18 UTC

[GitHub] [ignite] sk0x50 opened a new pull request #9352: IGNITE-15364 Fixed an issue that could break data consistency.

sk0x50 opened a new pull request #9352:
URL: https://github.com/apache/ignite/pull/9352


   Thank you for submitting the pull request to the Apache Ignite.
   
   In order to streamline the review of the contribution 
   we ask you to ensure the following steps have been taken:
   
   ### The Contribution Checklist
   - [ ] There is a single JIRA ticket related to the pull request. 
   - [ ] The web-link to the pull request is attached to the JIRA ticket.
   - [ ] The JIRA ticket has the _Patch Available_ state.
   - [ ] The pull request body describes changes that have been made. 
   The description explains _WHAT_ and _WHY_ was made instead of _HOW_.
   - [ ] The pull request title is treated as the final commit message. 
   The following pattern must be used: `IGNITE-XXXX Change summary` where `XXXX` - number of JIRA issue.
   - [ ] A reviewer has been mentioned through the JIRA comments 
   (see [the Maintainers list](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers)) 
   - [ ] The pull request has been checked by the Teamcity Bot and 
   the `green visa` attached to the JIRA ticket (see [TC.Bot: Check PR](https://mtcga.gridgain.com/prs.html))
   
   ### Notes
   - [How to Contribute](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute)
   - [Coding abbreviation rules](https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules)
   - [Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines)
   - [Apache Ignite Teamcity Bot](https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot)
   
   If you need any help, please email dev@ignite.apache.org or ask anу advice on http://asf.slack.com _#ignite_ channel.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite] sk0x50 closed pull request #9352: IGNITE-15364 Fixed an issue that could break data consistency.

Posted by GitBox <gi...@apache.org>.
sk0x50 closed pull request #9352:
URL: https://github.com/apache/ignite/pull/9352


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite] ascherbakoff commented on a change in pull request #9352: IGNITE-15364 Fixed an issue that could break data consistency.

Posted by GitBox <gi...@apache.org>.
ascherbakoff commented on a change in pull request #9352:
URL: https://github.com/apache/ignite/pull/9352#discussion_r762863800



##########
File path: modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java
##########
@@ -3311,15 +3311,8 @@ private void body0() throws InterruptedException, IgniteCheckedException {
                                 if (fut.topologyVersion().equals(lastAffChangedVer))
                                     exchFut = fut;
                                 else if (lastAffChangedVer.after(exchId.topologyVersion())) {
-                                    // There is a new exchange which should trigger rebalancing.
-                                    // This reassignment request can be skipped.
-                                    if (log.isInfoEnabled()) {
-                                        log.info("Partitions reassignment request skipped due to affinity was already changed" +
-                                            " [reassignTopVer=" + exchId.topologyVersion() +
-                                            ", lastAffChangedTopVer=" + lastAffChangedVer + ']');
-                                    }
-
-                                    continue;
+                                    exchFut = lastFut;

Review comment:
       1) This modification doesn't look related to a described in the ticket counter issue. Better to do it in a separate ticket or modify the ticket description.
   
   2) I'm not sure it's 100% correct, because lastAffChangedVer implies the rebalancing is requred. New affinity can contain incompatible assignments, and make the reassign obsolete. Can we keep this logic ?
   
   3) The fut is always null in the assertion above:
   ```
   assert fut.changedAffinity() :
        "Reassignment request started for exchange future which didn't change affinity " +
              "[exchId=" + exchId + ", fut=" + exchFut + ']';
   ```
   
   4) I would better rename `markNodeAsInapplicableForRebalance` to `copyInapplicableNodesFrom`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite] sk0x50 commented on a change in pull request #9352: IGNITE-15364 Fixed an issue that could break data consistency.

Posted by GitBox <gi...@apache.org>.
sk0x50 commented on a change in pull request #9352:
URL: https://github.com/apache/ignite/pull/9352#discussion_r762955358



##########
File path: modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java
##########
@@ -3311,15 +3311,8 @@ private void body0() throws InterruptedException, IgniteCheckedException {
                                 if (fut.topologyVersion().equals(lastAffChangedVer))
                                     exchFut = fut;
                                 else if (lastAffChangedVer.after(exchId.topologyVersion())) {
-                                    // There is a new exchange which should trigger rebalancing.
-                                    // This reassignment request can be skipped.
-                                    if (log.isInfoEnabled()) {
-                                        log.info("Partitions reassignment request skipped due to affinity was already changed" +
-                                            " [reassignTopVer=" + exchId.topologyVersion() +
-                                            ", lastAffChangedTopVer=" + lastAffChangedVer + ']');
-                                    }
-
-                                    continue;
+                                    exchFut = lastFut;

Review comment:
       > This modification doesn't look related to a described in the ticket counter issue. Better to do it in a separate ticket or modify the ticket description.
   Ok, you are right. I need to update the description or create a new ticket for this.
   
   > I'm not sure it's 100% correct, because lastAffChangedVer implies the rebalancing is requred.
   > New affinity can contain incompatible assignments, and make the reassign obsolete. Can we keep this logic ?
   In my understanding this is not quite correct. Let's consider the following scenario:
    a) `event-1` (the type of this event does not matter) triggers `exhange-1`, and corresponding rebalance `rebalance-fut-1`
    b) `event-2` triggers a new `exchnage-2` and `rebalance-fut-2` is compatible with `rebalance-fut-1`. this means that `rebalance-fut-1` will not be cancelled.
    c) supplier fails rebalancing (`rebalance-fut-1`) for some reason and so, `RebalanceReassignExchangeTask` will be posted to the exchange queue.
    d) the following code will skip new rebalancing and therefore rebalancing will not be completed at all.
    ```
       if (lastAffChangedVer.after(exchId.topologyVersion())) 
           continue.
   ```
   
   > The fut is always null in the assertion above:
   Thanks! will fix it.
   
   > I would better rename `markNodeAsInapplicableForRebalance `to `copyInapplicableNodesFrom`
   Makes sense to me. will rename this method.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite] sk0x50 commented on a change in pull request #9352: IGNITE-15364 Fixed an issue that could break data consistency.

Posted by GitBox <gi...@apache.org>.
sk0x50 commented on a change in pull request #9352:
URL: https://github.com/apache/ignite/pull/9352#discussion_r762955358



##########
File path: modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java
##########
@@ -3311,15 +3311,8 @@ private void body0() throws InterruptedException, IgniteCheckedException {
                                 if (fut.topologyVersion().equals(lastAffChangedVer))
                                     exchFut = fut;
                                 else if (lastAffChangedVer.after(exchId.topologyVersion())) {
-                                    // There is a new exchange which should trigger rebalancing.
-                                    // This reassignment request can be skipped.
-                                    if (log.isInfoEnabled()) {
-                                        log.info("Partitions reassignment request skipped due to affinity was already changed" +
-                                            " [reassignTopVer=" + exchId.topologyVersion() +
-                                            ", lastAffChangedTopVer=" + lastAffChangedVer + ']');
-                                    }
-
-                                    continue;
+                                    exchFut = lastFut;

Review comment:
       > This modification doesn't look related to a described in the ticket counter issue. Better to do it in a separate ticket or modify the ticket description.
   
   Ok, you are right. I need to update the description or create a new ticket for this.
   
   > I'm not sure it's 100% correct, because lastAffChangedVer implies the rebalancing is requred.
   > New affinity can contain incompatible assignments, and make the reassign obsolete. Can we keep this logic ?
   
   In my understanding this is not quite correct. Let's consider the following scenario:
    a) `event-1` (the type of this event does not matter) triggers `exhange-1`, and corresponding rebalance `rebalance-fut-1`
    b) `event-2` triggers a new `exchnage-2` and `rebalance-fut-2` is compatible with `rebalance-fut-1`. this means that `rebalance-fut-1` will not be cancelled.
    c) supplier fails rebalancing (`rebalance-fut-1`) for some reason and so, `RebalanceReassignExchangeTask` will be posted to the exchange queue.
    d) the following code will skip new rebalancing and therefore rebalancing will not be completed at all.
    ```
       if (lastAffChangedVer.after(exchId.topologyVersion())) 
           continue.
   ```
   
   > The fut is always null in the assertion above:
   
   Thanks! will fix it.
   
   > I would better rename `markNodeAsInapplicableForRebalance `to `copyInapplicableNodesFrom`
   
   Makes sense to me. will rename this method.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite] ascherbakoff commented on a change in pull request #9352: IGNITE-15364 Fixed an issue that could break data consistency.

Posted by GitBox <gi...@apache.org>.
ascherbakoff commented on a change in pull request #9352:
URL: https://github.com/apache/ignite/pull/9352#discussion_r703498967



##########
File path: modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java
##########
@@ -3311,15 +3312,19 @@ private void body0() throws InterruptedException, IgniteCheckedException {
                                 if (fut.topologyVersion().equals(lastAffChangedVer))
                                     exchFut = fut;
                                 else if (lastAffChangedVer.after(exchId.topologyVersion())) {
-                                    // There is a new exchange which should trigger rebalancing.
-                                    // This reassignment request can be skipped.
-                                    if (log.isInfoEnabled()) {
-                                        log.info("Partitions reassignment request skipped due to affinity was already changed" +
-                                            " [reassignTopVer=" + exchId.topologyVersion() +
-                                            ", lastAffChangedTopVer=" + lastAffChangedVer + ']');
-                                    }
+                                    if (lastFut.exchangeType() == ALL) {

Review comment:
       ```suggestion
                                       if (lastFut.exchangeType() == ALL && !exchFut.rebalanced()) {
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite] ascherbakoff commented on a change in pull request #9352: IGNITE-15364 Fixed an issue that could break data consistency.

Posted by GitBox <gi...@apache.org>.
ascherbakoff commented on a change in pull request #9352:
URL: https://github.com/apache/ignite/pull/9352#discussion_r695026293



##########
File path: modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtPartitionTopologyImpl.java
##########
@@ -596,7 +596,8 @@ else if (localNode(p, aff))
 
                 long updateSeq = this.updateSeq.incrementAndGet();
 
-                cntrMap.clear();
+                if (!exchFut.firstEvent().eventNode().isClient() || exchFut.changedAffinity())

Review comment:
       Shouldn't the condition be
   
   if (exchFut.exchangeType() == ALL && !exchFut.rebalanced())
         cntrMap.clear();




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite] sk0x50 closed pull request #9352: IGNITE-15364 Fixed an issue that could break data consistency.

Posted by GitBox <gi...@apache.org>.
sk0x50 closed pull request #9352:
URL: https://github.com/apache/ignite/pull/9352


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite] sk0x50 commented on a change in pull request #9352: IGNITE-15364 Fixed an issue that could break data consistency.

Posted by GitBox <gi...@apache.org>.
sk0x50 commented on a change in pull request #9352:
URL: https://github.com/apache/ignite/pull/9352#discussion_r762955358



##########
File path: modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java
##########
@@ -3311,15 +3311,8 @@ private void body0() throws InterruptedException, IgniteCheckedException {
                                 if (fut.topologyVersion().equals(lastAffChangedVer))
                                     exchFut = fut;
                                 else if (lastAffChangedVer.after(exchId.topologyVersion())) {
-                                    // There is a new exchange which should trigger rebalancing.
-                                    // This reassignment request can be skipped.
-                                    if (log.isInfoEnabled()) {
-                                        log.info("Partitions reassignment request skipped due to affinity was already changed" +
-                                            " [reassignTopVer=" + exchId.topologyVersion() +
-                                            ", lastAffChangedTopVer=" + lastAffChangedVer + ']');
-                                    }
-
-                                    continue;
+                                    exchFut = lastFut;

Review comment:
       > This modification doesn't look related to a described in the ticket counter issue. Better to do it in a separate ticket or modify the ticket description.
   
   Ok, you are right. I need to update the description or create a new ticket for this.
   
   > I'm not sure it's 100% correct, because lastAffChangedVer implies the rebalancing is requred.
   
   > New affinity can contain incompatible assignments, and make the reassign obsolete. Can we keep this logic ?
   In my understanding this is not quite correct. Let's consider the following scenario:
    a) `event-1` (the type of this event does not matter) triggers `exhange-1`, and corresponding rebalance `rebalance-fut-1`
    b) `event-2` triggers a new `exchnage-2` and `rebalance-fut-2` is compatible with `rebalance-fut-1`. this means that `rebalance-fut-1` will not be cancelled.
    c) supplier fails rebalancing (`rebalance-fut-1`) for some reason and so, `RebalanceReassignExchangeTask` will be posted to the exchange queue.
    d) the following code will skip new rebalancing and therefore rebalancing will not be completed at all.
    ```
       if (lastAffChangedVer.after(exchId.topologyVersion())) 
           continue.
   ```
   
   > The fut is always null in the assertion above:
   
   Thanks! will fix it.
   
   > I would better rename `markNodeAsInapplicableForRebalance `to `copyInapplicableNodesFrom`
   
   Makes sense to me. will rename this method.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [ignite] sk0x50 commented on a change in pull request #9352: IGNITE-15364 Fixed an issue that could break data consistency.

Posted by GitBox <gi...@apache.org>.
sk0x50 commented on a change in pull request #9352:
URL: https://github.com/apache/ignite/pull/9352#discussion_r702822784



##########
File path: modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtPartitionTopologyImpl.java
##########
@@ -596,7 +596,8 @@ else if (localNode(p, aff))
 
                 long updateSeq = this.updateSeq.incrementAndGet();
 
-                cntrMap.clear();
+                if (!exchFut.firstEvent().eventNode().isClient() || exchFut.changedAffinity())

Review comment:
       Yes, you are absolutely correct.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org