You are viewing a plain text version of this content. The canonical link for it is here.

Posted to notifications@asterixdb.apache.org by "Murtadha Hubail (Code Review)" <do...@asterixdb.incubator.apache.org> on 2018/03/25 01:59:52 UTC

Change in asterixdb[release-0.9.4-pre-rc]: [NO ISSUE][CLUS] Exclude Pending Removal Nodes From Location

Murtadha Hubail has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/2523

Change subject: [NO ISSUE][CLUS] Exclude Pending Removal Nodes From Location
......................................................................

[NO ISSUE][CLUS] Exclude Pending Removal Nodes From Location

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Exclude pending removel nodes from cluster locations
  to prevent new jobs from using them as compute locations.
- Add additional logging for dataset rebalance.

Change-Id: I0c9904b229aa086615a3a8bbdfaf44d51f1e1757
---
M asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/RebalanceUtil.java
M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/ClusterStateManager.java
2 files changed, 8 insertions(+), 0 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/23/2523/1

diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/RebalanceUtil.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/RebalanceUtil.java
index 7bb917f..702b6b4 100644
--- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/RebalanceUtil.java
+++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/RebalanceUtil.java
@@ -124,6 +124,9 @@
                 // The target dataset for rebalance.
                 targetDataset = sourceDataset.getTargetDatasetForRebalance(nodeGroupName);
 
+                LOGGER.info("Rebalancing dataset {} from node group {} with nodes {} to node group {} with nodes {}",
+                        sourceDataset.getDatasetName(), sourceDataset.getNodeGroupName(), sourceNodes,
+                        targetDataset.getNodeGroupName(), targetNcNames);
                 // Rebalances the source dataset into the target dataset.
                 rebalance(sourceDataset, targetDataset, metadataProvider, hcc, datasetRebalanceCallback);
             } else {
@@ -158,6 +161,7 @@
             // the source dataset.
             runMetadataTransaction(metadataProvider, () -> dropSourceDataset(sourceDataset, metadataProvider, hcc));
         });
+        LOGGER.info("Dataset {} rebalance completed successfully", datasetName);
     }
 
     @FunctionalInterface
@@ -238,6 +242,8 @@
                 (ActiveNotificationHandler) appCtx.getActiveNotificationHandler();
         IMetadataLockManager lockManager = appCtx.getMetadataLockManager();
         lockManager.upgradeDatasetLockToWrite(metadataProvider.getLocks(), DatasetUtil.getFullyQualifiedName(source));
+        LOGGER.info("Updating dataset {} node group from {} to {}", source.getDatasetName(), source.getNodeGroupName(),
+                target.getNodeGroupName());
         try {
             // Updates the dataset entry in the metadata storage
             MetadataManager.INSTANCE.updateDataset(mdTxnCtx, target);
@@ -248,6 +254,7 @@
                 }
             }
             MetadataManager.INSTANCE.commitTransaction(mdTxnCtx);
+            LOGGER.info("dataset {} node group updated to {}", target.getDatasetName(), target.getNodeGroupName());
         } finally {
             lockManager.downgradeDatasetLockToExclusiveModify(metadataProvider.getLocks(),
                     DatasetUtil.getFullyQualifiedName(target));
diff --git a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/ClusterStateManager.java b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/ClusterStateManager.java
index 7182204..73d6705 100644
--- a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/ClusterStateManager.java
+++ b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/ClusterStateManager.java
@@ -280,6 +280,7 @@
                 clusterActiveLocations.add(p.getActiveNodeId());
             }
         }
+        clusterActiveLocations.removeAll(pendingRemoval);
         clusterPartitionConstraint =
                 new AlgebricksAbsolutePartitionConstraint(clusterActiveLocations.toArray(new String[] {}));
     }

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2523
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0c9904b229aa086615a3a8bbdfaf44d51f1e1757
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: release-0.9.4-pre-rc
Gerrit-Owner: Murtadha Hubail <mh...@apache.org>