You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by "Murtadha Hubail (Code Review)" <do...@asterixdb.incubator.apache.org> on 2016/10/28 18:05:46 UTC
Change in asterixdb[master]: Fix sporadic failures in checkpoint test
Murtadha Hubail has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/1317
Change subject: Fix sporadic failures in checkpoint test
......................................................................
Fix sporadic failures in checkpoint test
Change-Id: Ic32b9221ba3f843e006931556bab52d184ab1d33
---
M asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java
1 file changed, 7 insertions(+), 5 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/17/1317/1
diff --git a/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java b/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java
index e3932ca..679d96e 100644
--- a/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java
+++ b/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java
@@ -30,6 +30,7 @@
import org.apache.asterix.common.configuration.Property;
import org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable;
import org.apache.asterix.common.transactions.DatasetId;
+import org.apache.asterix.common.transactions.IRecoveryManager;
import org.apache.asterix.common.transactions.ITransactionContext;
import org.apache.asterix.external.util.DataflowUtils;
import org.apache.asterix.metadata.entities.Dataset;
@@ -123,13 +124,14 @@
VSizeFrame frame = new VSizeFrame(ctx);
FrameTupleAppender tupleAppender = new FrameTupleAppender(frame);
+ IRecoveryManager recoveryManager = nc.getTransactionSubsystem().getRecoveryManager();
LogManager logManager = (LogManager) nc.getTransactionSubsystem().getLogManager();
// Number of log files after node startup should be one
int numberOfLogFiles = logManager.getLogFileIds().size();
Assert.assertEquals(1, numberOfLogFiles);
// Low-water mark LSN
- long lowWaterMarkLSN = nc.getTransactionSubsystem().getRecoveryManager().getMinFirstLSN();
+ long lowWaterMarkLSN = recoveryManager.getMinFirstLSN();
// Low-water mark log file id
long initialLowWaterMarkFileId = logManager.getLogFileId(lowWaterMarkLSN);
// Initial Low-water mark should be in the only available log file
@@ -142,7 +144,7 @@
}
// Check if the new low-water mark is still in the initial low-water mark log file
- lowWaterMarkLSN = nc.getTransactionSubsystem().getRecoveryManager().getMinFirstLSN();
+ lowWaterMarkLSN = recoveryManager.getMinFirstLSN();
long currentLowWaterMarkLogFileId = logManager.getLogFileId(lowWaterMarkLSN);
if (currentLowWaterMarkLogFileId == initialLowWaterMarkFileId) {
@@ -152,7 +154,7 @@
* recovery)
*/
int numberOfLogFilesBeforeCheckpoint = logManager.getLogFileIds().size();
- nc.getTransactionSubsystem().getRecoveryManager().checkpoint(false, logManager.getAppendLSN());
+ recoveryManager.checkpoint(false, logManager.getAppendLSN());
int numberOfLogFilesAfterCheckpoint = logManager.getLogFileIds().size();
Assert.assertEquals(numberOfLogFilesBeforeCheckpoint, numberOfLogFilesAfterCheckpoint);
@@ -164,7 +166,7 @@
while (currentLowWaterMarkLogFileId == initialLowWaterMarkFileId) {
ITupleReference tuple = tupleGenerator.next();
DataflowUtils.addTupleToFrame(tupleAppender, tuple, insertOp);
- lowWaterMarkLSN = nc.getTransactionSubsystem().getRecoveryManager().getMinFirstLSN();
+ lowWaterMarkLSN = recoveryManager.getMinFirstLSN();
currentLowWaterMarkLogFileId = logManager.getLogFileId(lowWaterMarkLSN);
}
}
@@ -173,7 +175,7 @@
* At this point, the low-water mark is not in the initialLowWaterMarkFileId, so
* a checkpoint should delete it.
*/
- nc.getTransactionSubsystem().getRecoveryManager().checkpoint(false, logManager.getAppendLSN());
+ recoveryManager.checkpoint(false, recoveryManager.getMinFirstLSN());
// Validate initialLowWaterMarkFileId was deleted
for (Long fileId : logManager.getLogFileIds()) {
--
To view, visit https://asterix-gerrit.ics.uci.edu/1317
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic32b9221ba3f843e006931556bab52d184ab1d33
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hu...@gmail.com>
Change in asterixdb[master]: Fix sporadic failures in checkpoint test
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: Fix sporadic failures in checkpoint test
......................................................................
Patch Set 1:
Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1032/
--
To view, visit https://asterix-gerrit.ics.uci.edu/1317
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic32b9221ba3f843e006931556bab52d184ab1d33
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hu...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: Fix sporadic failures in checkpoint test
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: Fix sporadic failures in checkpoint test
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3165/
--
To view, visit https://asterix-gerrit.ics.uci.edu/1317
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic32b9221ba3f843e006931556bab52d184ab1d33
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hu...@gmail.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: Fix sporadic failures in checkpoint test
Posted by "Ian Maxon (Code Review)" <do...@asterixdb.incubator.apache.org>.
Ian Maxon has posted comments on this change.
Change subject: Fix sporadic failures in checkpoint test
......................................................................
Patch Set 1:
(1 comment)
https://asterix-gerrit.ics.uci.edu/#/c/1317/1/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java
File asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java:
Line 178: recoveryManager.checkpoint(false, recoveryManager.getMinFirstLSN());
Does appendLSN() not guarantee the checkpoint to go into a new log file?
--
To view, visit https://asterix-gerrit.ics.uci.edu/1317
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic32b9221ba3f843e006931556bab52d184ab1d33
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hu...@gmail.com>
Gerrit-Reviewer: Ian Maxon <im...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: Yes
Change in asterixdb[master]: Fix sporadic failures in checkpoint test
Posted by "Murtadha Hubail (Code Review)" <do...@asterixdb.incubator.apache.org>.
Murtadha Hubail has posted comments on this change.
Change subject: Fix sporadic failures in checkpoint test
......................................................................
Patch Set 1:
(1 comment)
https://asterix-gerrit.ics.uci.edu/#/c/1317/1/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java
File asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java:
Line 178: recoveryManager.checkpoint(false, recoveryManager.getMinFirstLSN());
> Does appendLSN() not guarantee the checkpoint to go into a new log file?
There is a check in the checkpoint logic that checks if the passed LSN > the current minFirstLSN then the checkpoint doesn't happen and instead a flush is triggered.
--
To view, visit https://asterix-gerrit.ics.uci.edu/1317
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic32b9221ba3f843e006931556bab52d184ab1d33
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hu...@gmail.com>
Gerrit-Reviewer: Ian Maxon <im...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Murtadha Hubail <hu...@gmail.com>
Gerrit-HasComments: Yes
Change in asterixdb[master]: Fix sporadic failures in checkpoint test
Posted by "Murtadha Hubail (Code Review)" <do...@asterixdb.incubator.apache.org>.
Murtadha Hubail has submitted this change and it was merged.
Change subject: Fix sporadic failures in checkpoint test
......................................................................
Fix sporadic failures in checkpoint test
Change-Id: Ic32b9221ba3f843e006931556bab52d184ab1d33
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1317
Sonar-Qube: Jenkins <je...@fulliautomatix.ics.uci.edu>
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <je...@fulliautomatix.ics.uci.edu>
Reviewed-by: Ian Maxon <im...@apache.org>
---
M asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java
1 file changed, 7 insertions(+), 5 deletions(-)
Approvals:
Ian Maxon: Looks good to me, approved
Jenkins: Verified; No violations found; Verified
diff --git a/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java b/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java
index e3932ca..679d96e 100644
--- a/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java
+++ b/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java
@@ -30,6 +30,7 @@
import org.apache.asterix.common.configuration.Property;
import org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable;
import org.apache.asterix.common.transactions.DatasetId;
+import org.apache.asterix.common.transactions.IRecoveryManager;
import org.apache.asterix.common.transactions.ITransactionContext;
import org.apache.asterix.external.util.DataflowUtils;
import org.apache.asterix.metadata.entities.Dataset;
@@ -123,13 +124,14 @@
VSizeFrame frame = new VSizeFrame(ctx);
FrameTupleAppender tupleAppender = new FrameTupleAppender(frame);
+ IRecoveryManager recoveryManager = nc.getTransactionSubsystem().getRecoveryManager();
LogManager logManager = (LogManager) nc.getTransactionSubsystem().getLogManager();
// Number of log files after node startup should be one
int numberOfLogFiles = logManager.getLogFileIds().size();
Assert.assertEquals(1, numberOfLogFiles);
// Low-water mark LSN
- long lowWaterMarkLSN = nc.getTransactionSubsystem().getRecoveryManager().getMinFirstLSN();
+ long lowWaterMarkLSN = recoveryManager.getMinFirstLSN();
// Low-water mark log file id
long initialLowWaterMarkFileId = logManager.getLogFileId(lowWaterMarkLSN);
// Initial Low-water mark should be in the only available log file
@@ -142,7 +144,7 @@
}
// Check if the new low-water mark is still in the initial low-water mark log file
- lowWaterMarkLSN = nc.getTransactionSubsystem().getRecoveryManager().getMinFirstLSN();
+ lowWaterMarkLSN = recoveryManager.getMinFirstLSN();
long currentLowWaterMarkLogFileId = logManager.getLogFileId(lowWaterMarkLSN);
if (currentLowWaterMarkLogFileId == initialLowWaterMarkFileId) {
@@ -152,7 +154,7 @@
* recovery)
*/
int numberOfLogFilesBeforeCheckpoint = logManager.getLogFileIds().size();
- nc.getTransactionSubsystem().getRecoveryManager().checkpoint(false, logManager.getAppendLSN());
+ recoveryManager.checkpoint(false, logManager.getAppendLSN());
int numberOfLogFilesAfterCheckpoint = logManager.getLogFileIds().size();
Assert.assertEquals(numberOfLogFilesBeforeCheckpoint, numberOfLogFilesAfterCheckpoint);
@@ -164,7 +166,7 @@
while (currentLowWaterMarkLogFileId == initialLowWaterMarkFileId) {
ITupleReference tuple = tupleGenerator.next();
DataflowUtils.addTupleToFrame(tupleAppender, tuple, insertOp);
- lowWaterMarkLSN = nc.getTransactionSubsystem().getRecoveryManager().getMinFirstLSN();
+ lowWaterMarkLSN = recoveryManager.getMinFirstLSN();
currentLowWaterMarkLogFileId = logManager.getLogFileId(lowWaterMarkLSN);
}
}
@@ -173,7 +175,7 @@
* At this point, the low-water mark is not in the initialLowWaterMarkFileId, so
* a checkpoint should delete it.
*/
- nc.getTransactionSubsystem().getRecoveryManager().checkpoint(false, logManager.getAppendLSN());
+ recoveryManager.checkpoint(false, recoveryManager.getMinFirstLSN());
// Validate initialLowWaterMarkFileId was deleted
for (Long fileId : logManager.getLogFileIds()) {
--
To view, visit https://asterix-gerrit.ics.uci.edu/1317
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic32b9221ba3f843e006931556bab52d184ab1d33
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hu...@gmail.com>
Gerrit-Reviewer: Ian Maxon <im...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Murtadha Hubail <hu...@gmail.com>
Change in asterixdb[master]: Fix sporadic failures in checkpoint test
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: Fix sporadic failures in checkpoint test
......................................................................
Patch Set 1: Integration-Tests+1
Integration Tests Successful
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1032/ : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/1317
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic32b9221ba3f843e006931556bab52d184ab1d33
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hu...@gmail.com>
Gerrit-Reviewer: Ian Maxon <im...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: Fix sporadic failures in checkpoint test
Posted by "Ian Maxon (Code Review)" <do...@asterixdb.incubator.apache.org>.
Ian Maxon has posted comments on this change.
Change subject: Fix sporadic failures in checkpoint test
......................................................................
Patch Set 1: Code-Review+2
(1 comment)
https://asterix-gerrit.ics.uci.edu/#/c/1317/1/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java
File asterixdb/asterix-app/src/test/java/org/apache/asterix/test/logging/CheckpointingTest.java:
Line 178: recoveryManager.checkpoint(false, recoveryManager.getMinFirstLSN());
> There is a check in the checkpoint logic that checks if the passed LSN > th
I see, the minMCTFirstLSN bit for a nonsharp checkpoint.
--
To view, visit https://asterix-gerrit.ics.uci.edu/1317
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic32b9221ba3f843e006931556bab52d184ab1d33
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hu...@gmail.com>
Gerrit-Reviewer: Ian Maxon <im...@apache.org>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Murtadha Hubail <hu...@gmail.com>
Gerrit-HasComments: Yes