You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "jyotirmoy-gh (via GitHub)" <gi...@apache.org> on 2023/08/17 10:02:51 UTC
[GitHub] [ozone] jyotirmoy-gh opened a new pull request, #5194: HDDS-9132. [snapshot] Add unordered snapshot deletion unit-testcase
jyotirmoy-gh opened a new pull request, #5194:
URL: https://github.com/apache/ozone/pull/5194
## What changes were proposed in this pull request?
Add unordered snapshot deletion unit-testcase - `TestOzoneSnapshotRestore#testUnorderedDeletion`
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-9132
## How was this patch tested?
Testcase file - `hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/snapshot/TestOzoneSnapshotRestore.java`
```
mvn -l testlog.out -Dtest=TestOzoneSnapshotRestore test
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.ozone.om.snapshot.TestOzoneSnapshotRestore
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 213.91 s - in org.apache.hadoop.ozone.om.snapshot.TestOzoneSnapshotRestore
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sadanand48 commented on a diff in pull request #5194: HDDS-9132. [snapshot] Add unordered snapshot deletion unit-testcase
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #5194:
URL: https://github.com/apache/ozone/pull/5194#discussion_r1298069097
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/snapshot/TestOzoneSnapshotRestore.java:
##########
@@ -329,17 +334,65 @@ public void testRestoreSnapshotDifferentBucketLayout(
keyCopy(sourcePath + keyPrefix + i, destPath);
}
- assertDoesNotThrow(() -> waitForKeyCount(buck2, keyPrefix));
+ assertDoesNotThrow(() -> waitForKeyCount(buck2, keyPrefix, 5));
+ }
+
+ @ParameterizedTest
+ @MethodSource("bucketTypes")
+ public void testUnorderedDeletion(BucketLayout bucketLayoutTest)
+ throws Exception {
+ String volume = "vol-" + counter.incrementAndGet();
+ String bucket = "buck-" + counter.incrementAndGet();
+ store.createVolume(volume);
+ OzoneVolume vol = store.getVolume(volume);
+ BucketArgs bucketArgs = BucketArgs.newBuilder()
+ .setBucketLayout(bucketLayoutTest).build();
+ vol.createBucket(bucket, bucketArgs);
+ OzoneBucket buck = vol.getBucket(bucket);
+ // Create Key1 and take snapshot
+
+ String[] key = new String[10];
+ String[] snapshotName = new String[10];
+ String[] snapshotKeyPrefix = new String[10];
+
+ // create 10 incremental snapshots
+ for (int i = 0; i < 10; i++) {
+ key[i] = "key-" + counter.incrementAndGet();
+ snapshotName[i] = "snap-" + counter.incrementAndGet();
+ createFileKey(buck, key[i]);
+ snapshotKeyPrefix[i] = createSnapshot(volume, bucket, snapshotName[i]);
+ }
+
+ // delete multiple snapshots - 2nd, 5th , 8th
+ for (int i = 2; i < 10; i += 3) {
+ store.deleteSnapshot(volume, bucket, snapshotName[i]);
+ }
+
+ // delete all keys in bucket before restoring from snapshot
+ deleteKeys(buck);
+ int delKeyCount = keyCount(buck, "key-");
Review Comment:
nit : `keyCountAfterDelete` sounds better I guess .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sadanand48 commented on pull request #5194: HDDS-9132. [snapshot] Add unordered snapshot deletion unit-testcase
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on PR #5194:
URL: https://github.com/apache/ozone/pull/5194#issuecomment-1685838078
Thanks @jyotirmoy-gh for the test , @smengcl for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] jyotirmoy-gh commented on a diff in pull request #5194: HDDS-9132. [snapshot] Add unordered snapshot deletion unit-testcase
Posted by "jyotirmoy-gh (via GitHub)" <gi...@apache.org>.
jyotirmoy-gh commented on code in PR #5194:
URL: https://github.com/apache/ozone/pull/5194#discussion_r1298134669
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/snapshot/TestOzoneSnapshotRestore.java:
##########
@@ -329,17 +334,65 @@ public void testRestoreSnapshotDifferentBucketLayout(
keyCopy(sourcePath + keyPrefix + i, destPath);
}
- assertDoesNotThrow(() -> waitForKeyCount(buck2, keyPrefix));
+ assertDoesNotThrow(() -> waitForKeyCount(buck2, keyPrefix, 5));
+ }
+
+ @ParameterizedTest
+ @MethodSource("bucketTypes")
+ public void testUnorderedDeletion(BucketLayout bucketLayoutTest)
+ throws Exception {
+ String volume = "vol-" + counter.incrementAndGet();
+ String bucket = "buck-" + counter.incrementAndGet();
+ store.createVolume(volume);
+ OzoneVolume vol = store.getVolume(volume);
+ BucketArgs bucketArgs = BucketArgs.newBuilder()
+ .setBucketLayout(bucketLayoutTest).build();
+ vol.createBucket(bucket, bucketArgs);
+ OzoneBucket buck = vol.getBucket(bucket);
+ // Create Key1 and take snapshot
+
+ String[] key = new String[10];
+ String[] snapshotName = new String[10];
+ String[] snapshotKeyPrefix = new String[10];
+
+ // create 10 incremental snapshots
+ for (int i = 0; i < 10; i++) {
+ key[i] = "key-" + counter.incrementAndGet();
+ snapshotName[i] = "snap-" + counter.incrementAndGet();
+ createFileKey(buck, key[i]);
+ snapshotKeyPrefix[i] = createSnapshot(volume, bucket, snapshotName[i]);
+ }
+
+ // delete multiple snapshots - 2nd, 5th , 8th
+ for (int i = 2; i < 10; i += 3) {
+ store.deleteSnapshot(volume, bucket, snapshotName[i]);
+ }
+
+ // delete all keys in bucket before restoring from snapshot
+ deleteKeys(buck);
+ int delKeyCount = keyCount(buck, "key-");
Review Comment:
Fixed the variable name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sadanand48 merged pull request #5194: HDDS-9132. [snapshot] Add unordered snapshot deletion unit-testcase
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 merged PR #5194:
URL: https://github.com/apache/ozone/pull/5194
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] jyotirmoy-gh commented on pull request #5194: HDDS-9132. [snapshot] Add unordered snapshot deletion unit-testcase
Posted by "jyotirmoy-gh (via GitHub)" <gi...@apache.org>.
jyotirmoy-gh commented on PR #5194:
URL: https://github.com/apache/ozone/pull/5194#issuecomment-1682002264
Please review - @nilotpalnandi @smengcl @sadanand48
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] smengcl commented on pull request #5194: HDDS-9132. [snapshot] Add unordered snapshot deletion unit-testcase
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on PR #5194:
URL: https://github.com/apache/ozone/pull/5194#issuecomment-1683040060
Thanks @jyotirmoy-gh for the test case.
Can you address the findbugs warning:
https://github.com/apache/ozone/actions/runs/5889493468/job/15992065863?pr=5194#step:6:2365
```
M D ST: Write to static field org.apache.hadoop.ozone.om.snapshot.TestOzoneSnapshotRestore.counter from instance method org.apache.hadoop.ozone.om.snapshot.TestOzoneSnapshotRestore.init() At TestOzoneSnapshotRestore.java:[line 129]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] jyotirmoy-gh commented on pull request #5194: HDDS-9132. [snapshot] Add unordered snapshot deletion unit-testcase
Posted by "jyotirmoy-gh (via GitHub)" <gi...@apache.org>.
jyotirmoy-gh commented on PR #5194:
URL: https://github.com/apache/ozone/pull/5194#issuecomment-1683516845
> Thanks @jyotirmoy-gh for the test case.
>
> Can you address the findbugs warning:
>
> https://github.com/apache/ozone/actions/runs/5889493468/job/15992065863?pr=5194#step:6:2365
>
> ```
> M D ST: Write to static field org.apache.hadoop.ozone.om.snapshot.TestOzoneSnapshotRestore.counter from instance method org.apache.hadoop.ozone.om.snapshot.TestOzoneSnapshotRestore.init() At TestOzoneSnapshotRestore.java:[line 129]
> ```
Made some changes for above fix. Please review - @smengcl
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org