You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "adoroszlai (via GitHub)" <gi...@apache.org> on 2023/04/13 07:51:12 UTC
[GitHub] [ozone] adoroszlai opened a new pull request, #4564: HDDS-8370. Intermittent failure in TestOzoneFileSystem#testListStatusWithIntermediateDir
adoroszlai opened a new pull request, #4564:
URL: https://github.com/apache/ozone/pull/4564
## What changes were proposed in this pull request?
`testListStatusWithIntermediateDir` intermittently fails because it finds two entries in the root directory instead of one:
```
AssertionError: expected:<1> but was:<2>
...
at org.apache.hadoop.fs.ozone.TestOzoneFileSystem.testListStatusWithIntermediateDir(TestOzoneFileSystem.java:645)
```
Including the file status list in the assertion message reveals that the unexpected extra entry is `.Trash`:
```
AssertionFailedError: Found [
FileStatus{path=o3fs://bucket40939.volume15180/.Trash; isDirectory=true; modification_time=1680592792546; access_time=1680592792546; owner=runner; group=runner; permission=rwxrwxrwx; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false},
FileStatus{path=o3fs://bucket40939.volume15180/object-dir; isDirectory=true; modification_time=1680592792548; access_time=1680592792548; owner=runner; group=runner; permission=rwxrwxrwx; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false}
] with layout:LEGACY, fsPaths:false ==> expected: <1> but was: <2>
```
Since trash is handled by background thread (cleaning up deleted items from other tests in the same class), it may or may not be present at the time of the test. This PR changes the test to ignore it.
https://issues.apache.org/jira/browse/HDDS-8370
## How was this patch tested?
No failure of this test in 100/100 runs (but two other tests are still flaky):
https://github.com/adoroszlai/hadoop-ozone/actions/runs/4669634394
Regular CI:
https://github.com/adoroszlai/hadoop-ozone/actions/runs/4681320457
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #4564: HDDS-8370. Intermittent failure in TestOzoneFileSystem#testListStatusWithIntermediateDir
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #4564:
URL: https://github.com/apache/ozone/pull/4564#issuecomment-1509913396
Thanks @ayushtkn for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ayushtkn commented on a diff in pull request #4564: HDDS-8370. Intermittent failure in TestOzoneFileSystem#testListStatusWithIntermediateDir
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #4564:
URL: https://github.com/apache/ozone/pull/4564#discussion_r1166095830
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystem.java:
##########
@@ -639,10 +641,16 @@ public void testListStatusWithIntermediateDir() throws Exception {
}, 1000, 120000);
}
- FileStatus[] fileStatuses = fs.listStatus(parent);
+ List<FileStatus> fileStatuses = new ArrayList<>(Arrays.asList(
+ fs.listStatus(parent)));
+ Path trashRoot = new Path(OZONE_URI_DELIMITER, TRASH_PREFIX);
+ fileStatuses.removeIf(f -> trashRoot.equals(f.getPath()));
Review Comment:
Can we not use the ``listStatus`` with ``PathFilter`` option rather than doing this filtering ourselves?
Something like or a better variant as you like
```
fs.listStatus(parent, path -> path.toString().startsWith("/.Trash"));
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a diff in pull request #4564: HDDS-8370. Intermittent failure in TestOzoneFileSystem#testListStatusWithIntermediateDir
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #4564:
URL: https://github.com/apache/ozone/pull/4564#discussion_r1167186104
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFileSystem.java:
##########
@@ -639,10 +641,16 @@ public void testListStatusWithIntermediateDir() throws Exception {
}, 1000, 120000);
}
- FileStatus[] fileStatuses = fs.listStatus(parent);
+ List<FileStatus> fileStatuses = new ArrayList<>(Arrays.asList(
+ fs.listStatus(parent)));
+ Path trashRoot = new Path(OZONE_URI_DELIMITER, TRASH_PREFIX);
+ fileStatuses.removeIf(f -> trashRoot.equals(f.getPath()));
Review Comment:
Thanks for the suggestion, updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai merged pull request #4564: HDDS-8370. Intermittent failure in TestOzoneFileSystem#testListStatusWithIntermediateDir
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #4564:
URL: https://github.com/apache/ozone/pull/4564
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org