You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "sadanand48 (via GitHub)" <gi...@apache.org> on 2023/02/07 15:46:01 UTC
[GitHub] [hadoop] sadanand48 opened a new pull request, #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
sadanand48 opened a new pull request, #5364:
URL: https://github.com/apache/hadoop/pull/5364
### Description of PR
Currently in `DistcpSync` i.e the step which applies the diff b/w 2 provided snapshots as arguments to the distcp job with `-diff ` option, only `DistributedFilesystem` and `WebHDFS` filesystems are supported.
Now that Ozone supports snapshots, the change here is to make DistcpSync also support OzoneFilesystem (ofs).
If we add the check here in `DistcpSync `to check whether filesystem is instance of `BasicRootedOzoneFilesystem`,
we would need to introduce ozone dependency in hadoop code - but Ozone already has a hadoop dependency? To solve this , this PR introduces a job config by the name distcp.sync.class and the default value for this is the `DistcpSync` class itself. If a new filesystem wants to introduce their own implementation of `DistcpSync`, it just needs to extend the `DistcpSync `class and provide the class name to the config. The class will be initialised at run time, if no class is provided , it falls back to the original DistcpSync implementation and won't affect any current code.
The changes in this PR are mostly to be able to extend `DistcpSync` class in Ozone by making changing some of the access modifiers from private to public/protected so that it can be accessed from elsewhere.
### How was this patch tested?
Tested by building hadoop code and replacing distcp jars in Ozone via Unit tests in Ozone.
Also tested on a cluster.
### For code changes:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] sadanand48 commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1148389997
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java:
##########
@@ -286,6 +299,36 @@ private boolean getAllDiffs() throws IOException {
return false;
}
+ /**
+ * Check if the filesystem implementation has a method named
+ * getSnapshotDiffReport.
+ */
+ private static Method getSnapshotDiffReportMethod(FileSystem fs)
+ throws NoSuchMethodException {
+ return fs.getClass().getMethod(
+ "getSnapshotDiffReport", Path.class, String.class, String.class);
+ }
+
+ /**
+ * Get the snapshotDiff b/w the fromSnapshot & toSnapshot for the given
+ * filesystem.
+ */
+ private static SnapshotDiffReport getSnapshotDiffReport(
+ final FileSystem fs,
+ final Path snapshotDir,
+ final String fromSnapshot,
+ final String toSnapshot) throws IOException {
+ try {
+ return (SnapshotDiffReport) getSnapshotDiffReportMethod(fs).invoke(
+ fs, snapshotDir, fromSnapshot, toSnapshot);
+ } catch (InvocationTargetException e) {
+ throw new IOException(e.getCause());
+ } catch (NoSuchMethodException|IllegalAccessException e) {
Review Comment:
Done. Yes the exception will likely be detected earlier but these are checked exceptions so it requires either to define in `catch` block or `throws`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] sadanand48 commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1138906968
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java:
##########
@@ -122,8 +122,6 @@ private DistCpConstants() {
/* DistCp CopyListing class override param */
public static final String CONF_LABEL_COPY_LISTING_CLASS = "distcp.copy.listing.class";
- public static final String CONF_LABEL_DISTCP_SYNC_CLASS = "distcp.sync.class";
Review Comment:
Done. No change to this file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1480348050
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 18m 23s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 29m 0s | | trunk passed |
| +1 :green_heart: | compile | 29m 59s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 21m 45s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 4m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 47s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 30s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 16s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 54s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 35s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 16s | | the patch passed |
| +1 :green_heart: | compile | 24m 31s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 24m 31s | | the patch passed |
| +1 :green_heart: | compile | 21m 45s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 21m 45s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 52s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/8/artifact/out/results-checkstyle-root.txt) | root: The patch generated 3 new + 48 unchanged - 0 fixed = 51 total (was 48) |
| +1 :green_heart: | mvnsite | 1m 44s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 22s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 17s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 2s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 52s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 33s | | hadoop-hdfs-client in the patch passed. |
| -1 :x: | unit | 15m 3s | [/patch-unit-hadoop-tools_hadoop-distcp.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/8/artifact/out/patch-unit-hadoop-tools_hadoop-distcp.txt) | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 0m 52s | | The patch does not generate ASF License warnings. |
| | | 243m 41s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.tools.TestDistCpSync |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/8/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5364 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 5da565fdbbc2 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 345eac1bded740bf7ad00b40d24baa9b57d083ae |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/8/testReport/ |
| Max. process+thread count | 528 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/8/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1098966272
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java:
##########
@@ -165,6 +169,23 @@ private boolean preSyncCheck() throws IOException {
return true;
}
+ protected void checkFilesystemSupport(FileSystem srcFs, FileSystem tgtFs) {
Review Comment:
add javadocs
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java:
##########
@@ -286,6 +297,23 @@ private boolean getAllDiffs() throws IOException {
return false;
}
+ protected SnapshotDiffReport getSnapshotDiffReport(Path ssDir,
Review Comment:
javadocs
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java:
##########
@@ -122,6 +122,8 @@ private DistCpConstants() {
/* DistCp CopyListing class override param */
public static final String CONF_LABEL_COPY_LISTING_CLASS = "distcp.copy.listing.class";
+ public static final String CONF_LABEL_DISTCP_SYNC_CLASS = "distcp.sync.class";
Review Comment:
javadocs to explain what it does. the distcp markdown docs will need to mention it too, maybe, or at least add a section about working with ozone saying "read the ozone docs"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1480333212
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 1s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 20m 4s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 40s | | trunk passed |
| +1 :green_heart: | compile | 23m 1s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 20m 33s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 48s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 58s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 42s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 16s | | the patch passed |
| +1 :green_heart: | compile | 22m 23s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 22m 23s | | the patch passed |
| +1 :green_heart: | compile | 20m 33s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 20m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 34s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/7/artifact/out/results-checkstyle-root.txt) | root: The patch generated 3 new + 48 unchanged - 0 fixed = 51 total (was 48) |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 40s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 37s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 47s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 44s | | hadoop-hdfs-client in the patch passed. |
| -1 :x: | unit | 14m 9s | [/patch-unit-hadoop-tools_hadoop-distcp.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/7/artifact/out/patch-unit-hadoop-tools_hadoop-distcp.txt) | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 1m 1s | | The patch does not generate ASF License warnings. |
| | | 226m 5s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.tools.TestDistCpSync |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/7/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5364 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 4e8431556b86 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 345eac1bded740bf7ad00b40d24baa9b57d083ae |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/7/testReport/ |
| Max. process+thread count | 560 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] umamaheswararao merged pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "umamaheswararao (via GitHub)" <gi...@apache.org>.
umamaheswararao merged PR #5364:
URL: https://github.com/apache/hadoop/pull/5364
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1146801468
##########
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpSync.java:
##########
@@ -1303,12 +1303,13 @@ public void testSyncSnapshotDiffWithDummyFileSystem() throws Exception {
FileSystem dummyFs = FileSystem.get(URI.create("dummy:///"), conf);
Assert.assertTrue(dummyFs instanceof DummyFs);
new DistCp(conf, OptionsParser.parse(args)).execute();
+ } catch (UnsupportedOperationException e) {
+ Assert.fail("Dummy FS supports snapshots," +
Review Comment:
if you just rethrow e here, then the caller gets a full stack.
##########
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpSync.java:
##########
@@ -1276,4 +1283,64 @@ private void snapshotDiffWithPaths(Path sourceFSPath,
verifyCopyByFs(sourceFS, targetFS, sourceFS.getFileStatus(sourceFSPath),
targetFS.getFileStatus(targetFSPath), false);
}
+
+ @Test
+ public void testSyncSnapshotDiffWithLocalFileSystem() throws Exception {
+ String[] args = new String[]{"-update", "-diff", "s1", "s2",
+ "file:///source", "file:///target"};
+ LambdaTestUtils.intercept(
+ UnsupportedOperationException.class,
+ "The source file system file does not support snapshot",
+ () -> new DistCp(conf, OptionsParser.parse(args)).execute());
+ }
+
+ @Test
+ public void testSyncSnapshotDiffWithDummyFileSystem() throws Exception {
+ String[] args =
+ new String[] { "-update", "-diff", "s1", "s2", "dummy:///source",
+ "dummy:///target" };
+ try {
+ FileSystem dummyFs = FileSystem.get(URI.create("dummy:///"), conf);
+ Assert.assertTrue(dummyFs instanceof DummyFs);
Review Comment:
not very informative. I'd prefer an assertJ assertion that the class is an instance of,
```
assertThat(dummyfs).isInstanceOf()...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] sadanand48 commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1138909748
##########
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/SnapshotDiffReport.java:
##########
@@ -253,6 +253,11 @@ public String getFromSnapshot() {
return fromSnapshot;
}
+ /** @return {@link #toSnapshot} */
Review Comment:
On carefully looking , the getter for toSnapshot was already defined in getLaterSnapshotName, so removed this.
```
/** @return {@link #toSnapshot} */
public String getLaterSnapshotName() {
return toSnapshot;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] sadanand48 commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1135486313
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java:
##########
@@ -122,6 +122,8 @@ private DistCpConstants() {
/* DistCp CopyListing class override param */
public static final String CONF_LABEL_COPY_LISTING_CLASS = "distcp.copy.listing.class";
+ public static final String CONF_LABEL_DISTCP_SYNC_CLASS = "distcp.sync.class";
Review Comment:
Removed this approach for separate distcp sync class implementation, instead the correct filesystem implementation will be initialised at run time and getSnapshotDiffReport method would be called.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1468900264
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 17m 57s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 30m 43s | | trunk passed |
| +1 :green_heart: | compile | 23m 5s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 20m 26s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 50s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 4s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 46s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 32s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 38s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 16s | | the patch passed |
| +1 :green_heart: | compile | 22m 24s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 22m 24s | | the patch passed |
| +1 :green_heart: | compile | 20m 34s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 20m 34s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 37s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/4/artifact/out/results-checkstyle-root.txt) | root: The patch generated 3 new + 48 unchanged - 0 fixed = 51 total (was 48) |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 39s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 40s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 53s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 44s | | hadoop-hdfs-client in the patch passed. |
| +1 :green_heart: | unit | 14m 8s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 1m 2s | | The patch does not generate ASF License warnings. |
| | | 229m 32s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5364 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux abe64a2d431d 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6fd1f57bc530397d93785ccaac90fb4ebc15109e |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/4/testReport/ |
| Max. process+thread count | 556 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] sadanand48 commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1148389997
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java:
##########
@@ -286,6 +299,36 @@ private boolean getAllDiffs() throws IOException {
return false;
}
+ /**
+ * Check if the filesystem implementation has a method named
+ * getSnapshotDiffReport.
+ */
+ private static Method getSnapshotDiffReportMethod(FileSystem fs)
+ throws NoSuchMethodException {
+ return fs.getClass().getMethod(
+ "getSnapshotDiffReport", Path.class, String.class, String.class);
+ }
+
+ /**
+ * Get the snapshotDiff b/w the fromSnapshot & toSnapshot for the given
+ * filesystem.
+ */
+ private static SnapshotDiffReport getSnapshotDiffReport(
+ final FileSystem fs,
+ final Path snapshotDir,
+ final String fromSnapshot,
+ final String toSnapshot) throws IOException {
+ try {
+ return (SnapshotDiffReport) getSnapshotDiffReportMethod(fs).invoke(
+ fs, snapshotDir, fromSnapshot, toSnapshot);
+ } catch (InvocationTargetException e) {
+ throw new IOException(e.getCause());
+ } catch (NoSuchMethodException|IllegalAccessException e) {
Review Comment:
Thanks @umamaheswararao for the review.
Done. Yes the exception will likely be detected earlier but these are checked exceptions so it requires either to define in `catch` block or `throws`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] sadanand48 commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1146858970
##########
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpSync.java:
##########
@@ -1303,12 +1303,13 @@ public void testSyncSnapshotDiffWithDummyFileSystem() throws Exception {
FileSystem dummyFs = FileSystem.get(URI.create("dummy:///"), conf);
Assert.assertTrue(dummyFs instanceof DummyFs);
new DistCp(conf, OptionsParser.parse(args)).execute();
+ } catch (UnsupportedOperationException e) {
+ Assert.fail("Dummy FS supports snapshots," +
Review Comment:
Done.
##########
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpSync.java:
##########
@@ -1276,4 +1283,64 @@ private void snapshotDiffWithPaths(Path sourceFSPath,
verifyCopyByFs(sourceFS, targetFS, sourceFS.getFileStatus(sourceFSPath),
targetFS.getFileStatus(targetFSPath), false);
}
+
+ @Test
+ public void testSyncSnapshotDiffWithLocalFileSystem() throws Exception {
+ String[] args = new String[]{"-update", "-diff", "s1", "s2",
+ "file:///source", "file:///target"};
+ LambdaTestUtils.intercept(
+ UnsupportedOperationException.class,
+ "The source file system file does not support snapshot",
+ () -> new DistCp(conf, OptionsParser.parse(args)).execute());
+ }
+
+ @Test
+ public void testSyncSnapshotDiffWithDummyFileSystem() throws Exception {
+ String[] args =
+ new String[] { "-update", "-diff", "s1", "s2", "dummy:///source",
+ "dummy:///target" };
+ try {
+ FileSystem dummyFs = FileSystem.get(URI.create("dummy:///"), conf);
+ Assert.assertTrue(dummyFs instanceof DummyFs);
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] sadanand48 commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1138906146
##########
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpSync.java:
##########
@@ -1276,4 +1283,63 @@ private void snapshotDiffWithPaths(Path sourceFSPath,
verifyCopyByFs(sourceFS, targetFS, sourceFS.getFileStatus(sourceFSPath),
targetFS.getFileStatus(targetFSPath), false);
}
+
+ @Test
+ public void testSyncSnapshotDiffWithLocalFileSystem() throws Exception {
+ String[] args = new String[]{"-update", "-diff", "s1", "s2",
+ "file:///source", "file:///target"};
+ LambdaTestUtils.intercept(
+ IllegalArgumentException.class,
+ "The source file system file does not support snapshot",
+ () -> new DistCp(conf, OptionsParser.parse(args)).execute());
+ }
+
+ @Test
+ public void testSyncSnapshotDiffWithDummyFileSystem() throws Exception {
+ String[] args =
+ new String[] { "-update", "-diff", "s1", "s2", "dummy:///source",
+ "dummy:///target" };
+ try {
+ FileSystem dummyFs = FileSystem.get(URI.create("dummy:///"), conf);
+ Assert.assertTrue(dummyFs instanceof DummyFs);
+ new DistCp(conf, OptionsParser.parse(args)).execute();
+ } catch (Exception e) {
+ // can expect other exceptions as source and target paths
+ // are not created, assert if the exception is not arising
+ // due to the filesystem not supporting snapshots.
+ Assert.assertFalse(e.getMessage().contains("does not support snapshot"));
Review Comment:
Here I'm checking that a particular exception i.e exception stating that 'fs doesn't support snapshots' is **not** thrown, The intercept utility method checks whether the caught exception has a particular text, I think there is no util method to check its negation.
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java:
##########
@@ -169,20 +153,27 @@ protected boolean preSyncCheck() throws IOException {
return true;
}
- protected void checkFilesystemSupport(FileSystem srcFs, FileSystem tgtFs) {
- // currently we require both the source and the target file system are
- // DistributedFileSystem or (S)WebHdfsFileSystem.
- if (!(srcFs instanceof DistributedFileSystem
- || srcFs instanceof WebHdfsFileSystem)) {
- throw new IllegalArgumentException("Unsupported source file system: "
- + srcFs.getScheme() + "://. " +
- "Supported file systems: hdfs://, webhdfs:// and swebhdfs://.");
+ /**
+ * Check if the source and target filesystems support snapshots.
+ */
+ protected void checkFilesystemSupport(Path sourceDir, Path targetDir,
+ FileSystem srcFs, FileSystem tgtFs) throws IOException {
+ if (!srcFs.hasPathCapability(sourceDir,
+ CommonPathCapabilities.FS_SNAPSHOTS)) {
+ throw new IllegalArgumentException(
+ "The source file system " + srcFs.getScheme() + " does not support snapshot.");
+ }
+ if (!tgtFs.hasPathCapability(targetDir,
+ CommonPathCapabilities.FS_SNAPSHOTS)) {
+ throw new IllegalArgumentException(
+ "The target file system " + tgtFs.getScheme() + " does not support snapshot.");
}
- if (!(tgtFs instanceof DistributedFileSystem
- || tgtFs instanceof WebHdfsFileSystem)) {
- throw new IllegalArgumentException("Unsupported target file system: "
- + tgtFs.getScheme() + "://. " +
- "Supported file systems: hdfs://, webhdfs:// and swebhdfs://.");
+ try {
+ getSnapshotDiffReportMethod(srcFs);
+ getSnapshotDiffReportMethod(tgtFs);
+ } catch (NoSuchMethodException e) {
+ throw new IllegalArgumentException(
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1480682835
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 31s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 56s | | trunk passed |
| +1 :green_heart: | compile | 23m 14s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 20m 35s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 41s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 47s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 58s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 55s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 16s | | the patch passed |
| +1 :green_heart: | compile | 22m 21s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 22m 21s | | the patch passed |
| +1 :green_heart: | compile | 20m 31s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 20m 31s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 35s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/9/artifact/out/results-checkstyle-root.txt) | root: The patch generated 3 new + 48 unchanged - 0 fixed = 51 total (was 48) |
| +1 :green_heart: | mvnsite | 2m 4s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 41s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 38s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 9s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 46s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 44s | | hadoop-hdfs-client in the patch passed. |
| +1 :green_heart: | unit | 13m 51s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 1m 1s | | The patch does not generate ASF License warnings. |
| | | 223m 3s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/9/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5364 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux cce78eac795c 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 89da1b092bc82fa7f8e6f78065994296f214a350 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/9/testReport/ |
| Max. process+thread count | 741 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/9/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1483898294
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 59s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 51s | | trunk passed |
| +1 :green_heart: | compile | 23m 2s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 21m 2s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 41s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 45s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 35s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 1s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 53s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 17s | | the patch passed |
| +1 :green_heart: | compile | 22m 22s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 22m 22s | | the patch passed |
| +1 :green_heart: | compile | 20m 29s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 20m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 36s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/11/artifact/out/results-checkstyle-root.txt) | root: The patch generated 3 new + 48 unchanged - 0 fixed = 51 total (was 48) |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 41s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 38s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 58s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 44s | | hadoop-hdfs-client in the patch passed. |
| +1 :green_heart: | unit | 14m 0s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 1m 0s | | The patch does not generate ASF License warnings. |
| | | 222m 42s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/11/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5364 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c24b25c1efee 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a66c5f4108953a655a8cb70aa78d17574d1bf81d |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/11/testReport/ |
| Max. process+thread count | 761 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/11/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1468276427
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 17m 56s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 39s | | trunk passed |
| +1 :green_heart: | compile | 23m 10s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 20m 27s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 49s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 7s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 49s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 50s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 17s | | the patch passed |
| +1 :green_heart: | compile | 22m 25s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| -1 :x: | javac | 22m 25s | [/results-compile-javac-root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/3/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt) | root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 generated 2 new + 2824 unchanged - 0 fixed = 2826 total (was 2824) |
| +1 :green_heart: | compile | 20m 30s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| -1 :x: | javac | 20m 30s | [/results-compile-javac-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/3/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt) | root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 generated 2 new + 2623 unchanged - 0 fixed = 2625 total (was 2623) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 34s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/3/artifact/out/results-checkstyle-root.txt) | root: The patch generated 3 new + 48 unchanged - 0 fixed = 51 total (was 48) |
| +1 :green_heart: | mvnsite | 2m 2s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 39s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 38s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 0s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 43s | | hadoop-hdfs-client in the patch passed. |
| +1 :green_heart: | unit | 14m 10s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 1m 3s | | The patch does not generate ASF License warnings. |
| | | 227m 52s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5364 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c527582683ac 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / fd34ce6ced0729fcae98848598ea11fb7fc47221 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/3/testReport/ |
| Max. process+thread count | 687 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1472663326
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 17m 57s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 17m 37s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 31m 50s | | trunk passed |
| +1 :green_heart: | compile | 25m 13s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 21m 48s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 4m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 48s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 30s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 17s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 5s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 15s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 14s | | the patch passed |
| +1 :green_heart: | compile | 26m 7s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 26m 7s | | the patch passed |
| +1 :green_heart: | compile | 23m 22s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 23m 22s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 52s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/6/artifact/out/results-checkstyle-root.txt) | root: The patch generated 3 new + 48 unchanged - 0 fixed = 51 total (was 48) |
| +1 :green_heart: | mvnsite | 1m 44s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 21s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 16s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 12s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 42s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 40s | | hadoop-hdfs-client in the patch passed. |
| +1 :green_heart: | unit | 15m 58s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 0m 53s | | The patch does not generate ASF License warnings. |
| | | 265m 2s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5364 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 53ef61c08f5f 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ac44e1ccb1b7b0cb04a1929e1b0f8863d8c4f969 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/6/testReport/ |
| Max. process+thread count | 530 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] sadanand48 commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1135481494
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java:
##########
@@ -165,6 +169,23 @@ private boolean preSyncCheck() throws IOException {
return true;
}
+ protected void checkFilesystemSupport(FileSystem srcFs, FileSystem tgtFs) {
Review Comment:
Done.
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java:
##########
@@ -286,6 +297,23 @@ private boolean getAllDiffs() throws IOException {
return false;
}
+ protected SnapshotDiffReport getSnapshotDiffReport(Path ssDir,
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] sadanand48 commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1145254640
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java:
##########
@@ -161,20 +161,32 @@ private void checkFilesystemSupport(Path sourceDir, Path targetDir,
if (!srcFs.hasPathCapability(sourceDir,
CommonPathCapabilities.FS_SNAPSHOTS)) {
throw new IllegalArgumentException(
- "The source file system " + srcFs.getScheme() + " does not support snapshot.");
+ "The source file system " + srcFs.getScheme()
+ + " does not support snapshot.");
}
if (!tgtFs.hasPathCapability(targetDir,
CommonPathCapabilities.FS_SNAPSHOTS)) {
throw new IllegalArgumentException(
- "The target file system " + tgtFs.getScheme() + " does not support snapshot.");
+ "The target file system " + tgtFs.getScheme()
+ + " does not support snapshot.");
}
try {
getSnapshotDiffReportMethod(srcFs);
+ } catch (NoSuchMethodException e) {
+ throw new IllegalArgumentException(
Review Comment:
Done.
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java:
##########
@@ -161,20 +161,32 @@ private void checkFilesystemSupport(Path sourceDir, Path targetDir,
if (!srcFs.hasPathCapability(sourceDir,
CommonPathCapabilities.FS_SNAPSHOTS)) {
throw new IllegalArgumentException(
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1482101171
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 15s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 2s | | trunk passed |
| +1 :green_heart: | compile | 23m 3s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 20m 43s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 45s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 4s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 49s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 37s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 37s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 16s | | the patch passed |
| +1 :green_heart: | compile | 22m 19s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 22m 19s | | the patch passed |
| +1 :green_heart: | compile | 20m 38s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 20m 38s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 48s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/10/artifact/out/results-checkstyle-root.txt) | root: The patch generated 3 new + 48 unchanged - 0 fixed = 51 total (was 48) |
| +1 :green_heart: | mvnsite | 2m 7s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 40s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 38s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 11s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 48s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 45s | | hadoop-hdfs-client in the patch passed. |
| +1 :green_heart: | unit | 14m 4s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 1m 2s | | The patch does not generate ASF License warnings. |
| | | 223m 5s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/10/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5364 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d47e6da31e9a 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2a3a07a8c12e3f5d8dd2bbabb183cc9ffe582c2f |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/10/testReport/ |
| Max. process+thread count | 557 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/10/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] umamaheswararao commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "umamaheswararao (via GitHub)" <gi...@apache.org>.
umamaheswararao commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1146899680
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java:
##########
@@ -286,6 +299,36 @@ private boolean getAllDiffs() throws IOException {
return false;
}
+ /**
+ * Check if the filesystem implementation has a method named
+ * getSnapshotDiffReport.
+ */
+ private static Method getSnapshotDiffReportMethod(FileSystem fs)
+ throws NoSuchMethodException {
+ return fs.getClass().getMethod(
+ "getSnapshotDiffReport", Path.class, String.class, String.class);
+ }
+
+ /**
+ * Get the snapshotDiff b/w the fromSnapshot & toSnapshot for the given
+ * filesystem.
+ */
+ private static SnapshotDiffReport getSnapshotDiffReport(
+ final FileSystem fs,
+ final Path snapshotDir,
+ final String fromSnapshot,
+ final String toSnapshot) throws IOException {
+ try {
+ return (SnapshotDiffReport) getSnapshotDiffReportMethod(fs).invoke(
+ fs, snapshotDir, fromSnapshot, toSnapshot);
+ } catch (InvocationTargetException e) {
+ throw new IOException(e.getCause());
+ } catch (NoSuchMethodException|IllegalAccessException e) {
Review Comment:
NoSuchMethod sanity check already covered looks like before calling this API. IllegalAccess should noyt happen ideally. So runtime should be ok.
Tiny Nit: Message: "Failed to ...."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1472624677
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 12m 47s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 3s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 59s | | trunk passed |
| +1 :green_heart: | compile | 22m 56s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 20m 31s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 44s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 3s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 50s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 37s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 1s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 42s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 16s | | the patch passed |
| +1 :green_heart: | compile | 22m 27s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 22m 27s | | the patch passed |
| +1 :green_heart: | compile | 20m 33s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 20m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 39s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/5/artifact/out/results-checkstyle-root.txt) | root: The patch generated 3 new + 48 unchanged - 0 fixed = 51 total (was 48) |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 39s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 38s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 8s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 55s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 45s | | hadoop-hdfs-client in the patch passed. |
| +1 :green_heart: | unit | 14m 10s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 1m 2s | | The patch does not generate ASF License warnings. |
| | | 234m 35s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5364 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux f11a27007781 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ac44e1ccb1b7b0cb04a1929e1b0f8863d8c4f969 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/5/testReport/ |
| Max. process+thread count | 730 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] sadanand48 commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1145258696
##########
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpSync.java:
##########
@@ -1276,4 +1283,63 @@ private void snapshotDiffWithPaths(Path sourceFSPath,
verifyCopyByFs(sourceFS, targetFS, sourceFS.getFileStatus(sourceFSPath),
targetFS.getFileStatus(targetFSPath), false);
}
+
+ @Test
+ public void testSyncSnapshotDiffWithLocalFileSystem() throws Exception {
+ String[] args = new String[]{"-update", "-diff", "s1", "s2",
+ "file:///source", "file:///target"};
+ LambdaTestUtils.intercept(
+ IllegalArgumentException.class,
+ "The source file system file does not support snapshot",
+ () -> new DistCp(conf, OptionsParser.parse(args)).execute());
+ }
+
+ @Test
+ public void testSyncSnapshotDiffWithDummyFileSystem() throws Exception {
+ String[] args =
+ new String[] { "-update", "-diff", "s1", "s2", "dummy:///source",
+ "dummy:///target" };
+ try {
+ FileSystem dummyFs = FileSystem.get(URI.create("dummy:///"), conf);
+ Assert.assertTrue(dummyFs instanceof DummyFs);
+ new DistCp(conf, OptionsParser.parse(args)).execute();
+ } catch (Exception e) {
+ // can expect other exceptions as source and target paths
+ // are not created, assert if the exception is not arising
+ // due to the filesystem not supporting snapshots.
+ Assert.assertFalse(e.getMessage().contains("does not support snapshot"));
Review Comment:
After changing it to throw UnsupportedOperationException it became easier, The failure scenario for this test case is only if it gets UnsupportedException. It can expect other exceptions.
```java
catch (UnsupportedOperationException e) {
Assert.fail("Dummy FS supports snapshots," +
"did not expect UnsupportedOperationException");
} catch (Exception e) {
// can expect other exceptions as source and target paths
// are not created.
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1468311363
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 17m 41s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 17m 59s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 56s | | trunk passed |
| +1 :green_heart: | compile | 25m 11s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 21m 47s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 4m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 29s | | trunk passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 17s | | trunk passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 3m 54s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 37s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 24m 23s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| -1 :x: | javac | 24m 23s | [/results-compile-javac-root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/2/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt) | root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 generated 2 new + 2827 unchanged - 0 fixed = 2829 total (was 2827) |
| +1 :green_heart: | compile | 21m 47s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| -1 :x: | javac | 21m 47s | [/results-compile-javac-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/2/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt) | root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 generated 2 new + 2623 unchanged - 0 fixed = 2625 total (was 2623) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 4m 26s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/2/artifact/out/results-checkstyle-root.txt) | root: The patch generated 3 new + 48 unchanged - 0 fixed = 51 total (was 48) |
| +1 :green_heart: | mvnsite | 1m 57s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 25s | | the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 27s | | the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 33s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 32s | | hadoop-hdfs-client in the patch passed. |
| +1 :green_heart: | unit | 13m 56s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 0m 53s | | The patch does not generate ASF License warnings. |
| | | 257m 23s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5364 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux fa0bb4f87b46 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / fd34ce6ced0729fcae98848598ea11fb7fc47221 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/2/testReport/ |
| Max. process+thread count | 579 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1421351944
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 45s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 30m 46s | | trunk passed |
| +1 :green_heart: | compile | 23m 1s | | trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 20m 28s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 3m 44s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 4s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 47s | | trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javadoc | 1m 39s | | trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 55s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 33s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 29s | | the patch passed |
| +1 :green_heart: | compile | 22m 17s | | the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 22m 17s | | the patch passed |
| +1 :green_heart: | compile | 20m 39s | | the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 20m 39s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 41s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/1/artifact/out/results-checkstyle-root.txt) | root: The patch generated 3 new + 22 unchanged - 0 fixed = 25 total (was 22) |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| -1 :x: | javadoc | 0m 44s | [/results-javadoc-javadoc-hadoop-tools_hadoop-distcp-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/1/artifact/out/results-javadoc-javadoc-hadoop-tools_hadoop-distcp-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt) | hadoop-tools_hadoop-distcp-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 generated 9 new + 41 unchanged - 0 fixed = 50 total (was 41) |
| -1 :x: | javadoc | 0m 43s | [/results-javadoc-javadoc-hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_352-8u352-ga-1~20.04-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/1/artifact/out/results-javadoc-javadoc-hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_352-8u352-ga-1~20.04-b08.txt) | hadoop-tools_hadoop-distcp-jdkPrivateBuild-1.8.0_352-8u352-ga-1~20.04-b08 with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08 generated 9 new + 41 unchanged - 0 fixed = 50 total (was 41) |
| +1 :green_heart: | spotbugs | 4m 15s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 54s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 45s | | hadoop-hdfs-client in the patch passed. |
| +1 :green_heart: | unit | 21m 13s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 1m 3s | | The patch does not generate ASF License warnings. |
| | | 240m 33s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5364 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 976730f617b1 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b0742511a7d46fc56698a3276db5949aaf505c70 |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/1/testReport/ |
| Max. process+thread count | 566 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5364/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1144815944
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java:
##########
@@ -161,20 +161,32 @@ private void checkFilesystemSupport(Path sourceDir, Path targetDir,
if (!srcFs.hasPathCapability(sourceDir,
CommonPathCapabilities.FS_SNAPSHOTS)) {
throw new IllegalArgumentException(
- "The source file system " + srcFs.getScheme() + " does not support snapshot.");
+ "The source file system " + srcFs.getScheme()
+ + " does not support snapshot.");
}
if (!tgtFs.hasPathCapability(targetDir,
CommonPathCapabilities.FS_SNAPSHOTS)) {
throw new IllegalArgumentException(
- "The target file system " + tgtFs.getScheme() + " does not support snapshot.");
+ "The target file system " + tgtFs.getScheme()
+ + " does not support snapshot.");
}
try {
getSnapshotDiffReportMethod(srcFs);
+ } catch (NoSuchMethodException e) {
+ throw new IllegalArgumentException(
Review Comment:
move to UnsupportedOperationException
##########
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpSync.java:
##########
@@ -1305,7 +1305,7 @@ public void testSyncSnapshotDiffWithDummyFileSystem() throws Exception {
new DistCp(conf, OptionsParser.parse(args)).execute();
} catch (Exception e) {
Review Comment:
ok. catch `UnsupportedOperationException` after tightening the exception raised
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java:
##########
@@ -161,20 +161,32 @@ private void checkFilesystemSupport(Path sourceDir, Path targetDir,
if (!srcFs.hasPathCapability(sourceDir,
CommonPathCapabilities.FS_SNAPSHOTS)) {
throw new IllegalArgumentException(
Review Comment:
UnsupportedOperationException
##########
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpSync.java:
##########
@@ -1276,4 +1283,63 @@ private void snapshotDiffWithPaths(Path sourceFSPath,
verifyCopyByFs(sourceFS, targetFS, sourceFS.getFileStatus(sourceFSPath),
targetFS.getFileStatus(targetFSPath), false);
}
+
+ @Test
+ public void testSyncSnapshotDiffWithLocalFileSystem() throws Exception {
+ String[] args = new String[]{"-update", "-diff", "s1", "s2",
+ "file:///source", "file:///target"};
+ LambdaTestUtils.intercept(
+ IllegalArgumentException.class,
+ "The source file system file does not support snapshot",
+ () -> new DistCp(conf, OptionsParser.parse(args)).execute());
+ }
+
+ @Test
+ public void testSyncSnapshotDiffWithDummyFileSystem() throws Exception {
+ String[] args =
+ new String[] { "-update", "-diff", "s1", "s2", "dummy:///source",
+ "dummy:///target" };
+ try {
+ FileSystem dummyFs = FileSystem.get(URI.create("dummy:///"), conf);
+ Assert.assertTrue(dummyFs instanceof DummyFs);
+ new DistCp(conf, OptionsParser.parse(args)).execute();
+ } catch (Exception e) {
+ // can expect other exceptions as source and target paths
+ // are not created, assert if the exception is not arising
+ // due to the filesystem not supporting snapshots.
+ Assert.assertFalse(e.getMessage().contains("does not support snapshot"));
Review Comment:
ok.
Use `GenericTestUtils.assertExceptionContains(does not support snapshot", e)` so on a failure, the unexpected exception message and stack is not lost. whoever has to debug a test failure from an html test report will love you for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] sadanand48 commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1145254871
##########
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpSync.java:
##########
@@ -1305,7 +1305,7 @@ public void testSyncSnapshotDiffWithDummyFileSystem() throws Exception {
new DistCp(conf, OptionsParser.parse(args)).execute();
} catch (Exception e) {
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] umamaheswararao commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "umamaheswararao (via GitHub)" <gi...@apache.org>.
umamaheswararao commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1483136719
This patch looks good to me. If @steveloughran agree that his comments address, we can proceed to commit. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1471832418
thanks for the update; nothing to worry about. I was just concerned if some app was using distcp as a library (hive,...) and this would have broken it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a diff in pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on code in PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#discussion_r1137750976
##########
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpSync.java:
##########
@@ -1276,4 +1283,63 @@ private void snapshotDiffWithPaths(Path sourceFSPath,
verifyCopyByFs(sourceFS, targetFS, sourceFS.getFileStatus(sourceFSPath),
targetFS.getFileStatus(targetFSPath), false);
}
+
+ @Test
+ public void testSyncSnapshotDiffWithLocalFileSystem() throws Exception {
+ String[] args = new String[]{"-update", "-diff", "s1", "s2",
+ "file:///source", "file:///target"};
+ LambdaTestUtils.intercept(
+ IllegalArgumentException.class,
+ "The source file system file does not support snapshot",
+ () -> new DistCp(conf, OptionsParser.parse(args)).execute());
+ }
+
+ @Test
+ public void testSyncSnapshotDiffWithDummyFileSystem() throws Exception {
+ String[] args =
+ new String[] { "-update", "-diff", "s1", "s2", "dummy:///source",
+ "dummy:///target" };
+ try {
+ FileSystem dummyFs = FileSystem.get(URI.create("dummy:///"), conf);
+ Assert.assertTrue(dummyFs instanceof DummyFs);
+ new DistCp(conf, OptionsParser.parse(args)).execute();
+ } catch (Exception e) {
+ // can expect other exceptions as source and target paths
+ // are not created, assert if the exception is not arising
+ // due to the filesystem not supporting snapshots.
+ Assert.assertFalse(e.getMessage().contains("does not support snapshot"));
Review Comment:
are you expecting an exception here? if so, intercept() has an overload where you declare text to look for
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java:
##########
@@ -122,8 +122,6 @@ private DistCpConstants() {
/* DistCp CopyListing class override param */
public static final String CONF_LABEL_COPY_LISTING_CLASS = "distcp.copy.listing.class";
- public static final String CONF_LABEL_DISTCP_SYNC_CLASS = "distcp.sync.class";
Review Comment:
reinstate. api is tagged as @InterfaceAudience.LimitedPrivate("Distcp support tools"). if it's not supported tag as deprecated, but ideally have it keep working
##########
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/SnapshotDiffReport.java:
##########
@@ -253,6 +253,11 @@ public String getFromSnapshot() {
return fromSnapshot;
}
+ /** @return {@link #toSnapshot} */
Review Comment:
nit add a .
##########
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java:
##########
@@ -169,20 +153,27 @@ protected boolean preSyncCheck() throws IOException {
return true;
}
- protected void checkFilesystemSupport(FileSystem srcFs, FileSystem tgtFs) {
- // currently we require both the source and the target file system are
- // DistributedFileSystem or (S)WebHdfsFileSystem.
- if (!(srcFs instanceof DistributedFileSystem
- || srcFs instanceof WebHdfsFileSystem)) {
- throw new IllegalArgumentException("Unsupported source file system: "
- + srcFs.getScheme() + "://. " +
- "Supported file systems: hdfs://, webhdfs:// and swebhdfs://.");
+ /**
+ * Check if the source and target filesystems support snapshots.
+ */
+ protected void checkFilesystemSupport(Path sourceDir, Path targetDir,
+ FileSystem srcFs, FileSystem tgtFs) throws IOException {
+ if (!srcFs.hasPathCapability(sourceDir,
+ CommonPathCapabilities.FS_SNAPSHOTS)) {
+ throw new IllegalArgumentException(
+ "The source file system " + srcFs.getScheme() + " does not support snapshot.");
+ }
+ if (!tgtFs.hasPathCapability(targetDir,
+ CommonPathCapabilities.FS_SNAPSHOTS)) {
+ throw new IllegalArgumentException(
+ "The target file system " + tgtFs.getScheme() + " does not support snapshot.");
}
- if (!(tgtFs instanceof DistributedFileSystem
- || tgtFs instanceof WebHdfsFileSystem)) {
- throw new IllegalArgumentException("Unsupported target file system: "
- + tgtFs.getScheme() + "://. " +
- "Supported file systems: hdfs://, webhdfs:// and swebhdfs://.");
+ try {
+ getSnapshotDiffReportMethod(srcFs);
+ getSnapshotDiffReportMethod(tgtFs);
+ } catch (NoSuchMethodException e) {
+ throw new IllegalArgumentException(
Review Comment:
tell us the filesystem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] sadanand48 commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1470839515
> I'm worried that removing a constant and the ability to set a new sync class will break something.
Thanks @steveloughran for reviewing this PR. This constant is not present in the hadoop code but was added as part of a previous commit of this same PR, I'm no longer using that approach of setting the distcp.sync.class that I proposed earlier , instead just assigning the correct filesystem dynamically at run time did the job for me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] umamaheswararao commented on pull request #5364: HDFS-16911. Distcp with snapshot diff to support Ozone filesystem.
Posted by "umamaheswararao (via GitHub)" <gi...@apache.org>.
umamaheswararao commented on PR #5364:
URL: https://github.com/apache/hadoop/pull/5364#issuecomment-1502084228
If no other comments, i will proceed to merge this. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org