You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "haiyang1987 (via GitHub)" <gi...@apache.org> on 2023/06/05 07:55:37 UTC
[GitHub] [hadoop] haiyang1987 opened a new pull request, #5719: HDFS-16757. Add a new method copyBlockCrossNamespace to DataNode
haiyang1987 opened a new pull request, #5719:
URL: https://github.com/apache/hadoop/pull/5719
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
As described in https://issues.apache.org/jira/browse/HDFS-16757 ：
Add a new method copyBlockCrossNamespace to Datanode to support fast copy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-16757. Add a new method copyBlockCrossNamespace to DataNode [hadoop]
Posted by "LiuGuH (via GitHub)" <gi...@apache.org>.
LiuGuH commented on code in PR #5719:
URL: https://github.com/apache/hadoop/pull/5719#discussion_r1625937706
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java:
##########
@@ -1078,6 +1079,44 @@ public void blockGroupChecksum(final StripedBlockInfo stripedBlockInfo,
datanode.metrics.addBlockChecksumOp(elapsed());
}
+ @Override
+ public void copyBlockCrossNamespace(
+ final ExtendedBlock srcBlock, final Token<BlockTokenIdentifier> srcBlockToken,
+ final ExtendedBlock targetBlock, final Token<BlockTokenIdentifier> targetBlockToken,
+ final DatanodeInfo targetDN) throws IOException {
+ updateCurrentThreadName("Copying block cross namespace " + srcBlock);
+ final DataOutputStream reply = getBufferedOutputStream();
+ try {
+ // Check access
+ checkAccess(reply, true, srcBlock, srcBlockToken,
+ Op.COPY_BLOCK_CROSS_NAMESPACE, BlockTokenIdentifier.AccessMode.COPY);
Review Comment:
If we use namenode.getBlockLocations() , the blockToken AccessMode is BlockTokenIdentifier.AccessMode.READ
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-16757. Add a new method copyBlockCrossNamespace to DataNode [hadoop]
Posted by "LiuGuH (via GitHub)" <gi...@apache.org>.
LiuGuH commented on code in PR #5719:
URL: https://github.com/apache/hadoop/pull/5719#discussion_r1625937706
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java:
##########
@@ -1078,6 +1079,44 @@ public void blockGroupChecksum(final StripedBlockInfo stripedBlockInfo,
datanode.metrics.addBlockChecksumOp(elapsed());
}
+ @Override
+ public void copyBlockCrossNamespace(
+ final ExtendedBlock srcBlock, final Token<BlockTokenIdentifier> srcBlockToken,
+ final ExtendedBlock targetBlock, final Token<BlockTokenIdentifier> targetBlockToken,
+ final DatanodeInfo targetDN) throws IOException {
+ updateCurrentThreadName("Copying block cross namespace " + srcBlock);
+ final DataOutputStream reply = getBufferedOutputStream();
+ try {
+ // Check access
+ checkAccess(reply, true, srcBlock, srcBlockToken,
+ Op.COPY_BLOCK_CROSS_NAMESPACE, BlockTokenIdentifier.AccessMode.COPY);
Review Comment:
If we use namenode.getBlockLocations() , the blockToken AccessMode is BlockTokenIdentifier.AccessMode.READ. I also test in a real hdfs cluster with FastCopy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-16757. Add a new method copyBlockCrossNamespace to DataNode [hadoop]
Posted by "LiuGuH (via GitHub)" <gi...@apache.org>.
LiuGuH commented on code in PR #5719:
URL: https://github.com/apache/hadoop/pull/5719#discussion_r1627144059
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java:
##########
@@ -1078,6 +1079,44 @@ public void blockGroupChecksum(final StripedBlockInfo stripedBlockInfo,
datanode.metrics.addBlockChecksumOp(elapsed());
}
+ @Override
+ public void copyBlockCrossNamespace(
+ final ExtendedBlock srcBlock, final Token<BlockTokenIdentifier> srcBlockToken,
+ final ExtendedBlock targetBlock, final Token<BlockTokenIdentifier> targetBlockToken,
+ final DatanodeInfo targetDN) throws IOException {
+ updateCurrentThreadName("Copying block cross namespace " + srcBlock);
+ final DataOutputStream reply = getBufferedOutputStream();
+ try {
+ // Check access
+ checkAccess(reply, true, srcBlock, srcBlockToken,
+ Op.COPY_BLOCK_CROSS_NAMESPACE, BlockTokenIdentifier.AccessMode.COPY);
+ checkAccess(reply, true, targetBlock, targetBlockToken,
+ Op.COPY_BLOCK_CROSS_NAMESPACE, BlockTokenIdentifier.AccessMode.WRITE);
+
+ // Async copy the block
+ Future<?> result = datanode.internalCopyBlockCrossNamespace(srcBlock, targetBlock, targetDN);
+ try {
+ result.get(dnConf.getCopyBlockCrossNamespaceSocketTimeout(), TimeUnit.MILLISECONDS);
Review Comment:
If srcBlock targetBlock not in a same datanode, it will use DataTransfer . And DataTransfer will catch any exception in thread , so result.get() will always can not catch IOException.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5719: HDFS-16757. Add a new method copyBlockCrossNamespace to DataNode
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5719:
URL: https://github.com/apache/hadoop/pull/5719#issuecomment-1578329226
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 4 new or modified test files. |
|||| _ HDFS-2139 Compile Tests _ |
| +0 :ok: | mvndep | 16m 44s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 19m 36s | | HDFS-2139 passed |
| +1 :green_heart: | compile | 5m 13s | | HDFS-2139 passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 5m 10s | | HDFS-2139 passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 24s | | HDFS-2139 passed |
| +1 :green_heart: | mvnsite | 2m 14s | | HDFS-2139 passed |
| +1 :green_heart: | javadoc | 1m 51s | | HDFS-2139 passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 17s | | HDFS-2139 passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 5m 44s | | HDFS-2139 passed |
| +1 :green_heart: | shadedclient | 22m 30s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 51s | | the patch passed |
| +1 :green_heart: | compile | 5m 7s | | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | cc | 5m 7s | | the patch passed |
| +1 :green_heart: | javac | 5m 7s | | the patch passed |
| +1 :green_heart: | compile | 4m 58s | | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | cc | 4m 58s | | the patch passed |
| +1 :green_heart: | javac | 4m 58s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 1m 11s | | hadoop-hdfs-project: The patch generated 0 new + 621 unchanged - 3 fixed = 621 total (was 624) |
| +1 :green_heart: | mvnsite | 1m 56s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 25s | | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 1s | | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 5m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 37s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 24s | | hadoop-hdfs-client in the patch passed. |
| -1 :x: | unit | 238m 24s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5719/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not generate ASF License warnings. |
| | | 372m 15s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.TestRollingUpgrade |
| | hadoop.hdfs.server.datanode.TestDataNodeLifeline |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5719/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5719 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat xmllint |
| uname | Linux dc5c0f5b6215 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | HDFS-2139 / 5d806d0517210fb348d97408c91df044ff46af40 |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5719/2/testReport/ |
| Max. process+thread count | 3163 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5719/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] haiyang1987 commented on pull request #5719: HDFS-16757. Add a new method copyBlockCrossNamespace to DataNode
Posted by "haiyang1987 (via GitHub)" <gi...@apache.org>.
haiyang1987 commented on PR #5719:
URL: https://github.com/apache/hadoop/pull/5719#issuecomment-1578518452
@ZanderXu @ferhui @Hexiaoqiao @ayushtkn please help me review this pr when you are available, Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5719: HDFS-16757. Add a new method copyBlockCrossNamespace to DataNode
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #5719:
URL: https://github.com/apache/hadoop/pull/5719#issuecomment-1576883009
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 4 new or modified test files. |
|||| _ HDFS-2139 Compile Tests _ |
| +0 :ok: | mvndep | 18m 10s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 19m 43s | | HDFS-2139 passed |
| +1 :green_heart: | compile | 5m 19s | | HDFS-2139 passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 5m 5s | | HDFS-2139 passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 24s | | HDFS-2139 passed |
| +1 :green_heart: | mvnsite | 2m 13s | | HDFS-2139 passed |
| +1 :green_heart: | javadoc | 1m 50s | | HDFS-2139 passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 14s | | HDFS-2139 passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 5m 40s | | HDFS-2139 passed |
| +1 :green_heart: | shadedclient | 22m 15s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 50s | | the patch passed |
| +1 :green_heart: | compile | 5m 11s | | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | cc | 5m 11s | | the patch passed |
| +1 :green_heart: | javac | 5m 11s | | the patch passed |
| +1 :green_heart: | compile | 4m 54s | | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | cc | 4m 54s | | the patch passed |
| +1 :green_heart: | javac | 4m 54s | | the patch passed |
| -1 :x: | blanks | 0m 0s | [/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5719/1/artifact/out/blanks-eol.txt) | The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 10s | [/results-checkstyle-hadoop-hdfs-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5719/1/artifact/out/results-checkstyle-hadoop-hdfs-project.txt) | hadoop-hdfs-project: The patch generated 3 new + 621 unchanged - 3 fixed = 624 total (was 624) |
| +1 :green_heart: | mvnsite | 1m 56s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 27s | | the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 3s | | the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 5m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 32s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 22s | | hadoop-hdfs-client in the patch passed. |
| -1 :x: | unit | 237m 51s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5719/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 51s | | The patch does not generate ASF License warnings. |
| | | 373m 13s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.tools.TestHdfsConfigFields |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5719/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5719 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint bufcompat |
| uname | Linux e28c6c8db0a7 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | HDFS-2139 / a03ead02c9ef6be4480d00d39b6d2494261e928a |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5719/1/testReport/ |
| Max. process+thread count | 2825 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5719/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-16757. Add a new method copyBlockCrossNamespace to DataNode [hadoop]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #5719:
URL: https://github.com/apache/hadoop/pull/5719#issuecomment-3419090895
We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-16757. Add a new method copyBlockCrossNamespace to DataNode [hadoop]
Posted by "LiuGuH (via GitHub)" <gi...@apache.org>.
LiuGuH commented on PR #5719:
URL: https://github.com/apache/hadoop/pull/5719#issuecomment-2147469940
Hello ,sirs . For some reason, I now work with fastcopy and current progerss is that I now already finished fastcopy tools in standalone mode. Both replicated files and EC files are supported. And next is trying to finish distcp with fastcopy.
@haiyang1987 , are you still work on this ? If not ,I hope to finish the rest of the work. Thanks you.
And @ZanderXu @ayushtkn @Hexiaoqiao Sirs, can we continue to push this mission forward? FastCopy is more efficient
for data replication in a single hdfs cluster or a federation hdfs cluter with more nameservices. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-16757. Add a new method copyBlockCrossNamespace to DataNode [hadoop]
Posted by "haiyang1987 (via GitHub)" <gi...@apache.org>.
haiyang1987 commented on PR #5719:
URL: https://github.com/apache/hadoop/pull/5719#issuecomment-2149138284
> Hello ,sirs . For some reason, I now work with fastcopy and current progerss is that I now already finished fastcopy tools in standalone mode. Both replicated files and EC files are supported. And next is trying to finish distcp with fastcopy. @haiyang1987 , are you still work on this ? If not ,I hope to finish the rest of the work. Thanks you. And @ZanderXu @ayushtkn @Hexiaoqiao Sirs, can we continue to push this mission forward? FastCopy is more efficient for data replication in a single hdfs cluster or a federation hdfs cluter with more nameservices. Thanks!
Thanks @LiuGuH for your comment. I will check it based on your suggestion later.
Of course, you can finish to work other subtasks. Fast Copy is really a good feature, hope we continue to push this. thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] haiyang1987 commented on pull request #5719: HDFS-16757. Add a new method copyBlockCrossNamespace to DataNode
Posted by "haiyang1987 (via GitHub)" <gi...@apache.org>.
haiyang1987 commented on PR #5719:
URL: https://github.com/apache/hadoop/pull/5719#issuecomment-1578512841
The failed unit test seems unrelated to the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-16757. Add a new method copyBlockCrossNamespace to DataNode [hadoop]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #5719: HDFS-16757. Add a new method copyBlockCrossNamespace to DataNode
URL: https://github.com/apache/hadoop/pull/5719
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org