You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by st...@apache.org on 2020/05/15 12:52:05 UTC
[hadoop] branch branch-3.0 updated (a55f96c -> cb94911)
This is an automated email from the ASF dual-hosted git repository.
stevel pushed a change to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/hadoop.git.
from a55f96c HADOOP-16840. AliyunOSS: getFileStatus throws FileNotFoundException in versioning bucket. Contributed by wujinhu.
new 31c1a24 Revert "HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default)."
new cb94911 Revert "HADOOP-8143. Change distcp to have -pb on by default."
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../src/main/java/org/apache/hadoop/tools/OptionsParser.java | 3 ---
hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm | 2 +-
.../src/test/java/org/apache/hadoop/tools/TestOptionsParser.java | 2 +-
3 files changed, 2 insertions(+), 5 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 01/02: Revert "HADOOP-14557. Document HADOOP-8143 (Change
distcp to have -pb on by default)."
Posted by st...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
stevel pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 31c1a244263cb44a63ad1644d898c4e8327a1488
Author: Steve Loughran <st...@cloudera.com>
AuthorDate: Thu May 14 16:13:41 2020 +0100
Revert "HADOOP-14557. Document HADOOP-8143 (Change distcp to have -pb on by default)."
This reverts commit 44350fdf495f5cf1bb15b1fe6f6e9587d3de0a59.
It is related to the rollback of HADOOP-8143.
Change-Id: If48e3dd670c920ada702dc36461ff398fe9d35cc
---
hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm b/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
index b5062ba..788700d 100644
--- a/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
+++ b/hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
@@ -217,7 +217,7 @@ Command Line Options
| Flag | Description | Notes |
| ----------------- | ------------------------------------ | -------- |
-| `-p[rbugpcaxt]` | Preserve r: replication number b: block size u: user g: group p: permission c: checksum-type a: ACL x: XAttr t: timestamp | When `-update` is specified, status updates will **not** be synchronized unless the file sizes also differ (i.e. unless the file is re-created). If -pa is specified, DistCp preserves the permissions also because ACLs are a super-set of permissions. The option -pr is only valid if both source and target directory are not erasure coded. **Note:** I [...]
+| `-p[rbugpcaxt]` | Preserve r: replication number b: block size u: user g: group p: permission c: checksum-type a: ACL x: XAttr t: timestamp | When `-update` is specified, status updates will **not** be synchronized unless the file sizes also differ (i.e. unless the file is re-created). If -pa is specified, DistCp preserves the permissions also because ACLs are a super-set of permissions. The option -pr is only valid if both source and target directory are not erasure coded. |
| `-i` | Ignore failures | As explained in the Appendix, this option will keep more accurate statistics about the copy than the default case. It also preserves logs from failed copies, which can be valuable for debugging. Finally, a failing map will not cause the job to fail before all splits are attempted. |
| `-log <logdir>` | Write logs to \<logdir\> | DistCp keeps logs of each file it attempts to copy as map output. If a map fails, the log output will not be retained if it is re-executed. |
| `-v` | Log additional info (path, size) in the SKIP/COPY log | This option can only be used with -log option. |
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 02/02: Revert "HADOOP-8143. Change distcp to have -pb on
by default."
Posted by st...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
stevel pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit cb94911252dc9abb29b3e8efa9c95d04ca8603e6
Author: Steve Loughran <st...@cloudera.com>
AuthorDate: Thu May 14 16:06:24 2020 +0100
Revert "HADOOP-8143. Change distcp to have -pb on by default."
This reverts commit dd65eea74b1f9dde858ff34df8111e5340115511.
Change-Id: I74180cf59d5bbad8c9f66cb331535addcbea863e
---
.../src/main/java/org/apache/hadoop/tools/OptionsParser.java | 3 ---
.../src/test/java/org/apache/hadoop/tools/TestOptionsParser.java | 2 +-
2 files changed, 1 insertion(+), 4 deletions(-)
diff --git a/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java b/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
index 606ed32..267504f 100644
--- a/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
+++ b/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
@@ -189,9 +189,6 @@ public class OptionsParser {
if (command.hasOption(DistCpOptionSwitch.PRESERVE_STATUS.getSwitch())) {
builder.preserve(
getVal(command, DistCpOptionSwitch.PRESERVE_STATUS.getSwitch()));
- } else {
- // No "preserve" settings specified. Preserve block-size.
- builder.preserve(DistCpOptions.FileAttribute.BLOCKSIZE);
}
if (command.hasOption(DistCpOptionSwitch.FILE_LIMIT.getSwitch())) {
diff --git a/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java b/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
index 9361fc1..0a19e0c 100644
--- a/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
+++ b/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
@@ -413,7 +413,7 @@ public class TestOptionsParser {
"-f",
"hdfs://localhost:8020/source/first",
"hdfs://localhost:8020/target/"});
- Assert.assertTrue(options.shouldPreserve(FileAttribute.BLOCKSIZE));
+ Assert.assertFalse(options.shouldPreserve(FileAttribute.BLOCKSIZE));
Assert.assertFalse(options.shouldPreserve(FileAttribute.REPLICATION));
Assert.assertFalse(options.shouldPreserve(FileAttribute.PERMISSION));
Assert.assertFalse(options.shouldPreserve(FileAttribute.USER));
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org