You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2021/07/28 12:00:27 UTC

[GitHub] [hadoop] mehakmeet commented on a change in pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

mehakmeet commented on a change in pull request #3240:
URL: https://github.com/apache/hadoop/pull/3240#discussion_r678208587



##########
File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/contract/AbstractContractDistCpTest.java
##########
@@ -532,13 +549,15 @@ private Path distCpDeepDirectoryStructure(FileSystem srcFS,
    */
   private void largeFiles(FileSystem srcFS, Path srcDir, FileSystem dstFS,
       Path dstDir) throws Exception {
+    int fileSizeKb = conf.getInt(SCALE_TEST_DISTCP_FILE_SIZE_KB,
+        DEFAULT_DISTCP_SIZE_KB);
+    if (fileSizeKb < 1) {
+      skip("File size in " + SCALE_TEST_DISTCP_FILE_SIZE_KB + " too small");

Review comment:
       Nit: maybe, we should say to "make fileSize in `SCALE_TEST_DISTCP_FILE_SIZE_KB` be greater than or equal to 1" or "File Size in `SCALE_TEST_DISTCP_FILE_SIZE_KB` smaller than 1"

##########
File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/contract/AbstractContractDistCpTest.java
##########
@@ -612,6 +634,9 @@ public void testDirectWrite() throws Exception {
 
   @Test
   public void testNonDirectWrite() throws Exception {
+    if (directWriteAlways()) {
+      skip("not needed");

Review comment:
       Nit: maybe move below `describe()`, or mention in skip message what is being skipped. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org