You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by GitBox <gi...@apache.org> on 2021/03/15 13:51:10 UTC

[GitHub] [bigtop] iwasakims opened a new pull request #757: BIGTOP-3521. Drop QFS.

iwasakims opened a new pull request #757:
URL: https://github.com/apache/bigtop/pull/757


   https://issues.apache.org/jira/browse/BIGTOP-3521


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [bigtop] sekikn commented on pull request #757: BIGTOP-3521. Drop QFS.

Posted by GitBox <gi...@apache.org>.
sekikn commented on pull request #757:
URL: https://github.com/apache/bigtop/pull/757#issuecomment-801164984


   Thank you for updating the PR @iwasakims!
   I've just noticed there's still QFS-specific logic in bigtop-tests/test-artifacts/hadoop/src/main/groovy/org/apache/bigtop/itest/hadoop/mapreduce/TestHadoopExamples.groovy.
   Would you fix that too? Sorry for missing that in the first review.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [bigtop] iwasakims commented on pull request #757: BIGTOP-3521. Drop QFS.

Posted by GitBox <gi...@apache.org>.
iwasakims commented on pull request #757:
URL: https://github.com/apache/bigtop/pull/757#issuecomment-801192662


   I updated the TestHadoopExamples.groovy and ran the smoke-tests again.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [bigtop] iwasakims commented on a change in pull request #757: BIGTOP-3521. Drop QFS.

Posted by GitBox <gi...@apache.org>.
iwasakims commented on a change in pull request #757:
URL: https://github.com/apache/bigtop/pull/757#discussion_r596958700



##########
File path: bigtop-tests/test-artifacts/hadoop/src/main/groovy/org/apache/bigtop/itest/hadoop/mapreduce/TestHadoopExamples.groovy
##########
@@ -152,9 +143,7 @@ class TestHadoopExamples {
   public static LinkedHashMap<String, Object[]> generateTests() {
     LinkedHashMap<String, Object[]> res = [:];
     examples.each { k, v -> res[k] = [k.toString(), v.toString()] as Object[]; }
-    if (HADOOP_COMMAND != "hadoop-qfs") {
-      additional_examples.each { k, v -> res[k] = [k.toString(), v.toString()] as Object[]; }
-    }
+    additional_examples.each { k, v -> res[k] = [k.toString(), v.toString()] as Object[]; }

Review comment:
       Sure. I merged the maps.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [bigtop] sekikn merged pull request #757: BIGTOP-3521. Drop QFS.

Posted by GitBox <gi...@apache.org>.
sekikn merged pull request #757:
URL: https://github.com/apache/bigtop/pull/757


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [bigtop] sekikn commented on pull request #757: BIGTOP-3521. Drop QFS.

Posted by GitBox <gi...@apache.org>.
sekikn commented on pull request #757:
URL: https://github.com/apache/bigtop/pull/757#issuecomment-801054644


   I think we can also remove QFS-specific logic in provisioner/utils/smoke-tests.sh.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [bigtop] sekikn commented on pull request #757: BIGTOP-3521. Drop QFS.

Posted by GitBox <gi...@apache.org>.
sekikn commented on pull request #757:
URL: https://github.com/apache/bigtop/pull/757#issuecomment-802402552


   +1, I also confirmed that YARN and MR smoke tests passed on CentOS 7. Thanks @iwasakims!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [bigtop] iwasakims commented on pull request #757: BIGTOP-3521. Drop QFS.

Posted by GitBox <gi...@apache.org>.
iwasakims commented on pull request #757:
URL: https://github.com/apache/bigtop/pull/757#issuecomment-801147130


   Thanks, @sekikn. I updated the smoke-tests.sh and groovy scripts of mapreduce smoke-tests.
   
   I tested the change by running mapreduce smoke-tests via docker-hadoop.sh.
   ```
   ./docker-hadoop.sh \
    --create 3 \
    --image bigtop/puppet:trunk-centos-8 \
    --memory 8g \
    --repo file:///bigtop-home/output \
    --disable-gpg-check \
    --stack hdfs,yarn,mapreduce \
    --smoke-tests hdfs,yarn,mapreduce
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [bigtop] sekikn commented on a change in pull request #757: BIGTOP-3521. Drop QFS.

Posted by GitBox <gi...@apache.org>.
sekikn commented on a change in pull request #757:
URL: https://github.com/apache/bigtop/pull/757#discussion_r596451003



##########
File path: bigtop-tests/test-artifacts/hadoop/src/main/groovy/org/apache/bigtop/itest/hadoop/mapreduce/TestHadoopExamples.groovy
##########
@@ -152,9 +143,7 @@ class TestHadoopExamples {
   public static LinkedHashMap<String, Object[]> generateTests() {
     LinkedHashMap<String, Object[]> res = [:];
     examples.each { k, v -> res[k] = [k.toString(), v.toString()] as Object[]; }
-    if (HADOOP_COMMAND != "hadoop-qfs") {
-      additional_examples.each { k, v -> res[k] = [k.toString(), v.toString()] as Object[]; }
-    }
+    additional_examples.each { k, v -> res[k] = [k.toString(), v.toString()] as Object[]; }

Review comment:
       We can merge `additional_examples` into `examples`, because the former was separated from the latter so as to addressing QFS test failure originally. See the comment at line 129.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org