You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/12/29 02:54:33 UTC

[GitHub] [spark] Yikun opened a new pull request #35049: [SPARK-37757][BUILD] Enable Spark test scheduled job on ARM runner

Yikun opened a new pull request #35049:
URL: https://github.com/apache/spark/pull/35049


   ### What changes were proposed in this pull request?
   This patch adds the scheduled job on ARM runner.
   
   
   ### Why are the changes needed?
   Migrate Spark Arm Job from Jenkins to GitHub Actions.
   
   
   ### Does this PR introduce _any_ user-facing change?
   NO
   
   
   ### How was this patch tested?
   Trigger test in the repo: https://github.com/Yikun/spark/pull/47
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] Yikun commented on pull request #35049: [SPARK-37757][BUILD] Enable Spark test scheduled job on ARM runner

Posted by GitBox <gi...@apache.org>.
Yikun commented on pull request #35049:
URL: https://github.com/apache/spark/pull/35049#issuecomment-1003472856


   @srowen Thanks for review. And I am going to add self hosted runner to `apache/spark` first and then mark this as ready for review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] srowen commented on pull request #35049: [SPARK-37757][BUILD] Enable Spark test scheduled job on ARM runner

Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #35049:
URL: https://github.com/apache/spark/pull/35049#issuecomment-1003410130


   Looks good if it works


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] Yikun edited a comment on pull request #35049: [SPARK-37757][BUILD] Enable Spark test scheduled job on ARM runner

Posted by GitBox <gi...@apache.org>.
Yikun edited a comment on pull request #35049:
URL: https://github.com/apache/spark/pull/35049#issuecomment-1003472856


   @srowen Thanks for review. And I am going to add self hosted runner to `apache/spark` first and then mark this as ready for review. Here is the arm64 result: https://github.com/Yikun/spark/pull/51 .


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] martin-g commented on a change in pull request #35049: [SPARK-37757][BUILD] Enable Spark test scheduled job on ARM runner

Posted by GitBox <gi...@apache.org>.
martin-g commented on a change in pull request #35049:
URL: https://github.com/apache/spark/pull/35049#discussion_r800401719



##########
File path: .github/workflows/build_and_test.yml
##########
@@ -205,14 +217,17 @@ jobs:
           ${{ matrix.java }}-${{ matrix.hadoop }}-coursier-
     - name: Install Java ${{ matrix.java }}
       uses: actions/setup-java@v1
+      # actions/setup-java doesn'ts suppport arm64, it had been installed in self-hosted runner.
+      if: needs.configure-jobs.outputs.type != 'spark-arm64-scheduled'
       with:
         java-version: ${{ matrix.java }}
     - name: Install Python 3.8
       uses: actions/setup-python@v2
-      # We should install one Python that is higher then 3+ for SQL and Yarn because:
-      # - SQL component also has Python related tests, for example, IntegratedUDFTestUtils.
-      # - Yarn has a Python specific test too, for example, YarnClusterSuite.
-      if: contains(matrix.modules, 'yarn') || (contains(matrix.modules, 'sql') && !contains(matrix.modules, 'sql-'))
+      # * We should install one Python that is higher then 3+ for SQL and Yarn because:

Review comment:
       s/then/than/

##########
File path: .github/workflows/build_and_test.yml
##########
@@ -205,14 +217,17 @@ jobs:
           ${{ matrix.java }}-${{ matrix.hadoop }}-coursier-
     - name: Install Java ${{ matrix.java }}
       uses: actions/setup-java@v1
+      # actions/setup-java doesn'ts suppport arm64, it had been installed in self-hosted runner.
+      if: needs.configure-jobs.outputs.type != 'spark-arm64-scheduled'
       with:
         java-version: ${{ matrix.java }}
     - name: Install Python 3.8
       uses: actions/setup-python@v2
-      # We should install one Python that is higher then 3+ for SQL and Yarn because:
-      # - SQL component also has Python related tests, for example, IntegratedUDFTestUtils.
-      # - Yarn has a Python specific test too, for example, YarnClusterSuite.
-      if: contains(matrix.modules, 'yarn') || (contains(matrix.modules, 'sql') && !contains(matrix.modules, 'sql-'))
+      # * We should install one Python that is higher then 3+ for SQL and Yarn because:
+      #  - SQL component also has Python related tests, for example, IntegratedUDFTestUtils.
+      #  - Yarn has a Python specific test too, for example, YarnClusterSuite.
+      # * actions/setup-python doesn't support arm64, it had been installed in self-hosted runner.

Review comment:
       ```suggestion
         # * actions/setup-python doesn't support arm64, Python 3.8 is pre-installed in the self-hosted runner.
   ```

##########
File path: .github/workflows/build_and_test.yml
##########
@@ -205,14 +217,17 @@ jobs:
           ${{ matrix.java }}-${{ matrix.hadoop }}-coursier-
     - name: Install Java ${{ matrix.java }}
       uses: actions/setup-java@v1
+      # actions/setup-java doesn'ts suppport arm64, it had been installed in self-hosted runner.

Review comment:
       ```suggestion
         # actions/setup-java doesn'ts suppport arm64, JDK 8 is pre-installed in the self-hosted runner.
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org