[GitHub] [spark] HyukjinKwon commented on a change in pull request #30012: [SPARK-XXX][INFRA] Rebalance GitHub Action jobs

[GitBox <gi...@apache.org>]

HyukjinKwon commented on a change in pull request #30012:
URL: https://github.com/apache/spark/pull/30012#discussion_r503097144



##########
File path: .github/workflows/build_and_test.yml
##########
@@ -42,9 +42,11 @@ jobs:
             mllib-local, mllib,
             yarn, mesos, kubernetes, hadoop-cloud, spark-ganglia-lgpl
           - >-
-            pyspark-sql, pyspark-mllib, pyspark-resource
+            pyspark-sql, pyspark-resource
           - >-
-            pyspark-core, pyspark-streaming, pyspark-ml
+            pyspark-core, pyspark-streaming
+          - >-
+            pyspark-ml, pyspark-mllib

Review comment:
       @dongjoon-hyun, BTW do you have a GitHub Action workflow example of " The total execution time of two Python GitHub Action jobs is over 3 hours."? I just took a quick look and seems it's usually 1 hour 30 mins. For example, it has a change in core which probably runs most of tests: https://github.com/apache/spark/runs/1236699345




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org