[jira] [Created] (SPARK-32244) Build and run the Spark with test cases in Github Actions

["Hyukjin Kwon (Jira)" <ji...@apache.org>]

Hyukjin Kwon created SPARK-32244:
------------------------------------

             Summary: Build and run the Spark with test cases in Github Actions
                 Key: SPARK-32244
                 URL: https://issues.apache.org/jira/browse/SPARK-32244
             Project: Spark
          Issue Type: Umbrella
          Components: Project Infra
    Affects Versions: 3.0.0, 2.4.6, 3.1.0
            Reporter: Hyukjin Kwon


Last week and onwards, the Jenkins machines became very unstable for some reasons. 
  - Apparently, the machines became extremely slow. Almost all tests can't pass.
  - One machine (worker 4) started to have the corrupt .m2 which fails the build.
  - Documentation build fails time to time for an unknown reason in Jenkins machine specifically.

Almost all PRs are basically blocked by this instability currently.

This JIRA aims to run the tests in Github Actions.
  - To avoid depending on few persons who can access to the cluster. 
  - To reduce the elapsed time in the build - we could split the tests (e.g., SQL, ML, CORE), and run them in parallel so the total build time will significantly reduce.
  - To control the environment more flexibly. Currently, we should keep asking Shane to install or control something.
  - Other contributors can test and propose to fix Github Actions configurations so we can distribute this build management cost.





--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org