(spark) branch branch-4.x updated: [SPARK-57254][INFRA] Put CI-unrelated files in a module so CI won't be triggered

[ga...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gaogaotiantian pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new b52e114df4e2 [SPARK-57254][INFRA] Put CI-unrelated files in a module so CI won't be triggered
b52e114df4e2 is described below

commit b52e114df4e271df0d46d18b2ad0dbd23035092d
Author: Tian Gao <ga...@hotmail.com>
AuthorDate: Mon Jun 8 13:48:03 2026 -0700

    [SPARK-57254][INFRA] Put CI-unrelated files in a module so CI won't be triggered
    
    ### What changes were proposed in this pull request?
    
    Put all the CI-unrelated files in a module called `dev-tools` so they are correctly categorized - then they won't be considered part of "root" module and trigger a full CI.
    
    Notice that `lint` workflow is not impacted because it does not check changed files.
    
    ### Why are the changes needed?
    
    To reduce CI usage. According to Claude Code estimation, we could skip about 70 commits in the past year (2% of all commits).
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    CI.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #56312 from gaogaotiantian/ignore-some-files.
    
    Authored-by: Tian Gao <ga...@hotmail.com>
    Signed-off-by: Tian Gao <ga...@hotmail.com>
    (cherry picked from commit 952a283d39a6c90cce20f33812a0b77b5c18c239)
    Signed-off-by: Tian Gao <ga...@hotmail.com>
---
 dev/sparktestsupport/modules.py | 34 ++++++++++++++++++++++++++++++++++
 dev/sparktestsupport/utils.py   | 17 +----------------
 2 files changed, 35 insertions(+), 16 deletions(-)

diff --git a/dev/sparktestsupport/modules.py b/dev/sparktestsupport/modules.py
index f54b1cded011..ef5859bbfe43 100644
--- a/dev/sparktestsupport/modules.py
+++ b/dev/sparktestsupport/modules.py
@@ -1682,6 +1682,40 @@ docker_integration_tests = Module(
     test_tags=["org.apache.spark.tags.DockerTest"],
 )
 
+
+# dev_tools is a pseudo module that contains all the dev related files that
+# won't impact the CI build and tests (except for CI which is forced to
+# run anyway).
+# This module is created so modifying files in this module won't trigger any
+# tests to run.
+dev_tools = Module(
+    name="dev-tools",
+    dependencies=[],
+    source_file_regexes=[
+        ".*README.md",
+        ".*AGENTS.md",
+        r".*\.gitignore",
+        "CONTRIBUTING.md",
+        ".asf.yaml",
+        "SECURITY.md",
+        "NOTICE-binary",
+        "LICENSE-binary",
+        "ui-test/package.json",
+        "ui-test/package-lock.json",
+        "scalastyle-config.xml",
+        "dev/checkstyle.xml",
+        "dev/checkstyle-suppressions.xml",
+        "dev/spark-test-image/lint/Dockerfile",
+        "dev/lint-python",
+        "dev/lint-scala",
+        "dev/reformat-python",
+        "dev/structured_logging_style.py",
+        "dev/merge_spark_pr.py",
+        "dev/create_spark_jira.py",
+        "dev/create-release/",
+    ],
+)
+
 # The root module is a dummy module which is used to run all of the tests.
 # No other modules should directly depend on this module.
 root = Module(
diff --git a/dev/sparktestsupport/utils.py b/dev/sparktestsupport/utils.py
index b969b96a16c5..fe795303e63f 100755
--- a/dev/sparktestsupport/utils.py
+++ b/dev/sparktestsupport/utils.py
@@ -33,30 +33,15 @@ def determine_modules_for_files(filenames):
     """
     Given a list of filenames, return the set of modules that contain those files.
     If a file is not associated with a more specific submodule, then this method will consider that
-    file to belong to the 'root' module. `.github` directory is counted only in GitHub Actions,
-    and `README.md`, `AGENTS.md`, `CONTRIBUTING.md` are always ignored.
+    file to belong to the 'root' module. `.github` directory is counted only in GitHub Actions.
 
     >>> sorted(x.name for x in determine_modules_for_files(["python/pyspark/a.py", "sql/core/foo"]))
     ['pyspark-core', 'pyspark-errors', 'sql']
     >>> [x.name for x in determine_modules_for_files(["file_not_matched_by_any_subproject"])]
     ['root']
-    >>> [x.name for x in determine_modules_for_files(["sql/README.md"])]
-    []
-    >>> [x.name for x in determine_modules_for_files(["AGENTS.md"])]
-    []
-    >>> [x.name for x in determine_modules_for_files(["CONTRIBUTING.md"])]
-    []
     """
     changed_modules = set()
     for filename in filenames:
-        if filename.endswith(("README.md", "AGENTS.md", "CONTRIBUTING.md")):
-            continue
-        if filename in (
-            "scalastyle-config.xml",
-            "dev/checkstyle.xml",
-            "dev/checkstyle-suppressions.xml",
-        ):
-            continue
         if ("GITHUB_ACTIONS" not in os.environ) and filename.startswith(".github"):
             continue
         matched_at_least_one_module = False


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org