You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by mm...@apache.org on 2026/06/17 23:44:45 UTC

(pulsar) branch master updated: [improve][build] Run checkstyle/spotless without compiling and add quickCheck/sanityCheck tasks (#26050)

This is an automated email from the ASF dual-hosted git repository.

merlimat pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
     new 44173c6ae70 [improve][build] Run checkstyle/spotless without compiling and add quickCheck/sanityCheck tasks (#26050)
44173c6ae70 is described below

commit 44173c6ae709298b354c98f13e8b0d0006a031a8
Author: Lari Hotari <lh...@users.noreply.github.com>
AuthorDate: Thu Jun 18 02:44:38 2026 +0300

    [improve][build] Run checkstyle/spotless without compiling and add quickCheck/sanityCheck tasks (#26050)
---
 AGENTS.md                                          |  4 ++
 ARCHITECTURE.md                                    |  4 +-
 CODING.md                                          |  4 ++
 CONTRIBUTING.md                                    | 11 +++--
 README.md                                          |  2 +
 .../pulsar.code-quality-conventions.gradle.kts     | 46 +++++++++++++++---
 build.gradle.kts                                   | 56 ++++++++++++++++++++++
 pulsar-client-tools-test/build.gradle.kts          |  7 ---
 8 files changed, 115 insertions(+), 19 deletions(-)

diff --git a/AGENTS.md b/AGENTS.md
index 40aa15f98ae..f9b383dece3 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -58,6 +58,10 @@ A few rules matter specifically when an AI tool makes the change, on top of the
   platform-dependent). See [`CODING.md`](CODING.md#reproducing-concurrency--memory-visibility-bugs).
 - **Follow Java style guidance.** For Java conventions, including imports over fully qualified class
   names, follow [`CODING.md`](CODING.md#style).
+- **Check before claiming conformance.** Run `./gradlew quickCheck` for a fast source-only pass
+  (license headers + checkstyle, no compilation) or `./gradlew sanityCheck` to also
+  compile every module's main and test sources; neither builds shadow jars. See
+  [`CONTRIBUTING.md`](CONTRIBUTING.md#building).
 
 ## Critical rules
 
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
index f4c9ae75a97..cc064d1e91a 100644
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -146,5 +146,5 @@ When editing `build-logic/`, `settings.gradle.kts`, a module `build.gradle.kts`,
   which is plain text and cheaper to parse than the rendered HTML.
 
 Before finishing a build change, confirm the affected task and `./gradlew help` run clean with
-`--configuration-cache`, and that `assemble` and `rat spotlessCheck checkstyleMain checkstyleTest` pass
-(plus `checkBinaryLicense` if a dependency changed).
+`--configuration-cache`, and that `assemble` and `quickCheck` (license headers + checkstyle across
+all modules) pass (plus `checkBinaryLicense` if a dependency changed).
diff --git a/CODING.md b/CODING.md
index af05622f592..892b5a05168 100644
--- a/CODING.md
+++ b/CODING.md
@@ -13,6 +13,10 @@ for the agent-specific guardrails on top of it.
 - No `@author` tags in Javadoc.
 - Every `TODO` must reference a GitHub issue, e.g. `// TODO: https://github.com/apache/pulsar/issues/XXXX`.
 - Checkstyle config: `buildtools/src/main/resources/pulsar/checkstyle.xml`. Lombok is enabled.
+  Spotless enforces license headers only (no code formatting); checkstyle covers style/whitespace.
+  Both run on sources only (no compilation): `./gradlew quickCheck` runs the license-header and
+  checkstyle checks across all modules, and `./gradlew sanityCheck` also compiles main + test — see
+  [`CONTRIBUTING.md`](CONTRIBUTING.md#building).
 - Prefer imports over fully qualified class names in code. Use a fully qualified class name only when
   needed to disambiguate a name collision that imports cannot resolve.
 
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index db70206a1f7..88b0cfa8447 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -40,9 +40,14 @@ bypasses the check); `zip` is also needed. Use the bundled wrapper `./gradlew` (
 ./gradlew assemble
 ./gradlew :pulsar-broker:assemble
 
-# Lint / verify (license headers, formatting, checkstyle) — run before pushing
-./gradlew rat spotlessCheck checkstyleMain checkstyleTest
-./gradlew spotlessApply            # auto-fix license headers/formatting
+# Source-code conformance across all modules — license headers (rat + spotlessCheck) and checkstyle
+# (checkstyleMain + checkstyleTest). Fast: compiles nothing, never builds shadow jars.
+./gradlew quickCheck
+./gradlew spotlessApply            # auto-fix license headers
+
+# Pre-PR gate — quickCheck plus compiling main + test sources of every module. Modules that depend
+# on shaded artifacts on their compile classpath are skipped for compilation so no shadow jar is built.
+./gradlew sanityCheck
 
 # Verify bundled-dependency LICENSE/NOTICE coverage (after changing a runtime dependency)
 ./gradlew checkBinaryLicense
diff --git a/README.md b/README.md
index bff6e32ecd3..4f2b142d1ee 100644
--- a/README.md
+++ b/README.md
@@ -195,6 +195,8 @@ There is also a guide for [setting up the tooling for building Pulsar](https://p
 ./gradlew assemble                                                       # compile and assemble
 ./gradlew :pulsar-client-original:test --tests "ConsumerBuilderImplTest" # run a single test
 bin/pulsar standalone                                                    # run a standalone service
+./gradlew quickCheck                                                     # license headers + checkstyle, no compile
+./gradlew sanityCheck                                                    # quickCheck + compile main/test (pre-PR)
 ```
 
 For the full build, lint, test, and PR workflow — test groups, integration tests, Personal CI, and PR
diff --git a/build-logic/conventions/src/main/kotlin/pulsar.code-quality-conventions.gradle.kts b/build-logic/conventions/src/main/kotlin/pulsar.code-quality-conventions.gradle.kts
index de8caaaff81..f7efcec2354 100644
--- a/build-logic/conventions/src/main/kotlin/pulsar.code-quality-conventions.gradle.kts
+++ b/build-logic/conventions/src/main/kotlin/pulsar.code-quality-conventions.gradle.kts
@@ -34,6 +34,13 @@ configure<CheckstyleExtension> {
 }
 
 tasks.withType<Checkstyle>().configureEach {
+    // Checkstyle here analyzes source only; none of the configured checks need type
+    // resolution from compiled bytecode. By default Gradle wires the Checkstyle task
+    // classpath to `sourceSet.output + sourceSet.compileClasspath`, which forces this
+    // module (and every project dependency, including slow shaded/shadow jars) to be
+    // compiled and assembled just to run checkstyle. Clearing it lets checkstyle run
+    // directly on the sources without compiling anything.
+    classpath = files()
     // Broker module has very large files that need more heap
     maxHeapSize.set("1g")
     // Exclude generated source files (proto, lightproto, etc.)
@@ -44,6 +51,30 @@ tasks.withType<Checkstyle>().configureEach {
     exclude("**/proto/*")
 }
 
+// Markers identifying generated source roots (protobuf/lightproto/avro output, all under build/).
+// Files in these roots are already excluded from analysis above.
+val generatedSourceRootMarkers = listOf("/build/", "/generated-lightproto/", "/generated-sources/")
+
+// Run checkstyle straight from the hand-written source roots. Gradle wires the Checkstyle task's
+// source to `sourceSet.allJava`, which includes the generated-source roots registered by the
+// protobuf/lightproto/avro plugins. That makes checkstyle depend on those generator tasks, which in
+// turn resolve proto classpaths that compile dependency projects (and build shaded jars) just to run
+// checkstyle. Since the generated files are excluded from analysis anyway, restrict the source roots
+// to the non-generated ones so checkstyle depends on nothing but the sources it actually checks.
+plugins.withType<JavaBasePlugin>().configureEach {
+    the<SourceSetContainer>().configureEach {
+        val sourceSet = this
+        tasks.matching { it.name == sourceSet.getTaskName("checkstyle", null) }
+            .withType<Checkstyle>()
+            .configureEach {
+                val staticRoots = sourceSet.java.srcDirs.filter { dir ->
+                    generatedSourceRootMarkers.none { dir.invariantSeparatorsPath.contains(it) }
+                }
+                setSource(files(staticRoots).asFileTree.matching { include("**/*.java") })
+            }
+    }
+}
+
 // ── License header check (Spotless) ────────────────────────────────────────
 val asfLicenseHeader = rootProject.file("src/license-header.txt").readText()
 val asfLicenseHeaderJava = "/*\n" + asfLicenseHeader.lines()
@@ -52,13 +83,14 @@ val asfLicenseHeaderJava = "/*\n" + asfLicenseHeader.lines()
 val asfLicenseHeaderJavadoc = asfLicenseHeaderJava.replaceFirst("/*", "/**")
 configure<com.diffplug.gradle.spotless.SpotlessExtension> {
     java {
-        targetExclude(
-            "**/AbstractCASReferenceCounted.java",
-            "**/generated/**",
-            "**/generated-lightproto/**",
-            "**/generated-sources/**",
-            "build/**",
-        )
+        // Only the hand-written main and test sources. Spotless's default Java target is every source
+        // set's `allJava`, which includes the generated-proto roots (protobuf/lightproto/avro) and so
+        // makes spotlessJava depend on those generators — dragging proto generation and dependency
+        // compilation into what should be a source-only check. Targeting the static main/test roots
+        // keeps spotless (and the quickCheck/sanityCheck aggregates) from compiling anything. Sources
+        // outside src/main/java and src/test/java are still license-checked by the Apache RAT (`rat`) task.
+        target("src/main/java/**/*.java", "src/test/java/**/*.java")
+        targetExclude("**/AbstractCASReferenceCounted.java")
         licenseHeader(asfLicenseHeaderJava, "(\\n|package|import|public|class|module) ?")
     }
 
diff --git a/build.gradle.kts b/build.gradle.kts
index 9f4c7770ce7..55e0210f4b2 100644
--- a/build.gradle.kts
+++ b/build.gradle.kts
@@ -17,6 +17,7 @@
  * under the License.
  */
 import com.github.vlsi.gradle.git.dsl.gitignore
+import org.gradle.api.artifacts.ProjectDependency
 import org.jetbrains.gradle.ext.copyright
 import org.jetbrains.gradle.ext.settings
 
@@ -108,6 +109,61 @@ tasks.register("docker") {
     dependsOn(":docker:pulsar-docker-image:dockerBuild")
 }
 
+// ── Aggregate verification tasks (quickCheck / sanityCheck) ───────────────────
+// quickCheck   — fast, source-only conformance across every module: checkstyle + spotless + Apache
+//                RAT. Compiles nothing, so it never builds shadow jars.
+// sanityCheck  — pre-PR gate: quickCheck plus compiling main + test sources of every module that
+//                does NOT pull a shaded artifact onto its compile classpath. Compiling such a module
+//                would build a (slow) shadow jar, so those modules are skipped for compilation; their
+//                sources are still covered by the checkstyle/spotless part of quickCheck.
+//
+// Compiling a module builds a shadow jar only when it declares a shaded module (one applying the
+// shadow plugin) on its compile or test-compile classpath — compilation then needs that dependency's
+// relocated jar. The shade convention plugins (pulsar.shadow-conventions, pulsar.client-shade-conventions)
+// all apply `com.gradleup.shadow`, so the plugin id is the reliable marker for a shaded module.
+// A shaded module's OWN compileJava/compileTestJava does NOT build its shadow jar, so the shaded and
+// *-minimized modules are themselves compiled here; only their shadow-consuming dependents are skipped.
+val shadowPluginId = "com.gradleup.shadow"
+val compileScopeConfigurations = setOf(
+    "api", "implementation", "compileOnly", "testImplementation", "testCompileOnly",
+)
+fun Project.dependsOnShadowJar(): Boolean =
+    configurations
+        .filter { it.name in compileScopeConfigurations }
+        .any { configuration ->
+            configuration.dependencies.withType(ProjectDependency::class.java).any { dependency ->
+                rootProject.project(dependency.path).plugins.hasPlugin(shadowPluginId)
+            }
+        }
+
+// The cross-project wiring is deferred to `provider {}` so it only runs when these aggregates are
+// actually in the task graph, keeping configuration-on-demand intact for every other build.
+tasks.register("quickCheck") {
+    group = "verification"
+    description = "Fast source-code conformance check across all modules (checkstyle + spotless + " +
+        "Apache RAT). Compiles nothing and never builds shadow jars."
+    dependsOn("rat")
+    dependsOn(provider {
+        subprojects.flatMap { sub ->
+            listOf("checkstyleMain", "checkstyleTest", "spotlessCheck").mapNotNull { sub.tasks.findByName(it) }
+        }
+    })
+}
+
+tasks.register("sanityCheck") {
+    group = "verification"
+    description = "Pre-PR check: quickCheck plus compiling main + test sources of every module. " +
+        "Modules that depend on shaded artifacts are skipped for compilation so no shadow jar is built."
+    dependsOn("quickCheck")
+    dependsOn(provider {
+        subprojects
+            .filter { it.plugins.hasPlugin("java") && !it.dependsOnShadowJar() }
+            .flatMap { sub ->
+                listOf("compileJava", "compileTestJava").mapNotNull { sub.tasks.findByName(it) }
+            }
+    })
+}
+
 // ── Parent POM publication ──────────────────────────────────────────────────
 // Publishes org.apache.pulsar:pulsar as a POM-only parent artifact.
 // Child modules reference this via <parent> in their POMs, inheriting
diff --git a/pulsar-client-tools-test/build.gradle.kts b/pulsar-client-tools-test/build.gradle.kts
index 87365636aef..e89c505e2bd 100644
--- a/pulsar-client-tools-test/build.gradle.kts
+++ b/pulsar-client-tools-test/build.gradle.kts
@@ -51,10 +51,3 @@ val copyCustomCommandsNar by tasks.registering(Copy::class) {
 tasks.withType<Test> {
     dependsOn(copyCustomCommandsNar)
 }
-
-// checkstyleTest also scans test resources — ensure NAR copy runs first
-plugins.withId("checkstyle") {
-    tasks.named("checkstyleTest") {
-        dependsOn(copyCustomCommandsNar)
-    }
-}