You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@grails.apache.org by "jamesfredley (via GitHub)" <gi...@apache.org> on 2026/05/01 15:10:08 UTC
[PR] Verification harness for rendered guides (refs #354) [grails-static-website]
jamesfredley opened a new pull request, #457:
URL: https://github.com/apache/grails-static-website/pull/457
## Summary
Adds the verification harness for rendered guides as 5 new Gradle tasks plus an aggregate `verifyAllGuides` runner. Defaults to report-only mode; `-PverificationMode=hard-fail` switches to fail-on-violation.
## Tasks added
| Task | Group | Purpose |
|---|---|---|
| `asciidoctorWarningGate` | Migration tasks | Captures stdout/stderr from each `PublishGuide` task to `build/logs/asciidoctor-*.log` and matches against the canonical warning regex (`[WARNING]`, `[ERROR]`, `asciidoctor: WARN/ERROR/FAIL`, JVM exception/stack traces, `Caused by:`) with an explicit allowlist hook. |
| `crawlBuiltGuides` | Migration tasks | Walks rendered `build/dist/guides/<name>/<v>/*.html` for broken local `<a>`/`<img>`/`<link>`/`<script>` refs and missing in-document anchors. Jsoup-driven; ignores external https URLs (those are handled by `cspScan`). |
| `structuralDiffGuides` | Migration tasks | Extracts heading-tree + anchor-id + per-element counts as a fingerprint per (guide, version). Compares against captured baselines if present. **Production-baseline integration is a follow-up** -- this PR ships the extractor and the comparison harness, marked REVIEW in the acceptance report until the legacy `guides.grails.org` snapshot is wired in. |
| `acceptanceReport` | Migration tasks | Reads each gate's CSV and writes `build/reports/acceptance.csv` with one row per (guide, version) and a per-component verdict (VERIFIED / REVIEW / FAILED). |
| `verifyAllGuides` | Migration tasks | Aggregate. Depends on `buildAllGuides`, `asciidoctorWarningGate`, `crawlBuiltGuides`, `structuralDiffGuides`, `cspScan`, `acceptanceReport`. Default mode is report-only (exit 0); `-PverificationMode=hard-fail` flips it. |
## Existing infrastructure reused
- `cspScan` (existing) is wired as the 11e gate; `CspScanTask` group changed from `documentation` to `migration` so it shows up alongside the rest.
- `AdHocFixtureDiff` (Phase 8) extends with the structural-fingerprint extraction primitives.
- `RenderGuidesPlugin` and `GrailsWebsitePlugin` get the wiring hooks.
## Output locations
- `build/logs/asciidoctor-*.log` -- per-task captured output
- `build/reports/asciidoctor-warning-gate.csv` -- per-(guide, version) warning verdict
- `build/reports/crawl-built-guides.csv` -- per-(guide, version) crawl verdict
- `build/reports/structural-diff-guides.csv` -- per-(guide, version) structural-fingerprint verdict
- `build/reports/csp-scan.md` -- existing CSP scan output (unchanged)
- `build/reports/acceptance.csv` -- combined per-(guide, version) verdict across all gates
## Tests
Each new task has a Spock spec covering happy path, empty-input, and failure detection. `./gradlew :buildSrc:test` passes.
## Behavior verified locally
- `./gradlew tasks --group=migration` lists all 5 new tasks plus existing `cspScan`, `publishMainSite`, `validateGuides`, `buildAllGuides`, `vendor*`, `renderGuide_*`.
- `./gradlew :buildSrc:compileGroovy :buildSrc:compileTestGroovy :buildSrc:test` -- BUILD SUCCESSFUL.
## Out of scope (follow-ups)
- Production-baseline integration for `structuralDiffGuides` (capture `https://guides.grails.org/<g>/<v>/` snapshots and commit). All structural rows currently land as REVIEW pending that capture.
- Visual diff (Playwright). Plan keeps this advisory-only; not implemented here.
- Hard-fail mode allowlist tuning. Initial run will surface real findings; allowlist additions get owner approval per Appendix A policy.
Refs #354.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@grails.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Verification harness for rendered guides (refs #354) [grails-static-website]
Posted by "jamesfredley (via GitHub)" <gi...@apache.org>.
jamesfredley commented on code in PR #457:
URL: https://github.com/apache/grails-static-website/pull/457#discussion_r3173955042
##########
buildSrc/src/main/groovy/website/gradle/tasks/AsciidoctorWarningGateTask.groovy:
##########
@@ -0,0 +1,274 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package website.gradle.tasks
+
+import groovy.transform.CompileStatic
+
+import org.gradle.api.DefaultTask
+import org.gradle.api.GradleException
+import org.gradle.api.Project
+import org.gradle.api.Task
+import org.gradle.api.file.DirectoryProperty
+import org.gradle.api.file.RegularFileProperty
+import org.gradle.api.plugins.ExtraPropertiesExtension
+import org.gradle.api.provider.Property
+import org.gradle.api.provider.ListProperty
+import org.gradle.api.tasks.Input
+import org.gradle.api.tasks.InputDirectory
+import org.gradle.api.tasks.Optional
+import org.gradle.api.tasks.OutputFile
+import org.gradle.api.tasks.PathSensitive
+import org.gradle.api.tasks.PathSensitivity
+import org.gradle.api.tasks.TaskAction
+import org.gradle.api.tasks.TaskProvider
+import org.gradle.api.logging.StandardOutputListener
+
+import grails.doc.gradle.PublishGuideTask
+
+import java.util.regex.Pattern
+
+/**
+ * Scans captured PublishGuide output for unallowlisted warnings and errors and
+ * writes a per-guide summary to {@code build/reports/asciidoctor-warning-gate.csv}.
+ */
+@CompileStatic
+class AsciidoctorWarningGateTask extends DefaultTask {
+
+ static final String NAME = 'asciidoctorWarningGate'
+ static final String GROUP = 'migration'
+
+ private static final Pattern WARNING_PATTERN = ~/(?im)^.*(\[WARNING\]|\[ERROR\]|asciidoctor:\s*(WARNING|ERROR|FAIL)|\bWARN(?:ING)?:|\bERROR:|java\.lang\.\w+(?:Error|Exception)|java\.io\.\w+(?:Error|Exception)|java\.util\.\w+(?:Error|Exception)|org\.\w+(?:\.\w+)*\.\w+(?:Error|Exception)|^.*Caused by:)/
+
+ @Optional
+ @InputDirectory
+ @PathSensitive(PathSensitivity.RELATIVE)
+ final DirectoryProperty logDir = project.objects.directoryProperty()
+
+ @OutputFile
+ final RegularFileProperty reportFile = project.objects.fileProperty()
+
+ @Input
+ final Property<Boolean> failOnViolation = project.objects.property(Boolean)
+
+ @Input
+ final ListProperty<String> guideTaskMappings = project.objects.listProperty(String)
+
+ @TaskAction
+ void gate() {
+ File logsRoot = logDir.get().asFile
+ File report = reportFile.get().asFile
+
+ if (!logsRoot.isDirectory()) {
+ writeCsv(report, [])
+ return
+ }
+
+ Map<String, GuideTaskMetadata> metadataByTaskName = parseGuideTaskMappings(guideTaskMappings.get())
+ List<GuideSummary> results = []
+
+ for (GuideTaskMetadata metadata : metadataByTaskName.values()) {
+ File logFile = new File(logsRoot, "asciidoctor-${metadata.taskName}.log")
+ List<String> violations = logFile.isFile() ? findViolations(logFile) : []
+ String status = statusFor(violations.isEmpty())
+ String details = violations.isEmpty()
+ ? 'No unallowlisted warning or error lines detected.'
+ : abbreviate(violations.join(' | '))
+ results << new GuideSummary(
+ guide: metadata.guide,
+ version: metadata.version,
+ status: status,
+ issueCount: violations.size(),
+ details: details,
+ )
+ }
+
+ writeCsv(report, results)
+
+ List<GuideSummary> failed = results.findAll { GuideSummary result -> result.status == 'FAILED' } as List<GuideSummary>
+ if (!failed.isEmpty()) {
+ throw new GradleException("AsciiDoctor warnings detected. See ${report.absolutePath}.")
+ }
+ }
+
+ static TaskProvider<AsciidoctorWarningGateTask> register(Project project) {
+ wireLogCapture(project)
+ project.tasks.register(NAME, AsciidoctorWarningGateTask) { AsciidoctorWarningGateTask task ->
+ task.group = GROUP
+ task.description = 'Captures PublishGuide output and summarizes unallowlisted AsciiDoctor warnings and errors.'
+ task.notCompatibleWithConfigurationCache('Reads per-guide log capture emitted by vendored PublishGuide tasks and their finalizers.')
+ task.logDir.convention(project.layout.buildDirectory.dir('logs'))
+ task.reportFile.convention(project.layout.buildDirectory.file('reports/asciidoctor-warning-gate.csv'))
+ task.failOnViolation.convention(isHardFailMode(project))
+ task.guideTaskMappings.set(buildGuideTaskMappings(project))
+ task.dependsOn('buildAllGuides')
+ }
+ }
+
+ private static void wireLogCapture(Project project) {
+ project.tasks.withType(PublishGuideTask).all { PublishGuideTask renderTask ->
+ File logsRoot = project.layout.buildDirectory.dir('logs').get().asFile
+ File logFile = new File(logsRoot, "asciidoctor-${renderTask.name}.log")
+
+ renderTask.doFirst {
+ logsRoot.mkdirs()
+ logFile.text = ''
+
+ StandardOutputListener writeListener = ({ CharSequence message ->
+ logFile << message
+ } as StandardOutputListener)
+
+ renderTask.logging.addStandardOutputListener(writeListener)
+ renderTask.logging.addStandardErrorListener(writeListener)
+
+ ExtraPropertiesExtension extraProperties = renderTask.extensions.extraProperties
+ extraProperties.set('logFile', logFile)
+ extraProperties.set('logListener', writeListener)
+ }
+
+ String cleanupTaskName = "cleanup-${renderTask.name}-logs"
+ if (project.tasks.findByName(cleanupTaskName) == null) {
+ project.tasks.register(cleanupTaskName) { Task cleanupTask ->
+ cleanupTask.notCompatibleWithConfigurationCache('Removes StandardOutputListener instances from PublishGuide tasks.')
+ cleanupTask.doLast {
+ try {
+ ExtraPropertiesExtension extraProperties = renderTask.extensions.extraProperties
+ if (extraProperties.has('logListener')) {
+ StandardOutputListener writeListener = extraProperties.get('logListener') as StandardOutputListener
+ renderTask.logging.removeStandardOutputListener(writeListener)
+ renderTask.logging.removeStandardErrorListener(writeListener)
+ }
+ } catch (Exception e) {
+ project.logger.warn("Listener cleanup failed for ${renderTask.name}: ${e.message}")
+ }
+ }
+ }
+ }
+ renderTask.finalizedBy(cleanupTaskName)
+ }
+ }
+
+ private static List<String> findViolations(File logFile) {
+ List<String> violations = []
+ List<Pattern> excludeAllowlist = []
+ int lineNumber = 0
+ logFile.eachLine('UTF-8') { String line ->
+ lineNumber++
+ if (WARNING_PATTERN.matcher(line).find() && !isAllowlisted(line, excludeAllowlist)) {
+ violations.add("${logFile.name}:${lineNumber}:${sanitize(line)}".toString())
+ }
+ }
+ violations
+ }
+
+ private static boolean isAllowlisted(String line, List<Pattern> excludeAllowlist) {
+ for (Pattern pattern : excludeAllowlist) {
+ if (pattern.matcher(line).find()) {
+ return true
+ }
+ }
+ false
+ }
Review Comment:
Implemented in 3b77b70433. Added @Input @Optional ListProperty<String> allowlistPatterns wired via -PasciidoctorAllowlist=<comma-separated-regex>. The findViolations method now compiles each pattern and skips any line that matches. Default is an empty list (nothing allowlisted), with explicit owner approval required to extend it per the plan policy.
##########
buildSrc/src/main/groovy/website/gradle/tasks/CrawlBuiltGuidesTask.groovy:
##########
@@ -0,0 +1,353 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package website.gradle.tasks
+
+import groovy.transform.CompileStatic
+
+import org.gradle.api.DefaultTask
+import org.gradle.api.GradleException
+import org.gradle.api.Project
+import org.gradle.api.file.DirectoryProperty
+import org.gradle.api.file.RegularFileProperty
+import org.gradle.api.provider.Property
+import org.gradle.api.tasks.Input
+import org.gradle.api.tasks.InputDirectory
+import org.gradle.api.tasks.Optional
+import org.gradle.api.tasks.OutputFile
+import org.gradle.api.tasks.PathSensitive
+import org.gradle.api.tasks.PathSensitivity
+import org.gradle.api.tasks.TaskAction
+import org.gradle.api.tasks.TaskProvider
+
+import org.jsoup.Jsoup
+import org.jsoup.nodes.Document
+import org.jsoup.nodes.Element
+
+import java.net.URLDecoder
+import java.nio.charset.StandardCharsets
+import java.nio.file.Path
+
+/**
+ * Crawls rendered guide HTML looking for missing local files and broken anchors.
+ */
+@CompileStatic
+class CrawlBuiltGuidesTask extends DefaultTask {
+
+ static final String NAME = 'crawlBuiltGuides'
+ static final String GROUP = 'migration'
+
+ @Optional
+ @InputDirectory
+ @PathSensitive(PathSensitivity.RELATIVE)
+ final DirectoryProperty guidesDir = project.objects.directoryProperty()
+
+ @Optional
+ @InputDirectory
+ @PathSensitive(PathSensitivity.RELATIVE)
+ final DirectoryProperty siteRootDir = project.objects.directoryProperty()
+
+ @OutputFile
+ final RegularFileProperty reportFile = project.objects.fileProperty()
+
+ @Input
+ final Property<Boolean> failOnViolation = project.objects.property(Boolean)
+
+ @TaskAction
+ void crawl() {
+ File guidesRoot = guidesDir.get().asFile
+ File siteRoot = siteRootDir.get().asFile
+ File report = reportFile.get().asFile
+
+ if (!guidesRoot.isDirectory()) {
+ writeCsv(report, [])
+ return
+ }
+
+ Map<String, GuideSummary> summaries = discoverGuideVersions(guidesRoot)
+ Map<File, Document> documentCache = [:]
+ Map<File, Set<String>> anchorCache = [:]
+
+ for (GuideSummary summary : summaries.values()) {
+ List<File> htmlFiles = collectHtmlFiles(summary.versionDir)
+ if (htmlFiles.isEmpty()) {
+ summary.issues << 'No rendered HTML files were found for this guide version.'
+ continue
+ }
+ for (File htmlFile : htmlFiles) {
+ Document document = parseDocument(htmlFile, documentCache)
+ checkLinkElements(summary, htmlFile, document, siteRoot, documentCache, anchorCache)
+ }
+ }
+
+ List<GuideSummary> results = summaries.values() as List<GuideSummary>
+ for (GuideSummary summary : results) {
+ summary.status = determineStatus(summary.issues.isEmpty())
+ }
+ writeCsv(report, results)
+
+ List<GuideSummary> failed = results.findAll { GuideSummary summary -> summary.status == 'FAILED' } as List<GuideSummary>
+ if (!failed.isEmpty()) {
+ throw new GradleException("Broken rendered guide references detected. See ${report.absolutePath}.")
+ }
+ }
+
+ static TaskProvider<CrawlBuiltGuidesTask> register(Project project) {
+ project.tasks.register(NAME, CrawlBuiltGuidesTask) { CrawlBuiltGuidesTask task ->
+ task.group = GROUP
+ task.description = 'Smoke-crawls rendered guide HTML for broken local links, assets, and anchors.'
+ task.guidesDir.convention(project.layout.buildDirectory.dir('dist/guides'))
+ task.siteRootDir.convention(project.layout.buildDirectory.dir('dist'))
+ task.reportFile.convention(project.layout.buildDirectory.file('reports/crawl-built-guides.csv'))
+ task.failOnViolation.convention(isHardFailMode(project))
+ task.dependsOn('buildAllGuides')
+ }
+ }
+
+ private void checkLinkElements(GuideSummary summary, File htmlFile, Document document, File siteRoot,
+ Map<File, Document> documentCache, Map<File, Set<String>> anchorCache) {
+ checkElements(summary, htmlFile, document, siteRoot, documentCache, anchorCache, 'a[href]', 'href', true)
+ checkElements(summary, htmlFile, document, siteRoot, documentCache, anchorCache, 'img[src]', 'src', false)
+ checkElements(summary, htmlFile, document, siteRoot, documentCache, anchorCache, 'link[href]', 'href', false)
+ checkElements(summary, htmlFile, document, siteRoot, documentCache, anchorCache, 'script[src]', 'src', false)
+ }
+
+ private void checkElements(GuideSummary summary, File htmlFile, Document document, File siteRoot,
+ Map<File, Document> documentCache, Map<File, Set<String>> anchorCache,
+ String selector, String attribute, boolean checkAnchor) {
+
+ for (Element element : document.select(selector)) {
+ String reference = element.attr(attribute)?.trim()
+ if (shouldIgnore(reference)) {
+ continue
+ }
+
+ ReferenceParts parts = splitReference(reference)
+ if (parts.pathPart.isEmpty()) {
+ if (checkAnchor && !parts.fragment.isEmpty()) {
+ ensureAnchor(summary, htmlFile, htmlFile, parts.fragment, documentCache, anchorCache)
+ }
+ continue
+ }
+
+ File target = resolveTarget(htmlFile, siteRoot, parts.pathPart)
+ if (!target.isFile()) {
+ summary.issues.add("${relativePath(summary.versionDir, htmlFile)} -> missing ${attribute} target ${parts.pathPart}".toString())
+ continue
+ }
+ if (checkAnchor && !parts.fragment.isEmpty()) {
+ ensureAnchor(summary, htmlFile, target, parts.fragment, documentCache, anchorCache)
+ }
+ }
+ }
+
+ private void ensureAnchor(GuideSummary summary, File sourceFile, File targetFile, String fragment,
+ Map<File, Document> documentCache, Map<File, Set<String>> anchorCache) {
+ Set<String> anchors = anchorCache.computeIfAbsent(targetFile) {
+ extractAnchors(parseDocument(targetFile, documentCache))
+ }
+ if (!anchors.contains(fragment)) {
+ summary.issues.add("${relativePath(summary.versionDir, sourceFile)} -> missing anchor #${fragment} in ${relativePath(summary.versionDir, targetFile)}".toString())
+ }
+ }
+
+ private static Map<String, GuideSummary> discoverGuideVersions(File guidesRoot) {
+ Map<String, GuideSummary> summaries = [:]
+ File[] guideDirs = guidesRoot.listFiles()
+ if (guideDirs == null) {
+ return summaries
+ }
+ for (File guideDir : guideDirs) {
+ if (!guideDir.isDirectory()) {
+ continue
+ }
+ File[] versionDirs = guideDir.listFiles()
+ if (versionDirs == null) {
+ continue
+ }
+ for (File versionDir : versionDirs) {
+ if (!versionDir.isDirectory()) {
+ continue
+ }
+ GuideSummary summary = new GuideSummary(
+ guide: guideDir.name,
+ version: versionDir.name,
+ versionDir: versionDir,
+ )
+ summaries[keyFor(guideDir.name, versionDir.name)] = summary
+ }
+ }
+ summaries
+ }
+
+ private static List<File> collectHtmlFiles(File versionDir) {
+ List<File> htmlFiles = []
+ versionDir.eachFileRecurse { File candidate ->
+ if (candidate.isFile() && candidate.name.endsWith('.html')) {
+ htmlFiles << candidate
+ }
+ }
+ htmlFiles.sort { File left, File right -> left.absolutePath <=> right.absolutePath }
+ htmlFiles
+ }
+
+ private static Document parseDocument(File htmlFile, Map<File, Document> documentCache) {
+ documentCache.computeIfAbsent(htmlFile) {
+ Jsoup.parse(htmlFile, 'UTF-8')
+ }
+ }
+
+ private static Set<String> extractAnchors(Document document) {
+ Set<String> anchors = [] as LinkedHashSet<String>
+ for (Element element : document.select('[id]')) {
+ String id = element.attr('id')?.trim()
+ if (!id.isEmpty()) {
+ anchors << id
+ }
+ }
+ for (Element element : document.select('a[name]')) {
+ String name = element.attr('name')?.trim()
+ if (!name.isEmpty()) {
+ anchors << name
+ }
+ }
+ anchors
+ }
+
+ private static File resolveTarget(File htmlFile, File siteRoot, String rawPath) {
+ String decoded = URLDecoder.decode(rawPath, StandardCharsets.UTF_8)
+ Path candidate = decoded.startsWith('/')
+ ? siteRoot.toPath().resolve(decoded.substring(1))
+ : htmlFile.parentFile.toPath().resolve(decoded)
+ Path normalized = candidate.normalize()
+ File target = normalized.toFile()
+ if (target.isDirectory()) {
+ return normalized.resolve('index.html').toFile()
+ }
+ if (!target.exists() && !decoded.contains('.') && normalized.resolveSibling(normalized.fileName.toString() + '.html').toFile().isFile()) {
+ return normalized.resolveSibling(normalized.fileName.toString() + '.html').toFile()
+ }
+ if (!target.exists() && normalized.resolve('index.html').toFile().isFile()) {
+ return normalized.resolve('index.html').toFile()
Review Comment:
Fixed in 517b6aa4a5. resolveTarget now canonicalizes both the candidate and siteRoot via toAbsolutePath().normalize() and rejects anything that does not startsWith(siteRoot). Rejected paths return a sentinel under siteRoot/__path_traversal_rejected__/ so the caller still hits the missing-target branch and reports the broken reference, but parseDocument is never invoked on a path outside the deployed tree.
##########
buildSrc/src/main/groovy/website/gradle/tasks/CspScanTask.groovy:
##########
@@ -205,9 +205,16 @@ class CspScanTask extends DefaultTask {
task.allowlistFile.set(
project.rootProject.layout.projectDirectory.file('conf/csp-allowlist.yml'))
task.reportFile.set(project.layout.buildDirectory.file('reports/csp-scan.md'))
+ task.failOnViolation.convention(isHardFailMode(project))
+ task.dependsOn('buildAllGuides')
if (project.hasProperty('cspFailOnViolation')) {
task.failOnViolation.set(Boolean.parseBoolean(project.property('cspFailOnViolation') as String))
}
}
Review Comment:
Updated in 2137b78c67. Class-level Javadoc now documents the new precedence: cspFailOnViolation > verificationMode > default-false. The register() method already implements that precedence (convention(isHardFailMode) then conditional .set() override on cspFailOnViolation), so only the doc was stale.
##########
buildSrc/src/main/groovy/website/gradle/tasks/StructuralDiffGuidesTask.groovy:
##########
@@ -0,0 +1,331 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package website.gradle.tasks
+
+import groovy.json.JsonOutput
+import groovy.transform.CompileStatic
+
+import org.gradle.api.DefaultTask
+import org.gradle.api.GradleException
+import org.gradle.api.Project
+import org.gradle.api.file.DirectoryProperty
+import org.gradle.api.file.RegularFileProperty
+import org.gradle.api.provider.Property
+import org.gradle.api.tasks.Input
+import org.gradle.api.tasks.InputDirectory
+import org.gradle.api.tasks.Optional
+import org.gradle.api.tasks.OutputDirectory
+import org.gradle.api.tasks.OutputFile
+import org.gradle.api.tasks.PathSensitive
+import org.gradle.api.tasks.PathSensitivity
+import org.gradle.api.tasks.TaskAction
+import org.gradle.api.tasks.TaskProvider
+
+import org.jsoup.Jsoup
+import org.jsoup.nodes.Document
+import org.jsoup.nodes.Element
+
+import website.qa.AdHocFixtureDiff
+
+/**
+ * Extracts structural fingerprints from rendered guides and compares them to a
+ * baseline when one is available.
+ */
+@CompileStatic
+class StructuralDiffGuidesTask extends DefaultTask {
+
+ static final String NAME = 'structuralDiffGuides'
+ static final String GROUP = 'migration'
+
+ @Optional
+ @InputDirectory
+ @PathSensitive(PathSensitivity.RELATIVE)
+ final DirectoryProperty guidesDir = project.objects.directoryProperty()
+
+ @Input
+ final Property<String> baselineDirPath = project.objects.property(String)
+
Review Comment:
Fixed in e72ef6ccd7. Replaced @Input Property<String> baselineDirPath with @Optional @InputDirectory @PathSensitive(PathSensitivity.RELATIVE) DirectoryProperty baselineDir. The default convention only attaches when buildSrc/src/test/resources/structural-baseline/ exists at config time, so the property stays unset (and Gradle does not require the directory) when there is no baseline yet.
##########
buildSrc/src/main/groovy/website/gradle/tasks/AsciidoctorWarningGateTask.groovy:
##########
@@ -0,0 +1,274 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package website.gradle.tasks
+
+import groovy.transform.CompileStatic
+
+import org.gradle.api.DefaultTask
+import org.gradle.api.GradleException
+import org.gradle.api.Project
+import org.gradle.api.Task
+import org.gradle.api.file.DirectoryProperty
+import org.gradle.api.file.RegularFileProperty
+import org.gradle.api.plugins.ExtraPropertiesExtension
+import org.gradle.api.provider.Property
+import org.gradle.api.provider.ListProperty
+import org.gradle.api.tasks.Input
+import org.gradle.api.tasks.InputDirectory
+import org.gradle.api.tasks.Optional
+import org.gradle.api.tasks.OutputFile
+import org.gradle.api.tasks.PathSensitive
+import org.gradle.api.tasks.PathSensitivity
+import org.gradle.api.tasks.TaskAction
+import org.gradle.api.tasks.TaskProvider
+import org.gradle.api.logging.StandardOutputListener
+
+import grails.doc.gradle.PublishGuideTask
+
+import java.util.regex.Pattern
+
+/**
+ * Scans captured PublishGuide output for unallowlisted warnings and errors and
+ * writes a per-guide summary to {@code build/reports/asciidoctor-warning-gate.csv}.
+ */
+@CompileStatic
+class AsciidoctorWarningGateTask extends DefaultTask {
+
+ static final String NAME = 'asciidoctorWarningGate'
+ static final String GROUP = 'migration'
+
+ private static final Pattern WARNING_PATTERN = ~/(?im)^.*(\[WARNING\]|\[ERROR\]|asciidoctor:\s*(WARNING|ERROR|FAIL)|\bWARN(?:ING)?:|\bERROR:|java\.lang\.\w+(?:Error|Exception)|java\.io\.\w+(?:Error|Exception)|java\.util\.\w+(?:Error|Exception)|org\.\w+(?:\.\w+)*\.\w+(?:Error|Exception)|^.*Caused by:)/
+
+ @Optional
+ @InputDirectory
+ @PathSensitive(PathSensitivity.RELATIVE)
+ final DirectoryProperty logDir = project.objects.directoryProperty()
+
+ @OutputFile
+ final RegularFileProperty reportFile = project.objects.fileProperty()
+
+ @Input
+ final Property<Boolean> failOnViolation = project.objects.property(Boolean)
+
+ @Input
+ final ListProperty<String> guideTaskMappings = project.objects.listProperty(String)
+
+ @TaskAction
+ void gate() {
+ File logsRoot = logDir.get().asFile
+ File report = reportFile.get().asFile
+
+ if (!logsRoot.isDirectory()) {
+ writeCsv(report, [])
+ return
+ }
+
+ Map<String, GuideTaskMetadata> metadataByTaskName = parseGuideTaskMappings(guideTaskMappings.get())
+ List<GuideSummary> results = []
+
+ for (GuideTaskMetadata metadata : metadataByTaskName.values()) {
+ File logFile = new File(logsRoot, "asciidoctor-${metadata.taskName}.log")
+ List<String> violations = logFile.isFile() ? findViolations(logFile) : []
+ String status = statusFor(violations.isEmpty())
+ String details = violations.isEmpty()
+ ? 'No unallowlisted warning or error lines detected.'
+ : abbreviate(violations.join(' | '))
+ results << new GuideSummary(
Review Comment:
Fixed in 3b77b70433 (combined with the allowlist input since both touch this file). Missing per-guide log files now produce a row with status REVIEW, issueCount 0, and a descriptive detail "Missing captured log output for <task>; rerun the renderer or invalidate Gradle caches before trusting this verdict." This surfaces stale or skipped log capture instead of silently scoring VERIFIED.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@grails.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Verification harness for rendered guides (refs #354) [grails-static-website]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #457:
URL: https://github.com/apache/grails-static-website/pull/457#discussion_r3173746255
##########
buildSrc/src/main/groovy/website/gradle/tasks/CspScanTask.groovy:
##########
@@ -205,9 +205,16 @@ class CspScanTask extends DefaultTask {
task.allowlistFile.set(
project.rootProject.layout.projectDirectory.file('conf/csp-allowlist.yml'))
task.reportFile.set(project.layout.buildDirectory.file('reports/csp-scan.md'))
+ task.failOnViolation.convention(isHardFailMode(project))
+ task.dependsOn('buildAllGuides')
if (project.hasProperty('cspFailOnViolation')) {
task.failOnViolation.set(Boolean.parseBoolean(project.property('cspFailOnViolation') as String))
}
}
Review Comment:
`CspScanTask`’s class-level docs say violations fail the build by default and `-PcspFailOnViolation=false` demotes to warnings, but the task is now configured to default `failOnViolation` based on `-PverificationMode=hard-fail` (and otherwise be report-only). Please update the documentation to match the new default/override behavior (and document precedence between `verificationMode` and `cspFailOnViolation`).
##########
buildSrc/src/main/groovy/website/gradle/tasks/StructuralDiffGuidesTask.groovy:
##########
@@ -0,0 +1,331 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package website.gradle.tasks
+
+import groovy.json.JsonOutput
+import groovy.transform.CompileStatic
+
+import org.gradle.api.DefaultTask
+import org.gradle.api.GradleException
+import org.gradle.api.Project
+import org.gradle.api.file.DirectoryProperty
+import org.gradle.api.file.RegularFileProperty
+import org.gradle.api.provider.Property
+import org.gradle.api.tasks.Input
+import org.gradle.api.tasks.InputDirectory
+import org.gradle.api.tasks.Optional
+import org.gradle.api.tasks.OutputDirectory
+import org.gradle.api.tasks.OutputFile
+import org.gradle.api.tasks.PathSensitive
+import org.gradle.api.tasks.PathSensitivity
+import org.gradle.api.tasks.TaskAction
+import org.gradle.api.tasks.TaskProvider
+
+import org.jsoup.Jsoup
+import org.jsoup.nodes.Document
+import org.jsoup.nodes.Element
+
+import website.qa.AdHocFixtureDiff
+
+/**
+ * Extracts structural fingerprints from rendered guides and compares them to a
+ * baseline when one is available.
+ */
+@CompileStatic
+class StructuralDiffGuidesTask extends DefaultTask {
+
+ static final String NAME = 'structuralDiffGuides'
+ static final String GROUP = 'migration'
+
+ @Optional
+ @InputDirectory
+ @PathSensitive(PathSensitivity.RELATIVE)
+ final DirectoryProperty guidesDir = project.objects.directoryProperty()
+
+ @Input
+ final Property<String> baselineDirPath = project.objects.property(String)
+
Review Comment:
`baselineDirPath` is modeled as a plain `@Input Property<String>` even though it represents a directory whose contents influence the comparison. With the current setup, changes to baseline HTML files won’t invalidate Gradle up-to-date checks/build cache because Gradle only tracks the path string, not the directory contents. Consider switching this to an `@InputDirectory DirectoryProperty` (optional) with `@PathSensitive(PathSensitivity.RELATIVE)` and use it directly instead of `new File(baselineDirPath.get())`.
##########
buildSrc/src/main/groovy/website/gradle/tasks/AsciidoctorWarningGateTask.groovy:
##########
@@ -0,0 +1,274 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package website.gradle.tasks
+
+import groovy.transform.CompileStatic
+
+import org.gradle.api.DefaultTask
+import org.gradle.api.GradleException
+import org.gradle.api.Project
+import org.gradle.api.Task
+import org.gradle.api.file.DirectoryProperty
+import org.gradle.api.file.RegularFileProperty
+import org.gradle.api.plugins.ExtraPropertiesExtension
+import org.gradle.api.provider.Property
+import org.gradle.api.provider.ListProperty
+import org.gradle.api.tasks.Input
+import org.gradle.api.tasks.InputDirectory
+import org.gradle.api.tasks.Optional
+import org.gradle.api.tasks.OutputFile
+import org.gradle.api.tasks.PathSensitive
+import org.gradle.api.tasks.PathSensitivity
+import org.gradle.api.tasks.TaskAction
+import org.gradle.api.tasks.TaskProvider
+import org.gradle.api.logging.StandardOutputListener
+
+import grails.doc.gradle.PublishGuideTask
+
+import java.util.regex.Pattern
+
+/**
+ * Scans captured PublishGuide output for unallowlisted warnings and errors and
+ * writes a per-guide summary to {@code build/reports/asciidoctor-warning-gate.csv}.
+ */
+@CompileStatic
+class AsciidoctorWarningGateTask extends DefaultTask {
+
+ static final String NAME = 'asciidoctorWarningGate'
+ static final String GROUP = 'migration'
+
+ private static final Pattern WARNING_PATTERN = ~/(?im)^.*(\[WARNING\]|\[ERROR\]|asciidoctor:\s*(WARNING|ERROR|FAIL)|\bWARN(?:ING)?:|\bERROR:|java\.lang\.\w+(?:Error|Exception)|java\.io\.\w+(?:Error|Exception)|java\.util\.\w+(?:Error|Exception)|org\.\w+(?:\.\w+)*\.\w+(?:Error|Exception)|^.*Caused by:)/
+
+ @Optional
+ @InputDirectory
+ @PathSensitive(PathSensitivity.RELATIVE)
+ final DirectoryProperty logDir = project.objects.directoryProperty()
+
+ @OutputFile
+ final RegularFileProperty reportFile = project.objects.fileProperty()
+
+ @Input
+ final Property<Boolean> failOnViolation = project.objects.property(Boolean)
+
+ @Input
+ final ListProperty<String> guideTaskMappings = project.objects.listProperty(String)
+
+ @TaskAction
+ void gate() {
+ File logsRoot = logDir.get().asFile
+ File report = reportFile.get().asFile
+
+ if (!logsRoot.isDirectory()) {
+ writeCsv(report, [])
+ return
+ }
+
+ Map<String, GuideTaskMetadata> metadataByTaskName = parseGuideTaskMappings(guideTaskMappings.get())
+ List<GuideSummary> results = []
+
+ for (GuideTaskMetadata metadata : metadataByTaskName.values()) {
+ File logFile = new File(logsRoot, "asciidoctor-${metadata.taskName}.log")
+ List<String> violations = logFile.isFile() ? findViolations(logFile) : []
+ String status = statusFor(violations.isEmpty())
+ String details = violations.isEmpty()
+ ? 'No unallowlisted warning or error lines detected.'
+ : abbreviate(violations.join(' | '))
+ results << new GuideSummary(
+ guide: metadata.guide,
+ version: metadata.version,
+ status: status,
+ issueCount: violations.size(),
+ details: details,
+ )
+ }
+
+ writeCsv(report, results)
+
+ List<GuideSummary> failed = results.findAll { GuideSummary result -> result.status == 'FAILED' } as List<GuideSummary>
+ if (!failed.isEmpty()) {
+ throw new GradleException("AsciiDoctor warnings detected. See ${report.absolutePath}.")
+ }
+ }
+
+ static TaskProvider<AsciidoctorWarningGateTask> register(Project project) {
+ wireLogCapture(project)
+ project.tasks.register(NAME, AsciidoctorWarningGateTask) { AsciidoctorWarningGateTask task ->
+ task.group = GROUP
+ task.description = 'Captures PublishGuide output and summarizes unallowlisted AsciiDoctor warnings and errors.'
+ task.notCompatibleWithConfigurationCache('Reads per-guide log capture emitted by vendored PublishGuide tasks and their finalizers.')
+ task.logDir.convention(project.layout.buildDirectory.dir('logs'))
+ task.reportFile.convention(project.layout.buildDirectory.file('reports/asciidoctor-warning-gate.csv'))
+ task.failOnViolation.convention(isHardFailMode(project))
+ task.guideTaskMappings.set(buildGuideTaskMappings(project))
+ task.dependsOn('buildAllGuides')
+ }
+ }
+
+ private static void wireLogCapture(Project project) {
+ project.tasks.withType(PublishGuideTask).all { PublishGuideTask renderTask ->
+ File logsRoot = project.layout.buildDirectory.dir('logs').get().asFile
+ File logFile = new File(logsRoot, "asciidoctor-${renderTask.name}.log")
+
+ renderTask.doFirst {
+ logsRoot.mkdirs()
+ logFile.text = ''
+
+ StandardOutputListener writeListener = ({ CharSequence message ->
+ logFile << message
+ } as StandardOutputListener)
+
+ renderTask.logging.addStandardOutputListener(writeListener)
+ renderTask.logging.addStandardErrorListener(writeListener)
+
+ ExtraPropertiesExtension extraProperties = renderTask.extensions.extraProperties
+ extraProperties.set('logFile', logFile)
+ extraProperties.set('logListener', writeListener)
+ }
+
+ String cleanupTaskName = "cleanup-${renderTask.name}-logs"
+ if (project.tasks.findByName(cleanupTaskName) == null) {
+ project.tasks.register(cleanupTaskName) { Task cleanupTask ->
+ cleanupTask.notCompatibleWithConfigurationCache('Removes StandardOutputListener instances from PublishGuide tasks.')
+ cleanupTask.doLast {
+ try {
+ ExtraPropertiesExtension extraProperties = renderTask.extensions.extraProperties
+ if (extraProperties.has('logListener')) {
+ StandardOutputListener writeListener = extraProperties.get('logListener') as StandardOutputListener
+ renderTask.logging.removeStandardOutputListener(writeListener)
+ renderTask.logging.removeStandardErrorListener(writeListener)
+ }
+ } catch (Exception e) {
+ project.logger.warn("Listener cleanup failed for ${renderTask.name}: ${e.message}")
+ }
+ }
+ }
+ }
+ renderTask.finalizedBy(cleanupTaskName)
+ }
+ }
+
+ private static List<String> findViolations(File logFile) {
+ List<String> violations = []
+ List<Pattern> excludeAllowlist = []
+ int lineNumber = 0
+ logFile.eachLine('UTF-8') { String line ->
+ lineNumber++
+ if (WARNING_PATTERN.matcher(line).find() && !isAllowlisted(line, excludeAllowlist)) {
+ violations.add("${logFile.name}:${lineNumber}:${sanitize(line)}".toString())
+ }
+ }
+ violations
+ }
+
+ private static boolean isAllowlisted(String line, List<Pattern> excludeAllowlist) {
+ for (Pattern pattern : excludeAllowlist) {
+ if (pattern.matcher(line).find()) {
+ return true
+ }
+ }
+ false
+ }
Review Comment:
The PR description mentions an allowlist hook for warning lines, but `findViolations` currently always uses an empty `excludeAllowlist` and there’s no input/property to populate it. As written, nothing can ever be allowlisted. Either remove the dead allowlist plumbing, or implement a real allowlist input (e.g., a file or `ListProperty<String>` of regexes) and wire it into `findViolations`/`isAllowlisted` so known-benign lines can be suppressed intentionally.
##########
buildSrc/src/main/groovy/website/gradle/tasks/AsciidoctorWarningGateTask.groovy:
##########
@@ -0,0 +1,274 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package website.gradle.tasks
+
+import groovy.transform.CompileStatic
+
+import org.gradle.api.DefaultTask
+import org.gradle.api.GradleException
+import org.gradle.api.Project
+import org.gradle.api.Task
+import org.gradle.api.file.DirectoryProperty
+import org.gradle.api.file.RegularFileProperty
+import org.gradle.api.plugins.ExtraPropertiesExtension
+import org.gradle.api.provider.Property
+import org.gradle.api.provider.ListProperty
+import org.gradle.api.tasks.Input
+import org.gradle.api.tasks.InputDirectory
+import org.gradle.api.tasks.Optional
+import org.gradle.api.tasks.OutputFile
+import org.gradle.api.tasks.PathSensitive
+import org.gradle.api.tasks.PathSensitivity
+import org.gradle.api.tasks.TaskAction
+import org.gradle.api.tasks.TaskProvider
+import org.gradle.api.logging.StandardOutputListener
+
+import grails.doc.gradle.PublishGuideTask
+
+import java.util.regex.Pattern
+
+/**
+ * Scans captured PublishGuide output for unallowlisted warnings and errors and
+ * writes a per-guide summary to {@code build/reports/asciidoctor-warning-gate.csv}.
+ */
+@CompileStatic
+class AsciidoctorWarningGateTask extends DefaultTask {
+
+ static final String NAME = 'asciidoctorWarningGate'
+ static final String GROUP = 'migration'
+
+ private static final Pattern WARNING_PATTERN = ~/(?im)^.*(\[WARNING\]|\[ERROR\]|asciidoctor:\s*(WARNING|ERROR|FAIL)|\bWARN(?:ING)?:|\bERROR:|java\.lang\.\w+(?:Error|Exception)|java\.io\.\w+(?:Error|Exception)|java\.util\.\w+(?:Error|Exception)|org\.\w+(?:\.\w+)*\.\w+(?:Error|Exception)|^.*Caused by:)/
+
+ @Optional
+ @InputDirectory
+ @PathSensitive(PathSensitivity.RELATIVE)
+ final DirectoryProperty logDir = project.objects.directoryProperty()
+
+ @OutputFile
+ final RegularFileProperty reportFile = project.objects.fileProperty()
+
+ @Input
+ final Property<Boolean> failOnViolation = project.objects.property(Boolean)
+
+ @Input
+ final ListProperty<String> guideTaskMappings = project.objects.listProperty(String)
+
+ @TaskAction
+ void gate() {
+ File logsRoot = logDir.get().asFile
+ File report = reportFile.get().asFile
+
+ if (!logsRoot.isDirectory()) {
+ writeCsv(report, [])
+ return
+ }
+
+ Map<String, GuideTaskMetadata> metadataByTaskName = parseGuideTaskMappings(guideTaskMappings.get())
+ List<GuideSummary> results = []
+
+ for (GuideTaskMetadata metadata : metadataByTaskName.values()) {
+ File logFile = new File(logsRoot, "asciidoctor-${metadata.taskName}.log")
+ List<String> violations = logFile.isFile() ? findViolations(logFile) : []
+ String status = statusFor(violations.isEmpty())
+ String details = violations.isEmpty()
+ ? 'No unallowlisted warning or error lines detected.'
+ : abbreviate(violations.join(' | '))
+ results << new GuideSummary(
Review Comment:
When a per-guide log file is missing, the gate currently treats it as having zero violations (`logFile.isFile() ? findViolations(...) : []`) and can mark the guide `VERIFIED`. This can produce false positives (e.g., when `buildAllGuides` is up-to-date and no log capture ran, or logs were cleaned). Consider treating a missing log file for a mapped guide task as a violation (REVIEW/FAILED depending on mode) and include a clear detail like “missing captured log output for <task>”.
##########
buildSrc/src/main/groovy/website/gradle/tasks/CrawlBuiltGuidesTask.groovy:
##########
@@ -0,0 +1,353 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package website.gradle.tasks
+
+import groovy.transform.CompileStatic
+
+import org.gradle.api.DefaultTask
+import org.gradle.api.GradleException
+import org.gradle.api.Project
+import org.gradle.api.file.DirectoryProperty
+import org.gradle.api.file.RegularFileProperty
+import org.gradle.api.provider.Property
+import org.gradle.api.tasks.Input
+import org.gradle.api.tasks.InputDirectory
+import org.gradle.api.tasks.Optional
+import org.gradle.api.tasks.OutputFile
+import org.gradle.api.tasks.PathSensitive
+import org.gradle.api.tasks.PathSensitivity
+import org.gradle.api.tasks.TaskAction
+import org.gradle.api.tasks.TaskProvider
+
+import org.jsoup.Jsoup
+import org.jsoup.nodes.Document
+import org.jsoup.nodes.Element
+
+import java.net.URLDecoder
+import java.nio.charset.StandardCharsets
+import java.nio.file.Path
+
+/**
+ * Crawls rendered guide HTML looking for missing local files and broken anchors.
+ */
+@CompileStatic
+class CrawlBuiltGuidesTask extends DefaultTask {
+
+ static final String NAME = 'crawlBuiltGuides'
+ static final String GROUP = 'migration'
+
+ @Optional
+ @InputDirectory
+ @PathSensitive(PathSensitivity.RELATIVE)
+ final DirectoryProperty guidesDir = project.objects.directoryProperty()
+
+ @Optional
+ @InputDirectory
+ @PathSensitive(PathSensitivity.RELATIVE)
+ final DirectoryProperty siteRootDir = project.objects.directoryProperty()
+
+ @OutputFile
+ final RegularFileProperty reportFile = project.objects.fileProperty()
+
+ @Input
+ final Property<Boolean> failOnViolation = project.objects.property(Boolean)
+
+ @TaskAction
+ void crawl() {
+ File guidesRoot = guidesDir.get().asFile
+ File siteRoot = siteRootDir.get().asFile
+ File report = reportFile.get().asFile
+
+ if (!guidesRoot.isDirectory()) {
+ writeCsv(report, [])
+ return
+ }
+
+ Map<String, GuideSummary> summaries = discoverGuideVersions(guidesRoot)
+ Map<File, Document> documentCache = [:]
+ Map<File, Set<String>> anchorCache = [:]
+
+ for (GuideSummary summary : summaries.values()) {
+ List<File> htmlFiles = collectHtmlFiles(summary.versionDir)
+ if (htmlFiles.isEmpty()) {
+ summary.issues << 'No rendered HTML files were found for this guide version.'
+ continue
+ }
+ for (File htmlFile : htmlFiles) {
+ Document document = parseDocument(htmlFile, documentCache)
+ checkLinkElements(summary, htmlFile, document, siteRoot, documentCache, anchorCache)
+ }
+ }
+
+ List<GuideSummary> results = summaries.values() as List<GuideSummary>
+ for (GuideSummary summary : results) {
+ summary.status = determineStatus(summary.issues.isEmpty())
+ }
+ writeCsv(report, results)
+
+ List<GuideSummary> failed = results.findAll { GuideSummary summary -> summary.status == 'FAILED' } as List<GuideSummary>
+ if (!failed.isEmpty()) {
+ throw new GradleException("Broken rendered guide references detected. See ${report.absolutePath}.")
+ }
+ }
+
+ static TaskProvider<CrawlBuiltGuidesTask> register(Project project) {
+ project.tasks.register(NAME, CrawlBuiltGuidesTask) { CrawlBuiltGuidesTask task ->
+ task.group = GROUP
+ task.description = 'Smoke-crawls rendered guide HTML for broken local links, assets, and anchors.'
+ task.guidesDir.convention(project.layout.buildDirectory.dir('dist/guides'))
+ task.siteRootDir.convention(project.layout.buildDirectory.dir('dist'))
+ task.reportFile.convention(project.layout.buildDirectory.file('reports/crawl-built-guides.csv'))
+ task.failOnViolation.convention(isHardFailMode(project))
+ task.dependsOn('buildAllGuides')
+ }
+ }
+
+ private void checkLinkElements(GuideSummary summary, File htmlFile, Document document, File siteRoot,
+ Map<File, Document> documentCache, Map<File, Set<String>> anchorCache) {
+ checkElements(summary, htmlFile, document, siteRoot, documentCache, anchorCache, 'a[href]', 'href', true)
+ checkElements(summary, htmlFile, document, siteRoot, documentCache, anchorCache, 'img[src]', 'src', false)
+ checkElements(summary, htmlFile, document, siteRoot, documentCache, anchorCache, 'link[href]', 'href', false)
+ checkElements(summary, htmlFile, document, siteRoot, documentCache, anchorCache, 'script[src]', 'src', false)
+ }
+
+ private void checkElements(GuideSummary summary, File htmlFile, Document document, File siteRoot,
+ Map<File, Document> documentCache, Map<File, Set<String>> anchorCache,
+ String selector, String attribute, boolean checkAnchor) {
+
+ for (Element element : document.select(selector)) {
+ String reference = element.attr(attribute)?.trim()
+ if (shouldIgnore(reference)) {
+ continue
+ }
+
+ ReferenceParts parts = splitReference(reference)
+ if (parts.pathPart.isEmpty()) {
+ if (checkAnchor && !parts.fragment.isEmpty()) {
+ ensureAnchor(summary, htmlFile, htmlFile, parts.fragment, documentCache, anchorCache)
+ }
+ continue
+ }
+
+ File target = resolveTarget(htmlFile, siteRoot, parts.pathPart)
+ if (!target.isFile()) {
+ summary.issues.add("${relativePath(summary.versionDir, htmlFile)} -> missing ${attribute} target ${parts.pathPart}".toString())
+ continue
+ }
+ if (checkAnchor && !parts.fragment.isEmpty()) {
+ ensureAnchor(summary, htmlFile, target, parts.fragment, documentCache, anchorCache)
+ }
+ }
+ }
+
+ private void ensureAnchor(GuideSummary summary, File sourceFile, File targetFile, String fragment,
+ Map<File, Document> documentCache, Map<File, Set<String>> anchorCache) {
+ Set<String> anchors = anchorCache.computeIfAbsent(targetFile) {
+ extractAnchors(parseDocument(targetFile, documentCache))
+ }
+ if (!anchors.contains(fragment)) {
+ summary.issues.add("${relativePath(summary.versionDir, sourceFile)} -> missing anchor #${fragment} in ${relativePath(summary.versionDir, targetFile)}".toString())
+ }
+ }
+
+ private static Map<String, GuideSummary> discoverGuideVersions(File guidesRoot) {
+ Map<String, GuideSummary> summaries = [:]
+ File[] guideDirs = guidesRoot.listFiles()
+ if (guideDirs == null) {
+ return summaries
+ }
+ for (File guideDir : guideDirs) {
+ if (!guideDir.isDirectory()) {
+ continue
+ }
+ File[] versionDirs = guideDir.listFiles()
+ if (versionDirs == null) {
+ continue
+ }
+ for (File versionDir : versionDirs) {
+ if (!versionDir.isDirectory()) {
+ continue
+ }
+ GuideSummary summary = new GuideSummary(
+ guide: guideDir.name,
+ version: versionDir.name,
+ versionDir: versionDir,
+ )
+ summaries[keyFor(guideDir.name, versionDir.name)] = summary
+ }
+ }
+ summaries
+ }
+
+ private static List<File> collectHtmlFiles(File versionDir) {
+ List<File> htmlFiles = []
+ versionDir.eachFileRecurse { File candidate ->
+ if (candidate.isFile() && candidate.name.endsWith('.html')) {
+ htmlFiles << candidate
+ }
+ }
+ htmlFiles.sort { File left, File right -> left.absolutePath <=> right.absolutePath }
+ htmlFiles
+ }
+
+ private static Document parseDocument(File htmlFile, Map<File, Document> documentCache) {
+ documentCache.computeIfAbsent(htmlFile) {
+ Jsoup.parse(htmlFile, 'UTF-8')
+ }
+ }
+
+ private static Set<String> extractAnchors(Document document) {
+ Set<String> anchors = [] as LinkedHashSet<String>
+ for (Element element : document.select('[id]')) {
+ String id = element.attr('id')?.trim()
+ if (!id.isEmpty()) {
+ anchors << id
+ }
+ }
+ for (Element element : document.select('a[name]')) {
+ String name = element.attr('name')?.trim()
+ if (!name.isEmpty()) {
+ anchors << name
+ }
+ }
+ anchors
+ }
+
+ private static File resolveTarget(File htmlFile, File siteRoot, String rawPath) {
+ String decoded = URLDecoder.decode(rawPath, StandardCharsets.UTF_8)
+ Path candidate = decoded.startsWith('/')
+ ? siteRoot.toPath().resolve(decoded.substring(1))
+ : htmlFile.parentFile.toPath().resolve(decoded)
+ Path normalized = candidate.normalize()
+ File target = normalized.toFile()
+ if (target.isDirectory()) {
+ return normalized.resolve('index.html').toFile()
+ }
+ if (!target.exists() && !decoded.contains('.') && normalized.resolveSibling(normalized.fileName.toString() + '.html').toFile().isFile()) {
+ return normalized.resolveSibling(normalized.fileName.toString() + '.html').toFile()
+ }
+ if (!target.exists() && normalized.resolve('index.html').toFile().isFile()) {
+ return normalized.resolve('index.html').toFile()
Review Comment:
`resolveTarget(...)` normalizes the resolved path but doesn’t enforce that it stays within `siteRoot`. A crafted reference containing `..` segments (or an absolute `/../../...` path) can escape `build/dist` and cause the task to probe/parse arbitrary files on the build machine (notably via `ensureAnchor` -> `parseDocument`). Consider rejecting any resolved path that is outside `siteRoot` (treat as a broken reference) before calling `toFile()`/`isFile()`/`parseDocument`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@grails.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Verification harness for rendered guides (refs #354) [grails-static-website]
Posted by "jamesfredley (via GitHub)" <gi...@apache.org>.
jamesfredley merged PR #457:
URL: https://github.com/apache/grails-static-website/pull/457
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@grails.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org