You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "cloud-fan (via GitHub)" <gi...@apache.org> on 2026/05/12 04:58:10 UTC
[PR] [SPARK-56832][INFRA] Surface fatal javadoc errors in unidoc log summary and CI annotations [spark]
cloud-fan opened a new pull request, #55814:
URL: https://github.com/apache/spark/pull/55814
### What changes were proposed in this pull request?
After the noise filters from #55605, the Documentation generation CI log is around 4K lines on a failure run. The two-line per-file `error: reference not found` diagnostics are still buried in the middle of the log, and the GitHub Actions check panel for a failed doc-gen job only surfaces `Process completed with exit code 1`. Reviewers end up scrolling the raw log to find what actually broke.
This PR is purely additive in `docs/_plugins/build_api_docs.rb` -- no existing log lines are dropped. After the unidoc pipe closes:
1. A trailing `Fatal javadoc errors (N):` block is printed, listing each captured diagnostic with file, line, and message.
2. One `::error file=<path>,line=<line>,title=javadoc::<msg>` GitHub Actions workflow command is emitted per diagnostic, so they appear as inline annotations on the PR check panel instead of as a single opaque `exit code 1`.
Diagnostics are captured strictly within the Standard Doclet phase bracketed by `Building tree for all the packages and classes...` and `Building index for all classes...`, which is where doclint emits the build-failing diagnostics that count toward javadoc's exit code. Source-loading `error:` chatter outside that window is excluded -- it's already non-fatal and matches what javadoc's own `N errors` summary line counts.
As a self-check, the captured count is compared against javadoc's own `N errors` summary line. If they diverge -- e.g. because a future JDK changes the Standard Doclet phase wording -- a `::warning::` workflow command is emitted so the drift is surfaced without silently masking real failures.
### Why are the changes needed?
PR #55605 made the doc-gen log small enough to read, but the failure path is still discoverable only via grep. The per-file diagnostics emitted by doclint are the actionable content; promoting them to the PR check panel and a clearly delimited summary block makes a doc-gen failure self-explanatory without leaving the PR.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
End-to-end on this branch with deliberately broken references planted in two code paths (mirroring the test pattern from PR #55605):
- `ColumnarMap.java` (real Java source): `{@link org.apache.spark.deliberately.NoSuchClass}` and `{@link ColumnVector#nonExistentMethod()}`.
- `Partition.scala` (Scala source via genjavadoc): `[[Partition.index]]` -- the `.`-separator case that javadoc treats as inner-class lookup.
The Documentation generation job will fail with the expected `Fatal javadoc errors` summary block in the log and per-file inline annotations on this PR's check panel. The plant commit will be dropped before this PR is taken out of draft.
The state machine was also exercised locally against a captured log from a prior failing doc-gen run; the captured fatal count matches javadoc's `N errors` summary line.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude (Anthropic)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56832][INFRA] Surface fatal javadoc errors in unidoc log summary and CI annotations [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #55814:
URL: https://github.com/apache/spark/pull/55814#issuecomment-4438339504
thanks for review, merging to master/4.x/4.2!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56832][INFRA] Surface fatal javadoc errors in unidoc log summary and CI annotations [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #55814:
URL: https://github.com/apache/spark/pull/55814#issuecomment-4430765610
cc @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56832][INFRA] Surface fatal javadoc errors in unidoc log summary and CI annotations [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan closed pull request #55814: [SPARK-56832][INFRA] Surface fatal javadoc errors in unidoc log summary and CI annotations
URL: https://github.com/apache/spark/pull/55814
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56832][INFRA] Surface fatal javadoc errors in unidoc log summary and CI annotations [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #55814:
URL: https://github.com/apache/spark/pull/55814#issuecomment-4430719032
### End-to-end validation result
Validated on the prior branch state with the `VALIDATION (DO NOT MERGE)` plant commit (now dropped, force-pushed away — branch is now clean at `db6d05943a9`). The plant carried 3 deliberately broken references; the documentation generation job picked them up as follows.
**Summary block in the job log** (post-pipe, additive, drops no existing lines):
```
========================================================================
Fatal javadoc errors (3):
========================================================================
1. /__w/spark/spark/core/target/java/org/apache/spark/Partition.java:6: reference not found
* referencing the index field with the wrong separator: {@link Partition.index} --
^
2. /__w/spark/spark/sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ColumnarMap.java:26: reference not found
* a class that does not exist: {@link org.apache.spark.deliberately.NoSuchClass}
^
3. /__w/spark/spark/sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ColumnarMap.java:27: reference not found
* and a method that does not exist: {@link ColumnVector#nonExistentMethod()}.
^
========================================================================
```
**GitHub Actions check panel annotations** (one per fatal diagnostic, `title=javadoc`, multi-line message rendering the error + source snippet + caret):
| File | Line | Message |
| --- | --- | --- |
| `sql/catalyst/.../ColumnarMap.java` | 26 | `reference not found` + `* a class that does not exist: {@link org.apache.spark.deliberately.NoSuchClass}` + `^` |
| `sql/catalyst/.../ColumnarMap.java` | 27 | `reference not found` + `* and a method that does not exist: {@link ColumnVector#nonExistentMethod()}.` + `^` |
| `core/target/java/.../Partition.java` | 6 | `reference not found` + `* referencing the index field with the wrong separator: {@link Partition.index} --` + `^` |
This replaces the previous user experience where the PR check panel surfaced only `Process completed with exit code 1` and the actionable diagnostics required scrolling the raw job log.
**Self-check:** the captured count (3) matched javadoc's own `3 errors` summary line, so no `::warning::` drift signal was emitted. If a future JDK changes the Standard Doclet phase wording, the mismatch would surface that as a workflow warning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org