You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tomee.apache.org by ju...@apache.org on 2026/07/19 12:25:54 UTC
(tomee-tck) 04/07: Document the security-old baseline and its one product-gap exclusion
This is an automated email from the ASF dual-hosted git repository.
jungm pushed a commit to branch ee11
in repository https://gitbox.apache.org/repos/asf/tomee-tck.git
commit 848420b6610444fa5dc22dfb38e7697a6e90d8cf
Author: Markus Jung <ju...@apache.org>
AuthorDate: Sun Jul 19 10:53:10 2026 +0200
Document the security-old baseline and its one product-gap exclusion
---
KNOWN_ISSUES.md | 37 +++++++++++++++++++--------
runner-standalone/README.md | 10 +++++---
runner-standalone/exclusions/README.md | 6 ++---
runner-standalone/exclusions/security-old.txt | 18 +++++++++++++
4 files changed, 53 insertions(+), 18 deletions(-)
diff --git a/KNOWN_ISSUES.md b/KNOWN_ISSUES.md
index f0e2d51..2f120fe 100644
--- a/KNOWN_ISSUES.md
+++ b/KNOWN_ISSUES.md
@@ -38,6 +38,7 @@ Detail lives next to each runner:
| jsonb | 295 tests, 1 F + 1 E | 2 tests | 2 Johnzon 2.1.0 gaps |
| debugging | passes (4 SMAPs validated) | — | — |
| security | 132 tests, 5 F + 2 E; signature test passes | 7 tests | TomEE Jakarta Security |
+| security-old (JavaTest) | 83 tests, 82 pass, 1 F | 1 test | TomEE `SecurityContext.hasAccessToWebResource()` returns false for an authorized caller |
| authentication | 106 tests, 1 F; signature test passes | 1 method (spi `CheckMsgInfoKey`) | TCK challenge #219 (hard-codes a JACC requirement) |
| websocket | 737 tests, 3 E | 3 methods | Client container advertises permessage-deflate in the negotiated extension lists |
| faces (modern modules) | 327 tests, 45 F + 6 E (failsafe reruns inflate the counts; 10 distinct failing classes) | 10 classes | 1 TomEE faces-config gap + Mojarra/TomEE CDI-injection, whole-bean/method validation, and one Chrome ajax quirk |
@@ -114,20 +115,30 @@ Fixes belong in Apache TomEE (or Tomcat); each removes exclusion entries.
5. **Jakarta Security** — the BASIC mechanism answers 401 for valid
credentials in the decorated/custom-handler variants, and both OpenID
default modules fail token validation.
- [security.txt](runner-standalone/exclusions/security.txt). The runner drives
- the reactor's modern Arquillian app modules plus the signature test; the
- reactor's `old-tck` module (~68 legacy JavaTest tests) is not wired into
- the runner, so the recorded totals cover the modern suite only.
-6. **Persistence integration** — undeploy calls `close()` on an
+ [security.txt](runner-standalone/exclusions/security.txt). The `security`
+ runner drives the reactor's modern Arquillian app modules plus the signature
+ test; the reactor's `old-tck` module (legacy JavaTest suite,
+ `com.sun.ts.tests.securityapi`) is run by the separate `security-old` runner.
+6. **Jakarta Security `SecurityContext.hasAccessToWebResource()`** — the
+ programmatic access check returns `false` for a caller that is authorized
+ for the resource. The old-tck `securitycontext/callerdata` servlet reports
+ the correct caller and role membership, but `hasAccessToWebResource(
+ "/protectedServlet", "GET")` answers `false` where the spec requires `true`
+ for user `tom` (Manager role) against the `@HttpMethodConstraint("GET")`
+ resource; TomEE's `SecurityContext` is not wired to the servlet
+ authorization/`Policy` layer for this call. Excluded in
+ [security-old.txt](runner-standalone/exclusions/security-old.txt)
+ (`securitycontext/callerdata/Client.java#testSecurityContextHasAccessToWebResource`).
+7. **Persistence integration** — undeploy calls `close()` on an
already-closed `EntityManagerFactory` (fails the
`entityManagerFactoryCloseExceptions` vehicles), and the Jakarta
Persistence 3.2 CDI qualifier beans (`EntityManagerFactory`/
`EntityManager` etc. from `persistence.xml`) are not registered
(`ServletEMLookupTest`). Affects Plume and webprofile alike.
-7. **Jakarta Tags TLD registration** — the `jakarta.tags.*` URIs of the
+8. **Jakarta Tags TLD registration** — the `jakarta.tags.*` URIs of the
replacement Jakarta Tags 3.0 jar are not exposed to applications; all 50
Tags classes plus the EJB-Lite JSP vehicles fail as collateral.
-8. **Transactions — cross-request `UserTransaction` state leakage across
+9. **Transactions — cross-request `UserTransaction` state leakage across
pooled servlet requests.** A `UserTransaction` a servlet/jsp request leaves
in a non-clean state poisons the next request served on the same pooled
Tomcat exec thread; the victim sees an `IllegalStateException` that is not
@@ -148,14 +159,14 @@ Fixes belong in Apache TomEE (or Tomcat); each removes exclusion entries.
[transactions.txt](runner-standalone/exclusions/transactions.txt). The
Platform catalog additionally shows CDI `@Transactional` interceptors
failing propagation, rollback-rule, and `TransactionScoped` assertions.
-9. **Enterprise Beans** — timer callbacks expose incomplete/not-retried
+10. **Enterprise Beans** — timer callbacks expose incomplete/not-retried
transactions, `java:comp` is mutable where the spec requires
`OperationNotSupportedException`, and failed CDI/EJB deployments leak
deployment IDs (`DuplicateDeploymentIdException` in later apps).
-10. **webprofile ZIP signature leak** — the combined `jakartaee-api` jar
+11. **webprofile ZIP signature leak** — the combined `jakartaee-api` jar
exposes Jakarta Batch and Messaging packages although `javaee.level=web`
does not declare them; strip them or declare and certify them.
-11. **WebSocket 2.2 extension advertising (Tomcat)** — the server-side
+12. **WebSocket 2.2 extension advertising (Tomcat)** — the server-side
configurator reports the extensions the client requested and negotiated.
TomEE's client-side WebSocket container (Tomcat's `tomcat-websocket`)
always advertises its built-in `permessage-deflate` extension in the
@@ -168,7 +179,7 @@ Fixes belong in Apache TomEE (or Tomcat); each removes exclusion entries.
and pass: the runner's Arquillian extension tolerates the deployment
failure so each client probe still runs.
[websocket.txt](runner-standalone/exclusions/websocket.txt).
-12. **Bean Validation XML config broken on stock Plume** — the Plume
+13. **Bean Validation XML config broken on stock Plume** — the Plume
distribution ships EclipseLink MOXy (`eclipselink-5.0.1.jar`) and the JAXB
RI (`jaxb-runtime-4.0.4.jar`) side by side. EclipseLink registers a
`jakarta.xml.bind.JAXBContextFactory` service and wins ServiceLoader
@@ -256,6 +267,10 @@ Not product bugs — gaps in this repository's coverage.
ships an `old-tck-selenium` module (22 tests) that neither the `faces`
runner (which drives faces22/23/40/41 + faces-signaturetest) nor the
`faces-old` JavaTest runner executes; wire it into one of them.
+- **Security `security-old` not in CI**: the `security-old` runner (the
+ reactor's legacy JavaTest `com.sun.ts.tests.securityapi` suite) runs
+ locally via `run-standalone-suite.sh` and is green with the reviewed
+ exclusion; it joins the Jenkins branch list once its baseline has settled.
## What CI runs
diff --git a/runner-standalone/README.md b/runner-standalone/README.md
index 79f7fd7..243b163 100644
--- a/runner-standalone/README.md
+++ b/runner-standalone/README.md
@@ -13,10 +13,11 @@ runner-standalone/run-standalone-suite.sh concurrency
The Derby port is overridable everywhere through `-Dtck.derby.port`; the
`concurrency`, `data`, `servlet`, `pages`, `rest`, `websocket`,
-`transactions`, and `faces-old` runners additionally accept
+`transactions`, `faces-old`, and `security-old` runners additionally accept
`-Dtomee.http.port`, `-Dtomee.https.port`, and `-Dtomee.shutdown.port`
-(`transactions` and `faces-old` also `-Dtck.harness.log.port` for their
-JavaTest listeners), so they can run next to another harness instance. The
+(`transactions`, `faces-old`, and `security-old` also `-Dtck.harness.log.port`
+for their JavaTest listeners), so they can run next to another harness
+instance. The
`validation`, `cdi`, and `cdi-ee` runners parameterize only the HTTP and
shutdown ports (`-Dtomee.http.port`/`-Dtomee.shutdown.port`); their
arquillian.xml pins no `httpsPort`. The source-reactor runners security,
@@ -59,7 +60,8 @@ derived from; with the default exclusions applied these suites run green.
| Pages 4.0 | EFTL zip (installed as `jakarta.tck:jakarta-pages-tck:4.0.0`) | `pages` | **Passes 682/682 (0 failures, 0 errors, signature test included, no exclusions)** (2026-07-18). Needs the Central `jakarta.tck:common`/`signaturetest` support line at 11.1.1 with sigtest 2.6 (the TCK pom's own pins depend on the unpublished JavaTest harness), and the overlay removes TomEE's global UTF-8 default encodings from conf/web.xml because the TCK asserts the spec default ISO-8859-1. Its TomEE/ [...]
| Validation 3.1 | EFTL zip (installed as `jakarta.validation:validation-tck-tests:3.1.1`) | `validation` | **Runs: 1,049 tests, 0 failures, no exclusions; the mandatory API signature test runs and passes** against the `jakarta.validation` packages the Plume distribution ships. Needs AssertJ 3.7.0 pinned (the published TCK jar is compiled against its covariant signatures) and the JAXB RI pinned as `jakarta.xml.bind.JAXBContextFactory` in the server JVM (Plume ships both MOXy and the RI; [...]
| WebSocket 2.2 | EFTL zip (installed as `jakarta.tck:websocket-tck-spec-tests:2.2.0`) | `websocket` | **Runs: 737 tests, 0 failures, 3 errors** (2026-07-18) against Tomcat's WebSocket implementation, driven through the `tomcat-websocket` client container; the signature test passes. The negative-deployment classes run and pass: an invalid server endpoint aborts the whole webapp deployment as the spec requires, and the runner's Arquillian extension (`NegativeDeploymentToleranceObserver`) [...]
-| Security 4.0 | Source reactor zip 4.0.1 | `security` (Maven module) | **Runs: 26 app modules plus the signature test, 132 tests, 5 failures, 2 errors at baseline; green with the reviewed exclusions (all 27 invoker projects pass, verified end-to-end 2026-07-18)**. The runner downloads and patches the reactor, injects a tomee-remote profile (including the `trustStore.path`/`trustStore.password` properties the OpenID modules' keytool steps read), and drives every module through the Maven [...]
+| Security 4.0 | Source reactor zip 4.0.1 | `security` (Maven module) | **Runs: 26 app modules plus the signature test, 132 tests, 5 failures, 2 errors at baseline; green with the reviewed exclusions (all 27 invoker projects pass, verified end-to-end 2026-07-18)**. The runner downloads and patches the reactor, injects a tomee-remote profile (including the `trustStore.path`/`trustStore.password` properties the OpenID modules' keytool steps read), and drives every module through the Maven [...]
+| Security 4.0 (legacy old-tck) | Source reactor zip 4.0.1 (old-tck built from bundled sources) | `security-old` (Maven module) | **Baseline 83 tests, 82 pass, 1 failure (Plume, JDK 21, 2026-07-19); green with the reviewed exclusion.** The legacy JavaTest half exercises ~65 `com.sun.ts.tests.securityapi` Client classes across the idstore (in-war, database, LDAP), ham (HTTP authentication mechanism) and securitycontext trees. It deploys through the TCK's own `tomcat` handler (WAR copy int [...]
| Authentication 3.1 | Source reactor zip 3.1.2 | `authentication` (Maven module) | **Runs: 12 Web Profile modules plus the signature test; green with the reviewed exclusions (all 13 invoker projects pass, 0 failures, verified end-to-end 2026-07-18)**. The spi module registers the TCK's test `AuthConfigProvider` under Tomcat's JASPIC app-context naming (`Catalina/localhost /spitests_servlet_web`), so `ServletProfileSPITest` runs against Tomcat's `AuthConfigFactory` and passes 56 of its 5 [...]
| Faces 4.1 | Source reactor zip 4.1.2 | `faces` (Maven module) | **Runs: modern Arquillian modules (faces22/23/40/41) — 327 tests, 45 failures + 6 errors** (2026-07-18) on Plume's Mojarra; the failsafe rerun mechanism inflates those totals, covering 10 distinct failing classes, all in `exclusions/faces.txt`. One is a TomEE product gap (faces-config.xml unmarshaller rejecting the nonstandard *https* `xsi:schemaLocation`, `Spec1760IT`); the rest are Mojarra/TomEE integration behaviors — F [...]
| Faces 4.1 (legacy old-tck) | Source reactor zip 4.1.2 (old-tck built from bundled sources) | `faces-old` (Maven module) | **Passes 5,391/5,391 (standalone mode, no exclusions)** (2026-07-18). The recorded full run shows 5 failures in `htmloutcometargetbutton`, all from a foreign server answering port 8080 mid-run; they pass on re-run against TomEE. The JavaTest half deploys through the TCK's own `tomcat` handler (WAR copy into `webapps/`) against a TomEE instance the runner provisions [...]
diff --git a/runner-standalone/exclusions/README.md b/runner-standalone/exclusions/README.md
index f71ee3e..64644cc 100644
--- a/runner-standalone/exclusions/README.md
+++ b/runner-standalone/exclusions/README.md
@@ -24,9 +24,9 @@ Two formats, depending on how the runner executes tests:
both the declaring class and the concrete runtime class — several TCK
base classes contribute inherited test methods that only the subclass
name identifies.
-- **JavaTest-based runners** (`transactions`, `faces-old`): JavaTest jtx
- exclusion lines (`test/path/File.java#testid`), appended by the runner to
- the harness `ts.jtx` before the run.
+- **JavaTest-based runners** (`transactions`, `faces-old`, `security-old`):
+ JavaTest jtx exclusion lines (`test/path/File.java#testid`), appended by the
+ runner to the harness `ts.jtx` before the run.
To collect a full compatibility baseline without exclusions, run a suite with
`-Dtck.exclusions.file=$(pwd)/runner-standalone/exclusions/none.txt` (the
diff --git a/runner-standalone/exclusions/security-old.txt b/runner-standalone/exclusions/security-old.txt
new file mode 100644
index 0000000..50bb8a6
--- /dev/null
+++ b/runner-standalone/exclusions/security-old.txt
@@ -0,0 +1,18 @@
+# Reviewed known-gap exclusions for the legacy Jakarta Security 4.0 old-tck
+# (JavaTest) run against TomEE. Lines use the JavaTest jtx format
+# (test/dir/path#testid) and are appended to the mode's ts.jtx by the
+# runner-standalone-security-old module. See KNOWN_ISSUES.md for the
+# rationale behind each entry.
+#
+# Baseline 2026-07-19 (Plume, JDK 21): 83 tests, 82 passed, 1 failed,
+# 0 errors. The single failure below is a TomEE product gap.
+
+# TomEE product gap: SecurityContext.hasAccessToWebResource() returns false
+# for a caller that is authorized for the resource. The servlet reports the
+# correct caller ("context username: tom") and the authentication path works,
+# but the programmatic access check answers "has GET access ... : false" where
+# the spec requires true for user tom (Manager role) against the
+# @HttpMethodConstraint("GET") /protectedServlet. TomEE's SecurityContext is
+# not wired to the servlet authorization/Policy layer for
+# hasAccessToWebResource(). Assertions Security:JAVADOC:7/8, Security:SPEC:4.3-1.
+com/sun/ts/tests/securityapi/securitycontext/callerdata/Client.java#testSecurityContextHasAccessToWebResource