[jira] [Created] (FELIX-6870) Integration tests across the repository run against released bundles, not the code being built

["Paul Rütter (Jira)" <ji...@apache.org>]

Paul Rütter created FELIX-6870:
----------------------------------

             Summary: Integration tests across the repository run against released bundles, not the code being built
                 Key: FELIX-6870
                 URL: https://issues.apache.org/jira/browse/FELIX-6870
             Project: Felix
          Issue Type: Improvement
            Reporter: Paul Rütter


Most modules pin released versions of sibling Felix bundles as {{{}test{}}}-scoped dependencies, and pax-exam then runs their integration tests on those released artifacts rather than on the code in this repository. So those tests cannot catch a regression introduced here, and several run on a framework two major versions behind.

Found while fixing FELIX-6869, where exactly this hid a real bug: {{FilterImpl.WrapperCapability}} had {{Filter.matches(Map)}} throwing {{UnsupportedOperationException}} for any non-empty map, which broke whiteboard servlet registration. {{JettySniHostCheckIT}} passes on the pinned framework 7.0.5 and fails with {{404}} instead of {{200}} on the framework in this repository. The regression sat on master undetected until the HTTP tests were pointed at the local build (FELIX-554 / FELIX-6869).

Full survey, {{{}test{}}}-scoped pins of Felix artifacts:
||module||artifact||pinned||
|{{configadmin}}|framework|7.0.5|
|{{configadmin}}|framework.security|2.8.3|
|{{eventadmin/impl}}|framework|6.0.3|
|{{gogo/jline}}|gogo.command|1.1.2|
|{{healthcheck/core}}|framework|7.0.5|
|{{healthcheck/webconsoleplugin}}|healthcheck.core|2.0.0|
|{{http/itest}}|framework|7.0.5|
|{{http/jetty}}|framework|7.0.5 → fixed by FELIX-6869|
|{{http/jetty12}}|framework|7.0.5 → fixed by FELIX-6869|
|{{ipojo/manipulator/manipulator-it}}|configadmin|1.6.0|
|{{ipojo/runtime/composite-it}}|configadmin|1.8.0|
|{{ipojo/runtime/core-it}}|configadmin|1.8.0|
|{{ipojo/runtime/core}}|framework|4.2.1|
|{{resolver}}|utils|1.8.0|
|{{rootcause}}|framework|6.0.5|
|{{scr}}|configadmin|1.9.0|
|{{scr}}|framework|7.0.1|
|{{systemready}}|framework|6.0.5|

Plus {{ConfigAdminSecurityTest}} provisions {{org.apache.felix.framework.security}} *2.6.1* inline in the test source, a third version of the same artifact.

{*}Suggested approach{*}, following what FELIX-6869 did for the two Jetty modules:
 # Make the version a property, keeping the current release as the default so each module still builds standalone — the modules here are built independently, not through one reactor, so a {{SNAPSHOT}} default would break a fresh clone.
 # Have CI build and install the producing module first and override the property with the version it just built, e.g. {{{}-Dfelix.framework.version=$(mvn help:evaluate -Dexpression=project.version ...){}}}.
 # Order the CI steps accordingly, and make the producer step run when either it _or_ a consumer changed — otherwise a consumer-only change finds nothing to resolve.

{*}Expect this to surface failures{*}, and that is the point. Switching a module over should be verified per module rather than in one sweep. Two things known in advance:
 * {{org.apache.felix.framework.security}} is removed in the 8.0.0 framework line (FELIX-6807), so {{{}configadmin{}}}’s pins of it — in both the pom and the test source — have no counterpart there. {{ConfigAdminSecurityTest}} is already excluded above JDK 17 because it needs {{{}-Djava.security.manager=allow{}}}, fatal from Java 24 on (JEP 486).
 * {{{}scr{}}}’s integration harness could not be verified locally on Windows: {{ComponentTestBase.createBundleInputStream}} fails with {{IOException: Access is denied}} on both the pinned 7.0.1 and the local build, so that one needs checking in CI on Linux.

Some of these are moot if the module is retired — {{ipojo}} has a removal PR open and pins framework 4.2.1 — so it is worth triaging the list before changing anything.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)