You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "chennu2020 (via GitHub)" <gi...@apache.org> on 2025/07/25 14:28:37 UTC
[PR] JUnit5 support [beam]
chennu2020 opened a new pull request, #35688:
URL: https://github.com/apache/beam/pull/35688
**Please** add a meaningful description for your change here
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "chennu2020 (via GitHub)" <gi...@apache.org>.
chennu2020 commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2268057123
##########
CHANGES.md:
##########
@@ -103,6 +103,9 @@
* New highly anticipated feature X added to Python SDK ([#X](https://github.com/apache/beam/issues/X)).
* New highly anticipated feature Y added to Java SDK ([#Y](https://github.com/apache/beam/issues/Y)).
+* Added JUnit 5 support to Java SDK testing framework with new TestPipelineExtension class, now provided by a dedicated module (Java) ([#18733](https://github.com/apache/beam/issues/18733), [#35688](https://github.com/apache/beam/pull/35688)).
Review Comment:
Yes, conflict resolution showing up "Prism runner" in this PR. Removed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on PR #35688:
URL: https://github.com/apache/beam/pull/35688#issuecomment-3118070067
> [!IMPORTANT]
> **Installation incomplete:** to start using Gemini Code Assist, please ask the [organization owner(s)](https://docs.github.com/en/organizations/managing-peoples-access-to-your-organization-with-roles/roles-in-an-organization#organization-owners) to visit the [Gemini Code Assist Admin Console](https://codeassist.google/code-review/login) and sign the Terms of Services.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "chennu2020 (via GitHub)" <gi...@apache.org>.
chennu2020 commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2265479704
##########
sdks/java/io/jms/build.gradle:
##########
@@ -39,7 +39,8 @@ dependencies {
testImplementation library.java.activemq_kahadb_store
testImplementation library.java.activemq_client
testImplementation library.java.hamcrest
- testImplementation library.java.junit
+ // Explicitly specify JUnit 4 version to avoid conflicts with JUnit 5 BOM in core
+ testImplementation "junit:junit:4.13.2"
Review Comment:
@Abacn Created new module sdks/java/testing/junit and added JUnit5 related changes. Please let me know if any changes needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35688:
URL: https://github.com/apache/beam/pull/35688#issuecomment-3119853571
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35688:
URL: https://github.com/apache/beam/pull/35688#issuecomment-3120278510
Assigning reviewers:
R: @chamikaramj for label java.
Note: If you would like to opt out of this review, comment `assign to next reviewer`.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "idc101 (via GitHub)" <gi...@apache.org>.
idc101 commented on PR #35688:
URL: https://github.com/apache/beam/pull/35688#issuecomment-3347118918
I am trying to use this in v2.68 but the validation is still looking for the junit 4 Rule:
```
Is your TestPipeline declaration missing a @Rule annotation? Usage: @Rule public final transient TestPipeline pipeline = TestPipeline.create();
java.lang.IllegalStateException: Is your TestPipeline declaration missing a @Rule annotation? Usage: @Rule public final transient TestPipeline pipeline = TestPipeline.create();
at org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:512)
at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:414)
at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:366)
```
Recreate with:
```
import org.apache.beam.sdk.testing.NeedsRunner;
import org.apache.beam.sdk.testing.TestPipeline;
import org.apache.beam.sdk.testing.TestPipelineExtension;
import org.junit.experimental.categories.Category;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
@ExtendWith(TestPipelineExtension.class)
public class DeDuplicateTest implements Serializable {
@Test
@Category(NeedsRunner.class)
public void testBasicDeDuplicate(TestPipeline p) {
p.run();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2267784282
##########
CHANGES.md:
##########
@@ -119,6 +122,9 @@
## New Features / Improvements
* Added support for Processing time Timer in the Spark Classic runner ([#33633](https://github.com/apache/beam/issues/33633)).
+* Introduced a dedicated module for JUnit-based testing support: `sdks/java/testing/junit`, which provides `TestPipelineExtension` for JUnit 5 while maintaining backward compatibility with existing JUnit 4 `TestRule`-based tests (Java) ([#18733](https://github.com/apache/beam/issues/18733), [#35688](https://github.com/apache/beam/pull/35688)).
+ - Java core no longer carries JUnit 5 dependencies; production users of core will not receive test-only dependencies.
Review Comment:
The statement isn't accurate. Java core never carried JUnit 5 dependencies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2270146759
##########
CHANGES.md:
##########
@@ -103,6 +103,9 @@
* New highly anticipated feature X added to Python SDK ([#X](https://github.com/apache/beam/issues/X)).
* New highly anticipated feature Y added to Java SDK ([#Y](https://github.com/apache/beam/issues/Y)).
+* Added JUnit 5 support to Java SDK testing framework with new TestPipelineExtension class, now provided by a dedicated module (Java) ([#18733](https://github.com/apache/beam/issues/18733), [#35688](https://github.com/apache/beam/pull/35688)).
Review Comment:
> It goes to Beam 2.68.
Please check this and move the CHANGES.md item to 2.68.0 section
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn merged PR #35688:
URL: https://github.com/apache/beam/pull/35688
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2257219311
##########
sdks/java/core/build.gradle:
##########
@@ -130,3 +133,8 @@ project.tasks.compileTestJava {
// TODO: fix other places with warnings in tests and delete this option
options.compilerArgs += ['-Xlint:-rawtypes']
}
+
+// Configure test task to use both JUnit 4 and JUnit 5
+test {
+ useJUnit()
+}
Review Comment:
The comment indicates that this configuration is for both JUnit 4 and JUnit 5, but `useJUnit()` only supports JUnit 3 and 4. To correctly run both JUnit 4 and JUnit 5 tests, you should use `useJUnitPlatform()`. This will ensure that the test task discovers and executes tests from both frameworks.
```
test {
useJUnitPlatform()
}
```
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/testing/TestPipelineExtension.java:
##########
@@ -0,0 +1,227 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.testing;
+
+import static org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Preconditions.checkState;
+
+import java.lang.annotation.Annotation;
+import java.lang.reflect.Method;
+import java.util.Arrays;
+import java.util.Optional;
+import org.apache.beam.sdk.options.ApplicationNameOptions;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.testing.TestPipeline.PipelineAbandonedNodeEnforcement;
+import org.apache.beam.sdk.testing.TestPipeline.PipelineRunEnforcement;
+import org.junit.experimental.categories.Category;
+import org.junit.jupiter.api.extension.AfterEachCallback;
+import org.junit.jupiter.api.extension.BeforeEachCallback;
+import org.junit.jupiter.api.extension.ExtensionContext;
+import org.junit.jupiter.api.extension.ParameterContext;
+import org.junit.jupiter.api.extension.ParameterResolver;
+
+/**
+ * JUnit 5 extension for {@link TestPipeline} that provides the same functionality as the JUnit 4
+ * {@link org.junit.rules.TestRule} implementation.
+ *
+ * <p>Use this extension to test pipelines in JUnit 5:
+ *
+ * <pre><code>
+ * {@literal @}ExtendWith(TestPipelineExtension.class)
+ * class MyPipelineTest {
+ * {@literal @}Test
+ * {@literal @}Category(NeedsRunner.class)
+ * void myPipelineTest(TestPipeline pipeline) {
+ * final PCollection<String> pCollection = pipeline.apply(...)
+ * PAssert.that(pCollection).containsInAnyOrder(...);
+ * pipeline.run();
+ * }
+ * }
+ * </code></pre>
+ *
+ * <p>You can also create the extension yourself for more control:
+ *
+ * <pre><code>
+ * class MyPipelineTest {
+ * {@literal @}RegisterExtension
+ * final TestPipelineExtension pipeline = TestPipelineExtension.create();
+ *
+ * {@literal @}Test
+ * void testUsingPipeline() {
+ * pipeline.apply(...);
+ * pipeline.run();
+ * }
+ * }
+ * </code></pre>
+ */
+public class TestPipelineExtension
+ implements BeforeEachCallback, AfterEachCallback, ParameterResolver {
+
+ private static final ExtensionContext.Namespace NAMESPACE =
+ ExtensionContext.Namespace.create(TestPipelineExtension.class);
+ private static final String PIPELINE_KEY = "testPipeline";
+ private static final String ENFORCEMENT_KEY = "enforcement";
+
+ /** Creates a new TestPipelineExtension with default options. */
+ public static TestPipelineExtension create() {
+ return new TestPipelineExtension();
+ }
+
+ /** Creates a new TestPipelineExtension with custom options. */
+ public static TestPipelineExtension fromOptions(PipelineOptions options) {
+ return new TestPipelineExtension(options);
+ }
+
+ private TestPipeline testPipeline;
+
+ /** Creates a TestPipelineExtension with default options. */
+ public TestPipelineExtension() {
+ this.testPipeline = TestPipeline.create();
+ }
+
+ /** Creates a TestPipelineExtension with custom options. */
+ public TestPipelineExtension(PipelineOptions options) {
+ this.testPipeline = TestPipeline.fromOptions(options);
+ }
+
+ @Override
+ public boolean supportsParameter(
+ ParameterContext parameterContext, ExtensionContext extensionContext) {
+ return parameterContext.getParameter().getType() == TestPipeline.class;
+ }
+
+ @Override
+ public Object resolveParameter(
+ ParameterContext parameterContext, ExtensionContext extensionContext) {
+ if (this.testPipeline == null) {
+ return getOrCreateTestPipeline(extensionContext);
+ } else {
+ return this.testPipeline;
+ }
+ }
+
+ @Override
+ public void beforeEach(ExtensionContext context) throws Exception {
+ TestPipeline pipeline;
+
+ if (this.testPipeline != null) {
+ pipeline = this.testPipeline;
+ } else {
+ pipeline = getOrCreateTestPipeline(context);
+ }
+
+ Optional<PipelineRunEnforcement> enforcement = getOrCreateEnforcement(context);
+
+ // Set application name based on test method
+ String appName = getAppName(context);
+ pipeline.getOptions().as(ApplicationNameOptions.class).setAppName(appName);
+
+ // Set up enforcement based on annotations
+ setDeducedEnforcementLevel(context, pipeline, enforcement);
+ }
+
+ @Override
+ public void afterEach(ExtensionContext context) throws Exception {
+ Optional<PipelineRunEnforcement> enforcement = getEnforcement(context);
+ if (enforcement.isPresent()) {
+ enforcement.get().afterUserCodeFinished();
+ }
+ }
+
+ private TestPipeline getOrCreateTestPipeline(ExtensionContext context) {
+ return context
+ .getStore(NAMESPACE)
+ .getOrComputeIfAbsent(PIPELINE_KEY, key -> TestPipeline.create(), TestPipeline.class);
+ }
+
+ private Optional<PipelineRunEnforcement> getOrCreateEnforcement(ExtensionContext context) {
+ return context
+ .getStore(NAMESPACE)
+ .getOrComputeIfAbsent(
+ ENFORCEMENT_KEY, key -> Optional.<PipelineRunEnforcement>empty(), Optional.class);
+ }
+
+ private Optional<PipelineRunEnforcement> getEnforcement(ExtensionContext context) {
+ return context.getStore(NAMESPACE).get(ENFORCEMENT_KEY, Optional.class);
+ }
+
+ private void setEnforcement(
+ ExtensionContext context, Optional<PipelineRunEnforcement> enforcement) {
+ context.getStore(NAMESPACE).put(ENFORCEMENT_KEY, enforcement);
+ }
Review Comment:
Storing `Optional` instances in the `ExtensionContext.Store` is generally considered an anti-pattern as it can lead to confusing code and subtle bugs (e.g., `Optional<Optional<T>>`). It would be clearer and more robust to store the `PipelineRunEnforcement` instance directly (or `null`) and wrap the result with `Optional` upon retrieval.
A possible refactoring could look like this:
```java
private Optional<PipelineRunEnforcement> getEnforcement(ExtensionContext context) {
return Optional.ofNullable(
context.getStore(NAMESPACE).get(ENFORCEMENT_KEY, PipelineRunEnforcement.class));
}
private void setEnforcement(ExtensionContext context, PipelineRunEnforcement enforcement) {
context.getStore(NAMESPACE).put(ENFORCEMENT_KEY, enforcement);
}
```
This would require adjusting the call sites in `beforeEach`, `afterEach`, and `setDeducedEnforcementLevel` to work with these simplified helper methods. For example, `setDeducedEnforcementLevel` would no longer need to take an `Optional` parameter, and `getOrCreateEnforcement` could be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2267782089
##########
CHANGES.md:
##########
@@ -103,6 +103,9 @@
* New highly anticipated feature X added to Python SDK ([#X](https://github.com/apache/beam/issues/X)).
* New highly anticipated feature Y added to Java SDK ([#Y](https://github.com/apache/beam/issues/Y)).
+* Added JUnit 5 support to Java SDK testing framework with new TestPipelineExtension class, now provided by a dedicated module (Java) ([#18733](https://github.com/apache/beam/issues/18733), [#35688](https://github.com/apache/beam/pull/35688)).
Review Comment:
It goes to Beam 2.68. And the "Prism runner" item below is not relevant to this PR. Leftover of resolving merge conflict?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35688:
URL: https://github.com/apache/beam/pull/35688#issuecomment-3146467769
Reminder, please take a look at this pr: @chamikaramj
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "chennu2020 (via GitHub)" <gi...@apache.org>.
chennu2020 commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2270282161
##########
CHANGES.md:
##########
@@ -103,6 +103,9 @@
* New highly anticipated feature X added to Python SDK ([#X](https://github.com/apache/beam/issues/X)).
* New highly anticipated feature Y added to Java SDK ([#Y](https://github.com/apache/beam/issues/Y)).
+* Added JUnit 5 support to Java SDK testing framework with new TestPipelineExtension class, now provided by a dedicated module (Java) ([#18733](https://github.com/apache/beam/issues/18733), [#35688](https://github.com/apache/beam/pull/35688)).
Review Comment:
Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "chennu2020 (via GitHub)" <gi...@apache.org>.
chennu2020 commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2270210630
##########
CHANGES.md:
##########
@@ -103,6 +103,9 @@
* New highly anticipated feature X added to Python SDK ([#X](https://github.com/apache/beam/issues/X)).
* New highly anticipated feature Y added to Java SDK ([#Y](https://github.com/apache/beam/issues/Y)).
+* Added JUnit 5 support to Java SDK testing framework with new TestPipelineExtension class, now provided by a dedicated module (Java) ([#18733](https://github.com/apache/beam/issues/18733), [#35688](https://github.com/apache/beam/pull/35688)).
Review Comment:
"Prism runner" change notes already present in 2.68.0 hi lights. it was duplicated in 2.67.0, so removed from this section. Please let me know if I'm missing something.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "chennu2020 (via GitHub)" <gi...@apache.org>.
chennu2020 commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2268094637
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/testing/TestPipeline.java:
##########
@@ -97,6 +101,25 @@
* }
* </code></pre>
*
+ * <h3>JUnit Usage</h3>
Review Comment:
updated
##########
CHANGES.md:
##########
@@ -119,6 +122,9 @@
## New Features / Improvements
* Added support for Processing time Timer in the Spark Classic runner ([#33633](https://github.com/apache/beam/issues/33633)).
+* Introduced a dedicated module for JUnit-based testing support: `sdks/java/testing/junit`, which provides `TestPipelineExtension` for JUnit 5 while maintaining backward compatibility with existing JUnit 4 `TestRule`-based tests (Java) ([#18733](https://github.com/apache/beam/issues/18733), [#35688](https://github.com/apache/beam/pull/35688)).
+ - Java core no longer carries JUnit 5 dependencies; production users of core will not receive test-only dependencies.
Review Comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #35688:
URL: https://github.com/apache/beam/pull/35688#issuecomment-3347180332
> I am trying to use this in v2.68 but the validation is still looking for the junit 4 Rule:
Fixed in #36258. Unfortunately this new module isn't functioning in Beam 2.68. Will be fixed in the next release.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "chennu2020 (via GitHub)" <gi...@apache.org>.
chennu2020 commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2265479704
##########
sdks/java/io/jms/build.gradle:
##########
@@ -39,7 +39,8 @@ dependencies {
testImplementation library.java.activemq_kahadb_store
testImplementation library.java.activemq_client
testImplementation library.java.hamcrest
- testImplementation library.java.junit
+ // Explicitly specify JUnit 4 version to avoid conflicts with JUnit 5 BOM in core
+ testImplementation "junit:junit:4.13.2"
Review Comment:
@Abacn Created new module sdks/java/testing/junit and added JUnit5 related changes. Please let me know if any changes needed. Also removed jms gradle change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "chennu2020 (via GitHub)" <gi...@apache.org>.
chennu2020 commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2265479704
##########
sdks/java/io/jms/build.gradle:
##########
@@ -39,7 +39,8 @@ dependencies {
testImplementation library.java.activemq_kahadb_store
testImplementation library.java.activemq_client
testImplementation library.java.hamcrest
- testImplementation library.java.junit
+ // Explicitly specify JUnit 4 version to avoid conflicts with JUnit 5 BOM in core
+ testImplementation "junit:junit:4.13.2"
Review Comment:
Removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2257124240
##########
sdks/java/io/jms/build.gradle:
##########
@@ -39,7 +39,8 @@ dependencies {
testImplementation library.java.activemq_kahadb_store
testImplementation library.java.activemq_client
testImplementation library.java.hamcrest
- testImplementation library.java.junit
+ // Explicitly specify JUnit 4 version to avoid conflicts with JUnit 5 BOM in core
+ testImplementation "junit:junit:4.13.2"
Review Comment:
why only jms build.gradle need this change? Also this won't be effective as BeamModulePlugin forced all dependencies: https://github.com/apache/beam/blob/624b2fcd3fdd249d3a13b6faba6e71521e9fd0c1/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L1270
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35688:
URL: https://github.com/apache/beam/pull/35688#issuecomment-3159928507
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`:
R: @Abacn for label java.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #35688:
URL: https://github.com/apache/beam/pull/35688#issuecomment-3160239344
/gemini review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "chennu2020 (via GitHub)" <gi...@apache.org>.
chennu2020 commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2265482060
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/testing/TestPipelineExtension.java:
##########
@@ -0,0 +1,227 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.testing;
+
+import static org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Preconditions.checkState;
+
+import java.lang.annotation.Annotation;
+import java.lang.reflect.Method;
+import java.util.Arrays;
+import java.util.Optional;
+import org.apache.beam.sdk.options.ApplicationNameOptions;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.testing.TestPipeline.PipelineAbandonedNodeEnforcement;
+import org.apache.beam.sdk.testing.TestPipeline.PipelineRunEnforcement;
+import org.junit.experimental.categories.Category;
+import org.junit.jupiter.api.extension.AfterEachCallback;
+import org.junit.jupiter.api.extension.BeforeEachCallback;
+import org.junit.jupiter.api.extension.ExtensionContext;
+import org.junit.jupiter.api.extension.ParameterContext;
+import org.junit.jupiter.api.extension.ParameterResolver;
+
+/**
+ * JUnit 5 extension for {@link TestPipeline} that provides the same functionality as the JUnit 4
+ * {@link org.junit.rules.TestRule} implementation.
+ *
+ * <p>Use this extension to test pipelines in JUnit 5:
+ *
+ * <pre><code>
+ * {@literal @}ExtendWith(TestPipelineExtension.class)
+ * class MyPipelineTest {
+ * {@literal @}Test
+ * {@literal @}Category(NeedsRunner.class)
+ * void myPipelineTest(TestPipeline pipeline) {
+ * final PCollection<String> pCollection = pipeline.apply(...)
+ * PAssert.that(pCollection).containsInAnyOrder(...);
+ * pipeline.run();
+ * }
+ * }
+ * </code></pre>
+ *
+ * <p>You can also create the extension yourself for more control:
+ *
+ * <pre><code>
+ * class MyPipelineTest {
+ * {@literal @}RegisterExtension
+ * final TestPipelineExtension pipeline = TestPipelineExtension.create();
+ *
+ * {@literal @}Test
+ * void testUsingPipeline() {
+ * pipeline.apply(...);
+ * pipeline.run();
+ * }
+ * }
+ * </code></pre>
+ */
+public class TestPipelineExtension
+ implements BeforeEachCallback, AfterEachCallback, ParameterResolver {
+
+ private static final ExtensionContext.Namespace NAMESPACE =
+ ExtensionContext.Namespace.create(TestPipelineExtension.class);
+ private static final String PIPELINE_KEY = "testPipeline";
+ private static final String ENFORCEMENT_KEY = "enforcement";
+
+ /** Creates a new TestPipelineExtension with default options. */
+ public static TestPipelineExtension create() {
+ return new TestPipelineExtension();
+ }
+
+ /** Creates a new TestPipelineExtension with custom options. */
+ public static TestPipelineExtension fromOptions(PipelineOptions options) {
+ return new TestPipelineExtension(options);
+ }
+
+ private TestPipeline testPipeline;
+
+ /** Creates a TestPipelineExtension with default options. */
+ public TestPipelineExtension() {
+ this.testPipeline = TestPipeline.create();
+ }
+
+ /** Creates a TestPipelineExtension with custom options. */
+ public TestPipelineExtension(PipelineOptions options) {
+ this.testPipeline = TestPipeline.fromOptions(options);
+ }
+
+ @Override
+ public boolean supportsParameter(
+ ParameterContext parameterContext, ExtensionContext extensionContext) {
+ return parameterContext.getParameter().getType() == TestPipeline.class;
+ }
+
+ @Override
+ public Object resolveParameter(
+ ParameterContext parameterContext, ExtensionContext extensionContext) {
+ if (this.testPipeline == null) {
+ return getOrCreateTestPipeline(extensionContext);
+ } else {
+ return this.testPipeline;
+ }
+ }
+
+ @Override
+ public void beforeEach(ExtensionContext context) throws Exception {
+ TestPipeline pipeline;
+
+ if (this.testPipeline != null) {
+ pipeline = this.testPipeline;
+ } else {
+ pipeline = getOrCreateTestPipeline(context);
+ }
+
+ Optional<PipelineRunEnforcement> enforcement = getOrCreateEnforcement(context);
+
+ // Set application name based on test method
+ String appName = getAppName(context);
+ pipeline.getOptions().as(ApplicationNameOptions.class).setAppName(appName);
+
+ // Set up enforcement based on annotations
+ setDeducedEnforcementLevel(context, pipeline, enforcement);
+ }
+
+ @Override
+ public void afterEach(ExtensionContext context) throws Exception {
+ Optional<PipelineRunEnforcement> enforcement = getEnforcement(context);
+ if (enforcement.isPresent()) {
+ enforcement.get().afterUserCodeFinished();
+ }
+ }
+
+ private TestPipeline getOrCreateTestPipeline(ExtensionContext context) {
+ return context
+ .getStore(NAMESPACE)
+ .getOrComputeIfAbsent(PIPELINE_KEY, key -> TestPipeline.create(), TestPipeline.class);
+ }
+
+ private Optional<PipelineRunEnforcement> getOrCreateEnforcement(ExtensionContext context) {
+ return context
+ .getStore(NAMESPACE)
+ .getOrComputeIfAbsent(
+ ENFORCEMENT_KEY, key -> Optional.<PipelineRunEnforcement>empty(), Optional.class);
+ }
+
+ private Optional<PipelineRunEnforcement> getEnforcement(ExtensionContext context) {
+ return context.getStore(NAMESPACE).get(ENFORCEMENT_KEY, Optional.class);
+ }
+
+ private void setEnforcement(
+ ExtensionContext context, Optional<PipelineRunEnforcement> enforcement) {
+ context.getStore(NAMESPACE).put(ENFORCEMENT_KEY, enforcement);
+ }
Review Comment:
Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2267785719
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/testing/TestPipeline.java:
##########
@@ -97,6 +101,25 @@
* }
* </code></pre>
*
+ * <h3>JUnit Usage</h3>
Review Comment:
JUnit 5 Usage?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2270225521
##########
CHANGES.md:
##########
@@ -103,6 +103,9 @@
* New highly anticipated feature X added to Python SDK ([#X](https://github.com/apache/beam/issues/X)).
* New highly anticipated feature Y added to Java SDK ([#Y](https://github.com/apache/beam/issues/Y)).
+* Added JUnit 5 support to Java SDK testing framework with new TestPipelineExtension class, now provided by a dedicated module (Java) ([#18733](https://github.com/apache/beam/issues/18733), [#35688](https://github.com/apache/beam/pull/35688)).
Review Comment:
I mean the item "Added JUnit 5 support to Java SDK..." appears in "2.67.0" Section in CHANGES.md. It should be in under "2.68.0" section. Also, one CHANGES item is sufficient, we can deduplicate statements under "Highlights" and "New Features / Improvements"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] JUnit5 support [beam]
Posted by "chennu2020 (via GitHub)" <gi...@apache.org>.
chennu2020 commented on code in PR #35688:
URL: https://github.com/apache/beam/pull/35688#discussion_r2268057123
##########
CHANGES.md:
##########
@@ -103,6 +103,9 @@
* New highly anticipated feature X added to Python SDK ([#X](https://github.com/apache/beam/issues/X)).
* New highly anticipated feature Y added to Java SDK ([#Y](https://github.com/apache/beam/issues/Y)).
+* Added JUnit 5 support to Java SDK testing framework with new TestPipelineExtension class, now provided by a dedicated module (Java) ([#18733](https://github.com/apache/beam/issues/18733), [#35688](https://github.com/apache/beam/pull/35688)).
Review Comment:
Yes, conflict resolution showing up "Prism runner" in this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org