You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "Abacn (via GitHub)" <gi...@apache.org> on 2026/07/09 00:03:30 UTC

[PR] Support ACKNOWLEDGEMENT_MODE for JmsIO [beam]

Abacn opened a new pull request, #39253:
URL: https://github.com/apache/beam/pull/39253

   * CheckpointMark behavior in alignment with different ACKNOWLEDGEMENT_MODE
   
   * Ref count active checkpoint for quicker onClose that releases session
   
   * Fix hanging checkpoint when no incoming data in direct runner. This allows us to do an exact assert
   
   * Optimize long running unit test usign a short retry
   
   * Re-enable AMQP integration test after stuck unack messages resolved
   
   **Please** add a meaningful description for your change here
   
   Fix #26203
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
   
   To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/actions/workflows/build_wheels.yml/badge.svg?event=schedule&&?branch=master)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/actions/workflows/python_tests.yml/badge.svg?event=schedule&&?branch=master)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/actions/workflows/java_tests.yml/badge.svg?event=schedule&&?branch=master)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Go tests](https://github.com/apache/beam/actions/workflows/go_tests.yml/badge.svg?event=schedule&&?branch=master)](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3579861114


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   Question on the two new modes when finalization runs on a separate thread. In INDIVIDUAL_ACKNOWLEDGE and CLIENT_ACKNOWLEDGE_UNSAFE the mark holds Messages bound to the reader's single long lived session. My understanding is that finalizeCheckpoint() may be called on a different thread from the reader on some runners. You would know the Dataflow specifics far better than I do. If so, does message.acknowledge() here end up running concurrently with the reader's receive loop on the same Session, which JMS (section 4.4.6) says must have a single thread of control? CLIENT_ACKNOWLEDGE looks immune since each mark owns a private session. Could you confirm how ack is serialized against receive for the other two modes, or which providers this has been validated against? This ties into shunping's Dataflow question above.



##########
sdks/java/io/jms/src/test/java/org/apache/beam/sdk/io/jms/JmsIOIT.java:
##########
@@ -145,84 +149,145 @@ public interface JmsIOITOptions extends IOTestPipelineOptions, StreamingOptions
   @Parameterized.Parameters(name = "with client class {3}")
   public static Collection<Object[]> connectionFactories() {
     return ImmutableList.of(
-        new Object[] {
-          "vm://localhost", 5672, "jms.sendAcksAsync=false", ActiveMQConnectionFactory.class
-        });
-    // TODO(https://github.com/apache/beam/issues/26175) Test failure on direct runner due to
-    //  JmsIO read on amqp slow on CI (passed locally)
-    // new Object[] {
-    //   "amqp://localhost", 5672, "jms.forceAsyncAcks=false", JmsConnectionFactory.class
-    // });
+        new Object[] {"vm://localhost", "jms.sendAcksAsync=false", ActiveMQConnectionFactory.class},
+        new Object[] {"amqp://localhost", "jms.forceAsyncAcks=false", JmsConnectionFactory.class});
   }
 
-  private final CommonJms commonJms;
+  private static final Map<String, CommonJms> BROKERS = new ConcurrentHashMap<>();
+
+  private final String brokerUrl;
+  private final Integer brokerPort;
+  private final String forceAsyncAcksParam;
+  private final Class<? extends ConnectionFactory> connectionFactoryClassParam;
+  private CommonJms commonJms;
   private ConnectionFactory connectionFactory;
   private Class<? extends ConnectionFactory> connectionFactoryClass;
 
   public JmsIOIT(
       String brokerUrl,
-      Integer brokerPort,
       String forceAsyncAcksParam,
       Class<? extends ConnectionFactory> connectionFactoryClass) {
-    this.commonJms =
-        new CommonJms(
-            OPTIONS.isLocalJmsBrokerEnabled() ? brokerUrl : OPTIONS.getJmsBrokerHost(),
-            OPTIONS.isLocalJmsBrokerEnabled() ? brokerPort : OPTIONS.getJmsBrokerPort(),
-            forceAsyncAcksParam,
-            connectionFactoryClass);
+    this.brokerUrl = brokerUrl;
+    if (OPTIONS.isLocalJmsBrokerEnabled()) {
+      try {
+        this.brokerPort = NetworkTestHelper.getAvailableLocalPort();
+      } catch (IOException e) {
+        throw new RuntimeException("Failed to find available port", e);
+      }
+    } else {
+      this.brokerPort = OPTIONS.getJmsBrokerPort();
+    }
+    this.forceAsyncAcksParam = forceAsyncAcksParam;
+    this.connectionFactoryClassParam = connectionFactoryClass;
   }
 
   @Before
   public void setup() throws Exception {
     if (OPTIONS.isLocalJmsBrokerEnabled()) {
-      this.commonJms.startBroker();
-      connectionFactory = this.commonJms.createConnectionFactory();
-      connectionFactoryClass = this.commonJms.getConnectionFactoryClass();
-      // use a small number of record for local integration test
+      String key = brokerUrl + ":" + connectionFactoryClassParam.getName();
+      commonJms =
+          BROKERS.computeIfAbsent(
+              key,
+              k -> {
+                CommonJms broker =
+                    new CommonJms(
+                        brokerUrl, brokerPort, forceAsyncAcksParam, connectionFactoryClassParam);
+                try {
+                  broker.startBroker();
+                } catch (Exception e) {
+                  throw new RuntimeException(e);
+                }
+                return broker;
+              });
       OPTIONS.setNumberOfRecords(10000);
+    } else {
+      commonJms =
+          new CommonJms(
+              OPTIONS.getJmsBrokerHost(),
+              OPTIONS.getJmsBrokerPort(),
+              forceAsyncAcksParam,
+              connectionFactoryClassParam);
     }
   }
 
+  @AfterClass
+  public static void afterClass() throws Exception {
+    for (CommonJms broker : BROKERS.values()) {
+      try {
+        broker.stopBroker();
+      } catch (Exception e) {
+        // ignore errors on shutdown
+      }
+    }
+    BROKERS.clear();
+  }
+
+  private void setupConnection(JmsIO.AcknowledgeMode acknowledgeMode) throws Exception {
+    if (acknowledgeMode == JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE) {
+      connectionFactory = this.commonJms.createConnectionFactoryWithSyncAcksAndWithoutPrefetch();
+    } else {
+      connectionFactory = this.commonJms.createConnectionFactory();
+    }
+    connectionFactoryClass = this.commonJms.getConnectionFactoryClass();
+  }
+
   @After
   public void tearDown() throws Exception {
-    if (OPTIONS.isLocalJmsBrokerEnabled()) {
-      this.commonJms.stopBroker();
-      connectionFactory = null;
-      connectionFactoryClass = null;
-    }
+    connectionFactory = null;
+    connectionFactoryClass = null;
+  }
+
+  @Test
+  public void testPublishingThenReadingAll() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE);
+  }
+
+  @Test
+  public void testPublishingThenReadingAllIndividualAcknowledge() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.INDIVIDUAL_ACKNOWLEDGE);
   }
 
   @Test
-  public void testPublishingThenReadingAll() throws IOException, JMSException {
-    PipelineResult writeResult = publishingMessages();
+  public void testPublishingThenReadingAllClientAcknowledgeUnsafe() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE_UNSAFE);
+  }
+
+  private void runPublishingThenReadingAll(JmsIO.AcknowledgeMode acknowledgeMode) throws Exception {
+    setupConnection(acknowledgeMode);
+    String queue = QUEUE + "_" + acknowledgeMode.name();
+    PipelineResult writeResult = publishingMessages(queue);
     PipelineResult.State writeState = writeResult.waitUntilFinish();
     assertNotEquals(PipelineResult.State.FAILED, writeState);
 
-    PipelineResult readResult = readMessages();
-    PipelineResult.State readState =
-        readResult.waitUntilFinish(Duration.standardSeconds(OPTIONS.getReadTimeout()));
-    // A workaround to stop the pipeline for waiting for too long
+    PipelineResult readResult = readMessages(acknowledgeMode, queue);
+    MetricsReader metricsReader = new MetricsReader(readResult, NAMESPACE);
+    long startTime = System.currentTimeMillis();
+    long timeoutMillis = OPTIONS.getReadTimeout() * 1000L;
+    PipelineResult.State readState = readResult.getState();

Review Comment:
   Outside the streaming IO scope I was tagged for, but flagging it since I noticed it. readState comes from readResult.getState(), which on the direct runner returns RUNNING and never null. So when readTimeout expires with the pipeline still running (the amqp slow on CI case from #26175 that got this test disabled), cancelIfTimeouted, which only cancels when readState == null, is a no op. The asserts then fail while the pipeline keeps consuming from the class scoped shared BROKERS, leaking it into the next parameterized test. Suggest:
   
   ```java
   if (readState == null || !readState.isTerminal()) {
     readResult.cancel();
   }
   ```



##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsIO.java:
##########
@@ -783,21 +896,43 @@ public void close() {
     private void doClose() {
       try {
         closeAutoscaler();
-        closeConsumer();
-        ScheduledExecutorService executorService =
-            options.as(ExecutorOptions.class).getScheduledExecutorService();
-        executorService.schedule(
-            () -> {
-              LOG.debug("Closing connection after delay {}", source.spec.getCloseTimeout());
-              // Discard the checkpoints and set the reader as inactive
-              checkpointMarkPreparer.discard();
-              closeSession();
-              closeConnection();
-            },
-            source.spec.getCloseTimeout().getMillis(),
-            TimeUnit.MILLISECONDS);
+        // Discard the checkpoints and set the reader as inactive
+        checkpointMarkPreparer.discard();
+        if (source.spec.getAcknowledgeMode() == AcknowledgeMode.CLIENT_ACKNOWLEDGE) {
+          // checkpointMark holds session in CLIENT_ACKNOWLEDGE mode. Therefore
+          // we can close consumer and session immediately.
+          closeConsumer();
+          closeSession();
+        }
+        if (activeCheckpoints.get() == 0) {
+          closeConsumer();
+          closeSession();
+          closeConnection();
+        } else {
+          ScheduledExecutorService executorService =
+              options.as(ExecutorOptions.class).getScheduledExecutorService();
+          executorService.submit(
+              () -> {
+                long startTime = System.currentTimeMillis();
+                long timeoutMillis = source.spec.getCloseTimeout().getMillis();
+                while (activeCheckpoints.get() > 0
+                    && System.currentTimeMillis() - startTime < timeoutMillis) {
+                  try {
+                    Thread.sleep(1_000); // poll in 1 sec interval
+                  } catch (InterruptedException ignored) {
+                    break;
+                  }
+                }
+                LOG.debug(
+                    "Closing connection after checkpoints finalized or timeout: {}",
+                    source.spec.getCloseTimeout());
+                closeConsumer();
+                closeSession();
+                closeConnection();
+              });

Review Comment:
   The close path submit()s a while (...) Thread.sleep(1000) loop onto the shared ExecutorOptions scheduled pool. On any runner where checkpoint finalization is best effort or delayed, the count may never reach 0, so the loop runs up to the full closeTimeout (default 60s). When many readers close at once, for example on scale down, that parks a lot of pool threads and can starve other scheduled work. I am not sure how often Dataflow drops finalization in practice, you would know better, but even setting that aside the busy wait on a shared pool seems worth avoiding. Suggest rescheduling via schedule() with a short period, or having the last finalizeCheckpoint decrement trigger the close when a closed flag is set (last one out), with a single schedule(closeTimeout) fallback.



##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsIO.java:
##########
@@ -484,6 +510,23 @@ public Read<T> withRequiresDeduping() {
       return builder().setRequiresDeduping(true).build();
     }
 
+    /** Specify the {@link AcknowledgeMode} used for consuming and acknowledging JMS messages. */
+    public Read<T> withAcknowledgeMode(AcknowledgeMode acknowledgeMode) {
+      checkArgument(acknowledgeMode != null, "acknowledgeMode can not be null");
+      return builder().setAcknowledgeMode(acknowledgeMode).build();
+    }
+
+    /**
+     * Specify the custom integer code for individual message acknowledgment when using {@link
+     * AcknowledgeMode#INDIVIDUAL_ACKNOWLEDGE}.
+     *
+     * <p>Different JMS providers use different proprietary integer constants for individual
+     * acknowledgment (e.g., ActiveMQ uses 4, Qpid JMS / ActiveMQ Artemis / IBM MQ use 101).

Review Comment:
   The withIndividualAcknowledgeModeCode javadoc says "Qpid JMS / ActiveMQ Artemis / IBM MQ use 101". IBM MQ classes for JMS only support the standard acknowledgment modes (AUTO=1, CLIENT=2, DUPS_OK=3) and have no individual acknowledge constant, so 101 is not a valid mode there and createSession(false, 101) on IBM MQ would throw a JMSException rather than give individual ack. Suggest dropping IBM MQ from that list. For per message behavior on IBM MQ the guidance is CLIENT_ACKNOWLEDGE, which acks cumulatively. The "ActiveMQ uses 4" part checks out against ActiveMQSession.INDIVIDUAL_ACKNOWLEDGE. Minor, since getAckModeCode correctly throws for unrecognized providers, it might be worth a one line note that INDIVIDUAL mode on other providers requires an explicit withIndividualAcknowledgeModeCode.



##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsIO.java:
##########
@@ -752,16 +857,24 @@ public CheckpointMark getCheckpointMark() {
 
       MessageConsumer consumerToClose;
       Session sessionTofinalize;
+      AcknowledgeMode mode = source.spec.getAcknowledgeMode();
       synchronized (this) {
-        consumerToClose = consumer;
-        sessionTofinalize = session;
-      }
-      try {
-        recreateSession();
-      } catch (IOException e) {
-        throw new RuntimeException(e);
+        if (mode == AcknowledgeMode.CLIENT_ACKNOWLEDGE) {
+          consumerToClose = consumer;
+          sessionTofinalize = session;
+          try {
+            recreateSession();
+          } catch (IOException e) {
+            throw new RuntimeException(e);
+          }
+        } else {
+          consumerToClose = null;
+          sessionTofinalize = null;
+        }
       }
-      return checkpointMarkPreparer.newCheckpoint(consumerToClose, sessionTofinalize);
+      activeCheckpoints.incrementAndGet();

Review Comment:
   activeCheckpoints.incrementAndGet() runs outside the synchronized(this) block and unconditionally after the isEmpty() check. If doClose() or discard() races in between, newCheckpoint takes the discarded branch and returns emptyCheckpoint() whose activeCheckpoints is null (JmsCheckpointMark.java:265), so the increment is never balanced and the executor loop waits the full closeTimeout. In CLIENT_ACKNOWLEDGE, the old consumerToClose and sessionTofinalize captured just above (after recreateSession already swapped in a new session) are then silently dropped, leaking the old session and its prefetched messages. Suggest moving the increment inside newCheckpoint under the write lock on the non discarded branch, and closing the passed in consumer and session in the discarded branch.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3579861114


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   Question on the two new modes when finalization runs on a separate thread: 
   In `INDIVIDUAL_ACKNOWLEDGE` and `CLIENT_ACKNOWLEDGE_UNSAFE` the mark holds Messages bound to the reader's single long lived session. My understanding is that `finalizeCheckpoint()` may be called on a different thread from the reader on some runners. 
   You would know the Dataflow specifics far better than I do. If so, does `message.acknowledge()` here end up running concurrently with the reader's receive loop on the same Session, which [JMS (section 4.4.6)](https://docs.oracle.com/cd/E19957-01/816-5904-10/816-5904-10.pdf#page=57.36) says must have a single thread of control? `CLIENT_ACKNOWLEDGE` looks immune since each mark owns a private session. Could you confirm how ack is serialized against receive for the other two modes, or which providers this has been validated against? (This ties into @shunping 's Dataflow question above).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGEMENT_MODE for JmsIO [beam]

Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on PR #39253:
URL: https://github.com/apache/beam/pull/39253#issuecomment-4920321042

   ## Summary of Changes
   
   Hello, I'm Gemini Code Assist[^1]! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
   
   This pull request adds support for configurable acknowledgement modes in JmsIO, enabling better compatibility with various JMS providers and performance tuning. The changes involve a significant refactoring of how JmsIO manages JMS sessions and consumers, ensuring that resources are correctly handled across different acknowledgement strategies. Additionally, the PR improves the reliability of integration tests and addresses a bug in the direct runner's checkpoint finalization logic.
   
   ### Highlights
   
   * **Added AcknowledgeMode Support**: Introduced configurable AcknowledgeMode (CLIENT_ACKNOWLEDGE, CLIENT_ACKNOWLEDGE_UNSAFE, INDIVIDUAL_ACKNOWLEDGE) to JmsIO to support diverse JMS provider requirements.
   * **Refactored Checkpoint Management**: Updated JmsCheckpointMark and UnboundedJmsReader to track active checkpoints and manage session/consumer lifecycles appropriately for each acknowledgement mode.
   * **Improved Resource Cleanup**: Enhanced UnboundedJmsReader to delay connection closure until all active checkpoints are finalized, preventing premature resource release.
   * **Integration Test Enhancements**: Updated integration tests to validate new acknowledgement modes and improved test stability by fixing port management and cleanup logic.
   * **Bug Fixes**: Fixed a checkpoint finalization issue in UnboundedReadEvaluatorFactory to ensure proper cleanup of old checkpoints.
   
   <details>
   <summary><b>New Features</b></summary>
   
   🧠 You can now enable **Memory** (public preview) to help **Gemini Code Assist** learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. **Click [here](https://codeassist.google/code-review/login) to enable Memory in your admin console.**
   </details>
   
   <details>
   <summary><b>Using Gemini Code Assist</b></summary>
   <br>
   
   The full guide for Gemini Code Assist can be found on our [documentation page](https://developers.google.com/gemini-code-assist/docs/review-github-code), here are some quick tips.
   
   <b>Invoking Gemini</b>
   
   You can request assistance from Gemini at any point by creating a comment using either `/gemini <command>` or `@gemini-code-assist <command>`. Below is a summary of the supported commands on the current page.
   
   Feature | Command | Description
   --- | --- | ---
   Code Review | `/gemini review` | Performs a code review for the current pull request in its current state.
   Pull Request Summary | `/gemini summary` | Provides a summary of the current pull request in its current state.
   Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in pull request comments and review comments.
   Help | `/gemini help` | Displays a list of available commands.
   
   <b>Customization</b>
   
   To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a `.gemini/` folder in the base of the repository. Detailed instructions can be found [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github).
   
   <b>Limitations & Feedback</b>
   
   Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with :thumbsup: and :thumbsdown: on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up [here](https://google.qualtrics.com/jfe/form/SV_2cyuGuTWsEw84yG).
   </details>
   
   
   [^1]: Review the [Privacy Notices](https://policies.google.com/privacy), [Generative AI Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy), [Terms of Service](https://policies.google.com/terms), and learn how to configure Gemini Code Assist in GitHub [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github). Gemini can make mistakes, so double check it and [use code with caution](https://support.google.com/legal/answer/13505487).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3579861135


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsIO.java:
##########
@@ -752,16 +857,24 @@ public CheckpointMark getCheckpointMark() {
 
       MessageConsumer consumerToClose;
       Session sessionTofinalize;
+      AcknowledgeMode mode = source.spec.getAcknowledgeMode();
       synchronized (this) {
-        consumerToClose = consumer;
-        sessionTofinalize = session;
-      }
-      try {
-        recreateSession();
-      } catch (IOException e) {
-        throw new RuntimeException(e);
+        if (mode == AcknowledgeMode.CLIENT_ACKNOWLEDGE) {
+          consumerToClose = consumer;
+          sessionTofinalize = session;
+          try {
+            recreateSession();
+          } catch (IOException e) {
+            throw new RuntimeException(e);
+          }
+        } else {
+          consumerToClose = null;
+          sessionTofinalize = null;
+        }
       }
-      return checkpointMarkPreparer.newCheckpoint(consumerToClose, sessionTofinalize);
+      activeCheckpoints.incrementAndGet();

Review Comment:
   `activeCheckpoints.incrementAndGet()` runs outside the `synchronized(this)` block and unconditionally after the `isEmpty()` check. 
   If `doClose()` or `discard()` races in between, `newCheckpoint` takes the discarded branch and returns `emptyCheckpoint()` whose `activeCheckpoints` is null (JmsCheckpointMark.java:265), so the increment is never balanced and the executor loop waits the full `closeTimeout`. In `CLIENT_ACKNOWLEDGE`, the old `consumerToClose` and `sessionTofinalize` captured just above (after `recreateSession` already swapped in a new session) are then silently dropped, leaking the old session and its prefetched messages. Suggest moving the increment inside `newCheckpoint` under the write lock on the non discarded branch, and closing the passed in consumer and session in the discarded branch.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3579861114


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   Question on the two new modes when finalization runs on a separate thread: 
   In `INDIVIDUAL_ACKNOWLEDGE` and `CLIENT_ACKNOWLEDGE_UNSAFE` the mark holds Messages bound to the reader's single long lived session. My understanding is that `finalizeCheckpoint()` may be called on a different thread from the reader on some runners. 
   You would know the Dataflow specifics far better than I do. If so, does `message.acknowledge()` here end up running concurrently with the reader's receive loop on the same Session, which JMS (section 4.4.6) says must have a single thread of control? `CLIENT_ACKNOWLEDGE` looks immune since each mark owns a private session. Could you confirm how ack is serialized against receive for the other two modes, or which providers this has been validated against? (This ties into @shunping 's Dataflow question above).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3602925837


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   Thanks for pushing back @Abacn . 
   I checked the ActiveMQ and Qpid client internals and you are right in practice. 
   1. ActiveMQ Classic guards deliveredMessages on both the receive add and the individual ack remove and serializes the wire send through MutexTransport, and the individual ack deletes exactly the one message id without touching the CLIENT_ACKNOWLEDGE counters, so ack and receiveNoWait do not race there. 
   2. Qpid serializes everything on its single provider thread.
   3. Artemis seems safe in this interleaving too.
   
   The JMS spec does treat the whole session as single threaded including acknowledge, so this is provider tolerance rather than a spec guarantee. That is fine for the three auto detected providers, but for someone setting a custom code via `withIndividualAcknowledgeModeCode` it is not guaranteed. 
   
   A one line javadoc caveat there might be worth it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3586955042


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   Thanks, that mostly clears it up. One part I still want to nail down before approving.
   
   The CheckpointMark contract in UnboundedSource.java says finalizeCheckpoint "may be called from any thread, concurrently with calls to the UnboundedReader it was created from", and the reader is single threaded but finalize is exempted. For INDIVIDUAL_ACKNOWLEDGE and CLIENT_ACKNOWLEDGE_UNSAFE getCheckpointMark does not recreate the session, so the mark's messages stay bound to the reader's live session, and finalizeCheckpoint calls message.acknowledge() on it without taking the reader monitor. CLIENT_ACKNOWLEDGE avoids this because it hands each checkpoint its own session.
   
   On Dataflow, where commit finalization runs asynchronously on a pool thread while the cached reader keeps advancing, what stops acknowledge() from racing the reader's receiveNoWait() on the same session? Same question for the legacy Flink UnboundedSourceWrapper, which calls finalizeCheckpoint outside the checkpoint lock. The integration tests run on the direct runner, where finalize is on the reader thread, so they would not exercise this.
   
   If I am missing the mechanism that serializes them I am happy to be corrected. Otherwise acking under the reader monitor for these two modes, or giving them per checkpoint sessions like CLIENT_ACKNOWLEDGE, would close it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGEMENT_MODE for JmsIO [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #39253:
URL: https://github.com/apache/beam/pull/39253#issuecomment-4920433034

   possible reviewers R: @tkaymak (for streaming IO) @shunping would you mind taking a look?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3586880249


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsIO.java:
##########
@@ -484,6 +511,28 @@ public Read<T> withRequiresDeduping() {
       return builder().setRequiresDeduping(true).build();
     }
 
+    /**
+     * Specify the {@link AcknowledgeMode} used for consuming and acknowledging JMS messages.
+     *
+     * <p>To use {@link AcknowledgeMode#INDIVIDUAL_ACKNOWLEDGE}, providers other than ActiveMQ, Qpid
+     * JMS, ActiveMQ require configuring {@link #withIndividualAcknowledgeModeCode} explicitly.

Review Comment:
   ```suggestion
        * or JMS require configuring {@link #withIndividualAcknowledgeModeCode} explicitly.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3602766988


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   Thanks for pushing back. 
   I checked the ActiveMQ and Qpid client internals and you are right in practice. 
   1. ActiveMQ Classic guards deliveredMessages on both the receive add and the individual ack remove and serializes the wire send through MutexTransport, and the individual ack deletes exactly the one message id without touching the CLIENT_ACKNOWLEDGE counters, so ack and receiveNoWait do not race there. 
   2. Qpid serializes everything on its single provider thread. Artemis is gentle in this interleaving too.
   
   The JMS spec does treat the whole session as single threaded including acknowledge, so this seems provider tolerance rather than a spec guarantee. That is fine for the three auto detected providers, but for someone setting a custom code via `withIndividualAcknowledgeModeCode` it is not guaranteed. A one line javadoc caveat there could be worth it. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3602766988


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   Thanks for digging in. I checked the ActiveMQ and Qpid client internals and you are right in practice. ActiveMQ Classic guards deliveredMessages on both the receive add and the individual ack remove and serializes the wire send through MutexTransport, and the individual ack deletes exactly the one message id without touching the CLIENT_ACKNOWLEDGE counters, so ack and receiveNoWait do not race there. Qpid serializes everything on its single provider thread. Artemis is benign in this interleaving too.
   
   One nuance for the record. The JMS spec does treat the whole session as single threaded including acknowledge, so this is provider tolerance rather than a spec guarantee. That is fine for the three auto detected providers, but for someone setting a custom code via withIndividualAcknowledgeModeCode it is not guaranteed. A one line javadoc caveat there would be worth it. Not blocking.
   
   LGTM, approving.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGEMENT_MODE for JmsIO [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3548064684


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -93,14 +100,37 @@ public void finalizeCheckpoint() {
         LOG.info("Error closing JMS session. It may have already been closed.");
       }
     }
+
+    if (activeCheckpoints != null) {
+      activeCheckpoints.decrementAndGet();
+    }

Review Comment:
   Too defensive in opposite direction. Over estimate count is fine but we must not underestimate count. We only guarantee when the count is zero, there isn't active checkpoint.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn merged PR #39253:
URL: https://github.com/apache/beam/pull/39253


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3580416791


##########
runners/direct-java/src/main/java/org/apache/beam/runners/direct/UnboundedReadEvaluatorFactory.java:
##########
@@ -173,16 +173,17 @@ public void processElement(
         } else {
           Instant watermark = reader.getWatermark();
           if (watermark.isBefore(BoundedWindow.TIMESTAMP_MAX_VALUE)) {
-            // If the reader had no elements available, but the shard is not done, reuse it later
-            // Might be better to finalize old checkpoint.
+            // If the reader had no elements available, but the shard is not done, reuse it later.

Review Comment:
   Will follow up on Dataflow runner (need a remote ActiveMQ deployement)
   
   The original report where this issue was observed was on Direct runner: https://github.com/apache/beam/pull/30218#issuecomment-2037595552



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3586882328


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsIO.java:
##########
@@ -484,6 +511,28 @@ public Read<T> withRequiresDeduping() {
       return builder().setRequiresDeduping(true).build();
     }
 
+    /**
+     * Specify the {@link AcknowledgeMode} used for consuming and acknowledging JMS messages.
+     *
+     * <p>To use {@link AcknowledgeMode#INDIVIDUAL_ACKNOWLEDGE}, providers other than ActiveMQ, Qpid
+     * JMS, ActiveMQ require configuring {@link #withIndividualAcknowledgeModeCode} explicitly.

Review Comment:
   (nit: ActiveMQ was mentioned twice)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3595797109


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   @Abacn this is my only remaining point otherwise I can also approve it, just let me know! 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3580680090


##########
sdks/java/io/jms/src/test/java/org/apache/beam/sdk/io/jms/JmsIOIT.java:
##########
@@ -145,84 +149,145 @@ public interface JmsIOITOptions extends IOTestPipelineOptions, StreamingOptions
   @Parameterized.Parameters(name = "with client class {3}")
   public static Collection<Object[]> connectionFactories() {
     return ImmutableList.of(
-        new Object[] {
-          "vm://localhost", 5672, "jms.sendAcksAsync=false", ActiveMQConnectionFactory.class
-        });
-    // TODO(https://github.com/apache/beam/issues/26175) Test failure on direct runner due to
-    //  JmsIO read on amqp slow on CI (passed locally)
-    // new Object[] {
-    //   "amqp://localhost", 5672, "jms.forceAsyncAcks=false", JmsConnectionFactory.class
-    // });
+        new Object[] {"vm://localhost", "jms.sendAcksAsync=false", ActiveMQConnectionFactory.class},
+        new Object[] {"amqp://localhost", "jms.forceAsyncAcks=false", JmsConnectionFactory.class});
   }
 
-  private final CommonJms commonJms;
+  private static final Map<String, CommonJms> BROKERS = new ConcurrentHashMap<>();
+
+  private final String brokerUrl;
+  private final Integer brokerPort;
+  private final String forceAsyncAcksParam;
+  private final Class<? extends ConnectionFactory> connectionFactoryClassParam;
+  private CommonJms commonJms;
   private ConnectionFactory connectionFactory;
   private Class<? extends ConnectionFactory> connectionFactoryClass;
 
   public JmsIOIT(
       String brokerUrl,
-      Integer brokerPort,
       String forceAsyncAcksParam,
       Class<? extends ConnectionFactory> connectionFactoryClass) {
-    this.commonJms =
-        new CommonJms(
-            OPTIONS.isLocalJmsBrokerEnabled() ? brokerUrl : OPTIONS.getJmsBrokerHost(),
-            OPTIONS.isLocalJmsBrokerEnabled() ? brokerPort : OPTIONS.getJmsBrokerPort(),
-            forceAsyncAcksParam,
-            connectionFactoryClass);
+    this.brokerUrl = brokerUrl;
+    if (OPTIONS.isLocalJmsBrokerEnabled()) {
+      try {
+        this.brokerPort = NetworkTestHelper.getAvailableLocalPort();
+      } catch (IOException e) {
+        throw new RuntimeException("Failed to find available port", e);
+      }
+    } else {
+      this.brokerPort = OPTIONS.getJmsBrokerPort();
+    }
+    this.forceAsyncAcksParam = forceAsyncAcksParam;
+    this.connectionFactoryClassParam = connectionFactoryClass;
   }
 
   @Before
   public void setup() throws Exception {
     if (OPTIONS.isLocalJmsBrokerEnabled()) {
-      this.commonJms.startBroker();
-      connectionFactory = this.commonJms.createConnectionFactory();
-      connectionFactoryClass = this.commonJms.getConnectionFactoryClass();
-      // use a small number of record for local integration test
+      String key = brokerUrl + ":" + connectionFactoryClassParam.getName();
+      commonJms =
+          BROKERS.computeIfAbsent(
+              key,
+              k -> {
+                CommonJms broker =
+                    new CommonJms(
+                        brokerUrl, brokerPort, forceAsyncAcksParam, connectionFactoryClassParam);
+                try {
+                  broker.startBroker();
+                } catch (Exception e) {
+                  throw new RuntimeException(e);
+                }
+                return broker;
+              });
       OPTIONS.setNumberOfRecords(10000);
+    } else {
+      commonJms =
+          new CommonJms(
+              OPTIONS.getJmsBrokerHost(),
+              OPTIONS.getJmsBrokerPort(),
+              forceAsyncAcksParam,
+              connectionFactoryClassParam);
     }
   }
 
+  @AfterClass
+  public static void afterClass() throws Exception {
+    for (CommonJms broker : BROKERS.values()) {
+      try {
+        broker.stopBroker();
+      } catch (Exception e) {
+        // ignore errors on shutdown
+      }
+    }
+    BROKERS.clear();
+  }
+
+  private void setupConnection(JmsIO.AcknowledgeMode acknowledgeMode) throws Exception {
+    if (acknowledgeMode == JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE) {
+      connectionFactory = this.commonJms.createConnectionFactoryWithSyncAcksAndWithoutPrefetch();
+    } else {
+      connectionFactory = this.commonJms.createConnectionFactory();
+    }
+    connectionFactoryClass = this.commonJms.getConnectionFactoryClass();
+  }
+
   @After
   public void tearDown() throws Exception {
-    if (OPTIONS.isLocalJmsBrokerEnabled()) {
-      this.commonJms.stopBroker();
-      connectionFactory = null;
-      connectionFactoryClass = null;
-    }
+    connectionFactory = null;
+    connectionFactoryClass = null;
+  }
+
+  @Test
+  public void testPublishingThenReadingAll() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE);
+  }
+
+  @Test
+  public void testPublishingThenReadingAllIndividualAcknowledge() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.INDIVIDUAL_ACKNOWLEDGE);
   }
 
   @Test
-  public void testPublishingThenReadingAll() throws IOException, JMSException {
-    PipelineResult writeResult = publishingMessages();
+  public void testPublishingThenReadingAllClientAcknowledgeUnsafe() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE_UNSAFE);
+  }
+
+  private void runPublishingThenReadingAll(JmsIO.AcknowledgeMode acknowledgeMode) throws Exception {
+    setupConnection(acknowledgeMode);
+    String queue = QUEUE + "_" + acknowledgeMode.name();
+    PipelineResult writeResult = publishingMessages(queue);
     PipelineResult.State writeState = writeResult.waitUntilFinish();
     assertNotEquals(PipelineResult.State.FAILED, writeState);
 
-    PipelineResult readResult = readMessages();
-    PipelineResult.State readState =
-        readResult.waitUntilFinish(Duration.standardSeconds(OPTIONS.getReadTimeout()));
-    // A workaround to stop the pipeline for waiting for too long
+    PipelineResult readResult = readMessages(acknowledgeMode, queue);
+    MetricsReader metricsReader = new MetricsReader(readResult, NAMESPACE);
+    long startTime = System.currentTimeMillis();
+    long timeoutMillis = OPTIONS.getReadTimeout() * 1000L;
+    PipelineResult.State readState = readResult.getState();

Review Comment:
   Thanks for pointing that. Before #28014 direct runner returns null on querying state when running. This part was outdated and now fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3588867159


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsIO.java:
##########
@@ -484,6 +511,28 @@ public Read<T> withRequiresDeduping() {
       return builder().setRequiresDeduping(true).build();
     }
 
+    /**
+     * Specify the {@link AcknowledgeMode} used for consuming and acknowledging JMS messages.
+     *
+     * <p>To use {@link AcknowledgeMode#INDIVIDUAL_ACKNOWLEDGE}, providers other than ActiveMQ, Qpid
+     * JMS, ActiveMQ require configuring {@link #withIndividualAcknowledgeModeCode} explicitly.

Review Comment:
   Done. It's ActiveMQ Artemis



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3579861124


##########
sdks/java/io/jms/src/test/java/org/apache/beam/sdk/io/jms/JmsIOIT.java:
##########
@@ -145,84 +149,145 @@ public interface JmsIOITOptions extends IOTestPipelineOptions, StreamingOptions
   @Parameterized.Parameters(name = "with client class {3}")
   public static Collection<Object[]> connectionFactories() {
     return ImmutableList.of(
-        new Object[] {
-          "vm://localhost", 5672, "jms.sendAcksAsync=false", ActiveMQConnectionFactory.class
-        });
-    // TODO(https://github.com/apache/beam/issues/26175) Test failure on direct runner due to
-    //  JmsIO read on amqp slow on CI (passed locally)
-    // new Object[] {
-    //   "amqp://localhost", 5672, "jms.forceAsyncAcks=false", JmsConnectionFactory.class
-    // });
+        new Object[] {"vm://localhost", "jms.sendAcksAsync=false", ActiveMQConnectionFactory.class},
+        new Object[] {"amqp://localhost", "jms.forceAsyncAcks=false", JmsConnectionFactory.class});
   }
 
-  private final CommonJms commonJms;
+  private static final Map<String, CommonJms> BROKERS = new ConcurrentHashMap<>();
+
+  private final String brokerUrl;
+  private final Integer brokerPort;
+  private final String forceAsyncAcksParam;
+  private final Class<? extends ConnectionFactory> connectionFactoryClassParam;
+  private CommonJms commonJms;
   private ConnectionFactory connectionFactory;
   private Class<? extends ConnectionFactory> connectionFactoryClass;
 
   public JmsIOIT(
       String brokerUrl,
-      Integer brokerPort,
       String forceAsyncAcksParam,
       Class<? extends ConnectionFactory> connectionFactoryClass) {
-    this.commonJms =
-        new CommonJms(
-            OPTIONS.isLocalJmsBrokerEnabled() ? brokerUrl : OPTIONS.getJmsBrokerHost(),
-            OPTIONS.isLocalJmsBrokerEnabled() ? brokerPort : OPTIONS.getJmsBrokerPort(),
-            forceAsyncAcksParam,
-            connectionFactoryClass);
+    this.brokerUrl = brokerUrl;
+    if (OPTIONS.isLocalJmsBrokerEnabled()) {
+      try {
+        this.brokerPort = NetworkTestHelper.getAvailableLocalPort();
+      } catch (IOException e) {
+        throw new RuntimeException("Failed to find available port", e);
+      }
+    } else {
+      this.brokerPort = OPTIONS.getJmsBrokerPort();
+    }
+    this.forceAsyncAcksParam = forceAsyncAcksParam;
+    this.connectionFactoryClassParam = connectionFactoryClass;
   }
 
   @Before
   public void setup() throws Exception {
     if (OPTIONS.isLocalJmsBrokerEnabled()) {
-      this.commonJms.startBroker();
-      connectionFactory = this.commonJms.createConnectionFactory();
-      connectionFactoryClass = this.commonJms.getConnectionFactoryClass();
-      // use a small number of record for local integration test
+      String key = brokerUrl + ":" + connectionFactoryClassParam.getName();
+      commonJms =
+          BROKERS.computeIfAbsent(
+              key,
+              k -> {
+                CommonJms broker =
+                    new CommonJms(
+                        brokerUrl, brokerPort, forceAsyncAcksParam, connectionFactoryClassParam);
+                try {
+                  broker.startBroker();
+                } catch (Exception e) {
+                  throw new RuntimeException(e);
+                }
+                return broker;
+              });
       OPTIONS.setNumberOfRecords(10000);
+    } else {
+      commonJms =
+          new CommonJms(
+              OPTIONS.getJmsBrokerHost(),
+              OPTIONS.getJmsBrokerPort(),
+              forceAsyncAcksParam,
+              connectionFactoryClassParam);
     }
   }
 
+  @AfterClass
+  public static void afterClass() throws Exception {
+    for (CommonJms broker : BROKERS.values()) {
+      try {
+        broker.stopBroker();
+      } catch (Exception e) {
+        // ignore errors on shutdown
+      }
+    }
+    BROKERS.clear();
+  }
+
+  private void setupConnection(JmsIO.AcknowledgeMode acknowledgeMode) throws Exception {
+    if (acknowledgeMode == JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE) {
+      connectionFactory = this.commonJms.createConnectionFactoryWithSyncAcksAndWithoutPrefetch();
+    } else {
+      connectionFactory = this.commonJms.createConnectionFactory();
+    }
+    connectionFactoryClass = this.commonJms.getConnectionFactoryClass();
+  }
+
   @After
   public void tearDown() throws Exception {
-    if (OPTIONS.isLocalJmsBrokerEnabled()) {
-      this.commonJms.stopBroker();
-      connectionFactory = null;
-      connectionFactoryClass = null;
-    }
+    connectionFactory = null;
+    connectionFactoryClass = null;
+  }
+
+  @Test
+  public void testPublishingThenReadingAll() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE);
+  }
+
+  @Test
+  public void testPublishingThenReadingAllIndividualAcknowledge() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.INDIVIDUAL_ACKNOWLEDGE);
   }
 
   @Test
-  public void testPublishingThenReadingAll() throws IOException, JMSException {
-    PipelineResult writeResult = publishingMessages();
+  public void testPublishingThenReadingAllClientAcknowledgeUnsafe() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE_UNSAFE);
+  }
+
+  private void runPublishingThenReadingAll(JmsIO.AcknowledgeMode acknowledgeMode) throws Exception {
+    setupConnection(acknowledgeMode);
+    String queue = QUEUE + "_" + acknowledgeMode.name();
+    PipelineResult writeResult = publishingMessages(queue);
     PipelineResult.State writeState = writeResult.waitUntilFinish();
     assertNotEquals(PipelineResult.State.FAILED, writeState);
 
-    PipelineResult readResult = readMessages();
-    PipelineResult.State readState =
-        readResult.waitUntilFinish(Duration.standardSeconds(OPTIONS.getReadTimeout()));
-    // A workaround to stop the pipeline for waiting for too long
+    PipelineResult readResult = readMessages(acknowledgeMode, queue);
+    MetricsReader metricsReader = new MetricsReader(readResult, NAMESPACE);
+    long startTime = System.currentTimeMillis();
+    long timeoutMillis = OPTIONS.getReadTimeout() * 1000L;
+    PipelineResult.State readState = readResult.getState();

Review Comment:
   Outside the streaming IO scope I was tagged for, but flagging it since I noticed it. `readState` comes from `readResult.getState()`, which on the direct runner returns `RUNNING` and never `null`. So when `readTimeout` expires with the pipeline still running (the amqp slow on CI case from #26175 that got this test disabled), `cancelIfTimeouted`, which only cancels when `readState == null`, is a no op. The asserts then fail while the pipeline keeps consuming from the class scoped shared `BROKERS`, leaking it into the next parameterized test. Suggestion:
   
   ```java
   if (readState == null || !readState.isTerminal()) {
     readResult.cancel();
   }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGEMENT_MODE for JmsIO [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #39253:
URL: https://github.com/apache/beam/pull/39253#issuecomment-4920438654

   Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment `assign set of reviewers`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGEMENT_MODE for JmsIO [beam]

Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3548031933


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsIO.java:
##########
@@ -783,21 +896,43 @@ public void close() {
     private void doClose() {
       try {
         closeAutoscaler();
-        closeConsumer();
-        ScheduledExecutorService executorService =
-            options.as(ExecutorOptions.class).getScheduledExecutorService();
-        executorService.schedule(
-            () -> {
-              LOG.debug("Closing connection after delay {}", source.spec.getCloseTimeout());
-              // Discard the checkpoints and set the reader as inactive
-              checkpointMarkPreparer.discard();
-              closeSession();
-              closeConnection();
-            },
-            source.spec.getCloseTimeout().getMillis(),
-            TimeUnit.MILLISECONDS);
+        // Discard the checkpoints and set the reader as inactive
+        checkpointMarkPreparer.discard();
+        if (source.spec.getAcknowledgeMode() == AcknowledgeMode.CLIENT_ACKNOWLEDGE) {
+          // checkpointMark holds session in CLIENT_ACKNOWLEDGE mode. Therefore
+          // we can close consumer and session immediately.
+          closeConsumer();
+          closeSession();
+        }
+        if (activeCheckpoints.get() == 0) {
+          closeConsumer();
+          closeSession();
+          closeConnection();
+        } else {
+          ScheduledExecutorService executorService =
+              options.as(ExecutorOptions.class).getScheduledExecutorService();
+          executorService.submit(
+              () -> {
+                long startTime = System.currentTimeMillis();
+                long timeoutMillis = source.spec.getCloseTimeout().getMillis();
+                while (activeCheckpoints.get() > 0
+                    && System.currentTimeMillis() - startTime < timeoutMillis) {
+                  try {
+                    Thread.sleep(1_000); // poll in 1 sec interval
+                  } catch (InterruptedException ignored) {
+                    break;
+                  }
+                }
+                LOG.debug(
+                    "Closing connection after checkpoints finalized or timeout: {}",
+                    source.spec.getCloseTimeout());
+                closeConsumer();
+                closeSession();
+                closeConnection();
+              });
+        }

Review Comment:
   ![high](https://www.gstatic.com/codereviewagent/high-priority.svg)
   
   In `CLIENT_ACKNOWLEDGE` mode, `closeConsumer()` and `closeSession()` are called immediately on the main thread (lines 902-903). However, if there are active checkpoints (`activeCheckpoints.get() > 0`), the background thread submitted to the executor will also call `closeConsumer()` and `closeSession()` (lines 929-930) once those checkpoints are finalized or the timeout is reached.\n\nSince JMS `Session` and `MessageConsumer` are not thread-safe and calling `close()` concurrently or multiple times on different threads can lead to race conditions, exceptions, or undefined behavior in the JMS client library, we should ensure they are only closed once. We can introduce a boolean flag `isClientAck` and conditionally close them on the background thread only if they weren't already closed on the main thread.
   
   ```java
           final boolean isClientAck = source.spec.getAcknowledgeMode() == AcknowledgeMode.CLIENT_ACKNOWLEDGE;
           if (isClientAck) {
             // checkpointMark holds session in CLIENT_ACKNOWLEDGE mode. Therefore
             // we can close consumer and session immediately.
             closeConsumer();
             closeSession();
           }
           if (activeCheckpoints.get() == 0) {
             if (!isClientAck) {
               closeConsumer();
               closeSession();
             }
             closeConnection();
           } else {
             // Discard the checkpoints and set the reader as inactive
             checkpointMarkPreparer.discard();
             ScheduledExecutorService executorService =
                 options.as(ExecutorOptions.class).getScheduledExecutorService();
             executorService.submit(
                 () -> {
                   long startTime = System.currentTimeMillis();
                   long timeoutMillis = source.spec.getCloseTimeout().getMillis();
                   while (activeCheckpoints.get() > 0
                       && System.currentTimeMillis() - startTime < timeoutMillis) {
                     try {
                       Thread.sleep(1_000); // poll in 1 sec interval
                     } catch (InterruptedException ignored) {
                       break;
                     }
                   }
                   LOG.debug(
                       "Closing connection after checkpoints finalized or timeout: {}",
                       source.spec.getCloseTimeout());
                   if (!isClientAck) {
                     closeConsumer();
                     closeSession();
                   }
                   closeConnection();
                 });
           }
   ```



##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -93,14 +100,37 @@ public void finalizeCheckpoint() {
         LOG.info("Error closing JMS session. It may have already been closed.");
       }
     }
+
+    if (activeCheckpoints != null) {
+      activeCheckpoints.decrementAndGet();
+    }

Review Comment:
   ![medium](https://www.gstatic.com/codereviewagent/medium-priority.svg)
   
   If any `RuntimeException` is thrown during the message acknowledgment loop or during the closing of the consumer/session in `finalizeCheckpoint()`, the execution will bypass the `activeCheckpoints.decrementAndGet()` call. This would leave `activeCheckpoints` permanently greater than 0, causing the reader's close thread in `doClose()` to poll and block until the full `closeTimeout` expires.\n\nTo prevent this, the entire body of `finalizeCheckpoint()` should be wrapped in a `try-finally` block, ensuring that `activeCheckpoints.decrementAndGet()` is always executed in the `finally` block.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3586955042


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   Thank you, that mostly clears it up! One part I still want to nail/understand:
   
   The `CheckpointMark` contract in `UnboundedSource.java` says `finalizeCheckpoint` "may be called from any thread, concurrently with calls to the UnboundedReader it was created from", and the reader is single threaded but finalize is exempted. For `INDIVIDUAL_ACKNOWLEDGE` and `CLIENT_ACKNOWLEDGE_UNSAFE` `getCheckpointMark` does not recreate the session, so the mark's messages stay bound to the reader's live session, and `finalizeCheckpoint` calls `message.acknowledge()` on it without taking the reader monitor. `CLIENT_ACKNOWLEDGE` avoids this because it hands each checkpoint its own session.
   
   On Dataflow, where commit finalization runs asynchronously on a pool thread while the cached reader keeps advancing, what stops `acknowledge()` from racing the reader's `receiveNoWait()` on the same session? Same question for the legacy Flink `UnboundedSourceWrapper`, which calls `finalizeCheckpoint` outside the checkpoint lock. The integration tests run on the direct runner, where finalize is on the reader thread, so they would not exercise this. (Or am I missing something?)
   
   If I am missing the mechanism that serializes them I am happy to be corrected. Otherwise acking under the reader monitor for these two modes, or giving them per checkpoint sessions like `CLIENT_ACKNOWLEDGE`, would close this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3586955042


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   Thanks, that mostly clears it up. One part I still want to nail/understand:
   
   The `CheckpointMark` contract in `UnboundedSource.java` says `finalizeCheckpoint` "may be called from any thread, concurrently with calls to the UnboundedReader it was created from", and the reader is single threaded but finalize is exempted. For `INDIVIDUAL_ACKNOWLEDGE` and `CLIENT_ACKNOWLEDGE_UNSAFE` `getCheckpointMark` does not recreate the session, so the mark's messages stay bound to the reader's live session, and `finalizeCheckpoint` calls `message.acknowledge()` on it without taking the reader monitor. `CLIENT_ACKNOWLEDGE` avoids this because it hands each checkpoint its own session.
   
   On Dataflow, where commit finalization runs asynchronously on a pool thread while the cached reader keeps advancing, what stops `acknowledge()` from racing the reader's `receiveNoWait()` on the same session? Same question for the legacy Flink `UnboundedSourceWrapper`, which calls `finalizeCheckpoint` outside the checkpoint lock. The integration tests run on the direct runner, where finalize is on the reader thread, so they would not exercise this. (Or am I missing something?)
   
   If I am missing the mechanism that serializes them I am happy to be corrected. Otherwise acking under the reader monitor for these two modes, or giving them per checkpoint sessions like `CLIENT_ACKNOWLEDGE`, would close this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3579861146


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsIO.java:
##########
@@ -783,21 +896,43 @@ public void close() {
     private void doClose() {
       try {
         closeAutoscaler();
-        closeConsumer();
-        ScheduledExecutorService executorService =
-            options.as(ExecutorOptions.class).getScheduledExecutorService();
-        executorService.schedule(
-            () -> {
-              LOG.debug("Closing connection after delay {}", source.spec.getCloseTimeout());
-              // Discard the checkpoints and set the reader as inactive
-              checkpointMarkPreparer.discard();
-              closeSession();
-              closeConnection();
-            },
-            source.spec.getCloseTimeout().getMillis(),
-            TimeUnit.MILLISECONDS);
+        // Discard the checkpoints and set the reader as inactive
+        checkpointMarkPreparer.discard();
+        if (source.spec.getAcknowledgeMode() == AcknowledgeMode.CLIENT_ACKNOWLEDGE) {
+          // checkpointMark holds session in CLIENT_ACKNOWLEDGE mode. Therefore
+          // we can close consumer and session immediately.
+          closeConsumer();
+          closeSession();
+        }
+        if (activeCheckpoints.get() == 0) {
+          closeConsumer();
+          closeSession();
+          closeConnection();
+        } else {
+          ScheduledExecutorService executorService =
+              options.as(ExecutorOptions.class).getScheduledExecutorService();
+          executorService.submit(
+              () -> {
+                long startTime = System.currentTimeMillis();
+                long timeoutMillis = source.spec.getCloseTimeout().getMillis();
+                while (activeCheckpoints.get() > 0
+                    && System.currentTimeMillis() - startTime < timeoutMillis) {
+                  try {
+                    Thread.sleep(1_000); // poll in 1 sec interval
+                  } catch (InterruptedException ignored) {
+                    break;
+                  }
+                }
+                LOG.debug(
+                    "Closing connection after checkpoints finalized or timeout: {}",
+                    source.spec.getCloseTimeout());
+                closeConsumer();
+                closeSession();
+                closeConnection();
+              });

Review Comment:
   The close path `submit()`s a while (...) `Thread.sleep(1000)` loop onto the shared `ExecutorOptions` scheduled pool. On any runner where checkpoint finalization is best effort or delayed, the count may never reach 0, so the loop runs up to the full `closeTimeout` (default 60s). 
   When many readers close at once, for example on scale down, that parks a lot of pool threads and can starve other scheduled work. I am not sure how often Dataflow drops finalization in practice, you would know better, but even setting that aside the busy wait on a shared pool seems worth avoiding. 
   Suggest rescheduling via `schedule()` with a short period, or having the last `finalizeCheckpoint` decrement trigger the close when a closed flag is set (last one out), with a single `schedule(closeTimeout)` fallback.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3579861154


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsIO.java:
##########
@@ -484,6 +510,23 @@ public Read<T> withRequiresDeduping() {
       return builder().setRequiresDeduping(true).build();
     }
 
+    /** Specify the {@link AcknowledgeMode} used for consuming and acknowledging JMS messages. */
+    public Read<T> withAcknowledgeMode(AcknowledgeMode acknowledgeMode) {
+      checkArgument(acknowledgeMode != null, "acknowledgeMode can not be null");
+      return builder().setAcknowledgeMode(acknowledgeMode).build();
+    }
+
+    /**
+     * Specify the custom integer code for individual message acknowledgment when using {@link
+     * AcknowledgeMode#INDIVIDUAL_ACKNOWLEDGE}.
+     *
+     * <p>Different JMS providers use different proprietary integer constants for individual
+     * acknowledgment (e.g., ActiveMQ uses 4, Qpid JMS / ActiveMQ Artemis / IBM MQ use 101).

Review Comment:
   The `withIndividualAcknowledgeModeCode` javadoc says "Qpid JMS / ActiveMQ Artemis / IBM MQ use 101". IBM MQ classes for JMS only support the standard acknowledgment modes (AUTO=1, CLIENT=2, DUPS_OK=3) and have no individual acknowledge constant, so 101 is not a valid mode there and `createSession(false, 101)` on IBM MQ would throw a `JMSException` rather than give individual ack. 
   
   Suggest dropping IBM MQ from that list. For per message behavior on IBM MQ the guidance is `CLIENT_ACKNOWLEDGE`, which acks cumulatively. The "ActiveMQ uses 4" part checks out against `ActiveMQSession.INDIVIDUAL_ACKNOWLEDGE`. Minor, since `getAckModeCode` correctly throws for unrecognized providers, it might be worth a one line note that `INDIVIDUAL` mode on other providers requires an explicit `withIndividualAcknowledgeModeCode`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3580578679


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsIO.java:
##########
@@ -484,6 +510,23 @@ public Read<T> withRequiresDeduping() {
       return builder().setRequiresDeduping(true).build();
     }
 
+    /** Specify the {@link AcknowledgeMode} used for consuming and acknowledging JMS messages. */
+    public Read<T> withAcknowledgeMode(AcknowledgeMode acknowledgeMode) {
+      checkArgument(acknowledgeMode != null, "acknowledgeMode can not be null");
+      return builder().setAcknowledgeMode(acknowledgeMode).build();
+    }
+
+    /**
+     * Specify the custom integer code for individual message acknowledgment when using {@link
+     * AcknowledgeMode#INDIVIDUAL_ACKNOWLEDGE}.
+     *
+     * <p>Different JMS providers use different proprietary integer constants for individual
+     * acknowledgment (e.g., ActiveMQ uses 4, Qpid JMS / ActiveMQ Artemis / IBM MQ use 101).

Review Comment:
   Removed IBM MQ. checked that "IBM MQ use 101" was wrong and fixed `getAckModeCode` manually before pushing, but forgot to clean up the doc here.
   
   Added description in javadoc



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3580680090


##########
sdks/java/io/jms/src/test/java/org/apache/beam/sdk/io/jms/JmsIOIT.java:
##########
@@ -145,84 +149,145 @@ public interface JmsIOITOptions extends IOTestPipelineOptions, StreamingOptions
   @Parameterized.Parameters(name = "with client class {3}")
   public static Collection<Object[]> connectionFactories() {
     return ImmutableList.of(
-        new Object[] {
-          "vm://localhost", 5672, "jms.sendAcksAsync=false", ActiveMQConnectionFactory.class
-        });
-    // TODO(https://github.com/apache/beam/issues/26175) Test failure on direct runner due to
-    //  JmsIO read on amqp slow on CI (passed locally)
-    // new Object[] {
-    //   "amqp://localhost", 5672, "jms.forceAsyncAcks=false", JmsConnectionFactory.class
-    // });
+        new Object[] {"vm://localhost", "jms.sendAcksAsync=false", ActiveMQConnectionFactory.class},
+        new Object[] {"amqp://localhost", "jms.forceAsyncAcks=false", JmsConnectionFactory.class});
   }
 
-  private final CommonJms commonJms;
+  private static final Map<String, CommonJms> BROKERS = new ConcurrentHashMap<>();
+
+  private final String brokerUrl;
+  private final Integer brokerPort;
+  private final String forceAsyncAcksParam;
+  private final Class<? extends ConnectionFactory> connectionFactoryClassParam;
+  private CommonJms commonJms;
   private ConnectionFactory connectionFactory;
   private Class<? extends ConnectionFactory> connectionFactoryClass;
 
   public JmsIOIT(
       String brokerUrl,
-      Integer brokerPort,
       String forceAsyncAcksParam,
       Class<? extends ConnectionFactory> connectionFactoryClass) {
-    this.commonJms =
-        new CommonJms(
-            OPTIONS.isLocalJmsBrokerEnabled() ? brokerUrl : OPTIONS.getJmsBrokerHost(),
-            OPTIONS.isLocalJmsBrokerEnabled() ? brokerPort : OPTIONS.getJmsBrokerPort(),
-            forceAsyncAcksParam,
-            connectionFactoryClass);
+    this.brokerUrl = brokerUrl;
+    if (OPTIONS.isLocalJmsBrokerEnabled()) {
+      try {
+        this.brokerPort = NetworkTestHelper.getAvailableLocalPort();
+      } catch (IOException e) {
+        throw new RuntimeException("Failed to find available port", e);
+      }
+    } else {
+      this.brokerPort = OPTIONS.getJmsBrokerPort();
+    }
+    this.forceAsyncAcksParam = forceAsyncAcksParam;
+    this.connectionFactoryClassParam = connectionFactoryClass;
   }
 
   @Before
   public void setup() throws Exception {
     if (OPTIONS.isLocalJmsBrokerEnabled()) {
-      this.commonJms.startBroker();
-      connectionFactory = this.commonJms.createConnectionFactory();
-      connectionFactoryClass = this.commonJms.getConnectionFactoryClass();
-      // use a small number of record for local integration test
+      String key = brokerUrl + ":" + connectionFactoryClassParam.getName();
+      commonJms =
+          BROKERS.computeIfAbsent(
+              key,
+              k -> {
+                CommonJms broker =
+                    new CommonJms(
+                        brokerUrl, brokerPort, forceAsyncAcksParam, connectionFactoryClassParam);
+                try {
+                  broker.startBroker();
+                } catch (Exception e) {
+                  throw new RuntimeException(e);
+                }
+                return broker;
+              });
       OPTIONS.setNumberOfRecords(10000);
+    } else {
+      commonJms =
+          new CommonJms(
+              OPTIONS.getJmsBrokerHost(),
+              OPTIONS.getJmsBrokerPort(),
+              forceAsyncAcksParam,
+              connectionFactoryClassParam);
     }
   }
 
+  @AfterClass
+  public static void afterClass() throws Exception {
+    for (CommonJms broker : BROKERS.values()) {
+      try {
+        broker.stopBroker();
+      } catch (Exception e) {
+        // ignore errors on shutdown
+      }
+    }
+    BROKERS.clear();
+  }
+
+  private void setupConnection(JmsIO.AcknowledgeMode acknowledgeMode) throws Exception {
+    if (acknowledgeMode == JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE) {
+      connectionFactory = this.commonJms.createConnectionFactoryWithSyncAcksAndWithoutPrefetch();
+    } else {
+      connectionFactory = this.commonJms.createConnectionFactory();
+    }
+    connectionFactoryClass = this.commonJms.getConnectionFactoryClass();
+  }
+
   @After
   public void tearDown() throws Exception {
-    if (OPTIONS.isLocalJmsBrokerEnabled()) {
-      this.commonJms.stopBroker();
-      connectionFactory = null;
-      connectionFactoryClass = null;
-    }
+    connectionFactory = null;
+    connectionFactoryClass = null;
+  }
+
+  @Test
+  public void testPublishingThenReadingAll() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE);
+  }
+
+  @Test
+  public void testPublishingThenReadingAllIndividualAcknowledge() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.INDIVIDUAL_ACKNOWLEDGE);
   }
 
   @Test
-  public void testPublishingThenReadingAll() throws IOException, JMSException {
-    PipelineResult writeResult = publishingMessages();
+  public void testPublishingThenReadingAllClientAcknowledgeUnsafe() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE_UNSAFE);
+  }
+
+  private void runPublishingThenReadingAll(JmsIO.AcknowledgeMode acknowledgeMode) throws Exception {
+    setupConnection(acknowledgeMode);
+    String queue = QUEUE + "_" + acknowledgeMode.name();
+    PipelineResult writeResult = publishingMessages(queue);
     PipelineResult.State writeState = writeResult.waitUntilFinish();
     assertNotEquals(PipelineResult.State.FAILED, writeState);
 
-    PipelineResult readResult = readMessages();
-    PipelineResult.State readState =
-        readResult.waitUntilFinish(Duration.standardSeconds(OPTIONS.getReadTimeout()));
-    // A workaround to stop the pipeline for waiting for too long
+    PipelineResult readResult = readMessages(acknowledgeMode, queue);
+    MetricsReader metricsReader = new MetricsReader(readResult, NAMESPACE);
+    long startTime = System.currentTimeMillis();
+    long timeoutMillis = OPTIONS.getReadTimeout() * 1000L;
+    PipelineResult.State readState = readResult.getState();

Review Comment:
   Thanks for pointing that. Before ~#28014~ (edit: probably another earlier fix) direct runner returns null on querying state when running. This part was outdated and now fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3580909750


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsIO.java:
##########
@@ -752,16 +857,24 @@ public CheckpointMark getCheckpointMark() {
 
       MessageConsumer consumerToClose;
       Session sessionTofinalize;
+      AcknowledgeMode mode = source.spec.getAcknowledgeMode();
       synchronized (this) {
-        consumerToClose = consumer;
-        sessionTofinalize = session;
-      }
-      try {
-        recreateSession();
-      } catch (IOException e) {
-        throw new RuntimeException(e);
+        if (mode == AcknowledgeMode.CLIENT_ACKNOWLEDGE) {
+          consumerToClose = consumer;
+          sessionTofinalize = session;
+          try {
+            recreateSession();
+          } catch (IOException e) {
+            throw new RuntimeException(e);
+          }
+        } else {
+          consumerToClose = null;
+          sessionTofinalize = null;
+        }
       }
-      return checkpointMarkPreparer.newCheckpoint(consumerToClose, sessionTofinalize);
+      activeCheckpoints.incrementAndGet();

Review Comment:
   > moving the increment inside newCheckpoint under the write lock on the non discarded branch
   
   Done
   
   > closing the passed in consumer and session in the discarded branch
   
   I'm not sure if we should do this. On discard branch we don't mutate consumer and session, and it constructively it may be used later
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGEMENT_MODE for JmsIO [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3548055802


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsIO.java:
##########
@@ -783,21 +896,43 @@ public void close() {
     private void doClose() {
       try {
         closeAutoscaler();
-        closeConsumer();
-        ScheduledExecutorService executorService =
-            options.as(ExecutorOptions.class).getScheduledExecutorService();
-        executorService.schedule(
-            () -> {
-              LOG.debug("Closing connection after delay {}", source.spec.getCloseTimeout());
-              // Discard the checkpoints and set the reader as inactive
-              checkpointMarkPreparer.discard();
-              closeSession();
-              closeConnection();
-            },
-            source.spec.getCloseTimeout().getMillis(),
-            TimeUnit.MILLISECONDS);
+        // Discard the checkpoints and set the reader as inactive
+        checkpointMarkPreparer.discard();
+        if (source.spec.getAcknowledgeMode() == AcknowledgeMode.CLIENT_ACKNOWLEDGE) {
+          // checkpointMark holds session in CLIENT_ACKNOWLEDGE mode. Therefore
+          // we can close consumer and session immediately.
+          closeConsumer();
+          closeSession();
+        }
+        if (activeCheckpoints.get() == 0) {
+          closeConsumer();
+          closeSession();
+          closeConnection();
+        } else {
+          ScheduledExecutorService executorService =
+              options.as(ExecutorOptions.class).getScheduledExecutorService();
+          executorService.submit(
+              () -> {
+                long startTime = System.currentTimeMillis();
+                long timeoutMillis = source.spec.getCloseTimeout().getMillis();
+                while (activeCheckpoints.get() > 0
+                    && System.currentTimeMillis() - startTime < timeoutMillis) {
+                  try {
+                    Thread.sleep(1_000); // poll in 1 sec interval
+                  } catch (InterruptedException ignored) {
+                    break;
+                  }
+                }
+                LOG.debug(
+                    "Closing connection after checkpoints finalized or timeout: {}",
+                    source.spec.getCloseTimeout());
+                closeConsumer();
+                closeSession();
+                closeConnection();
+              });
+        }

Review Comment:
   Disagree. closeConsumer and closeSession are idempotent. It will set the resource to null



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGEMENT_MODE for JmsIO [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3548050530


##########
runners/direct-java/src/main/java/org/apache/beam/runners/direct/UnboundedReadEvaluatorFactory.java:
##########
@@ -173,16 +173,17 @@ public void processElement(
         } else {
           Instant watermark = reader.getWatermark();
           if (watermark.isBefore(BoundedWindow.TIMESTAMP_MAX_VALUE)) {
-            // If the reader had no elements available, but the shard is not done, reuse it later
-            // Might be better to finalize old checkpoint.
+            // If the reader had no elements available, but the shard is not done, reuse it later.

Review Comment:
   in a typical run, 5/6 tests observe small amount of messages are not acked due to the known issue noted here:
   
   https://github.com/apache/beam/blob/3c28e775ee06080bbab88f175b67549ab82b8471/sdks/java/io/jms/src/test/java/org/apache/beam/sdk/io/jms/JmsIOIT.java#L211
   
   
   - testPublishingThenReadingAllClientAcknowledgeUnsafe: 1
   - testPublishingThenReadingAllIndividualAcknowledge: 13
   - testPublishingThenReadingAllIndividualAcknowledge: 13
   - testPublishingThenReadingAll: 34
   - testPublishingThenReadingAll: 33
   
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGEMENT_MODE for JmsIO [beam]

Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3571841547


##########
runners/direct-java/src/main/java/org/apache/beam/runners/direct/UnboundedReadEvaluatorFactory.java:
##########
@@ -173,16 +173,17 @@ public void processElement(
         } else {
           Instant watermark = reader.getWatermark();
           if (watermark.isBefore(BoundedWindow.TIMESTAMP_MAX_VALUE)) {
-            // If the reader had no elements available, but the shard is not done, reuse it later
-            // Might be better to finalize old checkpoint.
+            // If the reader had no elements available, but the shard is not done, reuse it later.

Review Comment:
   How about on dataflow runner? Do we have the same problem?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3579861124


##########
sdks/java/io/jms/src/test/java/org/apache/beam/sdk/io/jms/JmsIOIT.java:
##########
@@ -145,84 +149,145 @@ public interface JmsIOITOptions extends IOTestPipelineOptions, StreamingOptions
   @Parameterized.Parameters(name = "with client class {3}")
   public static Collection<Object[]> connectionFactories() {
     return ImmutableList.of(
-        new Object[] {
-          "vm://localhost", 5672, "jms.sendAcksAsync=false", ActiveMQConnectionFactory.class
-        });
-    // TODO(https://github.com/apache/beam/issues/26175) Test failure on direct runner due to
-    //  JmsIO read on amqp slow on CI (passed locally)
-    // new Object[] {
-    //   "amqp://localhost", 5672, "jms.forceAsyncAcks=false", JmsConnectionFactory.class
-    // });
+        new Object[] {"vm://localhost", "jms.sendAcksAsync=false", ActiveMQConnectionFactory.class},
+        new Object[] {"amqp://localhost", "jms.forceAsyncAcks=false", JmsConnectionFactory.class});
   }
 
-  private final CommonJms commonJms;
+  private static final Map<String, CommonJms> BROKERS = new ConcurrentHashMap<>();
+
+  private final String brokerUrl;
+  private final Integer brokerPort;
+  private final String forceAsyncAcksParam;
+  private final Class<? extends ConnectionFactory> connectionFactoryClassParam;
+  private CommonJms commonJms;
   private ConnectionFactory connectionFactory;
   private Class<? extends ConnectionFactory> connectionFactoryClass;
 
   public JmsIOIT(
       String brokerUrl,
-      Integer brokerPort,
       String forceAsyncAcksParam,
       Class<? extends ConnectionFactory> connectionFactoryClass) {
-    this.commonJms =
-        new CommonJms(
-            OPTIONS.isLocalJmsBrokerEnabled() ? brokerUrl : OPTIONS.getJmsBrokerHost(),
-            OPTIONS.isLocalJmsBrokerEnabled() ? brokerPort : OPTIONS.getJmsBrokerPort(),
-            forceAsyncAcksParam,
-            connectionFactoryClass);
+    this.brokerUrl = brokerUrl;
+    if (OPTIONS.isLocalJmsBrokerEnabled()) {
+      try {
+        this.brokerPort = NetworkTestHelper.getAvailableLocalPort();
+      } catch (IOException e) {
+        throw new RuntimeException("Failed to find available port", e);
+      }
+    } else {
+      this.brokerPort = OPTIONS.getJmsBrokerPort();
+    }
+    this.forceAsyncAcksParam = forceAsyncAcksParam;
+    this.connectionFactoryClassParam = connectionFactoryClass;
   }
 
   @Before
   public void setup() throws Exception {
     if (OPTIONS.isLocalJmsBrokerEnabled()) {
-      this.commonJms.startBroker();
-      connectionFactory = this.commonJms.createConnectionFactory();
-      connectionFactoryClass = this.commonJms.getConnectionFactoryClass();
-      // use a small number of record for local integration test
+      String key = brokerUrl + ":" + connectionFactoryClassParam.getName();
+      commonJms =
+          BROKERS.computeIfAbsent(
+              key,
+              k -> {
+                CommonJms broker =
+                    new CommonJms(
+                        brokerUrl, brokerPort, forceAsyncAcksParam, connectionFactoryClassParam);
+                try {
+                  broker.startBroker();
+                } catch (Exception e) {
+                  throw new RuntimeException(e);
+                }
+                return broker;
+              });
       OPTIONS.setNumberOfRecords(10000);
+    } else {
+      commonJms =
+          new CommonJms(
+              OPTIONS.getJmsBrokerHost(),
+              OPTIONS.getJmsBrokerPort(),
+              forceAsyncAcksParam,
+              connectionFactoryClassParam);
     }
   }
 
+  @AfterClass
+  public static void afterClass() throws Exception {
+    for (CommonJms broker : BROKERS.values()) {
+      try {
+        broker.stopBroker();
+      } catch (Exception e) {
+        // ignore errors on shutdown
+      }
+    }
+    BROKERS.clear();
+  }
+
+  private void setupConnection(JmsIO.AcknowledgeMode acknowledgeMode) throws Exception {
+    if (acknowledgeMode == JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE) {
+      connectionFactory = this.commonJms.createConnectionFactoryWithSyncAcksAndWithoutPrefetch();
+    } else {
+      connectionFactory = this.commonJms.createConnectionFactory();
+    }
+    connectionFactoryClass = this.commonJms.getConnectionFactoryClass();
+  }
+
   @After
   public void tearDown() throws Exception {
-    if (OPTIONS.isLocalJmsBrokerEnabled()) {
-      this.commonJms.stopBroker();
-      connectionFactory = null;
-      connectionFactoryClass = null;
-    }
+    connectionFactory = null;
+    connectionFactoryClass = null;
+  }
+
+  @Test
+  public void testPublishingThenReadingAll() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE);
+  }
+
+  @Test
+  public void testPublishingThenReadingAllIndividualAcknowledge() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.INDIVIDUAL_ACKNOWLEDGE);
   }
 
   @Test
-  public void testPublishingThenReadingAll() throws IOException, JMSException {
-    PipelineResult writeResult = publishingMessages();
+  public void testPublishingThenReadingAllClientAcknowledgeUnsafe() throws Exception {
+    runPublishingThenReadingAll(JmsIO.AcknowledgeMode.CLIENT_ACKNOWLEDGE_UNSAFE);
+  }
+
+  private void runPublishingThenReadingAll(JmsIO.AcknowledgeMode acknowledgeMode) throws Exception {
+    setupConnection(acknowledgeMode);
+    String queue = QUEUE + "_" + acknowledgeMode.name();
+    PipelineResult writeResult = publishingMessages(queue);
     PipelineResult.State writeState = writeResult.waitUntilFinish();
     assertNotEquals(PipelineResult.State.FAILED, writeState);
 
-    PipelineResult readResult = readMessages();
-    PipelineResult.State readState =
-        readResult.waitUntilFinish(Duration.standardSeconds(OPTIONS.getReadTimeout()));
-    // A workaround to stop the pipeline for waiting for too long
+    PipelineResult readResult = readMessages(acknowledgeMode, queue);
+    MetricsReader metricsReader = new MetricsReader(readResult, NAMESPACE);
+    long startTime = System.currentTimeMillis();
+    long timeoutMillis = OPTIONS.getReadTimeout() * 1000L;
+    PipelineResult.State readState = readResult.getState();

Review Comment:
   Outside the streaming IO scope I was tagged for, but flagging it since I noticed it. `readState` comes from `readResult.getState()`, which on the direct runner returns `RUNNING` and never `null`. So when `readTimeout` expires with the pipeline still running (the amqp slow on CI case from #26175 that got this test disabled), `cancelIfTimeouted`, which only cancels when `readState == null`, is a no op. The asserts then fail while the pipeline keeps consuming from the class scoped shared `BROKERS`, leaking it into the next parameterized test. Suggest:
   
   ```java
   if (readState == null || !readState.isTerminal()) {
     readResult.cancel();
   }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3580462541


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   > My understanding is that finalizeCheckpoint() may be called on a different thread from the reader on some runners.
   
   Correct
   
   > does message.acknowledge() here running concurrently with the reader's receive loop on the same Session
   
   No for `INDIVIDUAL_ACKNOWLEDGE ` and `CLIENT_ACKNOWLEDGE_UNSAFE`. That's why the latter is "unsafe". The former isn't part of Jms spec but some providers' extended feature.
   
   > Could you confirm how ack is serialized against receive for the other two modes, or which providers this has been validated against? 
   
   The integration tests now covers this question, tested on ActiveMQ and Amqp provider.
   
   > `CLIENT_ACKNOWLEDGE` looks immune since each mark owns a private session
   
   Yes
   
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3580722272


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsIO.java:
##########
@@ -783,21 +896,43 @@ public void close() {
     private void doClose() {
       try {
         closeAutoscaler();
-        closeConsumer();
-        ScheduledExecutorService executorService =
-            options.as(ExecutorOptions.class).getScheduledExecutorService();
-        executorService.schedule(
-            () -> {
-              LOG.debug("Closing connection after delay {}", source.spec.getCloseTimeout());
-              // Discard the checkpoints and set the reader as inactive
-              checkpointMarkPreparer.discard();
-              closeSession();
-              closeConnection();
-            },
-            source.spec.getCloseTimeout().getMillis(),
-            TimeUnit.MILLISECONDS);
+        // Discard the checkpoints and set the reader as inactive
+        checkpointMarkPreparer.discard();
+        if (source.spec.getAcknowledgeMode() == AcknowledgeMode.CLIENT_ACKNOWLEDGE) {
+          // checkpointMark holds session in CLIENT_ACKNOWLEDGE mode. Therefore
+          // we can close consumer and session immediately.
+          closeConsumer();
+          closeSession();
+        }
+        if (activeCheckpoints.get() == 0) {
+          closeConsumer();
+          closeSession();
+          closeConnection();
+        } else {
+          ScheduledExecutorService executorService =
+              options.as(ExecutorOptions.class).getScheduledExecutorService();
+          executorService.submit(
+              () -> {
+                long startTime = System.currentTimeMillis();
+                long timeoutMillis = source.spec.getCloseTimeout().getMillis();
+                while (activeCheckpoints.get() > 0
+                    && System.currentTimeMillis() - startTime < timeoutMillis) {
+                  try {
+                    Thread.sleep(1_000); // poll in 1 sec interval
+                  } catch (InterruptedException ignored) {
+                    break;
+                  }
+                }
+                LOG.debug(
+                    "Closing connection after checkpoints finalized or timeout: {}",
+                    source.spec.getCloseTimeout());
+                closeConsumer();
+                closeSession();
+                closeConnection();
+              });

Review Comment:
   Agree this would theoretically happen, though in my observation this is rare.
   
   The `ScheduledExecutorService` only started when `activeCheckpoints > 0` (either there is active checkpoint or there is checkpoint never get finalized). JmsCheckpointMark.finalizeCheckpoint does not throw, so when it's executed, the counter is decremented.
   
   > Suggest rescheduling via schedule() with a short period
   
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3602766988


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   Thanks for pushing back. 
   I checked the ActiveMQ and Qpid client internals and you are right in practice. 
   1. ActiveMQ Classic guards deliveredMessages on both the receive add and the individual ack remove and serializes the wire send through MutexTransport, and the individual ack deletes exactly the one message id without touching the CLIENT_ACKNOWLEDGE counters, so ack and receiveNoWait do not race there. 
   2. Qpid serializes everything on its single provider thread. Artemis is gentle in this interleaving too.
   
   The JMS spec does treat the whole session as single threaded including acknowledge, so this seems provider tolerance rather than a spec guarantee. That is fine for the three auto detected providers, but for someone setting a custom code via `withIndividualAcknowledgeModeCode` it is not guaranteed. A one line javadoc caveat there could be worth it. 
   
   👍 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3602925837


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   Thanks for digging in. I checked the ActiveMQ and Qpid client internals and you are right in practice. ActiveMQ Classic guards deliveredMessages on both the receive add and the individual ack remove and serializes the wire send through MutexTransport, and the individual ack deletes exactly the one message id without touching the CLIENT_ACKNOWLEDGE counters, so ack and receiveNoWait do not race there. Qpid serializes everything on its single provider thread. Artemis is benign in this interleaving too.
   
   One nuance for the record. The JMS spec does treat the whole session as single threaded including acknowledge, so this is provider tolerance rather than a spec guarantee. That is fine for the three auto detected providers, but for someone setting a custom code via withIndividualAcknowledgeModeCode it is not guaranteed. A one line javadoc caveat there would be worth it. Not blocking.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "tkaymak (via GitHub)" <gi...@apache.org>.
tkaymak commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3602766988


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   Thanks for pushing back. 
   I checked the ActiveMQ and Qpid client internals and you are right in practice. 
   1. ActiveMQ Classic guards deliveredMessages on both the receive add and the individual ack remove and serializes the wire send through MutexTransport, and the individual ack deletes exactly the one message id without touching the CLIENT_ACKNOWLEDGE counters, so ack and receiveNoWait do not race there. 
   2. Qpid serializes everything on its single provider thread. Artemis is gentle in this interleaving too.
   
   The JMS spec does treat the whole session as single threaded including acknowledge, so this seems provider tolerance rather than a spec guarantee. That is fine for the three auto detected providers, but for someone setting a custom code via `withIndividualAcknowledgeModeCode` it is not guaranteed. A one line javadoc caveat there could be worth it. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Support ACKNOWLEDGE_MODE for JmsIO [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #39253:
URL: https://github.com/apache/beam/pull/39253#discussion_r3596343550


##########
sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java:
##########
@@ -41,29 +45,32 @@ class JmsCheckpointMark implements UnboundedSource.CheckpointMark, Serializable
   private static final Logger LOG = LoggerFactory.getLogger(JmsCheckpointMark.class);
 
   private Instant oldestMessageTimestamp;
-  private transient @Nullable Message lastMessage;
+  private transient @Nullable List<Message> messages;
   private transient @Nullable MessageConsumer consumer;
   private transient @Nullable Session session;
+  private transient @Nullable AtomicInteger activeCheckpoints;
 
   private JmsCheckpointMark(
       Instant oldestMessageTimestamp,
-      @Nullable Message lastMessage,
+      @Nullable List<Message> messages,
       @Nullable MessageConsumer consumer,
-      @Nullable Session session) {
+      @Nullable Session session,
+      @Nullable AtomicInteger activeCheckpoints) {
     this.oldestMessageTimestamp = oldestMessageTimestamp;
-    this.lastMessage = lastMessage;
+    this.messages = messages;
     this.consumer = consumer;
     this.session = session;
+    this.activeCheckpoints = activeCheckpoints;
   }
 
   /** Acknowledge all outstanding message. */
   @Override
   public void finalizeCheckpoint() {
     try {
-      // Jms spec will implicitly acknowledge _all_ messaged already received by the same
-      // session if one message in this session is being acknowledged.
-      if (lastMessage != null) {
-        lastMessage.acknowledge();
+      if (messages != null) {
+        for (Message message : messages) {
+          message.acknowledge();

Review Comment:
   I agree a race is theoretically possible. Jms spec's requirement for session operations running in single thread and Beam's asynchronous checkpointing resulted in this fundamental incompatibility. recreate session in CLIENT_ACKNOWLEDGE tried to workaround it
   
   * CLIENT_ACKNOWLEDGE_UNSAFE
   As noted previously, this mode is intended to restore the same behavior and timing prior to Beam 2.50.0, regardless race exists or not, based on the users report that certain Jms providers work with Beam<2.50.0 but not later.
   
   * INDIVIDUAL_ACKNOWLEDGE is an open question. I did more investigation
   
   public doc emphasizing "session not threadsafe" or "one thread per session" is referring to receive:
   
   > The JMS contract is that only 1 session is used by one thread at once - which if you’re using consumers means that only 1 consumer can receive messages at once if using the same session.
   
   (see https://activemq.apache.org/components/classic/documentation/multiple-consumers-on-a-queue)
   
   but none saying acknowledge and receive must be in same thread. I think AI generated answer based on mixing up concepts in the available doc
   
   If prompt is specific, here is an answer I got
   
   > Q: In Jms, Active MQ INDIVIDUAL_ACKNOELEDGE mode, is there a race if receive new message and acknowledge old message happens concurrently
   >
   > A: No, there is no race condition between receiving a new message and acknowledging an old message in INDIVIDUAL_ACKNOWLEDGE mode. ActiveMQ tracks and deletes messages on the broker based on the specific message object you call acknowledge() on.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org