You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "henrik242 (via GitHub)" <gi...@apache.org> on 2026/09/02 03:52:37 UTC

[PR] CAMEL-24583: camel-master - do not start the delegated consumer after leadership is lost [camel]

henrik242 opened a new pull request, #26028:
URL: https://github.com/apache/camel/pull/26028

   Also makes BackgroundTask.schedule unschedule itself once the task has completed or run out of budget, which camel-master cannot do component-side once its start task exhausts its retries.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24583: camel-master - do not start the delegated consumer after leadership is lost (CAMEL-24584) [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26028:
URL: https://github.com/apache/camel/pull/26028#issuecomment-5506444191

   :star2: Thank you for your contribution to the Apache Camel project! :star2:
     :robot: CI automation will test this PR automatically.
   
     :camel: Apache Camel Committers, please review the following items:
   
     * First-time contributors **require MANUAL approval** for the GitHub Actions to run
     * You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
     * You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
     * Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
     
     :warning: Be careful when sharing logs. Review their contents before sharing them publicly.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24583: camel-master - do not start the delegated consumer after leadership is lost (CAMEL-24584) [camel]

Posted by "atiaomar1978-hub (via GitHub)" <gi...@apache.org>.
atiaomar1978-hub commented on code in PR #26028:
URL: https://github.com/apache/camel/pull/26028#discussion_r3910873283


##########
components/camel-master/src/main/java/org/apache/camel/component/master/MasterConsumer.java:
##########
@@ -151,60 +162,105 @@ private BackgroundTask createTask() {
                 .build();
     }
 
-    private void onLeadershipTaken() throws Exception {
+    private void onLeadershipTaken() {
         lock.lock();
         try {
             if (!isRunAllowed()) {
                 return;
             }
 
-            if (delegatedConsumer != null) {
+            leadershipTaken = true;
+
+            if (delegatedConsumer != null || isStartPending()) {
                 return;
             }
 
-            final BackgroundTask leaderTask = createTask();
-            leaderTask.schedule(getEndpoint().getCamelContext(), () -> {
-                if (!isRunAllowed()) {
-                    return false;
-                }
-                LOG.info("Leadership taken. Attempt #{} to start consumer: {}", leaderTask.iteration(), delegatedEndpoint);
-
-                Exception cause = null;
-                try {
-                    if (delegatedConsumer == null) {
-                        delegatedConsumer = delegatedEndpoint.createConsumer(processor);
-                        if (delegatedConsumer instanceof StartupListener) {
-                            getEndpoint().getCamelContext().addStartupListener((StartupListener) delegatedConsumer);
-                        }
-                        if (delegatedConsumer instanceof ResumeAware resumeAwareConsumer && resumeStrategy != null) {
-                            LOG.debug("Setting up the resume adapter for the resume strategy in consumer");
-                            ResumeAdapter resumeAdapter
-                                    = AdapterHelper.eval(clusterService.getCamelContext(), resumeAwareConsumer,
-                                            resumeStrategy);
-                            resumeStrategy.setAdapter(resumeAdapter);
-
-                            LOG.debug("Setting up the resume strategy for consumer");
-                            resumeAwareConsumer.setResumeStrategy(resumeStrategy);
-                        }
-                    }
-                    ServiceHelper.startService(delegatedEndpoint, delegatedConsumer);
+            // a task from a previous leadership term may still be scheduled, drop it
+            cancelLeaderTask(false);
+

Review Comment:
   **Possible resource leak (non-blocking):** When leadership is lost after `createConsumer()` succeeded but before start completes, this path returns without shutting down `attempt.get()`. Consider cleaning up the attempt consumer when aborting due to `!leadershipTaken`.



##########
components/camel-master/src/main/java/org/apache/camel/component/master/MasterConsumer.java:
##########
@@ -151,60 +162,105 @@ private BackgroundTask createTask() {
                 .build();
     }
 
-    private void onLeadershipTaken() throws Exception {
+    private void onLeadershipTaken() {
         lock.lock();
         try {
             if (!isRunAllowed()) {
                 return;
             }
 
-            if (delegatedConsumer != null) {
+            leadershipTaken = true;
+
+            if (delegatedConsumer != null || isStartPending()) {
                 return;
             }
 
-            final BackgroundTask leaderTask = createTask();
-            leaderTask.schedule(getEndpoint().getCamelContext(), () -> {
-                if (!isRunAllowed()) {
-                    return false;
-                }
-                LOG.info("Leadership taken. Attempt #{} to start consumer: {}", leaderTask.iteration(), delegatedEndpoint);
-
-                Exception cause = null;
-                try {
-                    if (delegatedConsumer == null) {
-                        delegatedConsumer = delegatedEndpoint.createConsumer(processor);
-                        if (delegatedConsumer instanceof StartupListener) {
-                            getEndpoint().getCamelContext().addStartupListener((StartupListener) delegatedConsumer);
-                        }
-                        if (delegatedConsumer instanceof ResumeAware resumeAwareConsumer && resumeStrategy != null) {
-                            LOG.debug("Setting up the resume adapter for the resume strategy in consumer");
-                            ResumeAdapter resumeAdapter
-                                    = AdapterHelper.eval(clusterService.getCamelContext(), resumeAwareConsumer,
-                                            resumeStrategy);
-                            resumeStrategy.setAdapter(resumeAdapter);
-
-                            LOG.debug("Setting up the resume strategy for consumer");
-                            resumeAwareConsumer.setResumeStrategy(resumeStrategy);
-                        }
-                    }
-                    ServiceHelper.startService(delegatedEndpoint, delegatedConsumer);
+            // a task from a previous leadership term may still be scheduled, drop it
+            cancelLeaderTask(false);
+
+            final BackgroundTask task = createTask();
+            // the consumer is created once and re-used by the start attempts of this task
+            final AtomicReference<Consumer> attempt = new AtomicReference<>();
+            leaderTaskFuture = task.schedule(getEndpoint().getCamelContext(), () -> startDelegatedConsumer(task, attempt));
+        } finally {
+            lock.unlock();
+        }
+    }
+
+    private boolean startDelegatedConsumer(BackgroundTask task, AtomicReference<Consumer> attempt) {
+        try {
+            // interruptibly, so cancelling the task while this consumer is being stopped does not
+            // keep the leader pool thread waiting for a lock the stopping thread holds

Review Comment:
   **StartupListener registration:** `addStartupListener` runs once when the attempt consumer is first created and is reused across retry attempts. If leadership is lost mid-retry, the listener can remain registered even though `delegatedConsumer` is never published. Worth a brief code comment so this is not mistaken for a leak in a later refactor.



##########
components/camel-master/src/main/java/org/apache/camel/component/master/MasterConsumer.java:
##########
@@ -213,6 +269,15 @@ private void onLeadershipTaken() throws Exception {
     private void onLeadershipLost() {
         lock.lock();
         try {
+            leadershipTaken = false;
+            // a start scheduled by the leadership taken event may not have run yet, cancel it so it
+            // cannot start the consumer on a node that is no longer the leader
+            cancelLeaderTask(false);
+
+            if (delegatedConsumer == null) {
+                return;
+            }
+
             LOG.debug("Leadership lost. Stopping consumer: {}", delegatedEndpoint);
             try {
                 ServiceHelper.stopAndShutdownServices(delegatedConsumer, delegatedEndpoint);

Review Comment:
   **Good fix for event ordering.** Dispatching `onLeadershipLost` even when `delegatedConsumer` is still null (pending start) is the core of CAMEL-24583. Serialising under the consumer lock prevents concurrent leadership events from being applied out of order. The cluster-view lock-ordering note in the PR description is appreciated — agree it is pre-existing and out of scope here.



##########
core/camel-support/src/main/java/org/apache/camel/support/task/BackgroundTask.java:
##########
@@ -154,8 +161,25 @@ private void runTaskWrapper(CamelContext camelContext, BooleanSupplier supplier)
      */
     public Future<?> schedule(CamelContext camelContext, BooleanSupplier supplier) {
         running.set(true);
-        return service.scheduleWithFixedDelay(() -> runTaskWrapper(camelContext, supplier), budget.initialDelay(),
-                budget.interval(), TimeUnit.MILLISECONDS);
+        Future<?> future = service.scheduleWithFixedDelay(() -> runTaskWrapper(camelContext, supplier),
+                budget.initialDelay(), budget.interval(), TimeUnit.MILLISECONDS);
+        scheduledFuture.set(future);
+        if (latch.getCount() == 0) {
+            // the task already finished before the future was published, so it could not unschedule itself
+            unschedule();
+        }
+        return future;

Review Comment:
   **CAMEL-24584 looks correct.** Auto-`unschedule()` on completed/exhausted paths stops the no-op rescheduling that made `isStartPending()` unreliable on cancelled futures. The post-`schedule()` latch check handles the synchronous-finish race. Upgrade-guide entry documents the `Future.isCancelled()` behaviour change.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24583: camel-master - do not start the delegated consumer after leadership is lost (CAMEL-24584) [camel]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26028:
URL: https://github.com/apache/camel/pull/26028#issuecomment-5507986346

   <!-- ci-tested-modules -->
   :test_tube: **CI tested the following changed modules:**
   
   - `components/camel-master`
   - `core/camel-core`
   - `core/camel-support`
   - `docs`
   
   > :information_source: Dependent modules were not tested because the total number of affected modules exceeded the threshold (50). Use the `test-dependents` label to force testing all dependents.
   
   ---
   
   <details><summary>:microscope: Scalpel shadow comparison — Scalpel: 562 tested, 26 compile-only — current: 561 all tested</summary>
   
   [Maveniverse Scalpel](https://github.com/maveniverse/scalpel) detected **588 affected modules** (current approach: 561).
   
   <details><summary>:warning: Modules only in Scalpel (27)</summary>
   
   - `apache-camel`
   - `camel-allcomponents`
   - `camel-catalog`
   - `camel-catalog-console`
   - `camel-catalog-maven`
   - `camel-catalog-suggest`
   - `camel-componentdsl`
   - `camel-endpointdsl`
   - `camel-endpointdsl-support`
   - `camel-itest`
   - `camel-jbang-core`
   - `camel-jbang-it`
   - `camel-jbang-main`
   - `camel-jbang-plugin-edit`
   - `camel-jbang-plugin-generate`
   - `camel-jbang-plugin-kubernetes`
   - `camel-jbang-plugin-test`
   - `camel-kamelet-main`
   - `camel-launcher`
   - `camel-report-maven-plugin`
   - `camel-route-parser`
   - `camel-yaml-dsl`
   - `camel-yaml-dsl-deserializers`
   - `camel-yaml-dsl-maven-plugin`
   - `coverage`
   - `docs`
   - `dummy-component`
   
   </details>
   
   **Skip-tests mode would test 562 modules** (4 direct + 558 downstream), **skip tests for 26** (generated code, meta-modules)
   
   <details><summary>Modules Scalpel would test (562)</summary>
   
   - `archetypes`
   - `camel-a2a`
   - `camel-activemq`
   - `camel-activemq6`
   - `camel-ai-observability`
   - `camel-ai-observability-api`
   - `camel-ai-parent`
   - `camel-ai-resource`
   - `camel-ai-tool`
   - `camel-alibaba-common`
   - `camel-alibaba-eventbridge`
   - `camel-alibaba-fc`
   - `camel-alibaba-kms`
   - `camel-alibaba-mns`
   - `camel-alibaba-oss`
   - `camel-alibaba-ots`
   - `camel-alibaba-parent`
   - `camel-alibaba-sls`
   - `camel-alibaba-sms`
   - `camel-amqp`
   - `camel-api-component-maven-plugin`
   - `camel-arangodb`
   - `camel-archetype-api-component`
   - `camel-archetype-component`
   - `camel-archetype-dataformat`
   - `camel-archetype-java`
   - `camel-archetype-main`
   - `camel-as2`
   - `camel-as2-api`
   - `camel-as2-parent`
   - `camel-asn1`
   - `camel-asterisk`
   - `camel-atmosphere-websocket`
   - `camel-atom`
   - `camel-attachments`
   - `camel-avro`
   - `camel-avro-rpc`
   - `camel-avro-rpc-jetty`
   - `camel-avro-rpc-parent`
   - `camel-avro-rpc-spi`
   - `camel-aws-bedrock`
   - `camel-aws-cloudtrail`
   - `camel-aws-common`
   - `camel-aws-config`
   - `camel-aws-parameter-store`
   - `camel-aws-parent`
   - `camel-aws-secrets-manager`
   - `camel-aws-security-hub`
   - `camel-aws2-athena`
   - `camel-aws2-comprehend`
   - `camel-aws2-cw`
   - `camel-aws2-ddb`
   - `camel-aws2-ec2`
   - `camel-aws2-ecs`
   - `camel-aws2-eks`
   - `camel-aws2-eventbridge`
   - `camel-aws2-iam`
   - `camel-aws2-kinesis`
   - `camel-aws2-kms`
   - `camel-aws2-lambda`
   - `camel-aws2-mq`
   - `camel-aws2-msk`
   - `camel-aws2-polly`
   - `camel-aws2-redshift`
   - `camel-aws2-rekognition`
   - `camel-aws2-s3`
   - `camel-aws2-s3-vectors`
   - `camel-aws2-ses`
   - `camel-aws2-sns`
   - `camel-aws2-sqs`
   - `camel-aws2-step-functions`
   - `camel-aws2-sts`
   - `camel-aws2-textract`
   - `camel-aws2-timestream`
   - `camel-aws2-transcribe`
   - `camel-aws2-translate`
   - `camel-azure-common`
   - `camel-azure-cosmosdb`
   - `camel-azure-eventgrid`
   - `camel-azure-eventhubs`
   - `camel-azure-files`
   - `camel-azure-functions`
   - `camel-azure-key-vault`
   - `camel-azure-parent`
   - `camel-azure-schema-registry`
   - `camel-azure-servicebus`
   - `camel-azure-storage-blob`
   - `camel-azure-storage-datalake`
   - `camel-azure-storage-queue`
   - `camel-barcode`
   - `camel-base`
   - `camel-base-engine`
   - `camel-base64`
   - `camel-bean`
   - `camel-bean-validator`
   - `camel-beanio`
   - `camel-bindy`
   - `camel-bonita`
   - `camel-box`
   - `camel-box-api`
   - `camel-box-parent`
   - `camel-braintree`
   - `camel-browse`
   - `camel-caffeine`
   - `camel-camunda`
   - `camel-cassandraql`
   - `camel-catalog-common`
   - `camel-cbor`
   - `camel-chatscript`
   - `camel-chunk`
   - `camel-cli-connector`
   - `camel-cli-debug`
   - `camel-clickhouse`
   - `camel-clickup`
   - `camel-cloudevents`
   - `camel-cluster`
   - `camel-cm-sms`
   - `camel-coap`
   - `camel-cometd`
   - `camel-console`
   - `camel-consul`
   - `camel-controlbus`
   - `camel-core`
   - `camel-core-all`
   - `camel-core-engine`
   - `camel-core-languages`
   - `camel-core-model`
   - `camel-core-processor`
   - `camel-core-reifier`
   - `camel-core-xml`
   - `camel-couchbase`
   - `camel-couchdb`
   - `camel-cron`
   - `camel-crypto`
   - `camel-crypto-pgp`
   - `camel-csv`
   - `camel-cxf-common`
   - `camel-cxf-parent`
   - `camel-cxf-rest`
   - `camel-cxf-soap`
   - `camel-cxf-spring-common`
   - `camel-cxf-spring-rest`
   - `camel-cxf-spring-soap`
   - `camel-cxf-spring-transport`
   - `camel-cxf-transport`
   - `camel-cyberark-vault`
   - `camel-dapr`
   - `camel-dataformat`
   - `camel-dataset`
   - `camel-datasonnet`
   - `camel-dataweave`
   - `camel-debezium-common`
   - `camel-debezium-common-parent`
   - `camel-debezium-db2`
   - `camel-debezium-maven-plugin`
   - `camel-debezium-mongodb`
   - `camel-debezium-mysql`
   - `camel-debezium-oracle`
   - `camel-debezium-parent`
   - `camel-debezium-postgres`
   - `camel-debezium-sqlserver`
   - `camel-debug`
   - `camel-dfdl`
   - `camel-dhis2`
   - `camel-dhis2-api`
   - `camel-dhis2-parent`
   - `camel-diagram`
   - `camel-direct`
   - `camel-disruptor`
   - `camel-djl`
   - `camel-dns`
   - `camel-docker`
   - `camel-docling`
   - `camel-drill`
   - `camel-dropbox`
   - `camel-dsl-modeline`
   - `camel-dsl-support`
   - `camel-duckdb`
   - `camel-dynamic-router`
   - `camel-ehcache`
   - `camel-elasticsearch`
   - `camel-elasticsearch-rest-client`
   - `camel-event`
   - `camel-exec`
   - `camel-fastjson`
   - `camel-fhir`
   - `camel-fhir-api`
   - `camel-fhir-parent`
   - `camel-file`
   - `camel-file-watch`
   - `camel-flatpack`
   - `camel-flink`
   - `camel-flowable`
   - `camel-fop`
   - `camel-fory`
   - `camel-freemarker`
   - `camel-ftp`
   - `camel-ftp-common`
   - `camel-geocoder`
   - `camel-git`
   - `camel-github2`
   - `camel-google-bigquery`
   - `camel-google-calendar`
   - `camel-google-common`
   - `camel-google-drive`
   - `camel-google-firestore`
   - `camel-google-functions`
   - `camel-google-mail`
   - `camel-google-parent`
   - `camel-google-pubsub`
   - `camel-google-secret-manager`
   - `camel-google-sheets`
   - `camel-google-speech-to-text`
   - `camel-google-storage`
   - `camel-google-text-to-speech`
   - `camel-google-vertexai`
   - `camel-google-vision`
   - `camel-graphql`
   - `camel-grok`
   - `camel-groovy`
   - `camel-grpc`
   - `camel-gson`
   - `camel-hashicorp-vault`
   - `camel-hazelcast`
   - `camel-health`
   - `camel-hl7`
   - `camel-http`
   - `camel-http-base`
   - `camel-http-common`
   - `camel-huawei-parent`
   - `camel-huaweicloud-common`
   - `camel-huaweicloud-dms`
   - `camel-huaweicloud-frs`
   - `camel-huaweicloud-functiongraph`
   - `camel-huaweicloud-iam`
   - `camel-huaweicloud-imagerecognition`
   - `camel-huaweicloud-obs`
   - `camel-huaweicloud-smn`
   - `camel-huggingface`
   - `camel-ibm-cos`
   - `camel-ibm-parent`
   - `camel-ibm-secrets-manager`
   - `camel-ibm-watson-discovery`
   - `camel-ibm-watson-language`
   - `camel-ibm-watson-speech-to-text`
   - `camel-ibm-watson-text-to-speech`
   - `camel-ibm-watsonx-ai`
   - `camel-ibm-watsonx-data`
   - `camel-ical`
   - `camel-iggy`
   - `camel-ignite`
   - `camel-infinispan`
   - `camel-infinispan-common`
   - `camel-infinispan-embedded`
   - `camel-infinispan-parent`
   - `camel-influxdb`
   - `camel-influxdb2`
   - `camel-iso8583`
   - `camel-jackson`
   - `camel-jackson-avro`
   - `camel-jackson-protobuf`
   - `camel-jackson3`
   - `camel-jackson3-avro`
   - `camel-jackson3-protobuf`
   - `camel-jackson3xml`
   - `camel-jacksonxml`
   - `camel-jactl`
   - `camel-jandex`
   - `camel-jasypt`
   - `camel-java-io`
   - `camel-java-joor-dsl`
   - `camel-javascript`
   - `camel-jaxb`
   - `camel-jbang-console`
   - `camel-jbang-mcp`
   - `camel-jbang-plugin-mcp`
   - `camel-jbang-plugin-route-parser`
   - `camel-jbang-plugin-tui`
   - `camel-jbang-plugin-validate`
   - `camel-jcache`
   - `camel-jcr`
   - `camel-jdbc`
   - `camel-jetty`
   - `camel-jetty-common`
   - `camel-jfr`
   - `camel-jgroups`
   - `camel-jgroups-raft`
   - `camel-jira`
   - `camel-jms`
   - `camel-jmx`
   - `camel-jolt`
   - `camel-jooq`
   - `camel-joor`
   - `camel-jpa`
   - `camel-jq`
   - `camel-jsch`
   - `camel-jslt`
   - `camel-json-validator`
   - `camel-jsonapi`
   - `camel-jsonata`
   - `camel-jsonb`
   - `camel-jsonpath`
   - `camel-jsoup`
   - `camel-jt400`
   - `camel-jta`
   - `camel-jte`
   - `camel-kafka`
   - `camel-kamelet`
   - `camel-kamelet-main-support`
   - `camel-keycloak`
   - `camel-knative`
   - `camel-knative-api`
   - `camel-knative-http`
   - `camel-knative-parent`
   - `camel-kserve`
   - `camel-kubernetes`
   - `camel-kudu`
   - `camel-langchain4j-agent`
   - `camel-langchain4j-agent-api`
   - `camel-langchain4j-chat`
   - `camel-langchain4j-core`
   - `camel-langchain4j-embeddings`
   - `camel-langchain4j-embeddingstore`
   - `camel-langchain4j-embeddingstore-api`
   - `camel-langchain4j-tokenizer`
   - `camel-langchain4j-web-search`
   - `camel-language`
   - `camel-launcher-container`
   - `camel-ldap`
   - `camel-ldif`
   - `camel-log`
   - `camel-lra`
   - `camel-lucene`
   - `camel-lumberjack`
   - `camel-lzf`
   - `camel-mail`
   - `camel-mail-microsoft-oauth`
   - `camel-main`
   - `camel-management`
   - `camel-mapstruct`
   - `camel-master`
   - `camel-maven-plugin`
   - `camel-mcp-server`
   - `camel-mcp-server-api`
   - `camel-mdc`
   - `camel-metrics`
   - `camel-micrometer`
   - `camel-micrometer-observability`
   - `camel-micrometer-prometheus`
   - `camel-microprofile-config`
   - `camel-microprofile-fault-tolerance`
   - `camel-microprofile-health`
   - `camel-microprofile-parent`
   - `camel-milo`
   - `camel-milvus`
   - `camel-mina`
   - `camel-mina-sftp`
   - `camel-minio`
   - `camel-mllp`
   - `camel-mock`
   - `camel-mongodb`
   - `camel-mongodb-gridfs`
   - `camel-mustache`
   - `camel-mvel`
   - `camel-mybatis`
   - `camel-nats`
   - `camel-neo4j`
   - `camel-netty`
   - `camel-netty-http`
   - `camel-oaipmh`
   - `camel-oauth`
   - `camel-observability-services`
   - `camel-observation`
   - `camel-ocsf`
   - `camel-ognl`
   - `camel-olingo2`
   - `camel-olingo2-api`
   - `camel-olingo2-parent`
   - `camel-olingo4`
   - `camel-olingo4-api`
   - `camel-olingo4-parent`
   - `camel-once`
   - `camel-openai`
   - `camel-openapi-java`
   - `camel-openapi-rest-dsl-generator`
   - `camel-openapi-validator`
   - `camel-opensearch`
   - `camel-openstack`
   - `camel-opentelemetry`
   - `camel-opentelemetry-metrics`
   - `camel-opentelemetry2`
   - `camel-optaplanner`
   - `camel-paho`
   - `camel-paho-mqtt5`
   - `camel-parquet-avro`
   - `camel-pdf`
   - `camel-pg-replication-slot`
   - `camel-pgevent`
   - `camel-pgvector`
   - `camel-pinecone`
   - `camel-platform-http`
   - `camel-platform-http-jolokia`
   - `camel-platform-http-main`
   - `camel-platform-http-vertx`
   - `camel-plc4x`
   - `camel-pqc`
   - `camel-printer`
   - `camel-protobuf`
   - `camel-pubnub`
   - `camel-pulsar`
   - `camel-python`
   - `camel-python3`
   - `camel-qdrant`
   - `camel-quartz`
   - `camel-quickfix`
   - `camel-quickjs`
   - `camel-reactive-streams`
   - `camel-reactor`
   - `camel-redis`
   - `camel-ref`
   - `camel-resilience4j`
   - `camel-resilience4j-micrometer`
   - `camel-resourceresolver-github`
   - `camel-rest`
   - `camel-rest-openapi`
   - `camel-rest-postman`
   - `camel-restdsl-openapi-plugin`
   - `camel-robotframework`
   - `camel-rocketmq`
   - `camel-rss`
   - `camel-rxjava`
   - `camel-saga`
   - `camel-salesforce`
   - `camel-salesforce-codegen`
   - `camel-salesforce-maven-plugin`
   - `camel-salesforce-parent`
   - `camel-sap-netweaver`
   - `camel-saxon`
   - `camel-scheduler`
   - `camel-schematron`
   - `camel-seda`
   - `camel-servicenow`
   - `camel-servicenow-maven-plugin`
   - `camel-servicenow-parent`
   - `camel-servlet`
   - `camel-shell`
   - `camel-shiro`
   - `camel-sjms`
   - `camel-sjms2`
   - `camel-slack`
   - `camel-smb`
   - `camel-smooks`
   - `camel-smpp`
   - `camel-snakeyaml`
   - `camel-snmp`
   - `camel-soap`
   - `camel-solr`
   - `camel-spiffe`
   - `camel-splunk-hec`
   - `camel-spring`
   - `camel-spring-ai-chat`
   - `camel-spring-ai-embeddings`
   - `camel-spring-ai-image`
   - `camel-spring-ai-parent`
   - `camel-spring-ai-vector-store`
   - `camel-spring-batch`
   - `camel-spring-cloud-config`
   - `camel-spring-jdbc`
   - `camel-spring-ldap`
   - `camel-spring-main`
   - `camel-spring-parent`
   - `camel-spring-rabbitmq`
   - `camel-spring-redis`
   - `camel-spring-security`
   - `camel-spring-ws`
   - `camel-spring-xml`
   - `camel-sql`
   - `camel-ssh`
   - `camel-state-store`
   - `camel-state-store-parent`
   - `camel-stax`
   - `camel-stitch`
   - `camel-stream`
   - `camel-streamcaching-test`
   - `camel-stringtemplate`
   - `camel-stripe`
   - `camel-stub`
   - `camel-support`
   - `camel-swift`
   - `camel-syslog`
   - `camel-tahu`
   - `camel-tarfile`
   - `camel-telegram`
   - `camel-telemetry`
   - `camel-telemetry-dev`
   - `camel-tensorflow-serving`
   - `camel-test-infra-all`
   - `camel-test-infra-artemis`
   - `camel-test-infra-cli`
   - `camel-test-infra-core`
   - `camel-test-infra-smb`
   - `camel-test-junit5`
   - `camel-test-junit6`
   - `camel-test-main-junit5`
   - `camel-test-main-junit6`
   - `camel-test-parent`
   - `camel-test-spring-junit5`
   - `camel-test-spring-junit6`
   - `camel-thrift`
   - `camel-thymeleaf`
   - `camel-tika`
   - `camel-timer`
   - `camel-tooling-maven`
   - `camel-toon`
   - `camel-tracing`
   - `camel-twilio`
   - `camel-twitter`
   - `camel-ubl`
   - `camel-undertow`
   - `camel-undertow-spring-security`
   - `camel-univocity-parsers`
   - `camel-validator`
   - `camel-velocity`
   - `camel-vertx`
   - `camel-vertx-common`
   - `camel-vertx-http`
   - `camel-vertx-parent`
   - `camel-vertx-websocket`
   - `camel-wal`
   - `camel-wasm`
   - `camel-weather`
   - `camel-weaviate`
   - `camel-web3j`
   - `camel-webhook`
   - `camel-whatsapp`
   - `camel-wordpress`
   - `camel-workday`
   - `camel-xchange`
   - `camel-xj`
   - `camel-xml-io`
   - `camel-xml-io-dsl`
   - `camel-xml-jaxb`
   - `camel-xml-jaxb-dsl`
   - `camel-xml-jaxb-dsl-test-definition`
   - `camel-xml-jaxb-dsl-test-spring`
   - `camel-xml-jaxp`
   - `camel-xmlsecurity`
   - `camel-xmpp`
   - `camel-xpath`
   - `camel-xslt`
   - `camel-xslt-saxon`
   - `camel-yaml-dsl-common`
   - `camel-yaml-dsl-validator`
   - `camel-yaml-dsl-validator-maven-plugin`
   - `camel-yaml-io`
   - `camel-zendesk`
   - `camel-zip-deflater`
   - `camel-zipfile`
   - `camel-zookeeper`
   - `camel-zookeeper-master`
   - `components`
   - `docs`
   
   </details>
   
   <details><summary>Modules with tests skipped (26)</summary>
   
   - `apache-camel`
   - `camel-allcomponents`
   - `camel-catalog`
   - `camel-catalog-console`
   - `camel-catalog-maven`
   - `camel-catalog-suggest`
   - `camel-componentdsl`
   - `camel-endpointdsl`
   - `camel-endpointdsl-support`
   - `camel-itest`
   - `camel-jbang-core`
   - `camel-jbang-it`
   - `camel-jbang-main`
   - `camel-jbang-plugin-edit`
   - `camel-jbang-plugin-generate`
   - `camel-jbang-plugin-kubernetes`
   - `camel-jbang-plugin-test`
   - `camel-kamelet-main`
   - `camel-launcher`
   - `camel-report-maven-plugin`
   - `camel-route-parser`
   - `camel-yaml-dsl`
   - `camel-yaml-dsl-deserializers`
   - `camel-yaml-dsl-maven-plugin`
   - `coverage`
   - `dummy-component`
   
   </details>
   
   > :information_source: Shadow mode — Scalpel observes but does not affect test execution. [Learn more](https://github.com/maveniverse/scalpel)
   
   </details>
   
   :warning: **Some tests are disabled on GitHub Actions** (`@DisabledIfSystemProperty(named = "ci.env.name")`) and require manual verification:
   
   - `core/camel-core`: 2 test(s) disabled on GitHub Actions
   
   <details><summary>Build reactor — dependencies compiled but only changed modules were tested (587 modules)</summary>
   
   - Camel :: AI :: A2A
   - Camel :: AI :: ChatScript
   - Camel :: AI :: Deep Java Library
   - Camel :: AI :: Docling
   - Camel :: AI :: Hugging Face
   - Camel :: AI :: KServe
   - Camel :: AI :: LangChain4j :: Agent
   - Camel :: AI :: LangChain4j :: Agent :: API
   - Camel :: AI :: LangChain4j :: Chat
   - Camel :: AI :: LangChain4j :: Core
   - Camel :: AI :: LangChain4j :: Embedding
   - Camel :: AI :: LangChain4j :: Embedding Store :: API
   - Camel :: AI :: LangChain4j :: EmbeddingStore
   - Camel :: AI :: LangChain4j :: Tokenizer
   - Camel :: AI :: LangChain4j :: Web Search
   - Camel :: AI :: MCP Server
   - Camel :: AI :: MCP Server API
   - Camel :: AI :: Milvus
   - Camel :: AI :: Neo4j
   - Camel :: AI :: Observability
   - Camel :: AI :: Observability API
   - Camel :: AI :: OpenAI
   - Camel :: AI :: PGVector
   - Camel :: AI :: Parent
   - Camel :: AI :: Pinecone
   - Camel :: AI :: Qdrant
   - Camel :: AI :: Resource
   - Camel :: AI :: TensorFlow Serving
   - Camel :: AI :: Tool
   - Camel :: AI :: Weaviate
   - Camel :: AMQP
   - Camel :: AS2 :: API
   - Camel :: AS2 :: Component
   - Camel :: AS2 :: Parent
   - Camel :: ASN.1
   - Camel :: AWS :: Common
   - Camel :: AWS :: Parent
   - Camel :: AWS CloudTrail
   - Camel :: AWS Config
   - Camel :: AWS Redshift Data
   - Camel :: AWS Rekognition
   - Camel :: AWS Security Hub
   - Camel :: AWS Step Functions
   - Camel :: AWS Timestream
   - Camel :: AWS2 :: Transcribe
   - Camel :: AWS2 Athena
   - Camel :: AWS2 Bedrock
   - Camel :: AWS2 CW
   - Camel :: AWS2 Comprehend
   - Camel :: AWS2 DDB
   - Camel :: AWS2 EC2
   - Camel :: AWS2 ECS
   - Camel :: AWS2 EKS
   - Camel :: AWS2 Eventbridge
   - Camel :: AWS2 IAM
   - Camel :: AWS2 KMS
   - Camel :: AWS2 Kinesis
   - Camel :: AWS2 Lambda
   - Camel :: AWS2 MQ
   - Camel :: AWS2 MSK
   - Camel :: AWS2 Parameter Store
   - Camel :: AWS2 Polly
   - Camel :: AWS2 S3
   - Camel :: AWS2 S3 Vectors
   - Camel :: AWS2 SES
   - Camel :: AWS2 SNS
   - Camel :: AWS2 SQS
   - Camel :: AWS2 STS
   - Camel :: AWS2 Secrets Manager
   - Camel :: AWS2 Textract
   - Camel :: AWS2 Translate
   - Camel :: ActiveMQ 5.x
   - Camel :: ActiveMQ 6.x
   - Camel :: Alibaba Cloud :: Common
   - Camel :: Alibaba Cloud :: EventBridge
   - Camel :: Alibaba Cloud :: Function Compute (FC)
   - Camel :: Alibaba Cloud :: Key Management Service (KMS)
   - Camel :: Alibaba Cloud :: Message Service (MNS)
   - Camel :: Alibaba Cloud :: OSS
   - Camel :: Alibaba Cloud :: Parent
   - Camel :: Alibaba Cloud :: Short Message Service (SMS)
   - Camel :: Alibaba Cloud :: Simple Log Service (SLS)
   - Camel :: Alibaba Cloud :: Tablestore (OTS)
   - Camel :: All Components Sync point
   - Camel :: All Core Sync point
   - Camel :: ArangoDB
   - Camel :: Archetypes
   - Camel :: Archetypes :: API Component
   - Camel :: Archetypes :: Component
   - Camel :: Archetypes :: Data Format
   - Camel :: Archetypes :: Java Router
   - Camel :: Archetypes :: Main
   - Camel :: Assembly
   - Camel :: Asterisk
   - Camel :: Atmosphere WebSocket Servlet
   - Camel :: Atom
   - Camel :: Attachments
   - Camel :: Avro
   - Camel :: Avro RPC
   - Camel :: Avro RPC :: Jetty
   - Camel :: Avro RPC :: Parent
   - Camel :: Avro RPC :: Spi
   - Camel :: Azure :: Common
   - Camel :: Azure :: CosmosDB
   - Camel :: Azure :: Event Grid
   - Camel :: Azure :: Event Hubs
   - Camel :: Azure :: Files
   - Camel :: Azure :: Functions
   - Camel :: Azure :: Key Vault
   - Camel :: Azure :: Parent
   - Camel :: Azure :: Schema Registry
   - Camel :: Azure :: ServiceBus
   - Camel :: Azure :: Storage Blob
   - Camel :: Azure :: Storage Datalake
   - Camel :: Azure :: Storage Queue
   - Camel :: Barcode
   - Camel :: Base
   - Camel :: Base Engine
   - Camel :: Base64
   - Camel :: Bean
   - Camel :: Bean validator
   - Camel :: BeanIO
   - Camel :: Bindy
   - Camel :: Bonita
   - Camel :: Box :: API
   - Camel :: Box :: Component
   - Camel :: Box :: Parent
   - Camel :: Braintree
   - Camel :: Browse
   - Camel :: CBOR
   - Camel :: CM SMS
   - Camel :: CSV
   - Camel :: CXF :: Common
   - Camel :: CXF :: Common :: Spring
   - Camel :: CXF :: Parent
   - Camel :: CXF :: REST
   - Camel :: CXF :: REST :: Spring
   - Camel :: CXF :: SOAP
   - Camel :: CXF :: SOAP :: Spring
   - Camel :: CXF :: Transport
   - Camel :: CXF :: Transport :: Spring
   - Camel :: Caffeine
   - Camel :: Camunda
   - Camel :: Cassandra CQL
   - Camel :: Catalog :: Camel Catalog
   - Camel :: Catalog :: Camel Report Maven Plugin
   - Camel :: Catalog :: Camel Route Parser
   - Camel :: Catalog :: Common
   - Camel :: Catalog :: Console
   - Camel :: Catalog :: Dummy Component
   - Camel :: Catalog :: Maven
   - Camel :: Catalog :: Suggest
   - Camel :: Chunk
   - Camel :: ClickHouse
   - Camel :: ClickUp
   - Camel :: CloudEvents
   - Camel :: Cluster
   - Camel :: CoAP
   - Camel :: Cometd
   - Camel :: Common Telemetry
   - Camel :: Common Tracing (deprecated)
   - Camel :: Component DSL
   - Camel :: Components
   - Camel :: Console
   - Camel :: Consul
   - Camel :: Controlbus
   - Camel :: Core
   - Camel :: Core Engine
   - Camel :: Core Languages
   - Camel :: Core Model
   - Camel :: Core Processor
   - Camel :: Core Reifier
   - Camel :: Core XML
   - Camel :: CouchDB
   - Camel :: Couchbase
   - Camel :: Coverage
   - Camel :: Cron
   - Camel :: Crypto
   - Camel :: Crypto PGP
   - Camel :: CyberArk Vault
   - Camel :: DFDL
   - Camel :: DHIS2
   - Camel :: DHIS2 :: Parent
   - Camel :: DHIS2 API
   - Camel :: DNS
   - Camel :: DSL :: CLI Connector
   - Camel :: DSL :: CLI Debug
   - Camel :: DSL :: Modeline
   - Camel :: DSL :: Support
   - Camel :: Dapr
   - Camel :: DataSet
   - Camel :: DataSonnet
   - Camel :: DataWeave
   - Camel :: Dataformat
   - Camel :: Debezium :: Common
   - Camel :: Debezium :: Common :: Parent
   - Camel :: Debezium :: DB2
   - Camel :: Debezium :: Maven Plugin
   - Camel :: Debezium :: MongoDB
   - Camel :: Debezium :: MySQL
   - Camel :: Debezium :: Oracle
   - Camel :: Debezium :: Parent
   - Camel :: Debezium :: PostgreSQL
   - Camel :: Debezium :: SQL Server
   - Camel :: Debugging
   - Camel :: Diagram
   - Camel :: Direct
   - Camel :: Disruptor
   - Camel :: Docker
   - Camel :: Docs
   - Camel :: Drill
   - Camel :: Dropbox
   - Camel :: DuckDB
   - Camel :: Dynamic Router
   - Camel :: Ehcache
   - Camel :: ElasticSearch Rest Client
   - Camel :: Elasticsearch Java API Client
   - Camel :: Endpoint DSL
   - Camel :: Endpoint DSL :: Support
   - Camel :: Event
   - Camel :: Exec
   - Camel :: FHIR
   - Camel :: FHIR :: API
   - Camel :: FHIR :: Parent
   - Camel :: FOP
   - Camel :: FTP
   - Camel :: FTP Common
   - Camel :: Fastjson
   - Camel :: File
   - Camel :: File Watch
   - Camel :: FlatPack
   - Camel :: Flink
   - Camel :: Flowable
   - Camel :: Fory
   - Camel :: Freemarker
   - Camel :: Geocoder
   - Camel :: Git
   - Camel :: GitHub2
   - Camel :: Google :: BigQuery
   - Camel :: Google :: Calendar
   - Camel :: Google :: Common
   - Camel :: Google :: Drive
   - Camel :: Google :: Firestore
   - Camel :: Google :: Functions
   - Camel :: Google :: Mail
   - Camel :: Google :: Parent
   - Camel :: Google :: PubSub
   - Camel :: Google :: Secret Manager
   - Camel :: Google :: Sheets
   - Camel :: Google :: Speech To Text
   - Camel :: Google :: Storage
   - Camel :: Google :: Text To Speech
   - Camel :: Google :: Vertex AI
   - Camel :: Google :: Vision
   - Camel :: GraphQL
   - Camel :: Grok
   - Camel :: Groovy
   - Camel :: Gson
   - Camel :: HL7
   - Camel :: HTTP
   - Camel :: HTTP :: Base
   - Camel :: HTTP :: Common
   - Camel :: HashiCorp :: Key Vault
   - Camel :: HazelCast
   - Camel :: Health
   - Camel :: Huawei Cloud :: Common
   - Camel :: Huawei Cloud :: DMS
   - Camel :: Huawei Cloud :: FaceRecognition
   - Camel :: Huawei Cloud :: FunctionGraph
   - Camel :: Huawei Cloud :: IAM
   - Camel :: Huawei Cloud :: ImageRecognition
   - Camel :: Huawei Cloud :: OBS
   - Camel :: Huawei Cloud :: Parent
   - Camel :: Huawei Cloud :: SimpleNotification
   - Camel :: IBM :: Cloud Object Storage
   - Camel :: IBM :: Parent
   - Camel :: IBM :: Secrets Manager
   - Camel :: IBM :: Watson Discovery
   - Camel :: IBM :: Watson Language
   - Camel :: IBM :: Watson Speech to Text
   - Camel :: IBM :: Watson Text to Speech
   - Camel :: IBM :: watsonx.ai
   - Camel :: IBM :: watsonx.data
   - Camel :: ISO-8583
   - Camel :: Iggy
   - Camel :: Ignite
   - Camel :: Infinispan :: Common
   - Camel :: Infinispan :: Embedded
   - Camel :: Infinispan :: Parent
   - Camel :: Infinispan :: Remote
   - Camel :: InfluxDB
   - Camel :: InfluxDB2
   - Camel :: Integration Tests
   - Camel :: Integration Tests :: Stream Caching Tests
   - Camel :: JAXB
   - Camel :: JBang :: Console
   - Camel :: JBang :: Core
   - Camel :: JBang :: Integration tests
   - Camel :: JBang :: MCP
   - Camel :: JBang :: Main
   - Camel :: JBang :: Plugin :: Edit
   - Camel :: JBang :: Plugin :: Generate
   - Camel :: JBang :: Plugin :: Kubernetes
   - Camel :: JBang :: Plugin :: MCP
   - Camel :: JBang :: Plugin :: Route Parser
   - Camel :: JBang :: Plugin :: TUI
   - Camel :: JBang :: Plugin :: Testing
   - Camel :: JBang :: Plugin :: Validate
   - Camel :: JCR
   - Camel :: JCache
   - Camel :: JDBC
   - Camel :: JGroups
   - Camel :: JGroups Raft
   - Camel :: JIRA
   - Camel :: JMS
   - Camel :: JMX
   - Camel :: JOOQ
   - Camel :: JPA
   - Camel :: JQ
   - Camel :: JSON validator
   - Camel :: JSON-B
   - Camel :: JSONATA
   - Camel :: JSon Path
   - Camel :: JSonApi
   - Camel :: JSoup
   - Camel :: JTA
   - Camel :: Jackson
   - Camel :: Jackson 3
   - Camel :: Jackson 3 Avro
   - Camel :: Jackson 3 Protobuf
   - Camel :: Jackson 3 XML
   - Camel :: Jackson Avro
   - Camel :: Jackson Protobuf
   - Camel :: Jackson XML
   - Camel :: Jactl
   - Camel :: Jandex
   - Camel :: Jasypt
   - Camel :: Java DSL IO
   - Camel :: Java DSL with jOOR
   - Camel :: Java Flight Recorder
   - Camel :: Java Template Engine
   - Camel :: Java Toolbox for IBM i
   - Camel :: JavaScript
   - Camel :: Jetty
   - Camel :: Jetty :: Common
   - Camel :: Jolt
   - Camel :: Jsch
   - Camel :: Jslt
   - Camel :: Kafka
   - Camel :: Kamelet
   - Camel :: Kamelet Main
   - Camel :: Kamelet Main :: Support
   - Camel :: Keycloak
   - Camel :: Knative :: Parent
   - Camel :: Knative API
   - Camel :: Knative Component
   - Camel :: Knative HTTP
   - Camel :: Kubernetes
   - Camel :: Kudu
   - Camel :: LDAP
   - Camel :: LDIF
   - Camel :: LZF
   - Camel :: Language
   - Camel :: Launcher
   - Camel :: Launcher :: Container
   - Camel :: Log
   - Camel :: Long-Running-Action
   - Camel :: Lucene
   - Camel :: Lumberjack
   - Camel :: MDC
   - Camel :: MINA
   - Camel :: MINA SFTP
   - Camel :: MLLP
   - Camel :: MVEL
   - Camel :: Mail
   - Camel :: Mail :: Microsoft OAuth
   - Camel :: Main
   - Camel :: Management
   - Camel :: MapStruct
   - Camel :: Master
   - Camel :: Maven Plugins :: Camel API Component Plugin
   - Camel :: Maven Plugins :: Camel Maven Plugin
   - Camel :: Maven Plugins :: OpenApi REST DSL Generator
   - Camel :: Metrics
   - Camel :: MicroProfile :: Config
   - Camel :: MicroProfile :: Fault Tolerance
   - Camel :: MicroProfile :: Health
   - Camel :: MicroProfile :: Parent
   - Camel :: Micrometer
   - Camel :: Micrometer :: Observability 2
   - Camel :: Micrometer :: Observation (deprecated)
   - Camel :: Micrometer :: Prometheus
   - Camel :: Milo
   - Camel :: MinIO
   - Camel :: Mock
   - Camel :: MongoDB
   - Camel :: MongoDB GridFS
   - Camel :: Mustache
   - Camel :: MyBatis
   - Camel :: Nats
   - Camel :: Netty
   - Camel :: Netty HTTP
   - Camel :: OAIPMH
   - Camel :: OAuth
   - Camel :: OCSF
   - Camel :: OGNL (deprecated)
   - Camel :: Observability Services
   - Camel :: Olingo2 (Deprecated) :: API
   - Camel :: Olingo2 (Deprecated) :: Component
   - Camel :: Olingo2 (Deprecated) :: Parent
   - Camel :: Olingo4 (Deprecated) :: API
   - Camel :: Olingo4 (Deprecated) :: Component
   - Camel :: Olingo4 (Deprecated) :: Parent
   - Camel :: Once
   - Camel :: OpenAPI :: Validator
   - Camel :: OpenApi Java
   - Camel :: OpenSearch Java API Client
   - Camel :: OpenStack
   - Camel :: OpenTelemetry (deprecated)
   - Camel :: Opentelemetry 2
   - Camel :: Opentelemetry Metrics
   - Camel :: OptaPlanner
   - Camel :: PDF
   - Camel :: PLC4X
   - Camel :: PQC
   - Camel :: Paho (deprecated)
   - Camel :: Paho MQTT 5
   - Camel :: Parquet Avro
   - Camel :: PgEvent
   - Camel :: PgReplicationSlot
   - Camel :: Platform HTTP
   - Camel :: Platform HTTP :: Jolokia
   - Camel :: Platform HTTP :: Main
   - Camel :: Platform HTTP :: Vert.x
   - Camel :: Printer
   - Camel :: Protobuf
   - Camel :: PubNub
   - Camel :: Pulsar
   - Camel :: Python
   - Camel :: Python 3
   - Camel :: Quartz
   - Camel :: QuickFIX/J
   - Camel :: QuickJS
   - Camel :: REST
   - Camel :: REST OpenApi
   - Camel :: REST Postman
   - Camel :: RSS
   - Camel :: Reactive Streams
   - Camel :: Reactor
   - Camel :: Redis
   - Camel :: Ref
   - Camel :: Resilience4j
   - Camel :: Resilience4j :: Micrometer
   - Camel :: ResourceResolver GitHub
   - Camel :: RobotFramework
   - Camel :: RocketMQ
   - Camel :: RxJava
   - Camel :: SAP NetWeaver
   - Camel :: SMB
   - Camel :: SMPP
   - Camel :: SNMP
   - Camel :: SOAP
   - Camel :: SPIFFE
   - Camel :: SQL
   - Camel :: SSH
   - Camel :: SWIFT
   - Camel :: Saga
   - Camel :: Salesforce
   - Camel :: Salesforce :: CodeGen
   - Camel :: Salesforce :: Maven Plugin
   - Camel :: Salesforce :: Parent
   - Camel :: Saxon
   - Camel :: Scheduler
   - Camel :: Schematron
   - Camel :: Seda
   - Camel :: ServiceNow :: Component
   - Camel :: ServiceNow :: Maven Plugin
   - Camel :: ServiceNow :: Parent
   - Camel :: Servlet
   - Camel :: Shell
   - Camel :: Shiro
   - Camel :: Simple JMS
   - Camel :: Simple JMS2
   - Camel :: Slack
   - Camel :: Smooks :: Parent
   - Camel :: SnakeYAML
   - Camel :: Solr
   - Camel :: Splunk HEC
   - Camel :: Spring
   - Camel :: Spring :: Parent
   - Camel :: Spring AI :: Chat
   - Camel :: Spring AI :: Embeddings
   - Camel :: Spring AI :: Image
   - Camel :: Spring AI :: Parent
   - Camel :: Spring AI :: Vector Store
   - Camel :: Spring Batch
   - Camel :: Spring Cloud Config
   - Camel :: Spring JDBC
   - Camel :: Spring LDAP
   - Camel :: Spring Main
   - Camel :: Spring RabbitMQ
   - Camel :: Spring Redis
   - Camel :: Spring Security
   - Camel :: Spring Web Services
   - Camel :: Spring XML
   - Camel :: StAX
   - Camel :: State Store
   - Camel :: State Store :: Parent
   - Camel :: Stitch
   - Camel :: Stream
   - Camel :: StringTemplate
   - Camel :: Stripe
   - Camel :: Stub
   - Camel :: Support
   - Camel :: Syslog
   - Camel :: TOON
   - Camel :: Tahu
   - Camel :: Tar File
   - Camel :: Telegram
   - Camel :: Telemetry :: Dev
   - Camel :: Test :: JUnit5
   - Camel :: Test :: JUnit6
   - Camel :: Test :: Main :: JUnit5
   - Camel :: Test :: Main :: JUnit6
   - Camel :: Test :: Parent
   - Camel :: Test :: Spring :: JUnit5
   - Camel :: Test Infra :: All test services
   - Camel :: Test Infra :: Artemis
   - Camel :: Test Infra :: Cli (Camel CLI)
   - Camel :: Test Infra :: Core
   - Camel :: Test Infra :: Server Message Block
   - Camel :: Thrift
   - Camel :: Thymeleaf
   - Camel :: Tika
   - Camel :: Timer
   - Camel :: Tooling :: Maven
   - Camel :: Tooling :: OpenApi REST DSL Generator
   - Camel :: Twilio
   - Camel :: Twitter
   - Camel :: UBL
   - Camel :: Undertow
   - Camel :: Undertow Spring Security
   - Camel :: UniVocity Parsers
   - Camel :: Validator
   - Camel :: Velocity
   - Camel :: Vert.x :: Common
   - Camel :: Vert.x :: HTTP
   - Camel :: Vert.x :: Parent
   - Camel :: Vert.x :: WebSocket
   - Camel :: Vertx
   - Camel :: WAL
   - Camel :: Wasm
   - Camel :: Weather
   - Camel :: Web3j
   - Camel :: Webhook
   - Camel :: Whatsapp
   - Camel :: Wordpress
   - Camel :: Workday
   - Camel :: XChange
   - Camel :: XJ
   - Camel :: XML DSL Jaxb :: Test :: Definition
   - Camel :: XML DSL Jaxb :: Test :: Spring
   - Camel :: XML DSL with camel-xml-io
   - Camel :: XML DSL with camel-xml-jaxb
   - Camel :: XML IO
   - Camel :: XML JAXB
   - Camel :: XML JAXP
   - Camel :: XML Security
   - Camel :: XMPP
   - Camel :: XPath
   - Camel :: XSLT
   - Camel :: XSLT Saxon
   - Camel :: YAML DSL
   - Camel :: YAML DSL :: Common
   - Camel :: YAML DSL :: Deserializers
   - Camel :: YAML DSL :: Maven Plugins
   - Camel :: YAML DSL :: Validator
   - Camel :: YAML DSL :: Validator Maven Plugin
   - Camel :: YAML IO
   - Camel :: Zendesk
   - Camel :: Zip Deflater
   - Camel :: Zip File
   - Camel :: Zookeeper
   - Camel :: Zookeeper Master
   - Camel :: gRPC
   - Camel :: iCal
   - Camel :: jOOR
   
   </details>
   
   ---
   :gear: [View full build and test results](https://github.com/apache/camel/actions/runs/33588686927)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24583: camel-master - do not start the delegated consumer after leadership is lost (CAMEL-24584) [camel]

Posted by "atiaomar1978-hub (via GitHub)" <gi...@apache.org>.
atiaomar1978-hub commented on code in PR #26028:
URL: https://github.com/apache/camel/pull/26028#discussion_r3910874306


##########
components/camel-master/src/test/java/org/apache/camel/component/master/MasterConsumerLeadershipTest.java:
##########
@@ -0,0 +1,352 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.master;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.UUID;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+
+import org.apache.camel.Consumer;
+import org.apache.camel.Endpoint;
+import org.apache.camel.Processor;
+import org.apache.camel.Producer;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.cluster.CamelClusterMember;
+import org.apache.camel.impl.DefaultCamelContext;
+import org.apache.camel.support.DefaultComponent;
+import org.apache.camel.support.DefaultConsumer;
+import org.apache.camel.support.DefaultEndpoint;
+import org.apache.camel.support.cluster.AbstractCamelClusterService;
+import org.apache.camel.support.cluster.AbstractCamelClusterView;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.Timeout;
+
+import static org.awaitility.Awaitility.await;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+/**
+ * Verifies that the delegated consumer only ever runs while this node holds the leadership, also when the leadership
+ * changes while the start of the delegated consumer is still pending.
+ */
+public class MasterConsumerLeadershipTest {
+
+    private DefaultCamelContext context;
+    private TestClusterService clusterService;
+    private ProbeComponent probe;
+
+    @BeforeEach
+    void setUp() throws Exception {
+        probe = new ProbeComponent();
+        clusterService = new TestClusterService();
+
+        context = new DefaultCamelContext();
+        context.disableJMX();
+        context.addService(clusterService);
+        context.addComponent("probe", probe);
+
+        MasterComponent master = context.getComponent("master", MasterComponent.class);
+        // keep the retries short so an exhausted start does not dominate the test time
+        master.setBackOffDelay(200);
+        master.setBackOffMaxAttempts(2);
+
+        context.addRoutes(new RouteBuilder() {
+            @Override
+            public void configure() {
+                from("master:ns:probe:test").routeId("master-route").to("mock:result");
+            }
+        });
+
+        context.start();
+    }
+
+    @AfterEach
+    void tearDown() {
+        if (context != null) {
+            context.stop();
+        }
+    }
+
+    @Test
+    @Timeout(60)
+    void testLeadershipLostWhilePendingStartDoesNotStartConsumer() {
+        TestClusterView view = clusterService.getTestView();
+

Review Comment:
   **Strong test coverage.** The fake cluster view + probe consumer exercises pending start, in-progress start, post-success flap, stop-during-pending, and exhausted-retry recovery without real K8s/Pub/Sub infra. This gives good confidence the production race is fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24583: camel-master - do not start the delegated consumer after leadership is lost (CAMEL-24584) [camel]

Posted by "henrik242 (via GitHub)" <gi...@apache.org>.
henrik242 commented on code in PR #26028:
URL: https://github.com/apache/camel/pull/26028#discussion_r3916251553


##########
components/camel-master/src/main/java/org/apache/camel/component/master/MasterConsumer.java:
##########
@@ -151,60 +162,105 @@ private BackgroundTask createTask() {
                 .build();
     }
 
-    private void onLeadershipTaken() throws Exception {
+    private void onLeadershipTaken() {
         lock.lock();
         try {
             if (!isRunAllowed()) {
                 return;
             }
 
-            if (delegatedConsumer != null) {
+            leadershipTaken = true;
+
+            if (delegatedConsumer != null || isStartPending()) {
                 return;
             }
 
-            final BackgroundTask leaderTask = createTask();
-            leaderTask.schedule(getEndpoint().getCamelContext(), () -> {
-                if (!isRunAllowed()) {
-                    return false;
-                }
-                LOG.info("Leadership taken. Attempt #{} to start consumer: {}", leaderTask.iteration(), delegatedEndpoint);
-
-                Exception cause = null;
-                try {
-                    if (delegatedConsumer == null) {
-                        delegatedConsumer = delegatedEndpoint.createConsumer(processor);
-                        if (delegatedConsumer instanceof StartupListener) {
-                            getEndpoint().getCamelContext().addStartupListener((StartupListener) delegatedConsumer);
-                        }
-                        if (delegatedConsumer instanceof ResumeAware resumeAwareConsumer && resumeStrategy != null) {
-                            LOG.debug("Setting up the resume adapter for the resume strategy in consumer");
-                            ResumeAdapter resumeAdapter
-                                    = AdapterHelper.eval(clusterService.getCamelContext(), resumeAwareConsumer,
-                                            resumeStrategy);
-                            resumeStrategy.setAdapter(resumeAdapter);
-
-                            LOG.debug("Setting up the resume strategy for consumer");
-                            resumeAwareConsumer.setResumeStrategy(resumeStrategy);
-                        }
-                    }
-                    ServiceHelper.startService(delegatedEndpoint, delegatedConsumer);
+            // a task from a previous leadership term may still be scheduled, drop it
+            cancelLeaderTask(false);
+
+            final BackgroundTask task = createTask();
+            // the consumer is created once and re-used by the start attempts of this task
+            final AtomicReference<Consumer> attempt = new AtomicReference<>();
+            leaderTaskFuture = task.schedule(getEndpoint().getCamelContext(), () -> startDelegatedConsumer(task, attempt));
+        } finally {
+            lock.unlock();
+        }
+    }
+
+    private boolean startDelegatedConsumer(BackgroundTask task, AtomicReference<Consumer> attempt) {
+        try {
+            // interruptibly, so cancelling the task while this consumer is being stopped does not
+            // keep the leader pool thread waiting for a lock the stopping thread holds
+            lock.lockInterruptibly();
+        } catch (InterruptedException e) {
+            Thread.currentThread().interrupt();
+            return true; // no more attempts
+        }
+        try {
+            if (!isRunAllowed()) {
+                return false;
+            }
+
+            if (!leadershipTaken) {
+                // leadership was lost while this start was pending. Starting now would run the consumer on a
+                // node that is not the leader, and no further leadership event is coming to stop it again
+                LOG.debug("Leadership lost while the start was pending. Not starting consumer: {}", delegatedEndpoint);
+                return true; // no more attempts
+            }
 
-                } catch (Exception e) {
-                    cause = e;
+            if (delegatedConsumer != null) {
+                return true; // no more attempts
+            }
+
+            LOG.info("Leadership taken. Attempt #{} to start consumer: {}", task.iteration(), delegatedEndpoint);
+
+            Exception cause = null;
+            try {
+                Consumer consumer = attempt.get();
+                if (consumer == null) {
+                    consumer = delegatedEndpoint.createConsumer(processor);
+                    attempt.set(consumer);
+                    if (consumer instanceof StartupListener startupListener) {
+                        getEndpoint().getCamelContext().addStartupListener(startupListener);
+                    }
+                    if (consumer instanceof ResumeAware resumeAwareConsumer && resumeStrategy != null) {
+                        LOG.debug("Setting up the resume adapter for the resume strategy in consumer");
+                        ResumeAdapter resumeAdapter
+                                = AdapterHelper.eval(clusterService.getCamelContext(), resumeAwareConsumer,
+                                        resumeStrategy);
+                        resumeStrategy.setAdapter(resumeAdapter);
+
+                        LOG.debug("Setting up the resume strategy for consumer");
+                        resumeAwareConsumer.setResumeStrategy(resumeStrategy);
+                    }
                 }
+                ServiceHelper.startService(delegatedEndpoint, consumer);
+                // publish the consumer only once it is started, so a failed attempt cannot leave an unstarted
+                // consumer behind that makes every later leadership term believe there is nothing left to do
+                delegatedConsumer = consumer;
+
+            } catch (Exception e) {
+                cause = e;
+            }
 
-                if (cause != null) {
-                    String message = "Leadership taken. Attempt #" + leaderTask.iteration()
-                                     + " failed to start consumer due to: " + cause.getMessage();
-                    getExceptionHandler().handleException(message, cause);
-                    // make the task runner aware of the exception (will retry)
-                    throw new TaskRunFailureException(message, cause);
+            if (cause != null) {
+                String message = "Leadership taken. Attempt #" + task.iteration()
+                                 + " failed to start consumer due to: " + cause.getMessage();
+                getExceptionHandler().handleException(message, cause);
+                if (task.iteration() >= masterEndpoint.getComponent().getBackOffMaxAttempts()) {

Review Comment:
   confirmed, withUnlimitedDuration() added, log guarded by maxAttempts > 0, new test that fails without the one-line fix. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24583: camel-master - do not start the delegated consumer after leadership is lost (CAMEL-24584) [camel]

Posted by "henrik242 (via GitHub)" <gi...@apache.org>.
henrik242 commented on code in PR #26028:
URL: https://github.com/apache/camel/pull/26028#discussion_r3916293723


##########
components/camel-master/src/main/java/org/apache/camel/component/master/MasterConsumer.java:
##########
@@ -151,60 +162,105 @@ private BackgroundTask createTask() {
                 .build();
     }
 
-    private void onLeadershipTaken() throws Exception {
+    private void onLeadershipTaken() {
         lock.lock();
         try {
             if (!isRunAllowed()) {
                 return;
             }
 
-            if (delegatedConsumer != null) {
+            leadershipTaken = true;
+
+            if (delegatedConsumer != null || isStartPending()) {
                 return;
             }
 
-            final BackgroundTask leaderTask = createTask();
-            leaderTask.schedule(getEndpoint().getCamelContext(), () -> {
-                if (!isRunAllowed()) {
-                    return false;
-                }
-                LOG.info("Leadership taken. Attempt #{} to start consumer: {}", leaderTask.iteration(), delegatedEndpoint);
-
-                Exception cause = null;
-                try {
-                    if (delegatedConsumer == null) {
-                        delegatedConsumer = delegatedEndpoint.createConsumer(processor);
-                        if (delegatedConsumer instanceof StartupListener) {
-                            getEndpoint().getCamelContext().addStartupListener((StartupListener) delegatedConsumer);
-                        }
-                        if (delegatedConsumer instanceof ResumeAware resumeAwareConsumer && resumeStrategy != null) {
-                            LOG.debug("Setting up the resume adapter for the resume strategy in consumer");
-                            ResumeAdapter resumeAdapter
-                                    = AdapterHelper.eval(clusterService.getCamelContext(), resumeAwareConsumer,
-                                            resumeStrategy);
-                            resumeStrategy.setAdapter(resumeAdapter);
-
-                            LOG.debug("Setting up the resume strategy for consumer");
-                            resumeAwareConsumer.setResumeStrategy(resumeStrategy);
-                        }
-                    }
-                    ServiceHelper.startService(delegatedEndpoint, delegatedConsumer);
+            // a task from a previous leadership term may still be scheduled, drop it
+            cancelLeaderTask(false);
+

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24583: camel-master - do not start the delegated consumer after leadership is lost (CAMEL-24584) [camel]

Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #26028:
URL: https://github.com/apache/camel/pull/26028


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24583: camel-master - do not start the delegated consumer after leadership is lost (CAMEL-24584) [camel]

Posted by "henrik242 (via GitHub)" <gi...@apache.org>.
henrik242 commented on code in PR #26028:
URL: https://github.com/apache/camel/pull/26028#discussion_r3916255575


##########
components/camel-master/src/main/java/org/apache/camel/component/master/MasterConsumer.java:
##########
@@ -151,60 +162,105 @@ private BackgroundTask createTask() {
                 .build();
     }
 
-    private void onLeadershipTaken() throws Exception {
+    private void onLeadershipTaken() {
         lock.lock();
         try {
             if (!isRunAllowed()) {
                 return;
             }
 
-            if (delegatedConsumer != null) {
+            leadershipTaken = true;
+
+            if (delegatedConsumer != null || isStartPending()) {
                 return;
             }
 
-            final BackgroundTask leaderTask = createTask();
-            leaderTask.schedule(getEndpoint().getCamelContext(), () -> {
-                if (!isRunAllowed()) {
-                    return false;
-                }
-                LOG.info("Leadership taken. Attempt #{} to start consumer: {}", leaderTask.iteration(), delegatedEndpoint);
-
-                Exception cause = null;
-                try {
-                    if (delegatedConsumer == null) {
-                        delegatedConsumer = delegatedEndpoint.createConsumer(processor);
-                        if (delegatedConsumer instanceof StartupListener) {
-                            getEndpoint().getCamelContext().addStartupListener((StartupListener) delegatedConsumer);
-                        }
-                        if (delegatedConsumer instanceof ResumeAware resumeAwareConsumer && resumeStrategy != null) {
-                            LOG.debug("Setting up the resume adapter for the resume strategy in consumer");
-                            ResumeAdapter resumeAdapter
-                                    = AdapterHelper.eval(clusterService.getCamelContext(), resumeAwareConsumer,
-                                            resumeStrategy);
-                            resumeStrategy.setAdapter(resumeAdapter);
-
-                            LOG.debug("Setting up the resume strategy for consumer");
-                            resumeAwareConsumer.setResumeStrategy(resumeStrategy);
-                        }
-                    }
-                    ServiceHelper.startService(delegatedEndpoint, delegatedConsumer);
+            // a task from a previous leadership term may still be scheduled, drop it
+            cancelLeaderTask(false);
+
+            final BackgroundTask task = createTask();
+            // the consumer is created once and re-used by the start attempts of this task
+            final AtomicReference<Consumer> attempt = new AtomicReference<>();
+            leaderTaskFuture = task.schedule(getEndpoint().getCamelContext(), () -> startDelegatedConsumer(task, attempt));
+        } finally {
+            lock.unlock();
+        }
+    }
+
+    private boolean startDelegatedConsumer(BackgroundTask task, AtomicReference<Consumer> attempt) {
+        try {
+            // interruptibly, so cancelling the task while this consumer is being stopped does not
+            // keep the leader pool thread waiting for a lock the stopping thread holds
+            lock.lockInterruptibly();
+        } catch (InterruptedException e) {
+            Thread.currentThread().interrupt();
+            return true; // no more attempts
+        }
+        try {
+            if (!isRunAllowed()) {
+                return false;
+            }
+
+            if (!leadershipTaken) {
+                // leadership was lost while this start was pending. Starting now would run the consumer on a
+                // node that is not the leader, and no further leadership event is coming to stop it again
+                LOG.debug("Leadership lost while the start was pending. Not starting consumer: {}", delegatedEndpoint);
+                return true; // no more attempts
+            }
 
-                } catch (Exception e) {
-                    cause = e;
+            if (delegatedConsumer != null) {
+                return true; // no more attempts
+            }
+
+            LOG.info("Leadership taken. Attempt #{} to start consumer: {}", task.iteration(), delegatedEndpoint);
+
+            Exception cause = null;
+            try {
+                Consumer consumer = attempt.get();
+                if (consumer == null) {
+                    consumer = delegatedEndpoint.createConsumer(processor);
+                    attempt.set(consumer);
+                    if (consumer instanceof StartupListener startupListener) {
+                        getEndpoint().getCamelContext().addStartupListener(startupListener);
+                    }
+                    if (consumer instanceof ResumeAware resumeAwareConsumer && resumeStrategy != null) {
+                        LOG.debug("Setting up the resume adapter for the resume strategy in consumer");
+                        ResumeAdapter resumeAdapter
+                                = AdapterHelper.eval(clusterService.getCamelContext(), resumeAwareConsumer,
+                                        resumeStrategy);
+                        resumeStrategy.setAdapter(resumeAdapter);
+
+                        LOG.debug("Setting up the resume strategy for consumer");
+                        resumeAwareConsumer.setResumeStrategy(resumeStrategy);
+                    }
                 }
+                ServiceHelper.startService(delegatedEndpoint, consumer);

Review Comment:
   implemented exactly as suggested; if leadership went away during the start, the consumer is stopped rather than published.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24583: camel-master - do not start the delegated consumer after leadership is lost (CAMEL-24584) [camel]

Posted by "henrik242 (via GitHub)" <gi...@apache.org>.
henrik242 commented on code in PR #26028:
URL: https://github.com/apache/camel/pull/26028#discussion_r3916259695


##########
components/camel-master/src/main/java/org/apache/camel/component/master/MasterConsumer.java:
##########
@@ -225,30 +290,50 @@ private void onLeadershipLost() {
         }
     }
 
+    private boolean isStartPending() {
+        return leaderTaskFuture != null && !leaderTaskFuture.isDone();
+    }
+
+    private void cancelLeaderTask(boolean mayInterruptIfRunning) {
+        if (leaderTaskFuture != null) {
+            leaderTaskFuture.cancel(mayInterruptIfRunning);
+            leaderTaskFuture = null;
+        }
+    }
+
     // **************************************
     // Listener
     // **************************************
 
     private final class LeadershipListener implements CamelClusterEventListener.Leadership {
         @Override
         public void leadershipChanged(CamelClusterView view, CamelClusterMember leader) {
-            if (!isRunAllowed()) {
-                return;
-            }
-
-            if (view.getLocalMember().isLeader()) {
-                try {
-                    onLeadershipTaken();
-                } catch (Exception e) {
-                    getExceptionHandler().handleException("Error starting consumer while taking leadership", e);
+            lock.lock();

Review Comment:
   unlocked isRunAllowed() fast path restored, start moved off the lock, doStop cancels before removing the listener; residual is pre-existing and documented, follow-up JIRA.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24583: camel-master - do not start the delegated consumer after leadership is lost (CAMEL-24584) [camel]

Posted by "henrik242 (via GitHub)" <gi...@apache.org>.
henrik242 commented on code in PR #26028:
URL: https://github.com/apache/camel/pull/26028#discussion_r3916274672


##########
components/camel-master/src/main/java/org/apache/camel/component/master/MasterConsumer.java:
##########
@@ -103,6 +109,11 @@ protected void doStart() throws Exception {
     protected void doStop() throws Exception {
         super.doStop();
 
+        // a start can still be pending, cancel it first so it cannot start the delegated consumer
+        // after this consumer has been stopped
+        leadershipTaken = false;
+        cancelLeaderTask(true);
+
         if (view != null) {
             view.removeEventListener(leadershipListener);

Review Comment:
   both points stale; attribution line removed,and the branch is now genuinely the two ordered commits. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24583: camel-master - do not start the delegated consumer after leadership is lost (CAMEL-24584) [camel]

Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on code in PR #26028:
URL: https://github.com/apache/camel/pull/26028#discussion_r3911134943


##########
components/camel-master/src/main/java/org/apache/camel/component/master/MasterConsumer.java:
##########
@@ -103,6 +109,11 @@ protected void doStart() throws Exception {
     protected void doStop() throws Exception {
         super.doStop();
 
+        // a start can still be pending, cancel it first so it cannot start the delegated consumer
+        // after this consumer has been stopped
+        leadershipTaken = false;
+        cancelLeaderTask(true);
+
         if (view != null) {
             view.removeEventListener(leadershipListener);

Review Comment:
   Confirmed via `AbstractCamelClusterView`/`BaseService`: this call needs the view's `StampedLock` write lock while this thread already holds `MasterConsumer`'s `BaseService.lock` (from `stop()`). Meanwhile `AbstractCamelClusterView.doWithListener` dispatches `leadershipChanged` while holding the view's read lock, and (with this PR) that dispatch unconditionally acquires `MasterConsumer.lock` even when there's no consumer yet. That's a lock-order inversion between the two locks.
   
   This ordering pre-dates this PR (the "taken" path already re-entered `MasterConsumer.lock` from inside dispatch), but this change measurably widens exposure since `onLeadershipLost` is now dispatched — and takes the lock — on every lost event rather than only when `delegatedConsumer != null`. The PR description already discloses this ("Lock ordering between the cluster view and the consumer"), which is appreciated — suggesting a follow-up JIRA to track it, since that context won't survive in the source once this merges.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24583: camel-master - do not start the delegated consumer after leadership is lost (CAMEL-24584) [camel]

Posted by "Croway (via GitHub)" <gi...@apache.org>.
Croway commented on code in PR #26028:
URL: https://github.com/apache/camel/pull/26028#discussion_r3912089383


##########
components/camel-master/src/main/java/org/apache/camel/component/master/MasterConsumer.java:
##########
@@ -225,30 +290,50 @@ private void onLeadershipLost() {
         }
     }
 
+    private boolean isStartPending() {
+        return leaderTaskFuture != null && !leaderTaskFuture.isDone();
+    }
+
+    private void cancelLeaderTask(boolean mayInterruptIfRunning) {
+        if (leaderTaskFuture != null) {
+            leaderTaskFuture.cancel(mayInterruptIfRunning);
+            leaderTaskFuture = null;
+        }
+    }
+
     // **************************************
     // Listener
     // **************************************
 
     private final class LeadershipListener implements CamelClusterEventListener.Leadership {
         @Override
         public void leadershipChanged(CamelClusterView view, CamelClusterMember leader) {
-            if (!isRunAllowed()) {
-                return;
-            }
-
-            if (view.getLocalMember().isLeader()) {
-                try {
-                    onLeadershipTaken();
-                } catch (Exception e) {
-                    getExceptionHandler().handleException("Error starting consumer while taking leadership", e);
+            lock.lock();

Review Comment:
   **Lock-order inversion, can deadlock on stop.**
   
   `leadershipChanged` now takes `BaseService.lock` unconditionally. The view dispatches this listener while holding its `StampedLock` **read** lock (`AbstractCamelClusterView.doWithListener` -> `LockHelper.doWithReadLock`). On the other side, `MasterConsumer.stop()` holds `BaseService.lock` while `doStop()` calls `view.removeEventListener(...)`, which needs the `StampedLock` **write** lock.
   
   - Thread S: `stop()` holds `lock` -> `doStop()` -> `removeEventListener` -> waits for the read lock to be released.
   - Thread E (FileLock `tryLock` tick, Curator `LeaderSelector`, Kubernetes notifier): holds the read lock -> `leadershipChanged` -> `lock.lock()` -> waits for S.
   
   Both threads block forever, shutdown hangs, and every other master route on that view stops receiving events. Pre-PR the unlocked `isRunAllowed()` pre-check returned before contending for the lock once `STOPPING` was set (and lost-events never locked without a consumer), so the window was a microsecond race. Now every event contends, and combined with the start running under the lock (see below) the event thread can already be parked on `lock` for the whole delegated start when `stop()` arrives.
   
   Suggestion: keep an unlocked `isRunAllowed()` fast-path before taking the lock, and/or use `tryLock` with a bail-out here, and make sure nothing that runs under `BaseService.lock` needs the view's write lock (e.g. remove the listener before/outside the locked section of `doStop`).



##########
components/camel-master/src/main/java/org/apache/camel/component/master/MasterConsumer.java:
##########
@@ -151,60 +162,105 @@ private BackgroundTask createTask() {
                 .build();
     }
 
-    private void onLeadershipTaken() throws Exception {
+    private void onLeadershipTaken() {
         lock.lock();
         try {
             if (!isRunAllowed()) {
                 return;
             }
 
-            if (delegatedConsumer != null) {
+            leadershipTaken = true;
+
+            if (delegatedConsumer != null || isStartPending()) {
                 return;
             }
 
-            final BackgroundTask leaderTask = createTask();
-            leaderTask.schedule(getEndpoint().getCamelContext(), () -> {
-                if (!isRunAllowed()) {
-                    return false;
-                }
-                LOG.info("Leadership taken. Attempt #{} to start consumer: {}", leaderTask.iteration(), delegatedEndpoint);
-
-                Exception cause = null;
-                try {
-                    if (delegatedConsumer == null) {
-                        delegatedConsumer = delegatedEndpoint.createConsumer(processor);
-                        if (delegatedConsumer instanceof StartupListener) {
-                            getEndpoint().getCamelContext().addStartupListener((StartupListener) delegatedConsumer);
-                        }
-                        if (delegatedConsumer instanceof ResumeAware resumeAwareConsumer && resumeStrategy != null) {
-                            LOG.debug("Setting up the resume adapter for the resume strategy in consumer");
-                            ResumeAdapter resumeAdapter
-                                    = AdapterHelper.eval(clusterService.getCamelContext(), resumeAwareConsumer,
-                                            resumeStrategy);
-                            resumeStrategy.setAdapter(resumeAdapter);
-
-                            LOG.debug("Setting up the resume strategy for consumer");
-                            resumeAwareConsumer.setResumeStrategy(resumeStrategy);
-                        }
-                    }
-                    ServiceHelper.startService(delegatedEndpoint, delegatedConsumer);
+            // a task from a previous leadership term may still be scheduled, drop it
+            cancelLeaderTask(false);
+
+            final BackgroundTask task = createTask();
+            // the consumer is created once and re-used by the start attempts of this task
+            final AtomicReference<Consumer> attempt = new AtomicReference<>();
+            leaderTaskFuture = task.schedule(getEndpoint().getCamelContext(), () -> startDelegatedConsumer(task, attempt));
+        } finally {
+            lock.unlock();
+        }
+    }
+
+    private boolean startDelegatedConsumer(BackgroundTask task, AtomicReference<Consumer> attempt) {
+        try {
+            // interruptibly, so cancelling the task while this consumer is being stopped does not
+            // keep the leader pool thread waiting for a lock the stopping thread holds
+            lock.lockInterruptibly();
+        } catch (InterruptedException e) {
+            Thread.currentThread().interrupt();
+            return true; // no more attempts
+        }
+        try {
+            if (!isRunAllowed()) {
+                return false;
+            }
+
+            if (!leadershipTaken) {
+                // leadership was lost while this start was pending. Starting now would run the consumer on a
+                // node that is not the leader, and no further leadership event is coming to stop it again
+                LOG.debug("Leadership lost while the start was pending. Not starting consumer: {}", delegatedEndpoint);
+                return true; // no more attempts
+            }
 
-                } catch (Exception e) {
-                    cause = e;
+            if (delegatedConsumer != null) {
+                return true; // no more attempts
+            }
+
+            LOG.info("Leadership taken. Attempt #{} to start consumer: {}", task.iteration(), delegatedEndpoint);
+
+            Exception cause = null;
+            try {
+                Consumer consumer = attempt.get();
+                if (consumer == null) {
+                    consumer = delegatedEndpoint.createConsumer(processor);
+                    attempt.set(consumer);
+                    if (consumer instanceof StartupListener startupListener) {
+                        getEndpoint().getCamelContext().addStartupListener(startupListener);
+                    }
+                    if (consumer instanceof ResumeAware resumeAwareConsumer && resumeStrategy != null) {
+                        LOG.debug("Setting up the resume adapter for the resume strategy in consumer");
+                        ResumeAdapter resumeAdapter
+                                = AdapterHelper.eval(clusterService.getCamelContext(), resumeAwareConsumer,
+                                        resumeStrategy);
+                        resumeStrategy.setAdapter(resumeAdapter);
+
+                        LOG.debug("Setting up the resume strategy for consumer");
+                        resumeAwareConsumer.setResumeStrategy(resumeStrategy);
+                    }
                 }
+                ServiceHelper.startService(delegatedEndpoint, consumer);

Review Comment:
   **The delegated start now runs while holding `BaseService.lock`.**
   
   Before this PR the start lambda ran on the leader pool without the lock; only `schedule()` was under it. Now `createConsumer()` and `ServiceHelper.startService(delegatedEndpoint, consumer)` run with the lock held. A delegated JMS/Kafka consumer that spends 30 s+ in connect retries inside `doStart()` means:
   
   - `MasterConsumer.stop()` (context shutdown) blocks on `lock` until the start returns, so the `cancelLeaderTask(true)` in `doStop` can never interrupt an in-flight start (`lockInterruptibly` only helps a task that has not acquired the lock yet).
   - The view's dispatch thread parks on `lock` under the `StampedLock` read lock, so `addEventListener`/`removeEventListener` of every other `MasterConsumer` on the namespace block.
   - For `FileLockClusterView` the same `tryLock()` thread that fires the event also writes the leader heartbeat (`writeClusterLeaderInfo`), so other members can judge this leader stale and take over while this node is still starting.
   
   Suggestion: claim the start under the lock (check `leadershipTaken`, `delegatedConsumer == null`, set a "starting" marker), run create/start **off** the lock, then re-acquire to publish `delegatedConsumer`, and if leadership was lost in the meantime stop/shutdown the freshly started consumer instead of publishing it.



##########
components/camel-master/src/main/java/org/apache/camel/component/master/MasterConsumer.java:
##########
@@ -151,60 +162,105 @@ private BackgroundTask createTask() {
                 .build();
     }
 
-    private void onLeadershipTaken() throws Exception {
+    private void onLeadershipTaken() {
         lock.lock();
         try {
             if (!isRunAllowed()) {
                 return;
             }
 
-            if (delegatedConsumer != null) {
+            leadershipTaken = true;
+
+            if (delegatedConsumer != null || isStartPending()) {
                 return;
             }
 
-            final BackgroundTask leaderTask = createTask();
-            leaderTask.schedule(getEndpoint().getCamelContext(), () -> {
-                if (!isRunAllowed()) {
-                    return false;
-                }
-                LOG.info("Leadership taken. Attempt #{} to start consumer: {}", leaderTask.iteration(), delegatedEndpoint);
-
-                Exception cause = null;
-                try {
-                    if (delegatedConsumer == null) {
-                        delegatedConsumer = delegatedEndpoint.createConsumer(processor);
-                        if (delegatedConsumer instanceof StartupListener) {
-                            getEndpoint().getCamelContext().addStartupListener((StartupListener) delegatedConsumer);
-                        }
-                        if (delegatedConsumer instanceof ResumeAware resumeAwareConsumer && resumeStrategy != null) {
-                            LOG.debug("Setting up the resume adapter for the resume strategy in consumer");
-                            ResumeAdapter resumeAdapter
-                                    = AdapterHelper.eval(clusterService.getCamelContext(), resumeAwareConsumer,
-                                            resumeStrategy);
-                            resumeStrategy.setAdapter(resumeAdapter);
-
-                            LOG.debug("Setting up the resume strategy for consumer");
-                            resumeAwareConsumer.setResumeStrategy(resumeStrategy);
-                        }
-                    }
-                    ServiceHelper.startService(delegatedEndpoint, delegatedConsumer);
+            // a task from a previous leadership term may still be scheduled, drop it
+            cancelLeaderTask(false);
+
+            final BackgroundTask task = createTask();
+            // the consumer is created once and re-used by the start attempts of this task
+            final AtomicReference<Consumer> attempt = new AtomicReference<>();
+            leaderTaskFuture = task.schedule(getEndpoint().getCamelContext(), () -> startDelegatedConsumer(task, attempt));
+        } finally {
+            lock.unlock();
+        }
+    }
+
+    private boolean startDelegatedConsumer(BackgroundTask task, AtomicReference<Consumer> attempt) {
+        try {
+            // interruptibly, so cancelling the task while this consumer is being stopped does not
+            // keep the leader pool thread waiting for a lock the stopping thread holds
+            lock.lockInterruptibly();
+        } catch (InterruptedException e) {
+            Thread.currentThread().interrupt();
+            return true; // no more attempts
+        }
+        try {
+            if (!isRunAllowed()) {
+                return false;
+            }
+
+            if (!leadershipTaken) {
+                // leadership was lost while this start was pending. Starting now would run the consumer on a
+                // node that is not the leader, and no further leadership event is coming to stop it again
+                LOG.debug("Leadership lost while the start was pending. Not starting consumer: {}", delegatedEndpoint);
+                return true; // no more attempts
+            }
 
-                } catch (Exception e) {
-                    cause = e;
+            if (delegatedConsumer != null) {
+                return true; // no more attempts
+            }
+
+            LOG.info("Leadership taken. Attempt #{} to start consumer: {}", task.iteration(), delegatedEndpoint);
+
+            Exception cause = null;
+            try {
+                Consumer consumer = attempt.get();
+                if (consumer == null) {
+                    consumer = delegatedEndpoint.createConsumer(processor);
+                    attempt.set(consumer);
+                    if (consumer instanceof StartupListener startupListener) {
+                        getEndpoint().getCamelContext().addStartupListener(startupListener);
+                    }
+                    if (consumer instanceof ResumeAware resumeAwareConsumer && resumeStrategy != null) {
+                        LOG.debug("Setting up the resume adapter for the resume strategy in consumer");
+                        ResumeAdapter resumeAdapter
+                                = AdapterHelper.eval(clusterService.getCamelContext(), resumeAwareConsumer,
+                                        resumeStrategy);
+                        resumeStrategy.setAdapter(resumeAdapter);
+
+                        LOG.debug("Setting up the resume strategy for consumer");
+                        resumeAwareConsumer.setResumeStrategy(resumeStrategy);
+                    }
                 }
+                ServiceHelper.startService(delegatedEndpoint, consumer);
+                // publish the consumer only once it is started, so a failed attempt cannot leave an unstarted
+                // consumer behind that makes every later leadership term believe there is nothing left to do
+                delegatedConsumer = consumer;
+
+            } catch (Exception e) {
+                cause = e;
+            }
 
-                if (cause != null) {
-                    String message = "Leadership taken. Attempt #" + leaderTask.iteration()
-                                     + " failed to start consumer due to: " + cause.getMessage();
-                    getExceptionHandler().handleException(message, cause);
-                    // make the task runner aware of the exception (will retry)
-                    throw new TaskRunFailureException(message, cause);
+            if (cause != null) {
+                String message = "Leadership taken. Attempt #" + task.iteration()
+                                 + " failed to start consumer due to: " + cause.getMessage();
+                getExceptionHandler().handleException(message, cause);
+                if (task.iteration() >= masterEndpoint.getComponent().getBackOffMaxAttempts()) {

Review Comment:
   **This log never fires with default settings, `backOffMaxAttempts` is effectively ignored.**
   
   `createTask()` uses `Budgets.iterationTimeBudget()` without `withMaxDuration`/`withUnlimitedDuration`, so the builder's `DEFAULT_MAX_DURATION = 5000` ms applies, measured from task construction. With the defaults (`backOffDelay=5000`, `backOffMaxAttempts=10`, initial delay 1 s): attempt #1 runs at t~1 s and fails; the tick at t~6 s finds the budget exhausted and the task ends. `task.iteration()` is 1, never >= 10, so this ERROR is never emitted and the node silently sits as an idle leader (only `BackgroundTask`'s generic WARN appears). The budget config itself is pre-existing, but the PR's give-up contract and this message rely on it, and `MasterConsumerLeadershipTest` masks it with `backOffDelay=200` / `maxAttempts=2`, which fits inside 5 s.
   
   Also, `withMaxIterations()` ignores values <= 0, so `backOffMaxAttempts=0` keeps unlimited iterations but `iteration() >= 0` is always true here: "Giving up after 1 attempts" is logged on every failed attempt while retries continue.
   
   Suggestion: add `.withUnlimitedDuration()` (or a duration derived from `backOffDelay * backOffMaxAttempts`) in `createTask()`, and gate this log on the task actually being exhausted (its status/budget) rather than re-deriving the arithmetic here. A test with the default component settings would catch this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] CAMEL-24583: camel-master - do not start the delegated consumer after leadership is lost (CAMEL-24584) [camel]

Posted by "atiaomar1978-hub (via GitHub)" <gi...@apache.org>.
atiaomar1978-hub commented on PR #26028:
URL: https://github.com/apache/camel/pull/26028#issuecomment-5504658411

   ## PR review summary
   
   _AI-generated review on behalf of atiaomar1978-hub_
   
   **Verdict: Approve in principle** — this is a well-scoped, production-motivated fix with strong tests. The two-commit split (camel-support first, then camel-master) is the right dependency order.
   
   ### What works well
   
   - **Root cause addressed directly:** Leadership-lost during the 1s pending start window is no longer dropped; the scheduled task is cancelled and `leadershipTaken` is re-checked before starting the delegated consumer.
   - **CAMEL-24584 is a necessary foundation:** `BackgroundTask` auto-unschedule makes `isStartPending()` meaningful and stops lifetime no-op rescheduling.
   - **Failed-start recovery:** Publishing `delegatedConsumer` only after a successful `startService()` fixes the “stuck leader that never retries” failure mode — covered by `testConsumerStartsAfterLeadershipIsTakenAgainWhenAnEarlierStartFailed`.
   - **Operational visibility:** ERROR log when backoff is exhausted while still holding leadership is valuable for on-call.
   - **Tests:** `MasterConsumerLeadershipTest` is an excellent reproducer harness; the PR author’s note about verifying failures on unpatched `main` adds confidence.
   
   ### Inline notes (non-blocking follow-ups)
   
   1. **Partial attempt cleanup** — if `createConsumer()` succeeded but leadership is lost before publish, `attempt` may hold an unstarted consumer that is not shut down (see inline on `MasterConsumer`).
   2. **Upgrade guide** — consider a `camel-master` entry alongside the `camel-support` note for operators who hit the K8s/Pub/Sub follower-consumer symptom.
   3. **Pre-existing lock ordering** — agree with the PR description that cluster-view ↔ consumer lock inversion remains a separate follow-up.
   
   ### Testing / CI
   
   CI had not reported checks on the branch at review time. Locally the described matrix (7 new master tests + 2 BackgroundTask tests) is appropriate for this change set.
   
   Thanks @henrik242 for the thorough PR description and the transparent “known pre-existing issues” section — that made review much easier.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org