[jira] [Updated] (CAMEL-25846) camel-tahu - after its route is restarted, an Edge Node producer never connects again and drops every message (logged, the exchange completes)

["shashank (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CAMEL-25846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

shashank updated CAMEL-25846:
-----------------------------
    Description: 
{{TahuEdgeProducer}} (main 20cdb309f7d0) keeps the {{TahuEdgeClient}} of each Edge Node in a static map ({{descriptorClients}}, {{computeIfAbsent}} in {{createClient}}). When the Edge Node producer stops, it shuts its client down ({{TahuEdgeNodeProducer.doStop}}: {{tahuEdgeClient.shutdown()}}), but the client stays in the map.

A Tahu {{EdgeClient}} cannot run again after {{shutdown()}}: {{shutdown()}} sets {{stayRunning}} to false, only the constructor sets it to true, and {{run()}} returns at once when it is false (Tahu 1.0.21, read with javap). When the route is started again, Camel creates a new producer, whose constructor takes the shut down client from the map, and {{doStart}} submits it again ({{TahuEdgeClient.startup()}}). The client never connects to the MQTT server again: the Edge Node stays offline (its NDEATH was the last message), and every message sent to the endpoint is dropped. {{EdgeClient.publishSparkplugMessage}} catches the failure and logs {{Failed to publish message on topic=...}} at ERROR, so the exchange completes without an exception. Device producers of the same Edge Node, also in routes that were not restarted, keep the old client too. This lasts until the JVM restarts (the map is static, so a new CamelContext in the same JVM takes the shut down client as well). A route restart is enough: stopping and starting the route (JMX, control bus, a route policy, a supervising route controller).

*Fix*: the Edge Node producer removes its client from the map when it stops (and shuts down the client's executor), and creates a new client when it starts. A new client gets the metric maps of the Edge Node and of its Devices again (a producer keeps its metric map in a static map by descriptor from its start to its stop, and a producer that starts after its Edge Node got a new client adds its map to that client), and Device producers take the current client of their Edge Node when they publish. A Device producer never creates a client itself in that case.

*Tests*: new {{TahuEdgeProducerRestartTest}} (2 tests) with {{FakeMqttBroker}}, a small in-process MQTT 3.1.1 broker that acknowledges connections, subscriptions and publishes and records the published topics. {{edgeNodePublishesAgainAfterTheRouteIsRestarted}}: NBIRTH and one NDATA, stop and start the route, then a second NBIRTH and NDATA are expected. On main it fails after 20 seconds (Awaitility): {{ConditionTimeoutException ... not published; connects: 1, published: [spBv1.0/RestartGroup/NBIRTH/RestartNode, spBv1.0/RestartGroup/NDATA/RestartNode, spBv1.0/RestartGroup/DDEATH/RestartNode/D1, spBv1.0/RestartGroup/NDEATH/RestartNode]}}. {{deviceOfAnotherRoutePublishesAgainAfterTheEdgeNodeRouteIsRestarted}}: only the Edge Node route is restarted, and the Device route's DDATA is expected after the new DBIRTH; fails on main the same way ({{connects: 1}}). Both pass with the fix. camel-tahu unit tests with the fix: 9 run, 0 failures (the IT tests need a container and were not run).

*How it was found*: variant analysis of CAMEL-25838 (pattern: a client or connection that is shut down or replaced, while state that belongs to it, here the client in a static map and the Device metric maps, is kept or lost), then the JUnit tests.

*Affected*: main, camel-4.23.x, camel-4.18.x and camel-4.14.x (the same static map and shutdown; read, not run; the {{stayRunning}} behaviour was read in Tahu 1.0.21 only).

_Filed with Claude Code on behalf of allthingssecurity._


  was:
{{TahuEdgeProducer}} (main 20cdb309f7d0) keeps the {{TahuEdgeClient}} of each Edge Node in a static map ({{descriptorClients}}, {{computeIfAbsent}} in {{createClient}}). When the Edge Node producer stops, it shuts its client down ({{TahuEdgeNodeProducer.doStop}}: {{tahuEdgeClient.shutdown()}}), but the client stays in the map.

A Tahu {{EdgeClient}} cannot run again after {{shutdown()}}: {{shutdown()}} sets {{stayRunning}} to false, only the constructor sets it to true, and {{run()}} returns at once when it is false (Tahu 1.0.21, read with javap). When the route is started again, Camel creates a new producer, whose constructor takes the shut down client from the map, and {{doStart}} submits it again ({{TahuEdgeClient.startup()}}). The client never connects to the MQTT server again: the Edge Node stays offline (its NDEATH was the last message), and every message sent to the endpoint is dropped. {{EdgeClient.publishSparkplugMessage}} catches the failure and logs {{Failed to publish message on topic=...}} at ERROR, so the exchange completes without an exception. Device producers of the same Edge Node, also in routes that were not restarted, keep the old client too. This lasts until the JVM restarts (the map is static, so a new CamelContext in the same JVM takes the shut down client as well). A route restart is enough: stopping and starting the route (JMX, control bus, a route policy, a supervising route controller).

*Fix*: the Edge Node producer removes its client from the map when it stops (and shuts down the client's executor), and creates a new client when it starts. A new client gets the metric maps of the Edge Node and of its Devices again (kept in a static map by descriptor), and Device producers take the current client of their Edge Node when they publish. A Device producer never creates a client itself in that case.

*Tests*: new {{TahuEdgeProducerRestartTest}} (2 tests) with {{FakeMqttBroker}}, a small in-process MQTT 3.1.1 broker that acknowledges connections, subscriptions and publishes and records the published topics. {{edgeNodePublishesAgainAfterTheRouteIsRestarted}}: NBIRTH and one NDATA, stop and start the route, then a second NBIRTH and NDATA are expected. On main it fails: {{not published within 20 seconds; connects: 1, published: [spBv1.0/RestartGroup/NBIRTH/RestartNode, spBv1.0/RestartGroup/NDATA/RestartNode, spBv1.0/RestartGroup/DDEATH/RestartNode/D1, spBv1.0/RestartGroup/NDEATH/RestartNode]}}. {{deviceOfAnotherRoutePublishesAgainAfterTheEdgeNodeRouteIsRestarted}}: only the Edge Node route is restarted, and the Device route's DDATA is expected after the new DBIRTH; fails on main the same way ({{connects: 1}}). Both pass with the fix. camel-tahu unit tests with the fix: 9 run, 0 failures (the IT tests need a container and were not run).

*How it was found*: variant analysis of CAMEL-25838 (pattern: a client or connection that is shut down or replaced, while state that belongs to it, here the client in a static map and the Device metric maps, is kept or lost), then the JUnit tests.

*Affected*: main, camel-4.23.x, camel-4.18.x and camel-4.14.x (the same static map and shutdown; read, not run; the {{stayRunning}} behaviour was read in Tahu 1.0.21 only).

_Filed with Claude Code on behalf of allthingssecurity._



> camel-tahu - after its route is restarted, an Edge Node producer never connects again and drops every message (logged, the exchange completes)
> ----------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-25846
>                 URL: https://issues.apache.org/jira/browse/CAMEL-25846
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-tahu
>            Reporter: shashank
>            Assignee: shashank
>            Priority: Major
>
> {{TahuEdgeProducer}} (main 20cdb309f7d0) keeps the {{TahuEdgeClient}} of each Edge Node in a static map ({{descriptorClients}}, {{computeIfAbsent}} in {{createClient}}). When the Edge Node producer stops, it shuts its client down ({{TahuEdgeNodeProducer.doStop}}: {{tahuEdgeClient.shutdown()}}), but the client stays in the map.
> A Tahu {{EdgeClient}} cannot run again after {{shutdown()}}: {{shutdown()}} sets {{stayRunning}} to false, only the constructor sets it to true, and {{run()}} returns at once when it is false (Tahu 1.0.21, read with javap). When the route is started again, Camel creates a new producer, whose constructor takes the shut down client from the map, and {{doStart}} submits it again ({{TahuEdgeClient.startup()}}). The client never connects to the MQTT server again: the Edge Node stays offline (its NDEATH was the last message), and every message sent to the endpoint is dropped. {{EdgeClient.publishSparkplugMessage}} catches the failure and logs {{Failed to publish message on topic=...}} at ERROR, so the exchange completes without an exception. Device producers of the same Edge Node, also in routes that were not restarted, keep the old client too. This lasts until the JVM restarts (the map is static, so a new CamelContext in the same JVM takes the shut down client as well). A route restart is enough: stopping and starting the route (JMX, control bus, a route policy, a supervising route controller).
> *Fix*: the Edge Node producer removes its client from the map when it stops (and shuts down the client's executor), and creates a new client when it starts. A new client gets the metric maps of the Edge Node and of its Devices again (a producer keeps its metric map in a static map by descriptor from its start to its stop, and a producer that starts after its Edge Node got a new client adds its map to that client), and Device producers take the current client of their Edge Node when they publish. A Device producer never creates a client itself in that case.
> *Tests*: new {{TahuEdgeProducerRestartTest}} (2 tests) with {{FakeMqttBroker}}, a small in-process MQTT 3.1.1 broker that acknowledges connections, subscriptions and publishes and records the published topics. {{edgeNodePublishesAgainAfterTheRouteIsRestarted}}: NBIRTH and one NDATA, stop and start the route, then a second NBIRTH and NDATA are expected. On main it fails after 20 seconds (Awaitility): {{ConditionTimeoutException ... not published; connects: 1, published: [spBv1.0/RestartGroup/NBIRTH/RestartNode, spBv1.0/RestartGroup/NDATA/RestartNode, spBv1.0/RestartGroup/DDEATH/RestartNode/D1, spBv1.0/RestartGroup/NDEATH/RestartNode]}}. {{deviceOfAnotherRoutePublishesAgainAfterTheEdgeNodeRouteIsRestarted}}: only the Edge Node route is restarted, and the Device route's DDATA is expected after the new DBIRTH; fails on main the same way ({{connects: 1}}). Both pass with the fix. camel-tahu unit tests with the fix: 9 run, 0 failures (the IT tests need a container and were not run).
> *How it was found*: variant analysis of CAMEL-25838 (pattern: a client or connection that is shut down or replaced, while state that belongs to it, here the client in a static map and the Device metric maps, is kept or lost), then the JUnit tests.
> *Affected*: main, camel-4.23.x, camel-4.18.x and camel-4.14.x (the same static map and shutdown; read, not run; the {{stayRunning}} behaviour was read in Tahu 1.0.21 only).
> _Filed with Claude Code on behalf of allthingssecurity._



--
This message was sent by Atlassian Jira
(v8.20.10#820010)