[jira] [Created] (CAMEL-24525) camel-google-firestore: the realtime change buffer is unbounded

["Andrea Cosentino (Jira)" <ji...@apache.org>]

Andrea Cosentino created CAMEL-24525:
----------------------------------------

             Summary: camel-google-firestore: the realtime change buffer is unbounded
                 Key: CAMEL-24525
                 URL: https://issues.apache.org/jira/browse/CAMEL-24525
             Project: Camel
          Issue Type: Improvement
          Components: camel-google
            Reporter: Andrea Cosentino
            Assignee: Andrea Cosentino


The Firestore consumer buffers the document changes reported by the realtime snapshot listener in an unbounded queue, and drains it on the scheduled poll:

{code:java}
private final Queue<Exchange> pendingExchanges = new ConcurrentLinkedQueue<>();
{code}

The listener callback runs on a Firestore client thread and is never throttled by the route. Whenever changes arrive faster than the poll consumes them the queue grows without limit, and every element is a full Exchange holding the document data. The queue also keeps growing while the route is suspended, or when the consumer is configured with a long poll delay, so a burst of writes on a busy collection can exhaust the heap.

Add a {{maxPendingChanges}} consumer option that bounds the buffer:

* the default stays unbounded, so existing routes are unaffected;
* when a limit is configured and reached, the oldest buffered change is discarded - the route then keeps up with the current state of the collection - and a warning is logged;
* discarded exchanges are released back to the exchange pool instead of being dropped on the floor, and so is anything still buffered when the consumer stops.

Blocking the listener callback is deliberately not offered as a policy: that thread belongs to the Firestore gRPC client, and stalling it would stall the watch stream itself.

This is the item left open in CAMEL-24347, whose other findings are already fixed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)