[jira] [Work started] (CAMEL-24525) camel-google-firestore: the realtime change buffer is unbounded

["Andrea Cosentino (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CAMEL-24525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on CAMEL-24525 started by Andrea Cosentino.
------------------------------------------------
> camel-google-firestore: the realtime change buffer is unbounded
> ---------------------------------------------------------------
>
>                 Key: CAMEL-24525
>                 URL: https://issues.apache.org/jira/browse/CAMEL-24525
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-google
>            Reporter: Andrea Cosentino
>            Assignee: Andrea Cosentino
>            Priority: Minor
>
> The Firestore consumer buffers the document changes reported by the realtime snapshot listener in an unbounded queue, and drains it on the scheduled poll:
> {code:java}
> private final Queue<Exchange> pendingExchanges = new ConcurrentLinkedQueue<>();
> {code}
> The listener callback runs on a Firestore client thread and is never throttled by the route. Whenever changes arrive faster than the poll consumes them the queue grows without limit, and every element is a full Exchange holding the document data. The queue also keeps growing while the route is suspended, or when the consumer is configured with a long poll delay, so a burst of writes on a busy collection can exhaust the heap.
> Add a {{maxPendingChanges}} consumer option that bounds the buffer:
> * the default stays unbounded, so existing routes are unaffected;
> * when a limit is configured and reached, the oldest buffered change is discarded - the route then keeps up with the current state of the collection - and a warning is logged;
> * discarded exchanges are released back to the exchange pool instead of being dropped on the floor, and so is anything still buffered when the consumer stops.
> Blocking the listener callback is deliberately not offered as a policy: that thread belongs to the Firestore gRPC client, and stalling it would stall the watch stream itself.
> This is the item left open in CAMEL-24347, whose other findings are already fixed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)