[jira] [Commented] (CAMEL-25806) camel-jira - the newIssues consumer silently skips the older issues when more than maxResults issues are created between two polls

["Claus Ibsen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-25806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18129816#comment-18129816 ] 

Claus Ibsen commented on CAMEL-25806:
-------------------------------------

Merged via https://github.com/apache/camel/pull/28198

_Claude Code on behalf of davsclaus_

> camel-jira - the newIssues consumer silently skips the older issues when more than maxResults issues are created between two polls
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-25806
>                 URL: https://issues.apache.org/jira/browse/CAMEL-25806
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-jira
>            Reporter: shashank
>            Assignee: shashank
>            Priority: Major
>             Fix For: 4.24.0
>
>
> {{NewIssuesConsumer.getNewIssues}} (main 7aae27315946, lines 131-157) searches for {{created > "<minute of the latest issue>" AND <jql> ORDER BY created DESC}} through {{AbstractJiraConsumer.getIssues(jql)}}, which stops paging as soon as it holds {{maxResults}} issues (default 50; {{issues.size() >= maxResults}}, line 107). The consumer then makes the first (newest) issue of the answer its {{latestIssue}}:
> {code:java}
> Queue<Issue> issues = getIssues(jqlFilter);
> ...
> latestIssue = issues.peek();
> {code}
> So when more than {{maxResults}} issues were created since the previous poll, the poll routes the newest {{maxResults}} of them, and the next poll asks only for issues created after the newest one: the older ones are never returned again. Nothing is logged. This happens with a burst of new issues (a CSV import, a bulk clone, an automation rule) and, more often, at the first poll after Jira was unreachable or the polls failed for a while: all but the newest 50 issues of the outage are lost. The documentation says the consumer retrieves the new issues after the route is started; the option {{maxResults}} is described as "Max number of issues to search for", not as a limit that drops the rest.
> *Fix*: when there is a latest issue, the poll reads every page of that search ({{maxResults}}, capped at 50, is the page size); the startup search and the other consumers are not changed. Doc note in the component page, upgrade note in the 4.24 guide (a poll can now route more than {{maxResults}} issues).
> *Tests*: new {{NewIssuesConsumerMoreThanMaxResultsTest}} (mocked {{SearchRestClient}} that pages like the Jira search: 60 issues created after the latest issue at startup, default {{maxResults}}). Fails on main: {{mock://result Received message count. Expected: <60> but was: <50>}}; passes with the fix. camel-jira unit tests: 60 run, 0 failures (the existing tests of the order and the pagination of new issues are unchanged).
> *How it was found*: a Z3 model of the newIssues cursor over several polls (issues arrive between polls; the poll keeps the newest M; the cursor moves to the newest): "every issue is routed" holds with all pages read (no counterexample), and violated on main for every M as soon as M+1 issues arrive between two polls (N=51, M=50, one poll). Then the JUnit test above.
> *Affected*: main, camel-4.23.x, camel-4.22.x, camel-4.18.x and camel-4.14.x (same {{getNewIssues}} and the same cut in {{getIssues}}; read, not run).
> _Filed with Claude Code on behalf of allthingssecurity._



--
This message was sent by Atlassian Jira
(v8.20.10#820010)