[jira] [Created] (CAMEL-25808) camel-mail - after the IMAP folder connection dropped during a poll, the consumer marks (or deletes) another mail than the one it routed

["shashank (Jira)" <ji...@apache.org>]

shashank created CAMEL-25808:
--------------------------------

             Summary: camel-mail - after the IMAP folder connection dropped during a poll, the consumer marks (or deletes) another mail than the one it routed
                 Key: CAMEL-25808
                 URL: https://issues.apache.org/jira/browse/CAMEL-25808
             Project: Camel
          Issue Type: Bug
          Components: camel-mail
            Reporter: shashank
            Assignee: shashank


{{MailConsumer.processCommit}} (main 7aae27315946, lines 520-589) marks a mail once its exchange completed: it opens the folder again if it is closed, then sets SEEN or DELETED (and copies / moves the mail for {{copyTo}} / {{moveTo}}) on the mail object of the poll:

{code:java}
// ensure folder is open
if (!folder.isOpen()) {
    folder.open(Folder.READ_WRITE);
}
...
mail.setFlag(Flags.Flag.SEEN, true);   // or DELETED, after folder.copyMessages(new Message[] { mail }, dest)
{code}

The folder is closed at that point when its IMAP connection dropped while the mail was routed (an idle timeout of the server during a long exchange, a network failure): angus-mail cleans the folder up when it notices that, and {{isOpen()}} returns false. The mail object belongs to the earlier open. An angus-mail {{IMAPMessage}} sends its commands with {{getSequenceNumber()}}, which looks its message number up in the message cache of the *current* open of its folder ({{folder.messageCache.seqnumOf(msgnum)}}); the new open builds that cache from the mailbox as it is now. When mails were expunged between the two opens (another mail client, a second Camel consumer on the same mailbox with a shared {{idempotentRepository}} closing its folder, or a server that expunges a deleted mail at once), the number designates another mail:

* the mail that was routed is not marked: it stays unread and is routed again (a duplicate);
* another mail is marked instead. With the default {{unseen=true}} a mail marked SEEN is never polled again, so if it was not routed yet (for example a mail after the {{maxMessagesPerPoll}} limit) it is never consumed. With {{delete=true}} or {{moveTo}} it is deleted (moved) without being routed. Nothing is logged.

*Reproduction* ({{MailConsumerFolderReopenedTest}}, GreenMail and the real angus-mail client, as the other tests of the module): three mails, {{maxMessagesPerPoll=2}}, the consumer connects through a small TCP proxy. While the second mail is routed, another client removes the first one (already consumed) and the proxy drops the consumer's connections. On main the commit of the second mail marks the third mail SEEN: the mock receives "Message 0", "Message 1", "Message 1" and never "Message 2" ({{Message with body Message 2 was expected but not found in [Message 1]}}, 2 of 2 runs).

*Fix*: {{MailConsumer}} fetches the UIDs of the mails of a poll when it retrieves them (one FETCH), and counts the opens of the folder. When the folder was opened again after the mails of the batch were retrieved, the mail to mark is looked up by its UID (same UIDVALIDITY); when it is no longer there, nothing is marked (WARN). The batch is not routed further once the folder was opened again (the other mail objects are stale too, and with {{mapMailMessage=false}} their content would be read from other mails); those mails are given back as for a batch cut short by a stop (CAMEL-25710) and consumed by the next poll. POP3 is not changed: {{processCommit}} already finds a POP3 mail again by its key (see CAMEL-24434 for that key).

*Tests*: the new test fails on main and passes with the fix. camel-mail unit tests with the fix: 246 run, 0 failures, 0 errors, 4 skipped.

*How it was found*: deterministic simulation testing with a fake IMAP world (camel-test-dst, CAMEL-25738): the real {{MailConsumer}}, a fake IMAP server and client that follows RFC 3501 and the angus-mail message cache behaviour above, seeded connection drops, a second consumer, other clients that expunge and a server that expunges at once. Invariant: no mail is marked SEEN or DELETED, or copied, before its exchange completed. 384 of 5000 seeds failed on main, first failing seed 4 (a failing seed replays with the same trace hash); 0 of 5000 with the fix. The world re-finds CAMEL-25710 when that fix is reverted. Then the JUnit test above with the real client.

*Affected*: main, camel-4.23.x, camel-4.22.x, camel-4.18.x (the same {{processCommit}}; read, not run). Not run against a real IMAP server other than GreenMail.

_Filed with Claude Code on behalf of allthingssecurity._




--
This message was sent by Atlassian Jira
(v8.20.10#820010)