[jira] [Created] (CAMEL-25960) camel-file - a done file with a fixed name can be deleted by the last file of the previous poll when that file completes asynchronously while the next poll decides which done files to keep

["shashank (Jira)" <ji...@apache.org>]

shashank created CAMEL-25960:
--------------------------------

             Summary: camel-file - a done file with a fixed name can be deleted by the last file of the previous poll when that file completes asynchronously while the next poll decides which done files to keep
                 Key: CAMEL-25960
                 URL: https://issues.apache.org/jira/browse/CAMEL-25960
             Project: Camel
          Issue Type: Bug
          Components: camel-file
            Reporter: shashank
            Assignee: shashank


With a done file with a fixed name ({{doneFileName=ready}}), a poll that leaves files of a directory for the next poll keeps the done file, so the last file of the poll does not delete it ({{GenericFileOnCompletion.handleDoneFile}}, line 199: {{keptDoneFiles.isKept(doneFileName)}}).

When the next poll lists the directory completely, {{GenericFileKeptDoneFiles.startBatch(listed, true)}} clears the whole set (line 61, {{kept.clear()}}). The done files of the files this poll leaves are kept again only later in the same poll: in {{GenericFileConsumer.poll()}} for the non-eager {{maxMessagesPerPoll}} cut (line 249, after the exchanges are created and sorted), and in {{processBatch}} for a file that is not started (line 315) or that is left when the batch stops (line 322).

If the last file of the previous poll completes in between, on another thread (the route continues with {{threads()}}, {{seda}}, or an asynchronous producer, so the consumer's next poll can start before that exchange is done), {{handleDoneFile}} finds the done file not kept and deletes it. The files the new poll then leaves for the next one have no done file, so they are not consumed until a new done file is written.

CAMEL-25772 removed the same kind of window on the cut-listing path; this one is on the complete-listing path and is older (CAMEL-25728).

h3. Reproduction

Not reproduced by a test yet. Outline: {{doneFileName=ready}}, {{maxMessagesPerPoll=2}}, {{eagerMaxMessagesPerPoll=false}}, 3 files and {{ready}} in the directory; the route hands off with {{threads()}} and its processor for the second file blocks on a latch; release it after the next poll has called {{startBatch}} (complete listing) and before it re-keeps the done file. Expected on main: {{ready}} is deleted and the third file stays in the directory.

h3. Proposed fix

Decide the kept set of a poll before clearing the old one, or keep the old entries until the poll has re-kept its own (for example: compute the done files to keep, then replace the set in one step), so a completing exchange of the previous poll never sees a done file that both polls keep as not kept.

Affected: main (e6885df7894d). The kept done files exist since CAMEL-25728 (4.24 only).

_Filed with Claude Code on behalf of allthingssecurity._




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)