[jira] Assigned: (CAMEL-1824) GenericFileConsumer fails to pick up all files in a directory if there are more files than maxMessagePerPoll setting

["Claus Ibsen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/activemq/browse/CAMEL-1824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Claus Ibsen reassigned CAMEL-1824:
----------------------------------

    Assignee: Claus Ibsen

> GenericFileConsumer fails to pick up all files in a directory if there are more files than maxMessagePerPoll setting
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-1824
>                 URL: https://issues.apache.org/activemq/browse/CAMEL-1824
>             Project: Apache Camel
>          Issue Type: Bug
>          Components: camel-core
>    Affects Versions: 2.0-M2
>            Reporter: Alexander Lee
>            Assignee: Claus Ibsen
>
> Camel adds filenames it is processing to an in memory, in progress map.  Once it is finished processing it removes them.  If we specific a maxMessagesPerPoll number less then the files in a directory, it only processes up to the number of files we specify for each poll.  It then removes the rest from the in progress map and tries them again next poll.  This is the code (from GenericFileConsumer:processBatch()) for the last part:
>        for (int index = 0; index < exchanges.size() && isRunAllowed(); index++) {
>             GenericFileExchange<T> exchange = (GenericFileExchange<T>) exchanges.poll();
>             String key = exchange.getGenericFile().getFileName();
>             endpoint.getInProgressRepository().remove(key);
>         }
>  
> Unfortunately, as you can see it uses exchanges.size() to determine how many file names to remove (i.e. how many times to loop), however exchanges.poll() removes one from the head of exchanges for each loop.  This means that the exchanges.size() reduces by one for each loop, which means it only cleans up half of the filenames that are in exchanges, which means these files are never picked up again as Camel thinks it is still processing them.
> The fix is to replace the for loop with a while:
>         while ((exchanges.size() > 0) && isRunAllowed()) {
>             GenericFileExchange<T> exchange = (GenericFileExchange<T>) exchanges.poll();
>             String key = exchange.getGenericFile().getFileName();
>             endpoint.getInProgressRepository().remove(key);
>         }

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.