[jira] [Commented] (CAMEL-22168) PollEnrich with FTP ignores maxMessagesPerPoll

["Claus Ibsen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-22168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17983451#comment-17983451 ] 

Claus Ibsen commented on CAMEL-22168:
-------------------------------------

The poll / pollEnrich is for a single message only. The option maxMessages... is hardcoded to 1, and we will add an exception if you attempt to configure it to a higher value.

> PollEnrich with FTP ignores maxMessagesPerPoll
> ----------------------------------------------
>
>                 Key: CAMEL-22168
>                 URL: https://issues.apache.org/jira/browse/CAMEL-22168
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-file, camel-ftp
>    Affects Versions: 4.12.0
>            Reporter: Raymond
>            Assignee: Claus Ibsen
>            Priority: Minor
>
> The File and FTP components have the option {*}maxMessagesPerPoll{*}:
> | |To define a maximum messages to gather per poll. By default no maximum is set. Can be used to set a limit of e.g. 1000 to avoid when starting up the server that there are thousands of files. Set a value of 0 or negative to disabled it.|
> However, when the FTP component (or any other camel-file based component) is used in combination with the pollEnrich eip, this parameter is ignored.
> {code:java}
> <pollEnrich strategyRef="CurrentEnrichStrategy" timeout="5000">          <simple>sftp:myftpserver:2022/somepath?username=RAW(myuser)&amp;password=RAW(mypassword)&amp;include=RAW(.*.txt)&amp;maxMessagesPerPoll=3</simple> </pollEnrich>
>  {code}
> This is well known, as the pollEnrich can only pick up one file per poll. A solution can be found on StackOverflow:
> [https://stackoverflow.com/questions/38103035/apache-camel-pollenrich-is-not-copying-all-the-files]
> The problem is that you only can find out after running it. It's not clear that the maxMessagesPerPoll isn't active/ignored (and this may as well count for other parameters). 
> Besides, the maxMessagesPerPoll isn't working, even with the solution on StackOverflow. To work around this, something like this could be done:
> {code:java}
> <route id="674f52f7-981e-4b3c-95a1-cbdf299b9ee9">
>     <from uri="direct:start"/>
>     <setProperty propertyName="Enrich-MaxMessagesPerPoll">
>         <constant>3</constant>
>     </setProperty>
>     <loop doWhile="true">
>         <simple>${exchangeProperty.Enrich-MaxMessagesPerPoll} != 0</simple>
>         <pollEnrich strategyRef="CurrentEnrichStrategy" timeout="5000">
>             <simple>sftp:myftpserver:2022/somepath?username=RAW(myuser)&amp;password=RAW(mypassword)&amp;include=RAW(.*.txt)&amp;maxMessagesPerPoll=3</simple>
>         </pollEnrich>
>         <setProperty propertyName="Enrich-MaxMessagesPerPoll">
>             <simple>${exchangeProperty.Enrich-MaxMessagesPerPoll}--</simple>
>         </setProperty>
>         <choice>
>             <when>
>                 <simple>${body} == null</simple>
>                 <setProperty propertyName="Enrich-MaxMessagesPerPoll">
>                     <constant>0</constant>
>                 </setProperty>
>             </when>
>             <otherwise>
>                 <to uri="direct:out"/>
>                 <removeHeaders pattern="CamelFile*"/>
>             </otherwise>
>         </choice>
>     </loop>
> </route> {code}
> In the above example, -1 picks up all files, while a positive integer uses the maxMessagesPerPoll. However, this solution is suboptimal, because every loop needs to perform an FTP call (with large number this is resource intensive), and uses lots of DSL code of something that should be simple.
> I think at least a log warning should be printed when this combination is used that this parameter is not active (and maybe this counts for other parameters as well). The best thing would be if this would be possible (For example within a pollEnrich, or a producer option on FTP/File to pickup files, or some other solution). 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)