You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Daniel Langevin <da...@shq.gouv.qc.ca> on 2022/04/13 17:56:48 UTC
FTP or pollEnrich not working well in Camel 3.14.2
Hello,
I am converting routes from CAMEL 2.17.7 to Camel 3.14.2
This route performs a Directory List of an FTP Site and builds an XML result with the obtained list.
It's been working fine in Camel 2.17.7 for several years, but in Camel 3.14.2 I'm not able to get it to work properly anymore.
In the two cases I have 3 files in the FTP Directory (pointed on the same directory)
1)** Here is my route in CAMEL 2.17.7 in SPRING XML running in ServiceMix 7.0.1
<route id="rte.DESJ.FTPList">
<description>Camel 2.17.7 FTPlist </description>
<from uri="direct:desjFtpList" />
<setProperty propertyName="zzLoop"> <constant>0</constant></setProperty>
<setProperty propertyName="zzFtpDirList"><simple><DirList>\n</simple></setProperty>
<log message="Entering Loop 2.17.7" />
<loop doWhile="true">
<simple>${property.zzLoop} != ${header.CamelBatchSize}</simple>
<pollEnrich timeout="10000" >
<simple>ftp://{{serveur.DATA}}/{{desjSHQGroupe}}{{desjSHQrepertoireCheque}}?username={{serveur.user}}&password={{server.passwd}}&exclude=(?i)noseqchq.(?i)txt&download=false&timeout=30000&noop=true&sendEmptyMessageWhenIdle=true&disconnect=true&idempotent=false</simple>
</pollEnrich>
<log message="After pollEnrich" />
<log message="CamelBatchSize: ${header.CamelBatchSize} " />
<setProperty propertyName="zzLoop"><simple>${property.zzLoop}++</simple></setProperty>
<setProperty propertyName="zzFtpFicNom"><simple><Fichier>\n<Nom>${header.CamelFilename}</Nom>\n</simple></setProperty>
<setProperty propertyName="zzFtpFicDate"><simple><Date>${header.CamelFileLastModified}</Date>\n</Fichier></simple></setProperty>
<setProperty propertyName="zzFtpDirList"><simple>${property.zzFtpDirList}${property.zzFtpFicNom}${property.zzFtpFicDate}\n</simple></setProperty>
<log message="Loop:${property.zzLoop}, file:${header.CamelFileName}" />
</loop>
<log message="Outside loop" />
<setBody><simple>${property.zzFtpDirList}/n</DirList></simple></setBody>
<convertBodyTo type="java.lang.String" charset="UTF-8" />
<log message="Body:/n${body}" />
<log message="Ftp List Done" />
</route>
** trace Result for Camel 2.17.7 routes **
2022-04-13 11:57:32,478 | Entering Loop 2.17.7
2022-04-13 11:57:34,060 | After pollEnrich
2022-04-13 11:57:34,060 | CamelBatchSize: 3
2022-04-13 11:57:34,061 | Loop:1, file:C2202010 - Copie.psl
2022-04-13 11:57:34,062 | After pollEnrich
2022-04-13 11:57:34,062 | CamelBatchSize: 3
2022-04-13 11:57:34,062 | Loop:2, file:C2202010.psl
2022-04-13 11:57:34,062 | Apres le pollEnrich
2022-04-13 11:57:34,063 | CamelBatchSize: 3
2022-04-13 11:57:34,063 | Loop:3, file:C2202170.LOG
2022-04-13 11:57:34,070 | Outside loop
2022-04-13 11:57:34,082 | Body:
<DirList>
<Fichier>
<Nom>C2202010 - Copie.psl</Nom>
<Date>1642715460000</Date>
</Fichier>
<Fichier>
<Nom>C2202010.psl</Nom>
<Date>1642715460000</Date>
</Fichier>
<Fichier>
<Nom>C2202170.LOG</Nom>
<Date>1644506040000</Date>
</Fichier>
</DirList>
2022-04-13 11:57:34,082 | FTP List Done
After Converting a camel 3.14.2, the result is not as expected ????
Only first file in the FTP Directory is obtain ???
2)** Here is my route in CAMEL 3.14.2 in BLUEPRINT XML running in Karaf 4.3.3
<route id="rte.DESJ.FTPList">
<description>Camel 3.14.2 FTPlist </description>
<from uri="direct:desjFtpList" />
<setProperty name="zzLoop"> <constant>0</constant></setProperty>
<setProperty name="zzFtpDirList"><simple><DirList>\n</simple></setProperty>
<log message="Entering Loop 3.14.2" />
<loop doWhile="true">
<simple>${exchangeProperty.zzLoop} != ${header.CamelBatchSize} </simple>
<pollEnrich timeout="10000" >
<simple>ftp://{{serveur.DATA}}/{{desjSHQGroupe}}{{desjSHQrepertoireCheque}}?username={{serveur.user}}&password={{server.passwd}}&exclude=(?i)noseqchq.(?i)txt&download=false&timeout=30000&noop=true&sendEmptyMessageWhenIdle=true&disconnect=true&idempotent=false</simple>
</pollEnrich>
<log message="After pollEnrich" />
<log message="CamelBatchSize: ${header.CamelBatchSize} " />
<setProperty name="zzLoop"><simple>${exchangeProperty.zzLoop}++</simple></setProperty>
<setProperty name="zzFtpFicNom"><simple><Fichier>\n<Nom>${header.CamelFilename}</Nom>\n</simple></setProperty>
<setProperty name="zzFtpFicDate"><simple><Date>${header.CamelFileLastModified}</Date>\n</Fichier></simple></setProperty>
<setProperty name="zzFtpDirList"><simple>${exchangeProperty.zzFtpDirList}${exchangeProperty.zzFtpFicNom}${exchangeProperty.zzFtpFicDate}\n</simple></setProperty>
<log message="Loop:${exchangeProperty.zzLoop}, file:${header.CamelFileName}" />
</loop>
<log message="Outside loop" />
<setBody><simple>${exchangeProperty.zzFtpDirList}\n</DirList></simple></setBody>
<convertBodyTo type="java.lang.String" charset="UTF-8" />
<log message="Body:/n${body}" />
<log message="Ftp List Done" />
</route>
** trace Result for Camel 3.14.2 routes **
2022-04-13T13:33:39,340 | Entering Loop 3.14.2
2022-04-13T13:33:39,466 | After pollEnrich
2022-04-13T13:33:39,466 | CamelBatchSize: 1
2022-04-13T13:33:39,467 | Loop:1, file:C2202010 - Copie.psl
2022-04-13T13:33:39,467 | Outside loop
2022-04-13T13:33:39,467 | Body:
<DirList>
<Fichier>
<Nom>C2202010 - Copie.psl</Nom>
<Date>1642715460000</Date>
</Fichier>
</DirList>
2022-04-13T13:33:39,468 | Ftp List Done
As you may have noticed, only one file is read instead of the 3 files present in the Directory.
I've looked at this every which way, but I can't find what's wrong.
has the way of doing FTP or Enrich changed?
Someone can help !
Regards.
Daniel Langevin
Daniel Langevin
Direction de l'assistance et des technologies
Direction des ressources informationnelles et matérielles
RE: FTP or pollEnrich not working well in Camel 3.14.2
Posted by Daniel Langevin <da...@shq.gouv.qc.ca>.
Hi,
Finally I found a solution.
Not very pretty but it still works.
Here is what I did, if anyone is interested. Hope it will help someone else.
<route id="rte.DESJ.FTPList">
<description>Camel 3.14.2 FTPlist </description>
<from uri="direct:desjFtpList" />
<setProperty name="zzLoop"> <constant>0</constant></setProperty>
<setProperty name="zzFtpDirList"><simple><DirList>\n</simple></setProperty>
<setHeader name="finish"><simple>0</simple></setHeader> <!-- ** NEW -->
<log message="Entering Loop 3.14.2" />
<loop doWhile="true">
<simple>${header.finish} == 0</simple> <!-- ** NEW -->
<pollEnrich timeout="10000" >
<simple>ftp://{{serveur.DATA}}/{{desjSHQGroupe}}{{desjSHQrepertoireCheque}}?username={{serveur.user}}&password={{server.passwd}}&exclude=(?i)noseqchq.(?i)txt&download=false&timeout=30000&noop=true&sendEmptyMessageWhenIdle=true&disconnect=true&idempotent=false</simple>
</pollEnrich>
<log message="After pollEnrich" />
<log message="CamelBatchSize: ${header.CamelBatchSize} " />
<choice> <!-- ** NEW -->
<when> <simple>${header.CamelBatchSize} == 1 && ${header.CamelFileName} != null</simple> <!-- ** NEW -->
<setProperty name="zzLoop"><simple>${exchangeProperty.zzLoop}++</simple></setProperty>
<setProperty name="zzFtpFicNom"><simple><Fichier>\n<Nom>${header.CamelFilename}</Nom>\n</simple></setProperty>
<setProperty name="zzFtpFicDate"><simple><Date>${header.CamelFileLastModified}</Date>\n</Fichier></simple></setProperty>
<setProperty name="zzFtpDirList"><simple>${exchangeProperty.zzFtpDirList}${exchangeProperty.zzFtpFicNom}${exchangeProperty.zzFtpFicDate}\n</simple></setProperty>
<setHeader name="finish"><simple>0</simple></setHeader> <!-- ** NEW -->
</when> <!-- ** NEW -->
<otherwise> <!-- ** NEW -->
<setHeader name="finish"><simple>9</simple></setHeader> <!-- ** NEW -->
</otherwise> <!-- ** NEW -->
</choice> <!-- ** NEW -->
<log message="Loop:${exchangeProperty.zzLoop}, file:${header.CamelFileName}" />
</loop>
<log message="Outside loop" />
<setBody><simple>${exchangeProperty.zzFtpDirList}\n</DirList></simple></setBody>
<convertBodyTo type="java.lang.String" charset="UTF-8" />
<log message="Body:/n${body}" />
<log message="Ftp List Done" />
</route>
** trace Result after my modification for Camel 3.14.2 routes **
2022-04-21T10:50:17,689 | Entering Loop 3.14.2
2022-04-21T10:50:17,731 | After pollEnrich
2022-04-21T10:50:17,732 | CamelBatchSize: 1
2022-04-21T10:50:17,733 | Loop:1, file:C2202010 - Copie.psl
2022-04-21T10:50:17,760 | After pollEnrich
2022-04-21T10:50:17,761 | CamelBatchSize: 1
2022-04-21T10:50:17,762 | Loop:2, file:C2202010.psl
2022-04-21T10:50:17,776 | After pollEnrich
2022-04-21T10:50:17,777 | CamelBatchSize: 1
2022-04-21T10:50:17,777 | Loop:3, file:C2202170.LOG
2022-04-21T10:50:17,793 | After pollEnrich
2022-04-21T10:50:17,793 | CamelBatchSize: 1
2022-04-21T10:50:17,794 | Loop:3, file:
2022-04-21T10:50:17,794 | Outside loop
2022-04-21T10:50:17,796 | Body:
<DirList>
<Fichier>
<Nom>C2202010 - Copie.psl</Nom>
<Date>1642715460000</Date>
</Fichier>
<Fichier>
<Nom>C2202010.psl</Nom>
<Date>1642715460000</Date>
</Fichier>
<Fichier>
<Nom>C2202170.LOG</Nom>
<Date>1644506040000</Date>
</Fichier>
</DirList>
2022-04-21T10:50:17,796 | Ftp List Done
Regards
Daniel Langevin
-----Message d'origine-----
De : Daniel Langevin <da...@shq.gouv.qc.ca>
Envoyé : 13 avril 2022 13:57
À : users@camel.apache.org
Objet : FTP or pollEnrich not working well in Camel 3.14.2
Hello,
I am converting routes from CAMEL 2.17.7 to Camel 3.14.2
This route performs a Directory List of an FTP Site and builds an XML result with the obtained list.
It's been working fine in Camel 2.17.7 for several years, but in Camel 3.14.2 I'm not able to get it to work properly anymore.
In the two cases I have 3 files in the FTP Directory (pointed on the same directory)
1)** Here is my route in CAMEL 2.17.7 in SPRING XML running in ServiceMix 7.0.1
<route id="rte.DESJ.FTPList">
<description>Camel 2.17.7 FTPlist </description>
<from uri="direct:desjFtpList" />
<setProperty propertyName="zzLoop"> <constant>0</constant></setProperty>
<setProperty propertyName="zzFtpDirList"><simple><DirList>\n</simple></setProperty>
<log message="Entering Loop 2.17.7" />
<loop doWhile="true">
<simple>${property.zzLoop} != ${header.CamelBatchSize}</simple>
<pollEnrich timeout="10000" >
<simple>ftp://{{serveur.DATA}}/{{desjSHQGroupe}}{{desjSHQrepertoireCheque}}?username={{serveur.user}}&password={{server.passwd}}&exclude=(?i)noseqchq.(?i)txt&download=false&timeout=30000&noop=true&sendEmptyMessageWhenIdle=true&disconnect=true&idempotent=false</simple>
</pollEnrich>
<log message="After pollEnrich" />
<log message="CamelBatchSize: ${header.CamelBatchSize} " />
<setProperty propertyName="zzLoop"><simple>${property.zzLoop}++</simple></setProperty>
<setProperty propertyName="zzFtpFicNom"><simple><Fichier>\n<Nom>${header.CamelFilename}</Nom>\n</simple></setProperty>
<setProperty propertyName="zzFtpFicDate"><simple><Date>${header.CamelFileLastModified}</Date>\n</Fichier></simple></setProperty>
<setProperty propertyName="zzFtpDirList"><simple>${property.zzFtpDirList}${property.zzFtpFicNom}${property.zzFtpFicDate}\n</simple></setProperty>
<log message="Loop:${property.zzLoop}, file:${header.CamelFileName}" />
</loop>
<log message="Outside loop" />
<setBody><simple>${property.zzFtpDirList}/n</DirList></simple></setBody>
<convertBodyTo type="java.lang.String" charset="UTF-8" />
<log message="Body:/n${body}" />
<log message="Ftp List Done" />
</route>
** trace Result for Camel 2.17.7 routes **
2022-04-13 11:57:32,478 | Entering Loop 2.17.7
2022-04-13 11:57:34,060 | After pollEnrich
2022-04-13 11:57:34,060 | CamelBatchSize: 3
2022-04-13 11:57:34,061 | Loop:1, file:C2202010 - Copie.psl
2022-04-13 11:57:34,062 | After pollEnrich
2022-04-13 11:57:34,062 | CamelBatchSize: 3
2022-04-13 11:57:34,062 | Loop:2, file:C2202010.psl
2022-04-13 11:57:34,062 | Apres le pollEnrich
2022-04-13 11:57:34,063 | CamelBatchSize: 3
2022-04-13 11:57:34,063 | Loop:3, file:C2202170.LOG
2022-04-13 11:57:34,070 | Outside loop
2022-04-13 11:57:34,082 | Body:
<DirList>
<Fichier>
<Nom>C2202010 - Copie.psl</Nom>
<Date>1642715460000</Date>
</Fichier>
<Fichier>
<Nom>C2202010.psl</Nom>
<Date>1642715460000</Date>
</Fichier>
<Fichier>
<Nom>C2202170.LOG</Nom>
<Date>1644506040000</Date>
</Fichier>
</DirList>
2022-04-13 11:57:34,082 | FTP List Done
After Converting a camel 3.14.2, the result is not as expected ????
Only first file in the FTP Directory is obtain ???
2)** Here is my route in CAMEL 3.14.2 in BLUEPRINT XML running in Karaf 4.3.3
<route id="rte.DESJ.FTPList">
<description>Camel 3.14.2 FTPlist </description>
<from uri="direct:desjFtpList" />
<setProperty name="zzLoop"> <constant>0</constant></setProperty>
<setProperty name="zzFtpDirList"><simple><DirList>\n</simple></setProperty>
<log message="Entering Loop 3.14.2" />
<loop doWhile="true">
<simple>${exchangeProperty.zzLoop} != ${header.CamelBatchSize} </simple>
<pollEnrich timeout="10000" >
<simple>ftp://{{serveur.DATA}}/{{desjSHQGroupe}}{{desjSHQrepertoireCheque}}?username={{serveur.user}}&password={{server.passwd}}&exclude=(?i)noseqchq.(?i)txt&download=false&timeout=30000&noop=true&sendEmptyMessageWhenIdle=true&disconnect=true&idempotent=false</simple>
</pollEnrich>
<log message="After pollEnrich" />
<log message="CamelBatchSize: ${header.CamelBatchSize} " />
<setProperty name="zzLoop"><simple>${exchangeProperty.zzLoop}++</simple></setProperty>
<setProperty name="zzFtpFicNom"><simple><Fichier>\n<Nom>${header.CamelFilename}</Nom>\n</simple></setProperty>
<setProperty name="zzFtpFicDate"><simple><Date>${header.CamelFileLastModified}</Date>\n</Fichier></simple></setProperty>
<setProperty name="zzFtpDirList"><simple>${exchangeProperty.zzFtpDirList}${exchangeProperty.zzFtpFicNom}${exchangeProperty.zzFtpFicDate}\n</simple></setProperty>
<log message="Loop:${exchangeProperty.zzLoop}, file:${header.CamelFileName}" />
</loop>
<log message="Outside loop" />
<setBody><simple>${exchangeProperty.zzFtpDirList}\n</DirList></simple></setBody>
<convertBodyTo type="java.lang.String" charset="UTF-8" />
<log message="Body:/n${body}" />
<log message="Ftp List Done" />
</route>
** trace Result for Camel 3.14.2 routes **
2022-04-13T13:33:39,340 | Entering Loop 3.14.2
2022-04-13T13:33:39,466 | After pollEnrich
2022-04-13T13:33:39,466 | CamelBatchSize: 1
2022-04-13T13:33:39,467 | Loop:1, file:C2202010 - Copie.psl
2022-04-13T13:33:39,467 | Outside loop
2022-04-13T13:33:39,467 | Body:
<DirList>
<Fichier>
<Nom>C2202010 - Copie.psl</Nom>
<Date>1642715460000</Date>
</Fichier>
</DirList>
2022-04-13T13:33:39,468 | Ftp List Done
As you may have noticed, only one file is read instead of the 3 files present in the Directory.
I've looked at this every which way, but I can't find what's wrong.
has the way of doing FTP or Enrich changed?
Someone can help !
Regards.
Daniel Langevin
Daniel Langevin
Direction de l'assistance et des technologies Direction des ressources informationnelles et matérielles