You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Mikael Andersson Wigander <mi...@pm.me.INVALID> on 2020/10/21 08:00:15 UTC
File filter option using genericFileFilter is not working properly
We have a running SFTP route using the &filter option to accept files using a regular expression. This is working fine in Camel 2.25.0.
Migrating to Camel 3.5.0 it is not working at all. Every file in the directory is downloaded, disregarding the filter option.
I can't find any documentation about what might have changed EXCEPT that the &filter option has changed so that the bean reference should be without the "#" character prefix.
Bean definition
============
@Bean
public static StatusFileFilter<String> shuttleStatusFileFilter() {
return new StatusFileFilter<>();
}
Class definition
============
@Component
public class StatusFileFilter<T> implements GenericFileFilter<T> {
/**
* The Shuttle file.
*/
@Value("${shuttle.status-filename.regexp}")
private String shuttleFile;
@Override
public boolean accept(final GenericFile<T> genericFile) {
final long fileLength = genericFile.getFileLength();
final String fileName = genericFile.getFileName();
final boolean matches = fileName.matches(shuttleFile);
return (fileLength > 0 && matches);
}
}
The route:
========
final String fromStatusFilesStr = "{{shuttle.ftp.incoming-status-files.url}}" +
"&username={{shuttle.ftp.username}}" +
"&password={{shuttle.ftp.password}}" +
"&allowNullBody=false" +
"&filter=shuttleStatusFileFilter" +
"&bridgeErrorHandler=true" +
"&throwExceptionOnConnectFailed=true";
from(fromStatusFilesStr)
.to("log:StatusFiles?showHeaders=true&showProperties=true&multiline=true")
.choice()
.when(body().isNotNull())
.split(bodyAs(String.class).tokenize("\n")) //<.>
.parallelProcessing(true)
.streaming()
.to("{{jms.queue.statefiles}}")
.to("log:StateFiles?level=INFO&groupInterval=10000&groupActiveOnly=true")
.choice()
.when(simple("${header.CamelSplitComplete} == true"))
.log("Number of records split: ${header.CamelSplitSize}")
.log("Importing complete: ${header.CamelFileName}")
.endChoice()
.end();
I have tried to log whatever is going on in the bean but no log statements are present so that's why I think it is not working properly, the bean is not activated.
The documentation still points out the hash-char as prefix so that must still be wrong:
https://camel.apache.org/components/latest/file-component.html#_filter_using_org_apache_camel_component_file_genericfilefilter
Pls advice what could be wrong and/or point me to some documentation
Thx
/M
Re: File filter option using genericFileFilter is not working properly
Posted by Claus Ibsen <cl...@gmail.com>.
Some spring issue that it does not register your bean so Camel cant find it.
On Wed, Oct 21, 2020 at 10:31 AM Mikael Andersson Wigander
<mi...@pm.me.invalid> wrote:
>
> Thanks
>
> Using the # sign gives me this error
>
> Caused by: org.apache.camel.NoSuchBeanException: No bean could be found in the registry for: shuttleReconFileFilter of type: org.apache.camel.component.file.GenericFileFilter
>
> What am I missing, really?
>
>
> /M
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>
> On Wednesday 21 October 2020 kl. 10:14, Claus Ibsen <cl...@gmail.com> wrote:
>
> > Hi
> >
> > No you should use the # syntax. See the FromFtpFilterTest.
> >
> > On Wed, Oct 21, 2020 at 10:00 AM Mikael Andersson Wigander
> >
> > mikael.andersson.wigander@pm.me.invalid wrote:
> >
> > > We have a running SFTP route using the &filter option to accept files using a regular expression. This is working fine in Camel 2.25.0.
> > >
> > > Migrating to Camel 3.5.0 it is not working at all. Every file in the directory is downloaded, disregarding the filter option.
> > >
> > > I can't find any documentation about what might have changed EXCEPT that the &filter option has changed so that the bean reference should be without the "#" character prefix.
> > >
> > > Bean definition
> > > ===============
> > >
> > > @Bean
> > >
> > > public static StatusFileFilter<String> shuttleStatusFileFilter() {
> > >
> > > return new StatusFileFilter<>();
> > >
> > > }
> > >
> > > Class definition
> > > ================
> > >
> > > @Component
> > >
> > > public class StatusFileFilter<T> implements GenericFileFilter<T> {
> > >
> > > /**
> > >
> > > - The Shuttle file.
> > >
> > > */
> > >
> > > @Value("${shuttle.status-filename.regexp}")
> > >
> > > private String shuttleFile;
> > >
> > > @Override
> > >
> > > public boolean accept(final GenericFile<T> genericFile) {
> > >
> > > final long fileLength = genericFile.getFileLength();
> > >
> > > final String fileName = genericFile.getFileName();
> > >
> > > final boolean matches = fileName.matches(shuttleFile);
> > >
> > > return (fileLength > 0 && matches);
> > >
> > > }
> > >
> > > }
> > >
> > > The route:
> > > ==========
> > >
> > > final String fromStatusFilesStr = "{{shuttle.ftp.incoming-status-files.url}}" +
> > >
> > > "&username={{shuttle.ftp.username}}" +
> > >
> > > "&password={{shuttle.ftp.password}}" +
> > >
> > > "&allowNullBody=false" +
> > >
> > > "&filter=shuttleStatusFileFilter" +
> > >
> > > "&bridgeErrorHandler=true" +
> > >
> > > "&throwExceptionOnConnectFailed=true";
> > >
> > > from(fromStatusFilesStr)
> > >
> > > .to("log:StatusFiles?showHeaders=true&showProperties=true&multiline=true")
> > >
> > > .choice()
> > >
> > > .when(body().isNotNull())
> > >
> > > .split(bodyAs(String.class).tokenize("\n")) //<.>
> > >
> > > .parallelProcessing(true)
> > >
> > > .streaming()
> > >
> > > .to("{{jms.queue.statefiles}}")
> > >
> > > .to("log:StateFiles?level=INFO&groupInterval=10000&groupActiveOnly=true")
> > >
> > > .choice()
> > >
> > > .when(simple("${header.CamelSplitComplete} == true"))
> > >
> > > .log("Number of records split: ${header.CamelSplitSize}")
> > >
> > > .log("Importing complete: ${header.CamelFileName}")
> > >
> > > .endChoice()
> > >
> > > .end();
> > >
> > > I have tried to log whatever is going on in the bean but no log statements are present so that's why I think it is not working properly, the bean is not activated.
> > >
> > > The documentation still points out the hash-char as prefix so that must still be wrong:
> > >
> > > https://camel.apache.org/components/latest/file-component.html#_filter_using_org_apache_camel_component_file_genericfilefilter
> > >
> > > Pls advice what could be wrong and/or point me to some documentation
> > >
> > > Thx
> > >
> > > /M
> >
> > --
> >
> > Claus Ibsen
> > -----------
> >
> > http://davsclaus.com @davsclaus
> >
> > Camel in Action 2: https://www.manning.com/ibsen2
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: File filter option using genericFileFilter is not working properly
Posted by Mikael Andersson Wigander <mi...@pm.me.INVALID>.
Thanks
Using the # sign gives me this error
Caused by: org.apache.camel.NoSuchBeanException: No bean could be found in the registry for: shuttleReconFileFilter of type: org.apache.camel.component.file.GenericFileFilter
What am I missing, really?
/M
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday 21 October 2020 kl. 10:14, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> No you should use the # syntax. See the FromFtpFilterTest.
>
> On Wed, Oct 21, 2020 at 10:00 AM Mikael Andersson Wigander
>
> mikael.andersson.wigander@pm.me.invalid wrote:
>
> > We have a running SFTP route using the &filter option to accept files using a regular expression. This is working fine in Camel 2.25.0.
> >
> > Migrating to Camel 3.5.0 it is not working at all. Every file in the directory is downloaded, disregarding the filter option.
> >
> > I can't find any documentation about what might have changed EXCEPT that the &filter option has changed so that the bean reference should be without the "#" character prefix.
> >
> > Bean definition
> > ===============
> >
> > @Bean
> >
> > public static StatusFileFilter<String> shuttleStatusFileFilter() {
> >
> > return new StatusFileFilter<>();
> >
> > }
> >
> > Class definition
> > ================
> >
> > @Component
> >
> > public class StatusFileFilter<T> implements GenericFileFilter<T> {
> >
> > /**
> >
> > - The Shuttle file.
> >
> > */
> >
> > @Value("${shuttle.status-filename.regexp}")
> >
> > private String shuttleFile;
> >
> > @Override
> >
> > public boolean accept(final GenericFile<T> genericFile) {
> >
> > final long fileLength = genericFile.getFileLength();
> >
> > final String fileName = genericFile.getFileName();
> >
> > final boolean matches = fileName.matches(shuttleFile);
> >
> > return (fileLength > 0 && matches);
> >
> > }
> >
> > }
> >
> > The route:
> > ==========
> >
> > final String fromStatusFilesStr = "{{shuttle.ftp.incoming-status-files.url}}" +
> >
> > "&username={{shuttle.ftp.username}}" +
> >
> > "&password={{shuttle.ftp.password}}" +
> >
> > "&allowNullBody=false" +
> >
> > "&filter=shuttleStatusFileFilter" +
> >
> > "&bridgeErrorHandler=true" +
> >
> > "&throwExceptionOnConnectFailed=true";
> >
> > from(fromStatusFilesStr)
> >
> > .to("log:StatusFiles?showHeaders=true&showProperties=true&multiline=true")
> >
> > .choice()
> >
> > .when(body().isNotNull())
> >
> > .split(bodyAs(String.class).tokenize("\n")) //<.>
> >
> > .parallelProcessing(true)
> >
> > .streaming()
> >
> > .to("{{jms.queue.statefiles}}")
> >
> > .to("log:StateFiles?level=INFO&groupInterval=10000&groupActiveOnly=true")
> >
> > .choice()
> >
> > .when(simple("${header.CamelSplitComplete} == true"))
> >
> > .log("Number of records split: ${header.CamelSplitSize}")
> >
> > .log("Importing complete: ${header.CamelFileName}")
> >
> > .endChoice()
> >
> > .end();
> >
> > I have tried to log whatever is going on in the bean but no log statements are present so that's why I think it is not working properly, the bean is not activated.
> >
> > The documentation still points out the hash-char as prefix so that must still be wrong:
> >
> > https://camel.apache.org/components/latest/file-component.html#_filter_using_org_apache_camel_component_file_genericfilefilter
> >
> > Pls advice what could be wrong and/or point me to some documentation
> >
> > Thx
> >
> > /M
>
> --
>
> Claus Ibsen
> -----------
>
> http://davsclaus.com @davsclaus
>
> Camel in Action 2: https://www.manning.com/ibsen2
Re: File filter option using genericFileFilter is not working properly
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
No you should use the # syntax. See the FromFtpFilterTest.
On Wed, Oct 21, 2020 at 10:00 AM Mikael Andersson Wigander
<mi...@pm.me.invalid> wrote:
>
> We have a running SFTP route using the &filter option to accept files using a regular expression. This is working fine in Camel 2.25.0.
>
> Migrating to Camel 3.5.0 it is not working at all. Every file in the directory is downloaded, disregarding the filter option.
>
> I can't find any documentation about what might have changed EXCEPT that the &filter option has changed so that the bean reference should be without the "#" character prefix.
>
> Bean definition
> ============
>
> @Bean
> public static StatusFileFilter<String> shuttleStatusFileFilter() {
> return new StatusFileFilter<>();
> }
>
> Class definition
> ============
>
> @Component
> public class StatusFileFilter<T> implements GenericFileFilter<T> {
> /**
> * The Shuttle file.
> */
> @Value("${shuttle.status-filename.regexp}")
> private String shuttleFile;
>
> @Override
> public boolean accept(final GenericFile<T> genericFile) {
> final long fileLength = genericFile.getFileLength();
> final String fileName = genericFile.getFileName();
> final boolean matches = fileName.matches(shuttleFile);
> return (fileLength > 0 && matches);
> }
> }
>
> The route:
> ========
>
> final String fromStatusFilesStr = "{{shuttle.ftp.incoming-status-files.url}}" +
> "&username={{shuttle.ftp.username}}" +
> "&password={{shuttle.ftp.password}}" +
> "&allowNullBody=false" +
> "&filter=shuttleStatusFileFilter" +
> "&bridgeErrorHandler=true" +
> "&throwExceptionOnConnectFailed=true";
>
> from(fromStatusFilesStr)
> .to("log:StatusFiles?showHeaders=true&showProperties=true&multiline=true")
> .choice()
> .when(body().isNotNull())
> .split(bodyAs(String.class).tokenize("\n")) //<.>
> .parallelProcessing(true)
> .streaming()
> .to("{{jms.queue.statefiles}}")
> .to("log:StateFiles?level=INFO&groupInterval=10000&groupActiveOnly=true")
> .choice()
> .when(simple("${header.CamelSplitComplete} == true"))
> .log("Number of records split: ${header.CamelSplitSize}")
> .log("Importing complete: ${header.CamelFileName}")
> .endChoice()
> .end();
>
> I have tried to log whatever is going on in the bean but no log statements are present so that's why I think it is not working properly, the bean is not activated.
>
> The documentation still points out the hash-char as prefix so that must still be wrong:
> https://camel.apache.org/components/latest/file-component.html#_filter_using_org_apache_camel_component_file_genericfilefilter
>
> Pls advice what could be wrong and/or point me to some documentation
>
> Thx
>
> /M
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2