You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Klaus Johansen <kl...@gmail.com> on 2016/10/27 10:46:28 UTC
Is GenericFileOperationFailedException handled differently than other exceptions?
Hi all,
I have used a lot time configuring a deadLetterChannel errorHandler and a
custom PollingConsumerPollStrategy for our PollingConsumers (mostly sFTP
and FTPS).
In combination with "throwExceptionOnConnectFailed=true" and
"consumer.bridgeErrorHandler=true" we have a generally robust setup, where
the Camel error-handler handles both consumer and producer errors. It does
redelivery and sends an alert when exhausted. This allows us to handle e.g.
connection timeouts to FTP servers in both consumer and producer in the
same controlled way.
Recently we had several situations with FTP error code 150 (e.g. socket
timeouts) which ends up in an org.apache.camel.component.file.
GenericFileOperationFailedException.
Similar if the FTP server replies with "550 File not found, access denied."
it results in the following exception:
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
retrieve file: RemoteFile[TestFil.XML] from: Endpoint[ftp://127.0.0.1:21/?
autoCreate=false&binary=true&consumer.bridgeErrorHandler=
true&delete=true&disconnect=true&doneFileName=%24%7Bfile%3Aname%7D.SEM&
<ftp://127.0.0.1/?autoCreate=false&binary=true&consumer.bridgeErrorHandler=true&delete=true&disconnect=true&doneFileName=%24%7Bfile%3Aname%7D.SEM&>
include=%28%3Fi%29.*%5C.XML&localWorkDirectory=c%3A%5Ctest%5Cftpworkdir&
maximumReconnectAttempts=0&passiveMode=true&password=xxxxxx&pollStrategy=%
23poll30times&readLock=none&reconnectDelay=240000&
scheduler=spring&scheduler.cron=30+*+*+*+*+MON-FRI&stepwise=false&
throwExceptionOnConnectFailed=true&username=testuser]
According to the log, the GenericFileOperationFailedException is directly
followed by the configured "onPrepareFailure" processor and then handled by
the "FatalFallbackErrorHandler". As a result the "commit" method of the
custom PollingConsumerPollStrategy is invoked and no redelivery attempted.
Is GenericFileOperationFailedException handled differently by the Camel
Error Handler than other exceptions? - Or is it a bug?
If working as designed: Can I somehow configure an onException section or
something different to override this behaviour?
I'm currently using camel 2.17.3.
Best regards,
Klaus
Re: Is GenericFileOperationFailedException handled differently than
other exceptions?
Posted by Klaus Johansen <kl...@gmail.com>.
Hi again,
Let me rephrase: My problem is that GenericFileOperationFailedException goes
directly to the dead letter channel and and completely bypass redelivery.
I don't have any onException statements which reset maximumRedeliveries to
0.
Is this excepted behavior?
(In my earlier post I mentioned "FatalFallbackErrorHandler": It was invoked
because onPrepareFailure threw another exception. But it really doesn't make
any difference: the mentioned type of FTP related errors bypass redelivery
and goes directly to the dead letter channel with or without the
onPrepareFailure processor.)
Cheers,
Klaus
--
View this message in context: http://camel.465427.n5.nabble.com/Is-GenericFileOperationFailedException-handled-differently-than-other-exceptions-tp5789339p5789397.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Is GenericFileOperationFailedException handled differently than other exceptions?
Posted by John Taylor <jt...@gmail.com>.
Klaus Johansen <kl...@gmail.com> writes:
> Hi all,
>
> I have used a lot time configuring a deadLetterChannel errorHandler and a
> custom PollingConsumerPollStrategy for our PollingConsumers (mostly sFTP
> and FTPS).
>
> In combination with "throwExceptionOnConnectFailed=true" and
> "consumer.bridgeErrorHandler=true" we have a generally robust setup, where
> the Camel error-handler handles both consumer and producer errors. It does
> redelivery and sends an alert when exhausted. This allows us to handle e.g.
> connection timeouts to FTP servers in both consumer and producer in the
> same controlled way.
>
> Recently we had several situations with FTP error code 150 (e.g. socket
> timeouts) which ends up in an org.apache.camel.component.file.
> GenericFileOperationFailedException.
>
> Similar if the FTP server replies with "550 File not found, access denied."
> it results in the following exception:
>
> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
> retrieve file: RemoteFile[TestFil.XML] from: Endpoint[ftp://127.0.0.1:21/?
> autoCreate=false&binary=true&consumer.bridgeErrorHandler=
> true&delete=true&disconnect=true&doneFileName=%24%7Bfile%3Aname%7D.SEM&
> <ftp://127.0.0.1/?autoCreate=false&binary=true&consumer.bridgeErrorHandler=true&delete=true&disconnect=true&doneFileName=%24%7Bfile%3Aname%7D.SEM&>
> include=%28%3Fi%29.*%5C.XML&localWorkDirectory=c%3A%5Ctest%5Cftpworkdir&
> maximumReconnectAttempts=0&passiveMode=true&password=xxxxxx&pollStrategy=%
> 23poll30times&readLock=none&reconnectDelay=240000&
> scheduler=spring&scheduler.cron=30+*+*+*+*+MON-FRI&stepwise=false&
> throwExceptionOnConnectFailed=true&username=testuser]
>
> According to the log, the GenericFileOperationFailedException is directly
> followed by the configured "onPrepareFailure" processor and then handled by
> the "FatalFallbackErrorHandler". As a result the "commit" method of the
> custom PollingConsumerPollStrategy is invoked and no redelivery attempted.
>
> Is GenericFileOperationFailedException handled differently by the Camel
> Error Handler than other exceptions? - Or is it a bug?
>
> If working as designed: Can I somehow configure an onException section or
> something different to override this behaviour?
>
> I'm currently using camel 2.17.3.
>
> Best regards,
> Klaus
Re: Is GenericFileOperationFailedException handled differently than
other exceptions?
Posted by Klaus Johansen <kl...@gmail.com>.
Hi again,
I'm actually trying to recover from "java.net.SocketTimeoutException: Read
timed out" during download of files via FTP. Any ideas how to ensure
"redelivery" (or start another poll immediately) if download fails?
I done some more debugging to find out why some FTP consumer errors are
exhausted immediately and thereby completely bypass DeadLetterChannel
redelivery.
I'm using consumer.bridgeErrorHandler=true and the DeadLetterChannel
redelivery works perfectly for errors during FTP polling.
In these problem cases an exchange has already been created (trace says
"UnitOfWork created for ExchangeId..." ) when the error happens. Actually
the error happens when camel is trying to add data to the exchange.
Is there some kind of gap in the exception handling between FTP polling /
exchange creation and until the exchange has been populated with data (and
thereby being ready to go though the exchange life cycle inside Camel
error-handling boundaries)?
Since (data) sockets timeouts can be hard to reproduce I seems like same
situation occur if I change permission on the ftp server and disallows
download:
2016-10-31 10:34:30.425 TRACE 11780 --- [test/]
o.a.c.c.file.remote.FtpOperations : retrieveFile(test/TestFil2.XML)
2016-10-31 10:34:30.426 DEBUG 11780 --- [test/]
org.apache.camel.util.FileUtil : Retrying attempt 0 to delete
file: c:\camel-test\ftpworkdir\TestFil2.XML.inprogress
2016-10-31 10:34:30.427 DEBUG 11780 --- [test/]
org.apache.camel.util.FileUtil : Tried 1 to delete file:
c:\camel-test\ftpworkdir\TestFil2.XML.inprogress with result: true
2016-10-31 10:34:30.427 DEBUG 11780 --- [test/]
org.apache.camel.util.FileUtil : Retrying attempt 0 to delete
file: c:\camel-test\ftpworkdir\TestFil2.XML
2016-10-31 10:34:30.428 DEBUG 11780 --- [test/]
org.apache.camel.util.FileUtil : Tried 1 to delete file:
c:\camel-test\ftpworkdir\TestFil2.XML with result: true
2016-10-31 10:34:30.429 TRACE 11780 --- [test/]
o.a.c.c.file.remote.FtpOperations : Client retrieveFile:
test/TestFil2.XML
2016-10-31 10:34:30.440 DEBUG 11780 --- [test/]
o.a.c.c.file.remote.FtpOperations : Retrieve file to local work file
result: false
2016-10-31 10:34:30.445 TRACE 11780 --- [test/]
org.apache.camel.impl.MDCUnitOfWork : UnitOfWork created for
ExchangeId: ID-HOSTNAME-50073-1477906450300-0-1 with
Exchange[ID-HOSTNAME-50073-1477906450300-0-1]
2016-10-31 10:34:30.450 TRACE 11780 --- [test/]
org.apache.camel.impl.MDCUnitOfWork : UnitOfWork beforeRoute:
testRoute.cron for ExchangeId: ID-HOSTNAME-50073-1477906450300-0-1 with
Exchange[ID-HOSTNAME-50073-1477906450300-0-1]
2016-10-31 10:34:30.454 TRACE 11780 --- [test/]
o.a.c.processor.CamelInternalProcessor : Processing exchange for
exchangeId: ID-HOSTNAME-50073-1477906450300-0-1 ->
Exchange[ID-HOSTNAME-50073-1477906450300-0-1]
2016-10-31 10:34:30.454 TRACE 11780 --- [test/]
o.a.c.processor.CamelInternalProcessor : Processing exchange for
exchangeId: ID-HOSTNAME-50073-1477906450300-0-1 ->
Exchange[ID-HOSTNAME-50073-1477906450300-0-1]
2016-10-31 10:34:30.458 TRACE 11780 --- [test/]
o.a.camel.processor.DeadLetterChannel : isRunAllowed() -> true (Run
allowed if we are not stopped/stopping)
2016-10-31 10:34:30.458 TRACE 11780 --- [test/]
o.a.c.p.e.DefaultExceptionPolicyStrategy : Finding best suited exception
policy for thrown exception
org.apache.camel.component.file.GenericFileOperationFailedException
2016-10-31 10:34:30.458 TRACE 11780 --- [test/]
o.a.c.p.e.DefaultExceptionPolicyStrategy : Finding best suited exception
policy for thrown exception
org.apache.camel.component.file.GenericFileOperationFailedException
2016-10-31 10:34:30.458 TRACE 11780 --- [test/]
o.a.c.p.e.DefaultExceptionPolicyStrategy : Found 0 candidates
2016-10-31 10:34:30.461 INFO 11780 --- [test/] d.b.m.d.h.r.s.TestRouteRoute
: Failed delivery for (MessageId: ID-HOSTNAME-50073-1477906450300-0-2 on
ExchangeId: ID-HOSTNAME-50073-1477906450300-0-1). On delivery attempt: 0
caught: org.apache.camel.component.file.GenericFileOperationFailedException:
Cannot retrieve file: RemoteFile[TestFil2.XML] from:
Endpoint[ftp://lokalftp:21/test/?autoCreate=false&binary=true&consumer.bridgeErrorHandler=true&delete=true&disconnect=true&doneFileName=%24%7Bfile%3Aname%7D.SEM&filter=%23isBankDayFilter&include=%28%3Fi%29.*%5C.XML&localWorkDirectory=c%3A%5Ccamel-test%5Ccamel-test-hdp%5Cworkroot%5CtestRoute%5Cftpworkdir&maximumReconnectAttempts=0&passiveMode=true&password=xxxxxx&pollStrategy=%23poll30times&readLock=none&reconnectDelay=240000&scheduler=spring&scheduler.cron=0%2F30+*+*+*+*+MON-FRI&stepwise=false&streamDownload=true&throwExceptionOnConnectFailed=true&username=testuser]
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
retrieve file: RemoteFile[TestFil2.XML] from:
Endpoint[ftp://lokalftp:21/test/?autoCreate=false&binary=true&consumer.bridgeErrorHandler=true&delete=true&disconnect=true&doneFileName=%24%7Bfile%3Aname%7D.SEM&filter=%23isBankDayFilter&include=%28%3Fi%29.*%5C.XML&localWorkDirectory=c%3A%5Ccamel-test%5Ccamel-test-hdp%5Cworkroot%5CtestRoute%5Cftpworkdir&maximumReconnectAttempts=0&passiveMode=true&password=xxxxxx&pollStrategy=%23poll30times&readLock=none&reconnectDelay=240000&scheduler=spring&scheduler.cron=0%2F30+*+*+*+*+MON-FRI&stepwise=false&streamDownload=true&throwExceptionOnConnectFailed=true&username=testuser]
at
org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:430)
at
org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:137)
at
org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:226)
at
org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:190)
at
org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:175)
at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:102)
at
org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at
org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2016-10-31 10:34:30.461 TRACE 11780 --- [test/]
o.a.camel.processor.DeadLetterChannel : This exchange is marked as
redelivery exhausted: Exchange[ID-HOSTNAME-50073-1477906450300-0-1]
2016-10-31 10:34:30.461 TRACE 11780 --- [test/]
o.a.camel.processor.DeadLetterChannel : isRedeliveryAllowed() -> true (we
are not stopping/stopped)
2016-10-31 10:34:30.463 TRACE 11780 --- [test/]
o.a.camel.processor.DeadLetterChannel : Failure processor
FatalFallbackErrorHandler[sendTo(Endpoint[direct://exhausted] InOnly)] is
processing Exchange: Exchange[ID-HOSTNAME-50073-1477906450300-0-1]
Regards,
Klaus
--
View this message in context: http://camel.465427.n5.nabble.com/Is-GenericFileOperationFailedException-handled-differently-than-other-exceptions-tp5789339p5789513.html
Sent from the Camel - Users mailing list archive at Nabble.com.