You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Hilde <hi...@yahoo.de> on 2012/09/07 16:12:04 UTC
ftp: with option localWorkDirectory
Hello everybody!
Because we have to transfer large files from ftp server that caused
outOfMemory exceptions I have read that the ftp component option
*localWorkDirectory *can remedy this.
However could anybody please explain me why now some downloaded files via
ftp: component and option *localWorkDirectory *could not be saved randomly
into the final directory? There must be a logical reasoning in the camel
route for that failure.
Here comes the route:
from(ftp://00010@mecsoc-vm28/?noop=true&startingDirectoryMustExist=true&
sendEmptyMessageWhenIdle=true&maximumReconnectAttempts=4&reconnectDelay=10000&
binary=true&password=?76Hko*fPQ&localWorkDirectory=c:/temp&filter=#imageFileFilter).routeId(routeId)
.onException(Exception.class).handled(true)
.to("log:errorLog?level=ERROR&showAll=true&multiline=true")
.setHeader(DN1CamelHeader.ANY_MESSAGE, constant("Error occured while " +
"transfer was in progress!"))
.bean(transferPayloads, "payloadTransferFailure")
.to("activemq:topic:" + JmsDestinations.TOPIC_DASHBOARD_AUDITLOG)
.setBody(exceptionMessage())
.bean(stopRoute)
.end()
.onCompletion()
.setHeader(DN1CamelHeader.TRANSFER_PAYLOAD_TYPE, constant("STATUS"))
.to("seda:" + SedaDirectDestinations.SEDA_INFOS_WHILE_TRANSFER)
.choice()
.when(header("CamelBatchComplete").isEqualTo(Boolean.TRUE))
.setHeader(DN1CamelHeader.TRANSFER_PAYLOAD_TYPE, constant("COMPLETE"))
.setHeader(DN1CamelHeader.ANY_MESSAGE, simple("${header." +
DN1CamelHeader.TRANSFER_TYPE + "} transfer
completed!"))
.to("seda:" + SedaDirectDestinations.SEDA_INFOS_WHILE_TRANSFER)
.bean(stopRoute)
.end()
.end()
.setHeader(DN1CamelHeader.PRODUCTION, constant(production))
.setHeader(DN1CamelHeader.PROCESS, constant(process))
.setHeader(DN1CamelHeader.TRANSFER_TYPE, constant(dataType))
.choice()
.when(body().isNull())
.setHeader(DN1CamelHeader.TRANSFER_PAYLOAD_TYPE, constant("COMPLETE"))
.setHeader(DN1CamelHeader.ANY_MESSAGE,
simple("Directory is empty and transfer is
going to finish!"))
.to("seda:" + SedaDirectDestinations.SEDA_INFOS_WHILE_TRANSFER)
.bean(stopRoute)
.otherwise()
.choice()
.when(header("CamelBatchIndex").isEqualTo(Integer.valueOf(0)))
.setHeader(DN1CamelHeader.TRANSFER_PAYLOAD_TYPE, constant("OVERVIEW"))
.to("seda:" + SedaDirectDestinations.SEDA_INFOS_WHILE_TRANSFER)
.end()
.setHeader(DN1CamelHeader.TRANSFER_PAYLOAD_TYPE, constant("FILE"))
.to("seda:" + SedaDirectDestinations.SEDA_INFOS_WHILE_TRANSFER)
.to("file:" + receiptDir)
.end();
Inside the log file I can explore the following stacktrace:
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
store file: c:\dn1\inputDir\010.exe
at
org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:246)[118:org.apache.camel.camel-core:2.10.0]
at
org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:249)[118:org.apache.camel.camel-core:2.10.0]
at
org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:151)[118:org.apache.camel.camel-core:2.10.0]
Caused by: org.apache.camel.InvalidPayloadException: No body available of
type: java.io.InputStream but has value: GenericFile[010.exe] of type:
org.apache.camel.component.file.remote.RemoteFile on: 010.exe. Caused by:
Error during type conversion from type: java.io.File to the required type:
java.io.InputStream with value c:\temp\010.exe due
java.io.FileNotFoundException: c:\temp\010.exe (Das System kann die
angegebene Datei nicht finden). Exchange[010.exe]. Caused by:
[org.apache.camel.TypeConversionException - Error during type conversion
from type: java.io.File to the required type: java.io.InputStream with value
c:\temp\010.exe due java.io.FileNotFoundException: c:\temp\010.exe (Das
System kann die angegebene Datei nicht finden)]
at
org.apache.camel.impl.MessageSupport.getMandatoryBody(MessageSupport.java:101)[118:org.apache.camel.camel-core:2.10.0]
at
org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:237)[118:org.apache.camel.camel-core:2.10.0]
... 70 more
Caused by: org.apache.camel.TypeConversionException: Error during type
conversion from type: java.io.File to the required type: java.io.InputStream
with value c:\temp\010.exe due java.io.FileNotFoundException:
c:\temp\010.exe (Das System kann die angegebene Datei nicht finden)
at
org.apache.camel.impl.converter.BaseTypeConverterRegistry.convertTo(BaseTypeConverterRegistry.java:126)[118:org.apache.camel.camel-core:2.10.0]
at
org.apache.camel.core.osgi.OsgiTypeConverter.convertTo(OsgiTypeConverter.java:102)[120:org.apa
Thanks
Hilde
--
View this message in context: http://camel.465427.n5.nabble.com/ftp-with-option-localWorkDirectory-tp5718783.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: ftp: with option localWorkDirectory
Posted by Hilde <hi...@yahoo.de>.
Hello Pontus!
Thanks for your help and sorry for answering late.
Your suggestion to change from seda: to direct: made no differences. But we
figured out that
on Linux there are no troubles any more as prevously tested on a Windows
machine.
Cheers
Hilde
--
View this message in context: http://camel.465427.n5.nabble.com/ftp-with-option-localWorkDirectory-tp5718783p5721396.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: ftp: with option localWorkDirectory
Posted by Pontus Ullgren <ul...@gmail.com>.
Hello Hilde,
I would not say that it is not allowed to have anything between the to and from.
You can still have processors in between and you can send the request
to direct endpoints.
To me it seems like the first route (the one starting with FTP
endpoint) considers that the unit of work is done when the message is
passed to the seda endpoint, and there for the temporary local file is
removed.
So it does not exists when the route starting with the seda endpoint
tries to accesses it.
If you instead use a direct endpoint it will be executed in the same
thread (and unit of work) so the file will be left in the local work
dir until both routes are done.
It is possible that you can also use a seda endpoint with
"waitForTaskToComplete=Allways".
I stumbled on the same problem when I changed a set of routes that
looked something like this.
---
from("ftp:..").process(someprocessor).process(someother).to("direct:a");
from("direct:a").to("file://finaldest/");
---
To use seda instead:
---
from("ftp:..").process(someprocessor).process(someother).to("seda:a");
from("seda:a").to("file://finaldest/");
---
The first worked and the later gave the same error you are having.
I have a test case for this "problem" on my other computer I can push
it to a gist tomorrow if you are interested.
Best regards
Pontus Ullgren
On Mon, Sep 10, 2012 at 2:44 PM, Hilde <hi...@yahoo.de> wrote:
> Hello *!
>
> My assumption is that when using the option /localWorkDirectory/ it is not
> allowed that anything can be placed between from() and to() as the example
> will show:
>
> from("ftp://someone@someserver.com?password=secret&localWorkDirectory=/tmp").to("file://inbox");
>
> Though my case above has several /seda:/ components inside
>
> Can anybody confirm this?
>
> Hilde
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/ftp-with-option-localWorkDirectory-tp5718783p5719036.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: ftp: with option localWorkDirectory
Posted by Hilde <hi...@yahoo.de>.
Hello *!
My assumption is that when using the option /localWorkDirectory/ it is not
allowed that anything can be placed between from() and to() as the example
will show:
from("ftp://someone@someserver.com?password=secret&localWorkDirectory=/tmp").to("file://inbox");
Though my case above has several /seda:/ components inside
Can anybody confirm this?
Hilde
--
View this message in context: http://camel.465427.n5.nabble.com/ftp-with-option-localWorkDirectory-tp5718783p5719036.html
Sent from the Camel - Users mailing list archive at Nabble.com.