You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by "Pitre, Russell" <RP...@shawmut.com> on 2010/03/22 19:09:09 UTC
quartz + pollEnrich +sftp ==> 'move' parameter ignored
I'm using the Quartz scheduler component in conjunction with pollEnrich
to download a file at a specific time of day. It looks like the 'move'
parameter is being ignored, am I missing something?
public void configure() throws Exception {
from("quartz://myGroup/DownloadFilesTrigger?cron=0+03+14+*+*+?").
pollEnrich("sftp://" + sourceLocation() + "?password=" +
password + "&move=.done", 0).
to("file:" + INBOX_LOCATION );
}
Thanks,
Rusell Pitre
Re: quartz + pollEnrich +sftp ==> 'move' parameter ignored
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
If you want to store to a file directory directly from FTP you can use
localWorkDirectory option, while causes Camel to stream directly to a
temp file in the directory. But then if you do .to("file:xxx")
afterwards then Camel can do a fast IO move operation. Then I assume
the SFTP consumer would have better change doing the move operation
afterwards.
See more here
http://camel.apache.org/ftp2.html
On Mon, Mar 22, 2010 at 8:29 PM, Pitre, Russell <RP...@shawmut.com> wrote:
> Thanks for the quick response! No luck though. I know the sftp uri works because i can move it to the 'from' and all is well. Here's the stacktrace:
>
> ----------------------------------------------------------------
> from("quartz://myGroup/DownloadFilesTrigger?cron=0+23+15+*+*+?").
> pollEnrich("sftp://" + sourceLocation() + "?password=" + password + "&move=.done").
> to("file:" + INBOX_LOCATION );
> ----------------------------------------------------------------
>
>
>
> efaultQuartzScheduler_Worker-1] EventLog INFO Event: 67985393-5224-4834-85cc-07dbe5ff83ae exchange created: Exchange[Message: [Body is null]]
> [efaultQuartzScheduler_Worker-1] EventLog INFO Event: 67985393-5224-4834-85cc-07dbe5ff83ae exchange Exchange[Message: GenericFile[locations_test.xml]] sent to: file://\\dev-data\Intranet\adp\hris took: 28 ms.
> [efaultQuartzScheduler_Worker-1] EventLog INFO Event: 67985393-5224-4834-85cc-07dbe5ff83ae exchange completed: Exchange[Message: GenericFile[locations_test.xml]]
> [ove=.done&password=xxxxx] ilePollingConsumerPollStrategy WARN Trying to recover by disconnecting from remote server forcing a re-connect at next poll: sftp://pADPRsftpshawmutdes@server:22
> [efaultQuartzScheduler_Worker-1] GenericFileOnCompletion ERROR Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: /home/pADPRsftpshawmutdes]
> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: /home/pADPRsftpshawmutdes
> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:289)
> at org.apache.camel.component.file.remote.SftpOperations.buildDirectory(SftpOperations.java:244)
> at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.renameFile(GenericFileRenameProcessStrategy.java:85)
> at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:72)
> at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:122)
> at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
> at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
> at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:145)
> at org.apache.camel.processor.UnitOfWorkProcessor.done(UnitOfWorkProcessor.java:82)
> at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:71)
> at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
> at org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:41)
> at org.apache.camel.component.quartz.QuartzEndpoint.onJobExecute(QuartzEndpoint.java:104)
> at org.apache.camel.component.quartz.CamelJob.execute(CamelJob.java:33)
> at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
> at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:534)
> Caused by: 4:
> at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:1749)
> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:269)
> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:287)
> ... 16 more
> Caused by: java.io.IOException: inputstream is closed
> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2325)
> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2349)
> at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:1730)
> ... 18 more
> [ove=.done&password=xxxxx] ilePollingConsumerPollStrategy WARN Consumer Consumer[sftp://pADPRsftpshawmutdes@server:22/OUTBOUND?move=.done&password=xxxxx] could not poll endpoint: sftp://pADPRsftpshawmutdes@server:22/OUTBOUND?move=.done&password=xxxxx caused by: Cannot list directory: OUTBOUND
> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot list directory: OUTBOUND
> at org.apache.camel.component.file.remote.SftpOperations.listFiles(SftpOperations.java:314)
> at org.apache.camel.component.file.remote.SftpConsumer.pollDirectory(SftpConsumer.java:50)
> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:77)
> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:98)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: 2: /home/pADPRsftpshawmutdes/OUTBOUND/.done/OUTBOUND is not a valid file path
> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2287)
> at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:1739)
> at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:1756)
> at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1190)
> at org.apache.camel.component.file.remote.SftpOperations.listFiles(SftpOperations.java:305)
> ... 12 more
>
>
>
>
>
> -----Original Message-----
> From: Claus Ibsen [mailto:claus.ibsen@gmail.com]
> Sent: Monday, March 22, 2010 11:11 AM
> To: users@camel.apache.org
> Subject: Re: quartz + pollEnrich +sftp ==> 'move' parameter ignored
>
> Hi
>
> Yeah that was not supported. Its implemented in Camel 2.3.
>
> You can try the 2.3-SNAPSHOT
> http://camel.apache.org/download.html
>
>
> On Mon, Mar 22, 2010 at 7:09 PM, Pitre, Russell <RP...@shawmut.com> wrote:
>> I'm using the Quartz scheduler component in conjunction with pollEnrich
>> to download a file at a specific time of day. It looks like the 'move'
>> parameter is being ignored, am I missing something?
>>
>>
>>
>>
>>
>> public void configure() throws Exception {
>>
>>
>> from("quartz://myGroup/DownloadFilesTrigger?cron=0+03+14+*+*+?").
>>
>> pollEnrich("sftp://" + sourceLocation() + "?password=" +
>> password + "&move=.done", 0).
>>
>> to("file:" + INBOX_LOCATION );
>>
>> }
>>
>>
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Rusell Pitre
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
RE: quartz + pollEnrich +sftp ==> 'move' parameter ignored
Posted by "Pitre, Russell" <RP...@shawmut.com>.
Thanks for the quick response! No luck though. I know the sftp uri works because i can move it to the 'from' and all is well. Here's the stacktrace:
----------------------------------------------------------------
from("quartz://myGroup/DownloadFilesTrigger?cron=0+23+15+*+*+?").
pollEnrich("sftp://" + sourceLocation() + "?password=" + password + "&move=.done").
to("file:" + INBOX_LOCATION );
----------------------------------------------------------------
efaultQuartzScheduler_Worker-1] EventLog INFO Event: 67985393-5224-4834-85cc-07dbe5ff83ae exchange created: Exchange[Message: [Body is null]]
[efaultQuartzScheduler_Worker-1] EventLog INFO Event: 67985393-5224-4834-85cc-07dbe5ff83ae exchange Exchange[Message: GenericFile[locations_test.xml]] sent to: file://\\dev-data\Intranet\adp\hris took: 28 ms.
[efaultQuartzScheduler_Worker-1] EventLog INFO Event: 67985393-5224-4834-85cc-07dbe5ff83ae exchange completed: Exchange[Message: GenericFile[locations_test.xml]]
[ove=.done&password=xxxxx] ilePollingConsumerPollStrategy WARN Trying to recover by disconnecting from remote server forcing a re-connect at next poll: sftp://pADPRsftpshawmutdes@server:22
[efaultQuartzScheduler_Worker-1] GenericFileOnCompletion ERROR Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current directory to: /home/pADPRsftpshawmutdes]
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change current directory to: /home/pADPRsftpshawmutdes
at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:289)
at org.apache.camel.component.file.remote.SftpOperations.buildDirectory(SftpOperations.java:244)
at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.renameFile(GenericFileRenameProcessStrategy.java:85)
at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:72)
at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:122)
at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:145)
at org.apache.camel.processor.UnitOfWorkProcessor.done(UnitOfWorkProcessor.java:82)
at org.apache.camel.processor.UnitOfWorkProcessor.processNext(UnitOfWorkProcessor.java:71)
at org.apache.camel.processor.DelegateProcessor.process(DelegateProcessor.java:48)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:67)
at org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:41)
at org.apache.camel.component.quartz.QuartzEndpoint.onJobExecute(QuartzEndpoint.java:104)
at org.apache.camel.component.quartz.CamelJob.execute(CamelJob.java:33)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:534)
Caused by: 4:
at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:1749)
at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:269)
at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:287)
... 16 more
Caused by: java.io.IOException: inputstream is closed
at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2325)
at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2349)
at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:1730)
... 18 more
[ove=.done&password=xxxxx] ilePollingConsumerPollStrategy WARN Consumer Consumer[sftp://pADPRsftpshawmutdes@server:22/OUTBOUND?move=.done&password=xxxxx] could not poll endpoint: sftp://pADPRsftpshawmutdes@server:22/OUTBOUND?move=.done&password=xxxxx caused by: Cannot list directory: OUTBOUND
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot list directory: OUTBOUND
at org.apache.camel.component.file.remote.SftpOperations.listFiles(SftpOperations.java:314)
at org.apache.camel.component.file.remote.SftpConsumer.pollDirectory(SftpConsumer.java:50)
at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:77)
at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:98)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: 2: /home/pADPRsftpshawmutdes/OUTBOUND/.done/OUTBOUND is not a valid file path
at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2287)
at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:1739)
at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:1756)
at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1190)
at org.apache.camel.component.file.remote.SftpOperations.listFiles(SftpOperations.java:305)
... 12 more
-----Original Message-----
From: Claus Ibsen [mailto:claus.ibsen@gmail.com]
Sent: Monday, March 22, 2010 11:11 AM
To: users@camel.apache.org
Subject: Re: quartz + pollEnrich +sftp ==> 'move' parameter ignored
Hi
Yeah that was not supported. Its implemented in Camel 2.3.
You can try the 2.3-SNAPSHOT
http://camel.apache.org/download.html
On Mon, Mar 22, 2010 at 7:09 PM, Pitre, Russell <RP...@shawmut.com> wrote:
> I'm using the Quartz scheduler component in conjunction with pollEnrich
> to download a file at a specific time of day. It looks like the 'move'
> parameter is being ignored, am I missing something?
>
>
>
>
>
> public void configure() throws Exception {
>
>
> from("quartz://myGroup/DownloadFilesTrigger?cron=0+03+14+*+*+?").
>
> pollEnrich("sftp://" + sourceLocation() + "?password=" +
> password + "&move=.done", 0).
>
> to("file:" + INBOX_LOCATION );
>
> }
>
>
>
>
>
> Thanks,
>
>
>
> Rusell Pitre
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: quartz + pollEnrich +sftp ==> 'move' parameter ignored
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Yeah that was not supported. Its implemented in Camel 2.3.
You can try the 2.3-SNAPSHOT
http://camel.apache.org/download.html
On Mon, Mar 22, 2010 at 7:09 PM, Pitre, Russell <RP...@shawmut.com> wrote:
> I'm using the Quartz scheduler component in conjunction with pollEnrich
> to download a file at a specific time of day. It looks like the 'move'
> parameter is being ignored, am I missing something?
>
>
>
>
>
> public void configure() throws Exception {
>
>
> from("quartz://myGroup/DownloadFilesTrigger?cron=0+03+14+*+*+?").
>
> pollEnrich("sftp://" + sourceLocation() + "?password=" +
> password + "&move=.done", 0).
>
> to("file:" + INBOX_LOCATION );
>
> }
>
>
>
>
>
> Thanks,
>
>
>
> Rusell Pitre
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus