You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by skasish <as...@hotmail.com> on 2014/02/21 09:13:39 UTC
out of heap space if remote FTP site has too many files to pick up
out of heap space if remote FTP site has too many files to pick up.
Is there configuration problem.
My simple code is given below
import org.slf4j.*;
import org.apache.camel.*;
import org.apache.camel.builder.*;
import org.apache.camel.main.Main;
import org.apache.camel.component.dataset.*;
public class FileDemoCamel extends Main {
static Logger LOG = LoggerFactory.getLogger(FileDemoCamel.class);
public static void main(String[] args) throws Exception {
FileDemoCamel main = new FileDemoCamel();
main.enableHangupSupport();
main.addRouteBuilder(createRouteBuilder());
main.bind("sampleGenerator", createDataSet());
main.run(args);
}
static RouteBuilder createRouteBuilder() {
return new RouteBuilder() {
public void configure() {
from("ftp://admin@localhost:2121/MF?noop=true&password=admin&passiveMode=true&maxMessagesPerPoll=10000&doneFileName=${file:name.noext}.TRG")
.to("file://D:/temp");
}
};
}
static DataSet createDataSet() {
return new SimpleDataSet();
}
}
--
View this message in context: http://camel.465427.n5.nabble.com/out-of-heap-space-if-remote-FTP-site-has-too-many-files-to-pick-up-tp5747716.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: out of heap space if remote FTP site has too many files to pick up
Posted by Claus Ibsen <cl...@gmail.com>.
You can limit the number of files to pickup in the poll,
http://camel.apache.org/file2
See the maxMessagesPerPoll option
On Fri, Feb 21, 2014 at 9:13 AM, skasish <as...@hotmail.com> wrote:
> out of heap space if remote FTP site has too many files to pick up.
>
> Is there configuration problem.
>
> My simple code is given below
>
> import org.slf4j.*;
> import org.apache.camel.*;
> import org.apache.camel.builder.*;
> import org.apache.camel.main.Main;
> import org.apache.camel.component.dataset.*;
>
> public class FileDemoCamel extends Main {
> static Logger LOG = LoggerFactory.getLogger(FileDemoCamel.class);
> public static void main(String[] args) throws Exception {
> FileDemoCamel main = new FileDemoCamel();
> main.enableHangupSupport();
> main.addRouteBuilder(createRouteBuilder());
> main.bind("sampleGenerator", createDataSet());
> main.run(args);
> }
> static RouteBuilder createRouteBuilder() {
> return new RouteBuilder() {
> public void configure() {
>
> from("ftp://admin@localhost:2121/MF?noop=true&password=admin&passiveMode=true&maxMessagesPerPoll=10000&doneFileName=${file:name.noext}.TRG")
> .to("file://D:/temp");
> }
> };
> }
> static DataSet createDataSet() {
> return new SimpleDataSet();
> }
> }
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/out-of-heap-space-if-remote-FTP-site-has-too-many-files-to-pick-up-tp5747716.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Make your Camel applications look hawt, try: http://hawt.io
Re: out of heap space if remote FTP site has too many files to pick
up
Posted by skasish <as...@hotmail.com>.
Following error has been occurred
13:39:40.609 [Camel (camel-1) thread #0 - ftp://admin@localhost:2121/MF]
DEBUG o.a.c.c.file.GenericFileProducer - Wrote
[D:\temp\GT_aplist_ghiiwb02_apache_20140111.lst] to
[Endpoint[file://D:/temp]]
13:39:40.609 [Camel (camel-1) thread #0 - ftp://admin@localhost:2121/MF]
DEBUG o.a.c.c.file.GenericFileOnCompletion - Done processing file:
RemoteFile[GT_aplist_ghiiwb02_apache_20140111.lst] using exchange:
Exchange[GT_aplist_ghiiwb02_apache_20140111.lst]
13:39:40.781 [Camel (camel-1) thread #0 - ftp://admin@localhost:2121/MF]
WARN o.a.c.c.file.remote.FtpConsumer - Consumer
FtpConsumer[ftp://admin@localhost:2121/MF?doneFileName=%24%7Bfile%3Aname.noext%7D.TRG&maxMessagesPerPoll=10000&noop=true&passiveMode=true&password=xxxxxx]
failed polling endpoint:
Endpoint[ftp://admin@localhost:2121/MF?doneFileName=%24%7Bfile%3Aname.noext%7D.TRG&maxMessagesPerPoll=10000&noop=true&passiveMode=true&password=xxxxxx].
Will try again at next poll. Caused by: [java.lang.OutOfMemoryError - Java
heap space]
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2786) ~[na:1.6.0_13]
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
~[na:1.6.0_13]
at org.apache.commons.net.io.Util.copyStream(Util.java:119)
~[commons-net-2.0.jar:2.0]
at org.apache.commons.net.ftp.FTPClient.retrieveFile(FTPClient.java:1286)
~[commons-net-2.0.jar:2.0]
at
org.apache.camel.component.file.remote.FtpOperations.retrieveFileToStreamInBody(FtpOperations.java:353)
~[camel-ftp-2.12.1.jar:2.12.1]
at
org.apache.camel.component.file.remote.FtpOperations.retrieveFile(FtpOperations.java:301)
~[camel-ftp-2.12.1.jar:2.12.1]
at
org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:362)
~[camel-core-2.12.1.jar:2.12.1]
at
org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:99)
~[camel-ftp-2.12.1.jar:2.12.1]
at
org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:201)
~[camel-core-2.12.1.jar:2.12.1]
at
org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:165)
~[camel-core-2.12.1.jar:2.12.1]
at
org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:187)
[camel-core-2.12.1.jar:2.12.1]
at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:114)
[camel-core-2.12.1.jar:2.12.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
[na:1.6.0_13]
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
[na:1.6.0_13]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
[na:1.6.0_13]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
[na:1.6.0_13]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
[na:1.6.0_13]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
[na:1.6.0_13]
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[na:1.6.0_13]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[na:1.6.0_13]
at java.lang.Thread.run(Thread.java:619) [na:1.6.0_13]
13:39:41.296 [Camel (camel-1) thread #0 - ftp://admin@localhost:2121/MF]
DEBUG o.a.c.c.file.remote.FtpConsumer - Took 0.000 seconds to poll: MF
13:39:41.811 [Camel (camel-1) thread #0 - ftp://admin@localhost:2121/MF]
DEBUG o.a.c.c.file.remote.FtpConsumer - Took 0.000 seconds to poll: MF
13:39:42.326 [Camel (camel-1) thread #0 - ftp://admin@localhost:2121/MF]
DEBUG o.a.c.c.file.remote.FtpConsumer - Took 0.000 seconds to poll: MF
--
View this message in context: http://camel.465427.n5.nabble.com/out-of-heap-space-if-remote-FTP-site-has-too-many-files-to-pick-up-tp5747716p5747717.html
Sent from the Camel - Users mailing list archive at Nabble.com.