You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by rosenbergj <Ro...@crlcorp.com> on 2012/02/02 17:43:25 UTC
File Consumer Max File Size
I've run into the a problem that if the File Consumer tries to process a
large file (excess of 200MB) I run out of Heap Space.
Is there a way to tell the file2 Endpoint to ignore files over a certain
size?
Does the File Endpoint serialize the data into the message or does it just
provide a pointer to the file? If the latter, is there a way to
programatically inspect the size of the body of the message before loading
into a byte array?
I have the same questions for the ftp2 endpoint.
Thank you,
~Justin
--
View this message in context: http://camel.465427.n5.nabble.com/File-Consumer-Max-File-Size-tp5451246p5451246.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Consumer Max File Size
Posted by Claus Ibsen <cl...@gmail.com>.
On Thu, Feb 2, 2012 at 6:16 PM, rosenbergj <Ro...@crlcorp.com> wrote:
> Thank you for the quick reply. Your dedication to supporting Camel is much appreciated.
>
>
>
> That helps prevent the file from being processed. Is there any DSL magic I could do so that I could be alerted that the file is there, but not process it within camel?
>
>
>
> If I cannot do this in the same route, could I use another route that had the reverse filter, but somehow not load the file and just delete/move it?
>
I suggest to use a content based router then (eg its just like if ... else ...)
<choice>
<when>
<simple>${file:size} < 200000000</simple>
... process the small file here
</when>
<otherwise>
.. send alert we got a big file
</otherwise>
</route>
>
>
> ~Justin
>
>
>
> From: Claus Ibsen-2 [via Camel] [mailto:ml-node+s465427n5451319h1@n5.nabble.com]
> Sent: Thursday, February 02, 2012 11:06 AM
> To: Justin Rosenberg
> Subject: Re: File Consumer Max File Size
>
>
>
> Hi
>
> You can use a filter option, and implement the interface, where you
> return true|false.
> Then you can check the file size if its > 200mb and return false.
>
> See the section _Filter using
> org.apache.camel.component.file.GenericFileFilter_ at
> http://camel.apache.org/file2
>
>
> The file size is also provided in a CamelFileSize header, so you can
> also use the Filter EIP or Content Based Router EIP to route depending
> on the file size.
>
> Then you can use the simple language as a predicate
> <route>
> ...
> <filter>
> <simple>${file:size} < 2000000</simple>
> ... only process small files here
> </filter>
> </route>
>
>
> Notice the file size is in bytes, so insert a value that matches the 200mb
>
>
>
> On Thu, Feb 2, 2012 at 5:43 PM, rosenbergj <[hidden email]> wrote:
>
>
>> I've run into the a problem that if the File Consumer tries to process a
>> large file (excess of 200MB) I run out of Heap Space.
>>
>> Is there a way to tell the file2 Endpoint to ignore files over a certain
>> size?
>> Does the File Endpoint serialize the data into the message or does it just
>> provide a pointer to the file? If the latter, is there a way to
>> programatically inspect the size of the body of the message before loading
>> into a byte array?
>> I have the same questions for the ftp2 endpoint.
>>
>> Thank you,
>>
>> ~Justin
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.com/File-Consumer-Max-File-Size-tp5451246p5451246.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>
>
> --
> Claus Ibsen
> -----------------
> FuseSource
> Email: [hidden email]
> Web: http://fusesource.com
> Twitter: davsclaus, fusenews
> Blog: http://davsclaus.blogspot.com/
> Author of Camel in Action: http://www.manning.com/ibsen/
>
>
>
> ________________________________
>
> If you reply to this email, your message will be added to the discussion below:
>
> http://camel.465427.n5.nabble.com/Re-File-Consumer-Max-File-Size-tp5451319p5451319.html
>
> To start a new topic under Camel - Users, email ml-node+s465427n465428h80@n5.nabble.com
> To unsubscribe from Camel - Users, click here <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=465428&code=Um9zZW5iZXJnSkBjcmxjb3JwLmNvbXw0NjU0Mjh8LTUwMDM1NDcz> .
> NAML <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
> CONFIDENTIALITY NOTICE:
> The information in this message, and any attachment, is intended for the
> sole use of the individual and entity to whom it is addressed. This
> information may be privileged, confidential, and protected from
> disclosure. If you are not the intended recipient you are hereby notified
> that you have received this communication in error and that any review,
> disclosure, dissemination, distribution or copying of it, or its contents,
> is strictly prohibited. If you think that you have received this message
> in error please notify the sender and destroy all copies of this
> communication and any attachments. Thank you.
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Re-File-Consumer-Max-File-Size-tp5451319p5451352.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
RE: File Consumer Max File Size
Posted by rosenbergj <Ro...@crlcorp.com>.
Thank you for the quick reply. Your dedication to supporting Camel is much appreciated.
That helps prevent the file from being processed. Is there any DSL magic I could do so that I could be alerted that the file is there, but not process it within camel?
If I cannot do this in the same route, could I use another route that had the reverse filter, but somehow not load the file and just delete/move it?
~Justin
From: Claus Ibsen-2 [via Camel] [mailto:ml-node+s465427n5451319h1@n5.nabble.com]
Sent: Thursday, February 02, 2012 11:06 AM
To: Justin Rosenberg
Subject: Re: File Consumer Max File Size
Hi
You can use a filter option, and implement the interface, where you
return true|false.
Then you can check the file size if its > 200mb and return false.
See the section _Filter using
org.apache.camel.component.file.GenericFileFilter_ at
http://camel.apache.org/file2
The file size is also provided in a CamelFileSize header, so you can
also use the Filter EIP or Content Based Router EIP to route depending
on the file size.
Then you can use the simple language as a predicate
<route>
...
<filter>
<simple>${file:size} < 2000000</simple>
... only process small files here
</filter>
</route>
Notice the file size is in bytes, so insert a value that matches the 200mb
On Thu, Feb 2, 2012 at 5:43 PM, rosenbergj <[hidden email]> wrote:
> I've run into the a problem that if the File Consumer tries to process a
> large file (excess of 200MB) I run out of Heap Space.
>
> Is there a way to tell the file2 Endpoint to ignore files over a certain
> size?
> Does the File Endpoint serialize the data into the message or does it just
> provide a pointer to the file? If the latter, is there a way to
> programatically inspect the size of the body of the message before loading
> into a byte array?
> I have the same questions for the ftp2 endpoint.
>
> Thank you,
>
> ~Justin
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-Consumer-Max-File-Size-tp5451246p5451246.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
FuseSource
Email: [hidden email]
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
________________________________
If you reply to this email, your message will be added to the discussion below:
http://camel.465427.n5.nabble.com/Re-File-Consumer-Max-File-Size-tp5451319p5451319.html
To start a new topic under Camel - Users, email ml-node+s465427n465428h80@n5.nabble.com
To unsubscribe from Camel - Users, click here <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=465428&code=Um9zZW5iZXJnSkBjcmxjb3JwLmNvbXw0NjU0Mjh8LTUwMDM1NDcz> .
NAML <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
CONFIDENTIALITY NOTICE:
The information in this message, and any attachment, is intended for the
sole use of the individual and entity to whom it is addressed. This
information may be privileged, confidential, and protected from
disclosure. If you are not the intended recipient you are hereby notified
that you have received this communication in error and that any review,
disclosure, dissemination, distribution or copying of it, or its contents,
is strictly prohibited. If you think that you have received this message
in error please notify the sender and destroy all copies of this
communication and any attachments. Thank you.
--
View this message in context: http://camel.465427.n5.nabble.com/Re-File-Consumer-Max-File-Size-tp5451319p5451352.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Consumer Max File Size
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
You can use a filter option, and implement the interface, where you
return true|false.
Then you can check the file size if its > 200mb and return false.
See the section _Filter using
org.apache.camel.component.file.GenericFileFilter_ at
http://camel.apache.org/file2
The file size is also provided in a CamelFileSize header, so you can
also use the Filter EIP or Content Based Router EIP to route depending
on the file size.
Then you can use the simple language as a predicate
<route>
...
<filter>
<simple>${file:size} < 2000000</simple>
... only process small files here
</filter>
</route>
Notice the file size is in bytes, so insert a value that matches the 200mb
On Thu, Feb 2, 2012 at 5:43 PM, rosenbergj <Ro...@crlcorp.com> wrote:
> I've run into the a problem that if the File Consumer tries to process a
> large file (excess of 200MB) I run out of Heap Space.
>
> Is there a way to tell the file2 Endpoint to ignore files over a certain
> size?
> Does the File Endpoint serialize the data into the message or does it just
> provide a pointer to the file? If the latter, is there a way to
> programatically inspect the size of the body of the message before loading
> into a byte array?
> I have the same questions for the ftp2 endpoint.
>
> Thank you,
>
> ~Justin
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-Consumer-Max-File-Size-tp5451246p5451246.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/