You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by ebinsingh <eb...@VerizonWireless.com> on 2011/12/27 22:22:29 UTC
Using Zip dataformat
Hi, I am trying to use Zip / GZip data format to compress the output files
using Spring DSL implementation.
But looks like the files are not compressed. Not sure what mistake I am
making.
Appreciate your help.
<dataFormats>
<zip compressionLevel = "9" id = "zip"/>
</dataFormats>
<multicast stopOnException="true" executorServiceRef="mythreadpool">
<to uri="jms:queue:test_request_2"/>
<marshal ref="zip"/>
<to
uri="file:C:\\camelProject\\data\\outbox\\mars?fileName=${file:name}.zip"/>
</multicast>
Thanks & regards,
Ebe
--
View this message in context: http://camel.465427.n5.nabble.com/Using-Zip-dataformat-tp5103911p5103911.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Using Zip dataformat
Posted by Willem Jiang <wi...@gmail.com>.
I'm not sure what kind of message that you send to multicast, but as
you are using the multicast, you need to make sure the message body is
re-readable.
You can enable the stream caching[1] to cache the stream.
[1]http://camel.apache.org/stream-caching.html
On Wed Dec 28 05:22:29 2011, ebinsingh wrote:
> Hi, I am trying to use Zip / GZip data format to compress the output files
> using Spring DSL implementation.
> But looks like the files are not compressed. Not sure what mistake I am
> making.
>
> Appreciate your help.
>
> <dataFormats>
> <zip compressionLevel = "9" id = "zip"/>
> </dataFormats>
>
> <multicast stopOnException="true" executorServiceRef="mythreadpool">
> <to uri="jms:queue:test_request_2"/>
> <marshal ref="zip"/>
> <to
> uri="file:C:\\camelProject\\data\\outbox\\mars?fileName=${file:name}.zip"/>
> </multicast>
>
> Thanks& regards,
> Ebe
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Using-Zip-dataformat-tp5103911p5103911.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.javaeye.com (Chinese)
Twitter: willemjiang
Weibo: willemjiang
Re: Using Zip dataformat
Posted by ebinsingh <eb...@VerizonWireless.com>.
Apologies anf thanks.
But enabling Streamcache also did not solve the purpose.
Sorry to bother you, but is there something wrong with the below code.
Note: Input is a Large String.
<route streamCache="true">
<from ref="dispatch2" />
<choice>
<when>
<simple>${in.header.batchSize} == 2000 </simple>
<log message="processing batch of size == 2000"/>
<multicast stopOnException="true" executorServiceRef="mythreadpool"
parallelProcessing="true">
<to uri="jms:queue:test_request_2"/>
<marshal ref="zip"/>
<to
uri="file:C:\\camelProject\\data\\outbox\\mars?fileName=${file:name}.zip"/>
</multicast>
</when>
<when>
<simple>${in.header.batchSize} != 2000 </simple>
<log message="processing batch of size less than 2000"/>
<aggregate strategyRef="myAggregatorStrategy" completionSize="2000"
completionTimeout="3000">
<correlationExpression>
<constant>true</constant>
</correlationExpression>
<multicast stopOnException="true" executorServiceRef="mythreadpool"
parallelProcessing="true">
<to uri="jms:queue:test_request_2"/>
<marshal ref="zip"/>
<to
uri="file:C:\\camelProject\\data\\outbox\\mars?fileName=${file:name}.zip"/>
</multicast>
</aggregate>
</when>
</choice>
</route>
--
View this message in context: http://camel.465427.n5.nabble.com/Using-Zip-dataformat-tp5103911p5105513.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Using Zip dataformat
Posted by Claus Ibsen <cl...@gmail.com>.
On Wed, Dec 28, 2011 at 4:12 PM, ebinsingh
<eb...@verizonwireless.com> wrote:
> Yes, but according to the Camel documentation, it says "Multicast will
> implicitly cache streams to ensure that all the endpoints can access the
> message content "
>
> http://camel.apache.org/stream-caching.html
>
See the section title, that is only for *Camel 1.x*
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Using-Zip-dataformat-tp5103911p5105444.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Using Zip dataformat
Posted by ebinsingh <eb...@VerizonWireless.com>.
Yes, but according to the Camel documentation, it says "Multicast will
implicitly cache streams to ensure that all the endpoints can access the
message content "
http://camel.apache.org/stream-caching.html
--
View this message in context: http://camel.465427.n5.nabble.com/Using-Zip-dataformat-tp5103911p5105444.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Using Zip dataformat
Posted by Claus Ibsen <cl...@gmail.com>.
On Wed, Dec 28, 2011 at 3:52 PM, ebinsingh
<eb...@verizonwireless.com> wrote:
> I tried using the java DSL to compress file using zip and it did actually
> compress these files. But I was not able to extract them. Looks like they
> are compressed but not the same way as a normal zip file.
>
> What I am trying to do is something different. The input is actually a large
> String and not an actual file.
> In this case the compression did not happen.
>
> Note:
> Looks like the multicast in hamperring the compress. If I remove the
> multicast the String is actually compressed when written to a file.
>
Ah yeah see Willems answer about stream caching.
A stream is often only readable once, but when you multicast to 2+
processors/endpoints, then
you need to deal with this, by ensuring the stream can be re-read by
all the processors/endpoints
in the multicast.
And for that you can use stream caching from Camel, which will wrap
streams into a cached buffer
that is re-readable.
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Using-Zip-dataformat-tp5103911p5105405.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Using Zip dataformat
Posted by ebinsingh <eb...@VerizonWireless.com>.
I tried using the java DSL to compress file using zip and it did actually
compress these files. But I was not able to extract them. Looks like they
are compressed but not the same way as a normal zip file.
What I am trying to do is something different. The input is actually a large
String and not an actual file.
In this case the compression did not happen.
Note:
Looks like the multicast in hamperring the compress. If I remove the
multicast the String is actually compressed when written to a file.
--
View this message in context: http://camel.465427.n5.nabble.com/Using-Zip-dataformat-tp5103911p5105405.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Using Zip dataformat
Posted by Claus Ibsen <cl...@gmail.com>.
The Zip data format does not support files.
You need to use some specific API to work with files vs. non-files.
There is a JIRA ticket to improve this AFAIR.
On Tue, Dec 27, 2011 at 10:22 PM, ebinsingh
<eb...@verizonwireless.com> wrote:
> Hi, I am trying to use Zip / GZip data format to compress the output files
> using Spring DSL implementation.
> But looks like the files are not compressed. Not sure what mistake I am
> making.
>
> Appreciate your help.
>
> <dataFormats>
> <zip compressionLevel = "9" id = "zip"/>
> </dataFormats>
>
> <multicast stopOnException="true" executorServiceRef="mythreadpool">
> <to uri="jms:queue:test_request_2"/>
> <marshal ref="zip"/>
> <to
> uri="file:C:\\camelProject\\data\\outbox\\mars?fileName=${file:name}.zip"/>
> </multicast>
>
> Thanks & regards,
> Ebe
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Using-Zip-dataformat-tp5103911p5103911.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/