You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by abdiels <ab...@gmail.com> on 2013/08/29 03:23:26 UTC
Issue when using ZipSplitter and sending entries to a file endpoint
Hello,
I found this problems and I am still scratching my head, maybe you guys
can help me...I have a route like this:
//Compressed Files Route
//This route decompresses the files and sends them back to be
registered
from("seda:decompressFiles").routeId("DecompressFiles")
.split(new ZipSplitter()).streaming().shareUnitOfWork()
.log("LET'S SEE WHERE THIS GOES!!!")
.choice()
.when(simple("${property.CamelSplitComplete} == false"))
.log("ENTRIES!!!!")
.to("file:{{fmsSys.ClientFileArchiveRoot}}?fileName=${header." +
FMSHeaders.ARCHIVE_PARTIAL_PATH + "}/${header.zipFileName}")
//.to("activemq:queue:ZipChildFiles")
.when(simple("${property.CamelSplitComplete} == true"))
.log("ZIPFILE MULTIPLE ENTRIES CamelSplitComplete -
True")
.to("seda:CleanUp") //Todo: Make sure this is the
right place.
.otherwise()
.log("OHTERWISE!!!!")
.throwException(new FMSException("Unexpected Result in
Un-Zipping!!", null))
.end()
.log("AT THE END!!!!!!")
;
The Zip File that I am using to test contains two entries and I am using
Camel 2.11.0
When I run this I get:
[ad #0 - seda://decompressFiles] DecompressFiles INFO LET'S
SEE WHERE THIS GOES!!!
[ad #0 - seda://decompressFiles] DecompressFiles INFO
ENTRIES!!!!
[ad #0 - seda://decompressFiles] DecompressFiles INFO AT THE
END!!!!!!
[ad #0 - seda://decompressFiles] DecompressFiles INFO LET'S
SEE WHERE THIS GOES!!!
[ad #0 - seda://decompressFiles] DecompressFiles INFO
ENTRIES!!!!
[ad #0 - seda://decompressFiles] DecompressFiles INFO AT THE
END!!!!!!
Those print outs are for the entries...the Actual Zip file never gets in
there as I would expect. Now let's say comment out the to("file...") line
and replace for the active MQ line or pretty much anything else then I get
what I would expect:
[ad #0 - seda://decompressFiles] DecompressFiles INFO LET'S
SEE WHERE THIS GOES!!!
[ad #0 - seda://decompressFiles] DecompressFiles INFO
ENTRIES!!!!
[ad #0 - seda://decompressFiles] DecompressFiles INFO AT THE
END!!!!!!
[ad #0 - seda://decompressFiles] DecompressFiles INFO LET'S
SEE WHERE THIS GOES!!!
[ad #0 - seda://decompressFiles] DecompressFiles INFO
ENTRIES!!!!
[ad #0 - seda://decompressFiles] DecompressFiles INFO AT THE
END!!!!!!
[ad #0 - seda://decompressFiles] DecompressFiles INFO LET'S
SEE WHERE THIS GOES!!!
[ad #0 - seda://decompressFiles] DecompressFiles INFO
ZIPFILE MULTIPLE ENTRIES CamelSplitComplete - True
[ad #0 - seda://decompressFiles] DecompressFiles INFO AT THE
END!!!!!!
Some how when I put a to("file:...") it does not work as expected...I even
removed the choice and just left the splitting and still the does not work,
it works like the first example. I tried looking here in the forum, in the
book and all over the web and did not really stumble upon an answer. I
thought maybe it had to do with the end or endChoice, but that does not seem
to be it since I removed the choice, like I said and that did not do it. I
wonder if there is an issue with the combination ZipSplitter and
to("file:...).
Please advice,
Thank you,
Abdiel
--
View this message in context: http://camel.465427.n5.nabble.com/Issue-when-using-ZipSplitter-and-sending-entries-to-a-file-endpoint-tp5738179.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Issue when using ZipSplitter and sending entries to a file endpoint
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Yeah sounds good if you continue your investigations and then keep the
JIRA up to date when you have more details.
Its also sometimes a good idea from the JIRA ticket to add a link to
this thread so ppl can find back here.
On Thu, Aug 29, 2013 at 7:14 PM, abdiels <ab...@gmail.com> wrote:
> ok, after looking further at this, we discovered that we were under the wrong
> assumption on how the splitter in Camel works. We got that assumption after
> playing with the ZipSpliter. After running into this problem, we played
> with simple spliters, we figured that our assumption was wrong. I will
> explain...this is how the splitters usually work
>
> from("seda:decompressFiles").routeId("DecompressFiles")
> .split(body().tokenize(",")).streaming().shareUnitOfWork()
> .log("LET'S SEE WHERE THIS GOES!!!")
> .to("file:data/out?fileName=${header.zipFileName}")
>
> .to("log:?showExchangeId=true&showHeaders=true&showProperties=true&multiline=true")
> .end()
> .log("ORIGINAL FILE")
>
> .to("log:?showExchangeId=true&showHeaders=true&showProperties=true&multiline=true")
>
> feeding that route with this code:
> template.sendBodyAndHeaders("seda:decompressFiles", "One,Two", headers);
>
> You would get:
> [ad #0 - seda://decompressFiles] DecompressFiles INFO LET'S
> SEE WHERE THIS GOES!!!
> [ad #0 - seda://decompressFiles] DecompressFiles INFO LET'S
> SEE WHERE THIS GOES!!!
> [ad #0 - seda://decompressFiles] DecompressFiles INFO
> ORIGINAL FILE
>
> I removed the exchange print out, but you get the idea. Now, when using the
> zip splitter, it will work fine when you are sending the entries to a file;
> however, when you send the entries to a SEDA or something that does not
> immediately read the data from the stream then it does not work properly and
> you get an extra entry with the CamelZipComplete = true with a null body.
>
> So this is an issue with the ZipSplitter Specifically.
>
> Now, I can change the JIRA accordingly to explain the issue better; however,
> wanted to ask first since I don't know if you want to keep the original
> description or something. I would like to basically edit the subject and
> the description to explain this correctly.
> Also, we are looking into improving the ZipSplitter to behave better since
> we ended up going deeper into this and have a better picture of what is
> going on.
>
> Please let me know your thoughts.
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Issue-when-using-ZipSplitter-and-sending-entries-to-a-file-endpoint-tp5738179p5738242.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Issue when using ZipSplitter and sending entries to a file
endpoint
Posted by abdiels <ab...@gmail.com>.
ok, after looking further at this, we discovered that we were under the wrong
assumption on how the splitter in Camel works. We got that assumption after
playing with the ZipSpliter. After running into this problem, we played
with simple spliters, we figured that our assumption was wrong. I will
explain...this is how the splitters usually work
from("seda:decompressFiles").routeId("DecompressFiles")
.split(body().tokenize(",")).streaming().shareUnitOfWork()
.log("LET'S SEE WHERE THIS GOES!!!")
.to("file:data/out?fileName=${header.zipFileName}")
.to("log:?showExchangeId=true&showHeaders=true&showProperties=true&multiline=true")
.end()
.log("ORIGINAL FILE")
.to("log:?showExchangeId=true&showHeaders=true&showProperties=true&multiline=true")
feeding that route with this code:
template.sendBodyAndHeaders("seda:decompressFiles", "One,Two", headers);
You would get:
[ad #0 - seda://decompressFiles] DecompressFiles INFO LET'S
SEE WHERE THIS GOES!!!
[ad #0 - seda://decompressFiles] DecompressFiles INFO LET'S
SEE WHERE THIS GOES!!!
[ad #0 - seda://decompressFiles] DecompressFiles INFO
ORIGINAL FILE
I removed the exchange print out, but you get the idea. Now, when using the
zip splitter, it will work fine when you are sending the entries to a file;
however, when you send the entries to a SEDA or something that does not
immediately read the data from the stream then it does not work properly and
you get an extra entry with the CamelZipComplete = true with a null body.
So this is an issue with the ZipSplitter Specifically.
Now, I can change the JIRA accordingly to explain the issue better; however,
wanted to ask first since I don't know if you want to keep the original
description or something. I would like to basically edit the subject and
the description to explain this correctly.
Also, we are looking into improving the ZipSplitter to behave better since
we ended up going deeper into this and have a better picture of what is
going on.
Please let me know your thoughts.
--
View this message in context: http://camel.465427.n5.nabble.com/Issue-when-using-ZipSplitter-and-sending-entries-to-a-file-endpoint-tp5738179p5738242.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Issue when using ZipSplitter and sending entries to a file
endpoint
Posted by abdiels <ab...@gmail.com>.
Disregard my previous post. I figured out how to create the issue. Here is
the link to the JIRA issue:
https://issues.apache.org/jira/browse/CAMEL-6679
Let me know if you need additional information.
--
View this message in context: http://camel.465427.n5.nabble.com/Issue-when-using-ZipSplitter-and-sending-entries-to-a-file-endpoint-tp5738179p5738184.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Issue when using ZipSplitter and sending entries to a file
endpoint
Posted by abdiels <ab...@gmail.com>.
ok, I can do that, but how? I can see JIRA, but I don't think I am able to
add issues, is anybody able to? Also when you said submit a test case, do
you want an attachment with the test and the route?
--
View this message in context: http://camel.465427.n5.nabble.com/Issue-when-using-ZipSplitter-and-sending-entries-to-a-file-endpoint-tp5738179p5738183.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Issue when using ZipSplitter and sending entries to a file
endpoint
Posted by Willem jiang <wi...@gmail.com>.
It looks like a bug of camel.
Can you fill a JIRA and submit a simple test case to show the error?
--
Willem Jiang
Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/) (English)
http://jnn.iteye.com (http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On Thursday, August 29, 2013 at 9:23 AM, abdiels wrote:
> Hello,
>
> I found this problems and I am still scratching my head, maybe you guys
> can help me...I have a route like this:
>
> //Compressed Files Route
> //This route decompresses the files and sends them back to be
> registered
> from("seda:decompressFiles").routeId("DecompressFiles")
> .split(new ZipSplitter()).streaming().shareUnitOfWork()
> .log("LET'S SEE WHERE THIS GOES!!!")
> .choice()
> .when(simple("${property.CamelSplitComplete} == false"))
> .log("ENTRIES!!!!")
>
> .to("file:{{fmsSys.ClientFileArchiveRoot}}?fileName=${header." +
> FMSHeaders.ARCHIVE_PARTIAL_PATH + "}/${header.zipFileName}")
> //.to("activemq:queue:ZipChildFiles")
> .when(simple("${property.CamelSplitComplete} == true"))
> .log("ZIPFILE MULTIPLE ENTRIES CamelSplitComplete -
> True")
> .to("seda:CleanUp") //Todo: Make sure this is the
> right place.
> .otherwise()
> .log("OHTERWISE!!!!")
> .throwException(new FMSException("Unexpected Result in
> Un-Zipping!!", null))
> .end()
> .log("AT THE END!!!!!!")
> ;
>
> The Zip File that I am using to test contains two entries and I am using
> Camel 2.11.0
>
> When I run this I get:
> [ad #0 - seda://decompressFiles] DecompressFiles INFO LET'S
> SEE WHERE THIS GOES!!!
> [ad #0 - seda://decompressFiles] DecompressFiles INFO
> ENTRIES!!!!
> [ad #0 - seda://decompressFiles] DecompressFiles INFO AT THE
> END!!!!!!
> [ad #0 - seda://decompressFiles] DecompressFiles INFO LET'S
> SEE WHERE THIS GOES!!!
> [ad #0 - seda://decompressFiles] DecompressFiles INFO
> ENTRIES!!!!
> [ad #0 - seda://decompressFiles] DecompressFiles INFO AT THE
> END!!!!!!
>
> Those print outs are for the entries...the Actual Zip file never gets in
> there as I would expect. Now let's say comment out the to("file...") line
> and replace for the active MQ line or pretty much anything else then I get
> what I would expect:
>
> [ad #0 - seda://decompressFiles] DecompressFiles INFO LET'S
> SEE WHERE THIS GOES!!!
> [ad #0 - seda://decompressFiles] DecompressFiles INFO
> ENTRIES!!!!
> [ad #0 - seda://decompressFiles] DecompressFiles INFO AT THE
> END!!!!!!
> [ad #0 - seda://decompressFiles] DecompressFiles INFO LET'S
> SEE WHERE THIS GOES!!!
> [ad #0 - seda://decompressFiles] DecompressFiles INFO
> ENTRIES!!!!
> [ad #0 - seda://decompressFiles] DecompressFiles INFO AT THE
> END!!!!!!
> [ad #0 - seda://decompressFiles] DecompressFiles INFO LET'S
> SEE WHERE THIS GOES!!!
> [ad #0 - seda://decompressFiles] DecompressFiles INFO
> ZIPFILE MULTIPLE ENTRIES CamelSplitComplete - True
> [ad #0 - seda://decompressFiles] DecompressFiles INFO AT THE
> END!!!!!!
>
> Some how when I put a to("file:...") it does not work as expected...I even
> removed the choice and just left the splitting and still the does not work,
> it works like the first example. I tried looking here in the forum, in the
> book and all over the web and did not really stumble upon an answer. I
> thought maybe it had to do with the end or endChoice, but that does not seem
> to be it since I removed the choice, like I said and that did not do it. I
> wonder if there is an issue with the combination ZipSplitter and
> to("file:...).
>
> Please advice,
>
> Thank you,
>
> Abdiel
>
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Issue-when-using-ZipSplitter-and-sending-entries-to-a-file-endpoint-tp5738179.html
> Sent from the Camel - Users mailing list archive at Nabble.com (http://Nabble.com).