You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Jean Francois LE BESCONT <jf...@gmail.com> on 2013/05/15 16:36:45 UTC
stream:file closeOnDone not close all time
Hey !
A new option is available for the stream:file to close it when done
(closeOnDone). This option is appeared in 2.11 after my jira (
https://issues.apache.org/jira/browse/CAMEL-6147)
File unlock (or released or closed) looks to doesn't works fine if last
line is not passed to the endpoint
Example :
We have a CSV with X line. We want to write a part of it in a file
out_1.csv and a second part in a file out_2.csv according to a business
rule, in my example the rule is after two lines readed.
An example is :
from("file://C:/Temp/camel/rep1/?noop=true")
.log("start process file => ${file:name}")
.split()
.tokenize("\n")
.streaming()
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
// After 2 lines, next lines are rejected via an exchange
property
i++ ;
if( i > 2) {
exchange.setProperty("FILE_1", true );
} else {
exchange.setProperty("FILE_1", false);
}
}
})
.choice()
.when(property("FILE_1").isEqualTo(Boolean.TRUE))
.to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
.when(property("FILE_2").isEqualTo(Boolean.TRUE))
.to("stream:file?fileName=C:/Temp/camel/rep1/out/out_2.csv&closeOnDone=true")
.end()
.end()
.log("end process file => ${file:name}")
.end()
;
It create two files, and out_1.csv is still locked.
Should I update the jira or open a new ?
Thanks
Jeff
Re: stream:file closeOnDone not close all time
Posted by Jean Francois LE BESCONT <jf...@gmail.com>.
Hi,
Sorry if I spam ( I also speak in jira )
Hi,
( Sorry if I spam, I also speak in jira )
I have found a solution, based on the idea of send the exchange to both
endpoint :
public void configure() throws Exception {
// Properties
final String PROP_WRITE_IN_FILE_1 = "PROP_WRITE_IN_FILE_1";
final String PROP_WRITE_IN_FILE_2 = "PROP_WRITE_IN_FILE_2";
// Routes
final String ROUTE_FILE_1 = "direct:ROUTE_FILE_1" ;
final String ROUTE_FILE_2 = "direct:ROUTE_FILE_2" ;
// MAIN ROUTE
from("file://C:/Temp/camel/rep1/?noop=true")
.log("start process file => ${file:name}")
.split()
.tokenize("\n").streaming().process(new Processor() {
public void process(Exchange exchange) throws Exception {
// After 2 lines, next lines are rejected via an
// exchange property
i++;
if (i <= 2) {
exchange.setProperty(PROP_WRITE_IN_FILE_1, true);
exchange.setProperty(PROP_WRITE_IN_FILE_2, false);
} else {
exchange.setProperty(PROP_WRITE_IN_FILE_1, false);
exchange.setProperty(PROP_WRITE_IN_FILE_2, true);
}
}
})
.multicast().parallelProcessing()
.to(ROUTE_FILE_1 , ROUTE_FILE_2 )
.end()
.log("end process file => ${file:name}")
.end();
// OUT FILE 1
from(ROUTE_FILE_1)
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
if( ! exchange.getProperty(PROP_WRITE_IN_FILE_1,
Boolean.class).booleanValue())
exchange.getIn().setBody(null);
}
})
.to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
.end();
// OUT FILE 2
from( ROUTE_FILE_2)
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
if( ! exchange.getProperty(PROP_WRITE_IN_FILE_2,
Boolean.class).booleanValue())
exchange.getIn().setBody(null);
}
})
.to("stream:file?fileName=C:/Temp/camel/rep1/out/out_2.csv&closeOnDone=true")
.end();
}
It complicate the process but it works...
For me camel doesn't manage correctly the Content Based Router from the EIP
patterns in the file:stream component
Re: stream:file closeOnDone not close all time
Posted by Jean Francois LE BESCONT <jf...@gmail.com>.
Hi !
Please h o w can I pass by this limitation ?
Le 19 mai 2013 22:58, "Jean Francois LE BESCONT" <jf...@gmail.com> a
écrit :
> If it is possible !
>
>
> 2013/5/19 Jean Francois LE BESCONT <jf...@gmail.com>
>
>> Hi,
>>
>> Could you please tell me Claus how to use the addOnCompletion to close
>> the stream ?
>>
>> THanks
>>
>>
>> Jeff
>>
>>
>> 2013/5/16 Jean Francois LE BESCONT <jf...@gmail.com>
>>
>>> By the way, the problem appears not on if there are two files. It
>>> appears if the last exchange is not passed to the endpoint.
>>>
>>> Example with only one file :
>>>
>>> from("file://C:/Temp/camel/rep1/?noop=true")
>>> .split()
>>> .tokenize("\n")
>>> .streaming()
>>> .process(new Processor() {
>>>
>>> public void process(Exchange exchange) throws Exception {
>>> // After 2 lines, next lines are rejected via an
>>> // exchange property
>>> i++;
>>> if (i > 2) {
>>> exchange.setProperty("FILE_1", false);
>>> } else {
>>> exchange.setProperty("FILE_1", true);
>>> }
>>> }
>>> })
>>> .choice()
>>> .when(property("FILE_1").isEqualTo(Boolean.TRUE))
>>>
>>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
>>> .end()
>>> .end()
>>> .end();
>>>
>>>
>>> Jeff
>>>
>>>
>>> 2013/5/16 Jean Francois LE BESCONT <jf...@gmail.com>
>>>
>>>> Hi,
>>>>
>>>> Thanks Claus for the answer. I have logged a jira (
>>>> https://issues.apache.org/jira/browse/CAMEL-6367)
>>>>
>>>> I have not really understand how to implements your recommandation
>>>> about exchange.addOnCompletion. After downloaded the camel code to
>>>> understand how package org.apache.camel.converter.stream works, I
>>>> don't know how to access the fileInputStreamCache from the processor :
>>>>
>>>>
>>>> .process(new Processor() {
>>>>
>>>> public void process(Exchange exchange) throws Exception {
>>>> exchange.addOnCompletion(new SynchronizationAdapter() {
>>>> @Override
>>>> public void onDone(Exchange exchange) {
>>>> FileInputStreamCache fileInputStreamCache= ?????????? ;
>>>> try {
>>>> if (fileInputStreamCache != null) {
>>>> fileInputStreamCache.close();
>>>> }
>>>> close();
>>>> } catch (Exception e) {
>>>> LOG.warn("Error deleting temporary cache file: " + tempFile, e);
>>>> }
>>>> }
>>>> });
>>>> }
>>>>
>>>> Is it possible ?
>>>>
>>>>
>>>> Thanks
>>>>
>>>> Jeff
>>>>
>>>>
>>>> 2013/5/16 Claus Ibsen <cl...@gmail.com>
>>>>
>>>>> I have updated the wiki docs.
>>>>>
>>>>> Though we could improve the stream component to auto close the stream
>>>>> when the exchange is done at the end of routing as a fallback. Then
>>>>> your use-case with writing to 2+ files can be supported, as all the
>>>>> streams is closed when the routing is done at the end.
>>>>>
>>>>> This requires to add an onCompletion to the exchange
>>>>>
>>>>> exchange.addOnCompletion
>>>>>
>>>>> And then add logic there to close the stream
>>>>>
>>>>> Though a little challenge could be concurrency if you close the
>>>>> stream, and then another exchange is using it currently.
>>>>>
>>>>> Seems like the logic in stream producer should sync (lock) and not per
>>>>> method which is wrong.
>>>>> Or maybe better yet do not have a shared output stream.
>>>>>
>>>>> Fell free to log a JIRA ticket
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, May 16, 2013 at 8:07 AM, Claus Ibsen <cl...@gmail.com>
>>>>> wrote:
>>>>> > Hi
>>>>> >
>>>>> > That is because you do write to 2 files. The option was intended for
>>>>> > writing to the same file.
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > On Thu, May 16, 2013 at 7:43 AM, Jean Francois LE BESCONT
>>>>> > <jf...@gmail.com> wrote:
>>>>> >> Hi,
>>>>> >>
>>>>> >> what should I do ?
>>>>> >>
>>>>> >> Thanks
>>>>> >>
>>>>> >> Jeff
>>>>> >>
>>>>> >>
>>>>> >> 2013/5/15 Jean Francois LE BESCONT <jf...@gmail.com>
>>>>> >>
>>>>> >>> Hey !
>>>>> >>>
>>>>> >>> A new option is available for the stream:file to close it when done
>>>>> >>> (closeOnDone). This option is appeared in 2.11 after my jira (
>>>>> >>> https://issues.apache.org/jira/browse/CAMEL-6147)
>>>>> >>>
>>>>> >>> File unlock (or released or closed) looks to doesn't works fine if
>>>>> last
>>>>> >>> line is not passed to the endpoint
>>>>> >>>
>>>>> >>> Example :
>>>>> >>>
>>>>> >>> We have a CSV with X line. We want to write a part of it in a file
>>>>> >>> out_1.csv and a second part in a file out_2.csv according to a
>>>>> business
>>>>> >>> rule, in my example the rule is after two lines readed.
>>>>> >>>
>>>>> >>> An example is :
>>>>> >>>
>>>>> >>> from("file://C:/Temp/camel/rep1/?noop=true")
>>>>> >>> .log("start process file => ${file:name}")
>>>>> >>> .split()
>>>>> >>> .tokenize("\n")
>>>>> >>> .streaming()
>>>>> >>> .process(new Processor() {
>>>>> >>>
>>>>> >>> public void process(Exchange exchange) throws
>>>>> Exception {
>>>>> >>> // After 2 lines, next lines are rejected via an
>>>>> exchange
>>>>> >>> property
>>>>> >>> i++ ;
>>>>> >>> if( i > 2) {
>>>>> >>> exchange.setProperty("FILE_1", true );
>>>>> >>> } else {
>>>>> >>> exchange.setProperty("FILE_1", false);
>>>>> >>> }
>>>>> >>> }
>>>>> >>> })
>>>>> >>> .choice()
>>>>> >>> .when(property("FILE_1").isEqualTo(Boolean.TRUE))
>>>>> >>>
>>>>> >>>
>>>>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
>>>>> >>> .when(property("FILE_2").isEqualTo(Boolean.TRUE))
>>>>> >>>
>>>>> >>>
>>>>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_2.csv&closeOnDone=true")
>>>>> >>> .end()
>>>>> >>> .end()
>>>>> >>> .log("end process file => ${file:name}")
>>>>> >>> .end()
>>>>> >>> ;
>>>>> >>>
>>>>> >>> It create two files, and out_1.csv is still locked.
>>>>> >>>
>>>>> >>> Should I update the jira or open a new ?
>>>>> >>>
>>>>> >>> Thanks
>>>>> >>>
>>>>> >>> Jeff
>>>>> >>>
>>>>> >>>
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Claus Ibsen
>>>>> > -----------------
>>>>> > www.camelone.org: The open source integration conference.
>>>>> >
>>>>> > Red Hat, Inc.
>>>>> > FuseSource is now part of Red Hat
>>>>> > Email: cibsen@redhat.com
>>>>> > Web: http://fusesource.com
>>>>> > Twitter: davsclaus
>>>>> > Blog: http://davsclaus.com
>>>>> > Author of Camel in Action: http://www.manning.com/ibsen
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> -----------------
>>>>> www.camelone.org: The open source integration conference.
>>>>>
>>>>> Red Hat, Inc.
>>>>> FuseSource is now part of Red Hat
>>>>> Email: cibsen@redhat.com
>>>>> Web: http://fusesource.com
>>>>> Twitter: davsclaus
>>>>> Blog: http://davsclaus.com
>>>>> Author of Camel in Action: http://www.manning.com/ibsen
>>>>>
>>>>
>>>>
>>>
>>
>
Re: stream:file closeOnDone not close all time
Posted by Jean Francois LE BESCONT <jf...@gmail.com>.
If it is possible !
2013/5/19 Jean Francois LE BESCONT <jf...@gmail.com>
> Hi,
>
> Could you please tell me Claus how to use the addOnCompletion to close
> the stream ?
>
> THanks
>
>
> Jeff
>
>
> 2013/5/16 Jean Francois LE BESCONT <jf...@gmail.com>
>
>> By the way, the problem appears not on if there are two files. It appears
>> if the last exchange is not passed to the endpoint.
>>
>> Example with only one file :
>>
>> from("file://C:/Temp/camel/rep1/?noop=true")
>> .split()
>> .tokenize("\n")
>> .streaming()
>> .process(new Processor() {
>>
>> public void process(Exchange exchange) throws Exception {
>> // After 2 lines, next lines are rejected via an
>> // exchange property
>> i++;
>> if (i > 2) {
>> exchange.setProperty("FILE_1", false);
>> } else {
>> exchange.setProperty("FILE_1", true);
>> }
>> }
>> })
>> .choice()
>> .when(property("FILE_1").isEqualTo(Boolean.TRUE))
>>
>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
>> .end()
>> .end()
>> .end();
>>
>>
>> Jeff
>>
>>
>> 2013/5/16 Jean Francois LE BESCONT <jf...@gmail.com>
>>
>>> Hi,
>>>
>>> Thanks Claus for the answer. I have logged a jira (
>>> https://issues.apache.org/jira/browse/CAMEL-6367)
>>>
>>> I have not really understand how to implements your recommandation about
>>> exchange.addOnCompletion. After downloaded the camel code to understand
>>> how package org.apache.camel.converter.stream works, I don't know how
>>> to access the fileInputStreamCache from the processor :
>>>
>>>
>>> .process(new Processor() {
>>>
>>> public void process(Exchange exchange) throws Exception {
>>> exchange.addOnCompletion(new SynchronizationAdapter() {
>>> @Override
>>> public void onDone(Exchange exchange) {
>>> FileInputStreamCache fileInputStreamCache= ?????????? ;
>>> try {
>>> if (fileInputStreamCache != null) {
>>> fileInputStreamCache.close();
>>> }
>>> close();
>>> } catch (Exception e) {
>>> LOG.warn("Error deleting temporary cache file: " + tempFile, e);
>>> }
>>> }
>>> });
>>> }
>>>
>>> Is it possible ?
>>>
>>>
>>> Thanks
>>>
>>> Jeff
>>>
>>>
>>> 2013/5/16 Claus Ibsen <cl...@gmail.com>
>>>
>>>> I have updated the wiki docs.
>>>>
>>>> Though we could improve the stream component to auto close the stream
>>>> when the exchange is done at the end of routing as a fallback. Then
>>>> your use-case with writing to 2+ files can be supported, as all the
>>>> streams is closed when the routing is done at the end.
>>>>
>>>> This requires to add an onCompletion to the exchange
>>>>
>>>> exchange.addOnCompletion
>>>>
>>>> And then add logic there to close the stream
>>>>
>>>> Though a little challenge could be concurrency if you close the
>>>> stream, and then another exchange is using it currently.
>>>>
>>>> Seems like the logic in stream producer should sync (lock) and not per
>>>> method which is wrong.
>>>> Or maybe better yet do not have a shared output stream.
>>>>
>>>> Fell free to log a JIRA ticket
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, May 16, 2013 at 8:07 AM, Claus Ibsen <cl...@gmail.com>
>>>> wrote:
>>>> > Hi
>>>> >
>>>> > That is because you do write to 2 files. The option was intended for
>>>> > writing to the same file.
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Thu, May 16, 2013 at 7:43 AM, Jean Francois LE BESCONT
>>>> > <jf...@gmail.com> wrote:
>>>> >> Hi,
>>>> >>
>>>> >> what should I do ?
>>>> >>
>>>> >> Thanks
>>>> >>
>>>> >> Jeff
>>>> >>
>>>> >>
>>>> >> 2013/5/15 Jean Francois LE BESCONT <jf...@gmail.com>
>>>> >>
>>>> >>> Hey !
>>>> >>>
>>>> >>> A new option is available for the stream:file to close it when done
>>>> >>> (closeOnDone). This option is appeared in 2.11 after my jira (
>>>> >>> https://issues.apache.org/jira/browse/CAMEL-6147)
>>>> >>>
>>>> >>> File unlock (or released or closed) looks to doesn't works fine if
>>>> last
>>>> >>> line is not passed to the endpoint
>>>> >>>
>>>> >>> Example :
>>>> >>>
>>>> >>> We have a CSV with X line. We want to write a part of it in a file
>>>> >>> out_1.csv and a second part in a file out_2.csv according to a
>>>> business
>>>> >>> rule, in my example the rule is after two lines readed.
>>>> >>>
>>>> >>> An example is :
>>>> >>>
>>>> >>> from("file://C:/Temp/camel/rep1/?noop=true")
>>>> >>> .log("start process file => ${file:name}")
>>>> >>> .split()
>>>> >>> .tokenize("\n")
>>>> >>> .streaming()
>>>> >>> .process(new Processor() {
>>>> >>>
>>>> >>> public void process(Exchange exchange) throws Exception
>>>> {
>>>> >>> // After 2 lines, next lines are rejected via an
>>>> exchange
>>>> >>> property
>>>> >>> i++ ;
>>>> >>> if( i > 2) {
>>>> >>> exchange.setProperty("FILE_1", true );
>>>> >>> } else {
>>>> >>> exchange.setProperty("FILE_1", false);
>>>> >>> }
>>>> >>> }
>>>> >>> })
>>>> >>> .choice()
>>>> >>> .when(property("FILE_1").isEqualTo(Boolean.TRUE))
>>>> >>>
>>>> >>>
>>>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
>>>> >>> .when(property("FILE_2").isEqualTo(Boolean.TRUE))
>>>> >>>
>>>> >>>
>>>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_2.csv&closeOnDone=true")
>>>> >>> .end()
>>>> >>> .end()
>>>> >>> .log("end process file => ${file:name}")
>>>> >>> .end()
>>>> >>> ;
>>>> >>>
>>>> >>> It create two files, and out_1.csv is still locked.
>>>> >>>
>>>> >>> Should I update the jira or open a new ?
>>>> >>>
>>>> >>> Thanks
>>>> >>>
>>>> >>> Jeff
>>>> >>>
>>>> >>>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Claus Ibsen
>>>> > -----------------
>>>> > www.camelone.org: The open source integration conference.
>>>> >
>>>> > Red Hat, Inc.
>>>> > FuseSource is now part of Red Hat
>>>> > Email: cibsen@redhat.com
>>>> > Web: http://fusesource.com
>>>> > Twitter: davsclaus
>>>> > Blog: http://davsclaus.com
>>>> > Author of Camel in Action: http://www.manning.com/ibsen
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> -----------------
>>>> www.camelone.org: The open source integration conference.
>>>>
>>>> Red Hat, Inc.
>>>> FuseSource is now part of Red Hat
>>>> Email: cibsen@redhat.com
>>>> Web: http://fusesource.com
>>>> Twitter: davsclaus
>>>> Blog: http://davsclaus.com
>>>> Author of Camel in Action: http://www.manning.com/ibsen
>>>>
>>>
>>>
>>
>
Re: stream:file closeOnDone not close all time
Posted by Jean Francois LE BESCONT <jf...@gmail.com>.
Hi,
Could you please tell me Claus how to use the addOnCompletion to close the
stream ?
THanks
Jeff
2013/5/16 Jean Francois LE BESCONT <jf...@gmail.com>
> By the way, the problem appears not on if there are two files. It appears
> if the last exchange is not passed to the endpoint.
>
> Example with only one file :
>
> from("file://C:/Temp/camel/rep1/?noop=true")
> .split()
> .tokenize("\n")
> .streaming()
> .process(new Processor() {
>
> public void process(Exchange exchange) throws Exception {
> // After 2 lines, next lines are rejected via an
> // exchange property
> i++;
> if (i > 2) {
> exchange.setProperty("FILE_1", false);
> } else {
> exchange.setProperty("FILE_1", true);
> }
> }
> })
> .choice()
> .when(property("FILE_1").isEqualTo(Boolean.TRUE))
>
> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
> .end()
> .end()
> .end();
>
>
> Jeff
>
>
> 2013/5/16 Jean Francois LE BESCONT <jf...@gmail.com>
>
>> Hi,
>>
>> Thanks Claus for the answer. I have logged a jira (
>> https://issues.apache.org/jira/browse/CAMEL-6367)
>>
>> I have not really understand how to implements your recommandation about
>> exchange.addOnCompletion. After downloaded the camel code to understand
>> how package org.apache.camel.converter.stream works, I don't know how to
>> access the fileInputStreamCache from the processor :
>>
>>
>> .process(new Processor() {
>>
>> public void process(Exchange exchange) throws Exception {
>> exchange.addOnCompletion(new SynchronizationAdapter() {
>> @Override
>> public void onDone(Exchange exchange) {
>> FileInputStreamCache fileInputStreamCache= ?????????? ;
>> try {
>> if (fileInputStreamCache != null) {
>> fileInputStreamCache.close();
>> }
>> close();
>> } catch (Exception e) {
>> LOG.warn("Error deleting temporary cache file: " + tempFile, e);
>> }
>> }
>> });
>> }
>>
>> Is it possible ?
>>
>>
>> Thanks
>>
>> Jeff
>>
>>
>> 2013/5/16 Claus Ibsen <cl...@gmail.com>
>>
>>> I have updated the wiki docs.
>>>
>>> Though we could improve the stream component to auto close the stream
>>> when the exchange is done at the end of routing as a fallback. Then
>>> your use-case with writing to 2+ files can be supported, as all the
>>> streams is closed when the routing is done at the end.
>>>
>>> This requires to add an onCompletion to the exchange
>>>
>>> exchange.addOnCompletion
>>>
>>> And then add logic there to close the stream
>>>
>>> Though a little challenge could be concurrency if you close the
>>> stream, and then another exchange is using it currently.
>>>
>>> Seems like the logic in stream producer should sync (lock) and not per
>>> method which is wrong.
>>> Or maybe better yet do not have a shared output stream.
>>>
>>> Fell free to log a JIRA ticket
>>>
>>>
>>>
>>>
>>>
>>> On Thu, May 16, 2013 at 8:07 AM, Claus Ibsen <cl...@gmail.com>
>>> wrote:
>>> > Hi
>>> >
>>> > That is because you do write to 2 files. The option was intended for
>>> > writing to the same file.
>>> >
>>> >
>>> >
>>> >
>>> > On Thu, May 16, 2013 at 7:43 AM, Jean Francois LE BESCONT
>>> > <jf...@gmail.com> wrote:
>>> >> Hi,
>>> >>
>>> >> what should I do ?
>>> >>
>>> >> Thanks
>>> >>
>>> >> Jeff
>>> >>
>>> >>
>>> >> 2013/5/15 Jean Francois LE BESCONT <jf...@gmail.com>
>>> >>
>>> >>> Hey !
>>> >>>
>>> >>> A new option is available for the stream:file to close it when done
>>> >>> (closeOnDone). This option is appeared in 2.11 after my jira (
>>> >>> https://issues.apache.org/jira/browse/CAMEL-6147)
>>> >>>
>>> >>> File unlock (or released or closed) looks to doesn't works fine if
>>> last
>>> >>> line is not passed to the endpoint
>>> >>>
>>> >>> Example :
>>> >>>
>>> >>> We have a CSV with X line. We want to write a part of it in a file
>>> >>> out_1.csv and a second part in a file out_2.csv according to a
>>> business
>>> >>> rule, in my example the rule is after two lines readed.
>>> >>>
>>> >>> An example is :
>>> >>>
>>> >>> from("file://C:/Temp/camel/rep1/?noop=true")
>>> >>> .log("start process file => ${file:name}")
>>> >>> .split()
>>> >>> .tokenize("\n")
>>> >>> .streaming()
>>> >>> .process(new Processor() {
>>> >>>
>>> >>> public void process(Exchange exchange) throws Exception {
>>> >>> // After 2 lines, next lines are rejected via an
>>> exchange
>>> >>> property
>>> >>> i++ ;
>>> >>> if( i > 2) {
>>> >>> exchange.setProperty("FILE_1", true );
>>> >>> } else {
>>> >>> exchange.setProperty("FILE_1", false);
>>> >>> }
>>> >>> }
>>> >>> })
>>> >>> .choice()
>>> >>> .when(property("FILE_1").isEqualTo(Boolean.TRUE))
>>> >>>
>>> >>>
>>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
>>> >>> .when(property("FILE_2").isEqualTo(Boolean.TRUE))
>>> >>>
>>> >>>
>>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_2.csv&closeOnDone=true")
>>> >>> .end()
>>> >>> .end()
>>> >>> .log("end process file => ${file:name}")
>>> >>> .end()
>>> >>> ;
>>> >>>
>>> >>> It create two files, and out_1.csv is still locked.
>>> >>>
>>> >>> Should I update the jira or open a new ?
>>> >>>
>>> >>> Thanks
>>> >>>
>>> >>> Jeff
>>> >>>
>>> >>>
>>> >
>>> >
>>> >
>>> > --
>>> > Claus Ibsen
>>> > -----------------
>>> > www.camelone.org: The open source integration conference.
>>> >
>>> > Red Hat, Inc.
>>> > FuseSource is now part of Red Hat
>>> > Email: cibsen@redhat.com
>>> > Web: http://fusesource.com
>>> > Twitter: davsclaus
>>> > Blog: http://davsclaus.com
>>> > Author of Camel in Action: http://www.manning.com/ibsen
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> -----------------
>>> www.camelone.org: The open source integration conference.
>>>
>>> Red Hat, Inc.
>>> FuseSource is now part of Red Hat
>>> Email: cibsen@redhat.com
>>> Web: http://fusesource.com
>>> Twitter: davsclaus
>>> Blog: http://davsclaus.com
>>> Author of Camel in Action: http://www.manning.com/ibsen
>>>
>>
>>
>
Re: stream:file closeOnDone not close all time
Posted by Jean Francois LE BESCONT <jf...@gmail.com>.
By the way, the problem appears not on if there are two files. It appears
if the last exchange is not passed to the endpoint.
Example with only one file :
from("file://C:/Temp/camel/rep1/?noop=true")
.split()
.tokenize("\n")
.streaming()
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
// After 2 lines, next lines are rejected via an
// exchange property
i++;
if (i > 2) {
exchange.setProperty("FILE_1", false);
} else {
exchange.setProperty("FILE_1", true);
}
}
})
.choice()
.when(property("FILE_1").isEqualTo(Boolean.TRUE))
.to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
.end()
.end()
.end();
Jeff
2013/5/16 Jean Francois LE BESCONT <jf...@gmail.com>
> Hi,
>
> Thanks Claus for the answer. I have logged a jira (
> https://issues.apache.org/jira/browse/CAMEL-6367)
>
> I have not really understand how to implements your recommandation about
> exchange.addOnCompletion. After downloaded the camel code to understand
> how package org.apache.camel.converter.stream works, I don't know how to
> access the fileInputStreamCache from the processor :
>
>
> .process(new Processor() {
>
> public void process(Exchange exchange) throws Exception {
> exchange.addOnCompletion(new SynchronizationAdapter() {
> @Override
> public void onDone(Exchange exchange) {
> FileInputStreamCache fileInputStreamCache= ?????????? ;
> try {
> if (fileInputStreamCache != null) {
> fileInputStreamCache.close();
> }
> close();
> } catch (Exception e) {
> LOG.warn("Error deleting temporary cache file: " + tempFile, e);
> }
> }
> });
> }
>
> Is it possible ?
>
>
> Thanks
>
> Jeff
>
>
> 2013/5/16 Claus Ibsen <cl...@gmail.com>
>
>> I have updated the wiki docs.
>>
>> Though we could improve the stream component to auto close the stream
>> when the exchange is done at the end of routing as a fallback. Then
>> your use-case with writing to 2+ files can be supported, as all the
>> streams is closed when the routing is done at the end.
>>
>> This requires to add an onCompletion to the exchange
>>
>> exchange.addOnCompletion
>>
>> And then add logic there to close the stream
>>
>> Though a little challenge could be concurrency if you close the
>> stream, and then another exchange is using it currently.
>>
>> Seems like the logic in stream producer should sync (lock) and not per
>> method which is wrong.
>> Or maybe better yet do not have a shared output stream.
>>
>> Fell free to log a JIRA ticket
>>
>>
>>
>>
>>
>> On Thu, May 16, 2013 at 8:07 AM, Claus Ibsen <cl...@gmail.com>
>> wrote:
>> > Hi
>> >
>> > That is because you do write to 2 files. The option was intended for
>> > writing to the same file.
>> >
>> >
>> >
>> >
>> > On Thu, May 16, 2013 at 7:43 AM, Jean Francois LE BESCONT
>> > <jf...@gmail.com> wrote:
>> >> Hi,
>> >>
>> >> what should I do ?
>> >>
>> >> Thanks
>> >>
>> >> Jeff
>> >>
>> >>
>> >> 2013/5/15 Jean Francois LE BESCONT <jf...@gmail.com>
>> >>
>> >>> Hey !
>> >>>
>> >>> A new option is available for the stream:file to close it when done
>> >>> (closeOnDone). This option is appeared in 2.11 after my jira (
>> >>> https://issues.apache.org/jira/browse/CAMEL-6147)
>> >>>
>> >>> File unlock (or released or closed) looks to doesn't works fine if
>> last
>> >>> line is not passed to the endpoint
>> >>>
>> >>> Example :
>> >>>
>> >>> We have a CSV with X line. We want to write a part of it in a file
>> >>> out_1.csv and a second part in a file out_2.csv according to a
>> business
>> >>> rule, in my example the rule is after two lines readed.
>> >>>
>> >>> An example is :
>> >>>
>> >>> from("file://C:/Temp/camel/rep1/?noop=true")
>> >>> .log("start process file => ${file:name}")
>> >>> .split()
>> >>> .tokenize("\n")
>> >>> .streaming()
>> >>> .process(new Processor() {
>> >>>
>> >>> public void process(Exchange exchange) throws Exception {
>> >>> // After 2 lines, next lines are rejected via an
>> exchange
>> >>> property
>> >>> i++ ;
>> >>> if( i > 2) {
>> >>> exchange.setProperty("FILE_1", true );
>> >>> } else {
>> >>> exchange.setProperty("FILE_1", false);
>> >>> }
>> >>> }
>> >>> })
>> >>> .choice()
>> >>> .when(property("FILE_1").isEqualTo(Boolean.TRUE))
>> >>>
>> >>>
>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
>> >>> .when(property("FILE_2").isEqualTo(Boolean.TRUE))
>> >>>
>> >>>
>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_2.csv&closeOnDone=true")
>> >>> .end()
>> >>> .end()
>> >>> .log("end process file => ${file:name}")
>> >>> .end()
>> >>> ;
>> >>>
>> >>> It create two files, and out_1.csv is still locked.
>> >>>
>> >>> Should I update the jira or open a new ?
>> >>>
>> >>> Thanks
>> >>>
>> >>> Jeff
>> >>>
>> >>>
>> >
>> >
>> >
>> > --
>> > Claus Ibsen
>> > -----------------
>> > www.camelone.org: The open source integration conference.
>> >
>> > Red Hat, Inc.
>> > FuseSource is now part of Red Hat
>> > Email: cibsen@redhat.com
>> > Web: http://fusesource.com
>> > Twitter: davsclaus
>> > Blog: http://davsclaus.com
>> > Author of Camel in Action: http://www.manning.com/ibsen
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> www.camelone.org: The open source integration conference.
>>
>> Red Hat, Inc.
>> FuseSource is now part of Red Hat
>> Email: cibsen@redhat.com
>> Web: http://fusesource.com
>> Twitter: davsclaus
>> Blog: http://davsclaus.com
>> Author of Camel in Action: http://www.manning.com/ibsen
>>
>
>
Re: stream:file closeOnDone not close all time
Posted by Jean Francois LE BESCONT <jf...@gmail.com>.
Hi,
Thanks Claus for the answer. I have logged a jira (
https://issues.apache.org/jira/browse/CAMEL-6367)
I have not really understand how to implements your recommandation about
exchange.addOnCompletion. After downloaded the camel code to
understand how package
org.apache.camel.converter.stream works, I don't know how to access
the fileInputStreamCache from the processor :
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
exchange.addOnCompletion(new SynchronizationAdapter() {
@Override
public void onDone(Exchange exchange) {
FileInputStreamCache fileInputStreamCache= ?????????? ;
try {
if (fileInputStreamCache != null) {
fileInputStreamCache.close();
}
close();
} catch (Exception e) {
LOG.warn("Error deleting temporary cache file: " + tempFile, e);
}
}
});
}
Is it possible ?
Thanks
Jeff
2013/5/16 Claus Ibsen <cl...@gmail.com>
> I have updated the wiki docs.
>
> Though we could improve the stream component to auto close the stream
> when the exchange is done at the end of routing as a fallback. Then
> your use-case with writing to 2+ files can be supported, as all the
> streams is closed when the routing is done at the end.
>
> This requires to add an onCompletion to the exchange
>
> exchange.addOnCompletion
>
> And then add logic there to close the stream
>
> Though a little challenge could be concurrency if you close the
> stream, and then another exchange is using it currently.
>
> Seems like the logic in stream producer should sync (lock) and not per
> method which is wrong.
> Or maybe better yet do not have a shared output stream.
>
> Fell free to log a JIRA ticket
>
>
>
>
>
> On Thu, May 16, 2013 at 8:07 AM, Claus Ibsen <cl...@gmail.com>
> wrote:
> > Hi
> >
> > That is because you do write to 2 files. The option was intended for
> > writing to the same file.
> >
> >
> >
> >
> > On Thu, May 16, 2013 at 7:43 AM, Jean Francois LE BESCONT
> > <jf...@gmail.com> wrote:
> >> Hi,
> >>
> >> what should I do ?
> >>
> >> Thanks
> >>
> >> Jeff
> >>
> >>
> >> 2013/5/15 Jean Francois LE BESCONT <jf...@gmail.com>
> >>
> >>> Hey !
> >>>
> >>> A new option is available for the stream:file to close it when done
> >>> (closeOnDone). This option is appeared in 2.11 after my jira (
> >>> https://issues.apache.org/jira/browse/CAMEL-6147)
> >>>
> >>> File unlock (or released or closed) looks to doesn't works fine if last
> >>> line is not passed to the endpoint
> >>>
> >>> Example :
> >>>
> >>> We have a CSV with X line. We want to write a part of it in a file
> >>> out_1.csv and a second part in a file out_2.csv according to a business
> >>> rule, in my example the rule is after two lines readed.
> >>>
> >>> An example is :
> >>>
> >>> from("file://C:/Temp/camel/rep1/?noop=true")
> >>> .log("start process file => ${file:name}")
> >>> .split()
> >>> .tokenize("\n")
> >>> .streaming()
> >>> .process(new Processor() {
> >>>
> >>> public void process(Exchange exchange) throws Exception {
> >>> // After 2 lines, next lines are rejected via an
> exchange
> >>> property
> >>> i++ ;
> >>> if( i > 2) {
> >>> exchange.setProperty("FILE_1", true );
> >>> } else {
> >>> exchange.setProperty("FILE_1", false);
> >>> }
> >>> }
> >>> })
> >>> .choice()
> >>> .when(property("FILE_1").isEqualTo(Boolean.TRUE))
> >>>
> >>>
> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
> >>> .when(property("FILE_2").isEqualTo(Boolean.TRUE))
> >>>
> >>>
> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_2.csv&closeOnDone=true")
> >>> .end()
> >>> .end()
> >>> .log("end process file => ${file:name}")
> >>> .end()
> >>> ;
> >>>
> >>> It create two files, and out_1.csv is still locked.
> >>>
> >>> Should I update the jira or open a new ?
> >>>
> >>> Thanks
> >>>
> >>> Jeff
> >>>
> >>>
> >
> >
> >
> > --
> > Claus Ibsen
> > -----------------
> > www.camelone.org: The open source integration conference.
> >
> > Red Hat, Inc.
> > FuseSource is now part of Red Hat
> > Email: cibsen@redhat.com
> > Web: http://fusesource.com
> > Twitter: davsclaus
> > Blog: http://davsclaus.com
> > Author of Camel in Action: http://www.manning.com/ibsen
>
>
>
> --
> Claus Ibsen
> -----------------
> www.camelone.org: The open source integration conference.
>
> Red Hat, Inc.
> FuseSource is now part of Red Hat
> Email: cibsen@redhat.com
> Web: http://fusesource.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
>
Re: stream:file closeOnDone not close all time
Posted by Claus Ibsen <cl...@gmail.com>.
I have updated the wiki docs.
Though we could improve the stream component to auto close the stream
when the exchange is done at the end of routing as a fallback. Then
your use-case with writing to 2+ files can be supported, as all the
streams is closed when the routing is done at the end.
This requires to add an onCompletion to the exchange
exchange.addOnCompletion
And then add logic there to close the stream
Though a little challenge could be concurrency if you close the
stream, and then another exchange is using it currently.
Seems like the logic in stream producer should sync (lock) and not per
method which is wrong.
Or maybe better yet do not have a shared output stream.
Fell free to log a JIRA ticket
On Thu, May 16, 2013 at 8:07 AM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> That is because you do write to 2 files. The option was intended for
> writing to the same file.
>
>
>
>
> On Thu, May 16, 2013 at 7:43 AM, Jean Francois LE BESCONT
> <jf...@gmail.com> wrote:
>> Hi,
>>
>> what should I do ?
>>
>> Thanks
>>
>> Jeff
>>
>>
>> 2013/5/15 Jean Francois LE BESCONT <jf...@gmail.com>
>>
>>> Hey !
>>>
>>> A new option is available for the stream:file to close it when done
>>> (closeOnDone). This option is appeared in 2.11 after my jira (
>>> https://issues.apache.org/jira/browse/CAMEL-6147)
>>>
>>> File unlock (or released or closed) looks to doesn't works fine if last
>>> line is not passed to the endpoint
>>>
>>> Example :
>>>
>>> We have a CSV with X line. We want to write a part of it in a file
>>> out_1.csv and a second part in a file out_2.csv according to a business
>>> rule, in my example the rule is after two lines readed.
>>>
>>> An example is :
>>>
>>> from("file://C:/Temp/camel/rep1/?noop=true")
>>> .log("start process file => ${file:name}")
>>> .split()
>>> .tokenize("\n")
>>> .streaming()
>>> .process(new Processor() {
>>>
>>> public void process(Exchange exchange) throws Exception {
>>> // After 2 lines, next lines are rejected via an exchange
>>> property
>>> i++ ;
>>> if( i > 2) {
>>> exchange.setProperty("FILE_1", true );
>>> } else {
>>> exchange.setProperty("FILE_1", false);
>>> }
>>> }
>>> })
>>> .choice()
>>> .when(property("FILE_1").isEqualTo(Boolean.TRUE))
>>>
>>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
>>> .when(property("FILE_2").isEqualTo(Boolean.TRUE))
>>>
>>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_2.csv&closeOnDone=true")
>>> .end()
>>> .end()
>>> .log("end process file => ${file:name}")
>>> .end()
>>> ;
>>>
>>> It create two files, and out_1.csv is still locked.
>>>
>>> Should I update the jira or open a new ?
>>>
>>> Thanks
>>>
>>> Jeff
>>>
>>>
>
>
>
> --
> Claus Ibsen
> -----------------
> www.camelone.org: The open source integration conference.
>
> Red Hat, Inc.
> FuseSource is now part of Red Hat
> Email: cibsen@redhat.com
> Web: http://fusesource.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
--
Claus Ibsen
-----------------
www.camelone.org: The open source integration conference.
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: stream:file closeOnDone not close all time
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
That is because you do write to 2 files. The option was intended for
writing to the same file.
On Thu, May 16, 2013 at 7:43 AM, Jean Francois LE BESCONT
<jf...@gmail.com> wrote:
> Hi,
>
> what should I do ?
>
> Thanks
>
> Jeff
>
>
> 2013/5/15 Jean Francois LE BESCONT <jf...@gmail.com>
>
>> Hey !
>>
>> A new option is available for the stream:file to close it when done
>> (closeOnDone). This option is appeared in 2.11 after my jira (
>> https://issues.apache.org/jira/browse/CAMEL-6147)
>>
>> File unlock (or released or closed) looks to doesn't works fine if last
>> line is not passed to the endpoint
>>
>> Example :
>>
>> We have a CSV with X line. We want to write a part of it in a file
>> out_1.csv and a second part in a file out_2.csv according to a business
>> rule, in my example the rule is after two lines readed.
>>
>> An example is :
>>
>> from("file://C:/Temp/camel/rep1/?noop=true")
>> .log("start process file => ${file:name}")
>> .split()
>> .tokenize("\n")
>> .streaming()
>> .process(new Processor() {
>>
>> public void process(Exchange exchange) throws Exception {
>> // After 2 lines, next lines are rejected via an exchange
>> property
>> i++ ;
>> if( i > 2) {
>> exchange.setProperty("FILE_1", true );
>> } else {
>> exchange.setProperty("FILE_1", false);
>> }
>> }
>> })
>> .choice()
>> .when(property("FILE_1").isEqualTo(Boolean.TRUE))
>>
>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
>> .when(property("FILE_2").isEqualTo(Boolean.TRUE))
>>
>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_2.csv&closeOnDone=true")
>> .end()
>> .end()
>> .log("end process file => ${file:name}")
>> .end()
>> ;
>>
>> It create two files, and out_1.csv is still locked.
>>
>> Should I update the jira or open a new ?
>>
>> Thanks
>>
>> Jeff
>>
>>
--
Claus Ibsen
-----------------
www.camelone.org: The open source integration conference.
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: stream:file closeOnDone not close all time
Posted by Jean Francois LE BESCONT <jf...@gmail.com>.
Hi,
what should I do ?
Thanks
Jeff
2013/5/15 Jean Francois LE BESCONT <jf...@gmail.com>
> Hey !
>
> A new option is available for the stream:file to close it when done
> (closeOnDone). This option is appeared in 2.11 after my jira (
> https://issues.apache.org/jira/browse/CAMEL-6147)
>
> File unlock (or released or closed) looks to doesn't works fine if last
> line is not passed to the endpoint
>
> Example :
>
> We have a CSV with X line. We want to write a part of it in a file
> out_1.csv and a second part in a file out_2.csv according to a business
> rule, in my example the rule is after two lines readed.
>
> An example is :
>
> from("file://C:/Temp/camel/rep1/?noop=true")
> .log("start process file => ${file:name}")
> .split()
> .tokenize("\n")
> .streaming()
> .process(new Processor() {
>
> public void process(Exchange exchange) throws Exception {
> // After 2 lines, next lines are rejected via an exchange
> property
> i++ ;
> if( i > 2) {
> exchange.setProperty("FILE_1", true );
> } else {
> exchange.setProperty("FILE_1", false);
> }
> }
> })
> .choice()
> .when(property("FILE_1").isEqualTo(Boolean.TRUE))
>
> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
> .when(property("FILE_2").isEqualTo(Boolean.TRUE))
>
> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_2.csv&closeOnDone=true")
> .end()
> .end()
> .log("end process file => ${file:name}")
> .end()
> ;
>
> It create two files, and out_1.csv is still locked.
>
> Should I update the jira or open a new ?
>
> Thanks
>
> Jeff
>
>