You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by rdifrango <ro...@gmail.com> on 2012/04/27 15:32:00 UTC
File Processor Not deleting the files
I have the following route:
context.addRoutes(new RouteBuilder {
// Read the file(s) from the directory
"file:perf?delete=true" ==> {
// Split the file up at each line
split(_.getIn().getBody(classOf[String]).split("\n")) {
// The further split up the processing across multiple parsers
loadbalance roundrobin {
// Reference to my internal components
to("direct:x")
to("direct:y")
to("direct:z")
}
// Join back the results from aboce
// then split it out again turning off commit level changes
}.loadbalance roundrobin {
to("jdbc:dataSource?resetAutoCommit=false")
to("jdbc:dataSource?resetAutoCommit=false")
to("jdbc:dataSource?resetAutoCommit=false")
}
}
The behavior I'm see is that if I have multiple files, they aren't deleted
once they are processed. Any reason why not?
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5670301.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Processor Not deleting the files
Posted by rdifrango <ro...@gmail.com>.
Just as a wrap up on this one, I think splitting things up into multiple
routes along with the 2.10 release solved this issue. Thanks again for your
assistance.
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5697666.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Processor Not deleting the files
Posted by rdifrango <ro...@gmail.com>.
I was doing just that, what I found that works is to create multiple routes
that are in effect chained off one of another like the following works
[meaning the files get properly deleted]:
context.addRoutes(new RouteBuilder {
"file:perf?delete=true" ==> {
process(myProcessor).to("file:perf_outbox")
}
"file:perf_outbox?delete=true" ==> {
process(insertProcessor).to("file:perf_insert")
}
"file:perf_insert?delete=true" ==> {
split(_.getIn().getBody(classOf[String]).split("\n")).to("jdbc:dataSource")
}
})
Where the initial processor loads everything and perform the initial split:
val myProcessor = (exchange: Exchange) => {
val lines =
Source.fromFile(exchange.getIn().getBody(classOf[File])).getLines
val body = new ArrayBuffer[String]()
lines.foreach(line => {
val lineSplit = line.split("\\[\\]\\: ")
if (lineSplit.size >= 2) {
val split = lineSplit(1).split(" ")
if (split.length >= 11)
body += split(0) + "," + split(1) + "," + split(2) + "," +
split(3) + "," + split(4) + "," + split(5) + "," + split(7) + "," +
split(10)
else
println("Secondary Split Not Enough Line Data")
} else println("Initial Split Not Enough Line Data")
})
exchange.getIn().setBody(body.mkString("\n"))
}
Then the secondary split takes that and turns it into SQL statements:
val format = "INSERT INTO EVT_PERF ( EVT_PERF_ID, CRLTN_ID, USER_ID,
APPN_SYS_CD, HOST_NM, WEBLOGIC_INSTNC_NM, WEB_ANLYTCS_CRLTN_ID,
LOGGER_CLASS_NM, EXEC_TIME ) values ( EVT_PERF_ID_SEQ.NEXTVAL, ''{0}'',
''{1}'', ''{2}'', ''{3}'', ''{4}'', ''{5}'', ''{6}'', {7} )"
val insertProcessor = (exchange: Exchange) => {
val lines =
Source.fromFile(exchange.getIn().getBody(classOf[File])).getLines
val body = for {
line <- lines
val split = line.split(",")
// Build the insert statement
val x = MessageFormat.format(format, split(0), split(1), split(2),
split(3), split(4), split(5), split(6), split(7))
// Replace the incoming message with the insert statement
} yield x
exchange.getIn().setBody(body.mkString("\n"))
}
The interesting thing to me is that the following route works as works as
expected:
"file:perf_insert?delete=true" ==> {
split(_.getIn().getBody(classOf[String]).split("\n")).to("jdbc:dataSource")
}
Meaning, that the split properly splits up the input and sends it to the
JDBC component, then deletes the file once it is done processing.
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5694084.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Processor Not deleting the files
Posted by rdifrango <ro...@gmail.com>.
I'll try that and see if it works. I have attached my Scala and Java code
for your review of course the user name and password are blanked out.
http://camel.465427.n5.nabble.com/file/n5691459/JavaSplitPerfProcessor.java
JavaSplitPerfProcessor.java
http://camel.465427.n5.nabble.com/file/n5691459/SplitPerfProcessor.scala
SplitPerfProcessor.scala
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5691459.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Processor Not deleting the files
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, May 7, 2012 at 6:11 PM, rdifrango <ro...@gmail.com> wrote:
> OK...so it does not seem to fully correct the issue:
>
> public void configure() {
>
> from("file:perf?delete=true&idempotent=true").convertBodyTo(String.class)
> .split(body().tokenize("\n")).streaming()
> .process(lineSplitter)
> .to("jdbc:dataSource?resetAutoCommit=false").end();
> }
>
> I'll try loading the entire file into memory next.
>
convertBodyTo loads the file into memory.
Can you try to remove pieces of your route, until you get a point
where the file is not locked anymore?
And if you enable DEBUG or TRACE logging level on
org.apache.camel.util.FileUtil then it ought to log delete attempts.
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5691470.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: File Processor Not deleting the files
Posted by rdifrango <ro...@gmail.com>.
OK...so it does not seem to fully correct the issue:
public void configure() {
from("file:perf?delete=true&idempotent=true").convertBodyTo(String.class)
.split(body().tokenize("\n")).streaming()
.process(lineSplitter)
.to("jdbc:dataSource?resetAutoCommit=false").end();
}
I'll try loading the entire file into memory next.
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5691470.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Processor Not deleting the files
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
I dont know what you do in those routes you call with the direct endpoint.
But I suggest to load the file into memory in the start of the route.
Then there is no input stream to the file handle in use, which
otherwise could lock the file.
You can add a
.convertBodyTo(String.class)
in the start of your route, eg just after the from
On Mon, May 7, 2012 at 5:20 PM, rdifrango <ro...@gmail.com> wrote:
> I downloaded the snapshot version today and ran both my Java and Scala
> versions and it still appears to have the same problem. Here are the
> libraries that I used:
>
> camel-core-2.10-SNAPSHOT.jar
> camel-ftp-2.10-SNAPSHOT.jar
> camel-jdbc-2.10-SNAPSHOT.jar
> camel-jms-2.10-SNAPSHOT.jar
> camel-scala-2.10-SNAPSHOT.jar
> optional/log4j-1.2.16.jar
> slf4j-api-1.6.1.jar
> optional/slf4j-log4j12-1.6.4.jar
> spring/spring-beans-3.0.7.RELEASE.jar
> spring/spring-context-3.0.7.RELEASE.jar
> spring/spring-core-3.0.7.RELEASE.jar
> spring/spring-jdbc-3.0.7.RELEASE.jar
> spring/spring-jms-3.0.7.RELEASE.jar
> spring/spring-tx-3.0.7.RELEASE.jar
>
Well
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5691340.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: File Processor Not deleting the files
Posted by rdifrango <ro...@gmail.com>.
I downloaded the snapshot version today and ran both my Java and Scala
versions and it still appears to have the same problem. Here are the
libraries that I used:
camel-core-2.10-SNAPSHOT.jar
camel-ftp-2.10-SNAPSHOT.jar
camel-jdbc-2.10-SNAPSHOT.jar
camel-jms-2.10-SNAPSHOT.jar
camel-scala-2.10-SNAPSHOT.jar
optional/log4j-1.2.16.jar
slf4j-api-1.6.1.jar
optional/slf4j-log4j12-1.6.4.jar
spring/spring-beans-3.0.7.RELEASE.jar
spring/spring-context-3.0.7.RELEASE.jar
spring/spring-core-3.0.7.RELEASE.jar
spring/spring-jdbc-3.0.7.RELEASE.jar
spring/spring-jms-3.0.7.RELEASE.jar
spring/spring-tx-3.0.7.RELEASE.jar
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5691340.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Processor Not deleting the files
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, May 4, 2012 at 5:23 PM, rdifrango <ro...@gmail.com> wrote:
> Has this been baked into a snapshot release that is available or source build
> only?
>
Yeah its in the trunk and 2.9 branches.
http://camel.apache.org/source.html
The 2.10 SNAPSHOT can be downloaded
http://camel.apache.org/download.html
Or you can build from source
http://camel.apache.org/building.html
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5686206.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: File Processor Not deleting the files
Posted by rdifrango <ro...@gmail.com>.
Has this been baked into a snapshot release that is available or source build
only?
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5686206.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Processor Not deleting the files
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, May 1, 2012 at 12:52 PM, rdifrango <ro...@gmail.com> wrote:
> Great, Thanks! Will this fix also cover
> the Scala DSL?
>
Yes the fix is in camel-core which also applies for Scala.
> On May 1, 2012, at 3:08 AM, "Claus Ibsen-2 [via Camel]" <
> ml-node+s465427n5677551h11@n5.nabble.com> wrote:
>
> Hi
>
> I committed a fix to trunk and 2.9 branches
> https://issues.apache.org/jira/browse/CAMEL-5235
>
> You are welcome to try on your windows system with a SNAPSHOT version.
> Make sure to restart your windows box beforehand so it has no file
> locks from the start.
>
> Also if you do a full unit tests of camel-core, eg from the source code
> cd camel-core
> mvn clean install
>
> Then afte the testing you should be able to do a
> mvn clean
>
> To delete all the temporary files created, to make sure Windows dont
> have any locks.
>
>
>
> On Mon, Apr 30, 2012 at 8:58 PM, rdifrango <[hidden
> email]</user/SendEmail.jtp?type=node&node=5677551&i=0>>
> wrote:
>> I wonder if using one the Marshalling frameworks would solve this?
>>
>> --
>> View this message in context:
> http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5676645.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>
> --
> Claus Ibsen
> -----------------
> CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
> FuseSource
> Email: [hidden email] </user/SendEmail.jtp?type=node&node=5677551&i=1>
> Web: http://fusesource.com
> Twitter: davsclaus, fusenews
> Blog: http://davsclaus.blogspot.com/
> Author of Camel in Action: http://www.manning.com/ibsen/
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5677551.html
> To unsubscribe from File Processor Not deleting the files, click
> here<http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5670301&code=cm9uLmRpZnJhbmdvQGdtYWlsLmNvbXw1NjcwMzAxfC03NDczMjc1NzY=>
> .
> NAML<http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5677821.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: File Processor Not deleting the files
Posted by rdifrango <ro...@gmail.com>.
Great, Thanks! Will this fix also cover
the Scala DSL?
On May 1, 2012, at 3:08 AM, "Claus Ibsen-2 [via Camel]" <
ml-node+s465427n5677551h11@n5.nabble.com> wrote:
Hi
I committed a fix to trunk and 2.9 branches
https://issues.apache.org/jira/browse/CAMEL-5235
You are welcome to try on your windows system with a SNAPSHOT version.
Make sure to restart your windows box beforehand so it has no file
locks from the start.
Also if you do a full unit tests of camel-core, eg from the source code
cd camel-core
mvn clean install
Then afte the testing you should be able to do a
mvn clean
To delete all the temporary files created, to make sure Windows dont
have any locks.
On Mon, Apr 30, 2012 at 8:58 PM, rdifrango <[hidden
email]</user/SendEmail.jtp?type=node&node=5677551&i=0>>
wrote:
> I wonder if using one the Marshalling frameworks would solve this?
>
> --
> View this message in context:
http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5676645.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: [hidden email] </user/SendEmail.jtp?type=node&node=5677551&i=1>
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
------------------------------
If you reply to this email, your message will be added to the discussion
below:
http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5677551.html
To unsubscribe from File Processor Not deleting the files, click
here<http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5670301&code=cm9uLmRpZnJhbmdvQGdtYWlsLmNvbXw1NjcwMzAxfC03NDczMjc1NzY=>
.
NAML<http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5677821.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Processor Not deleting the files
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
I committed a fix to trunk and 2.9 branches
https://issues.apache.org/jira/browse/CAMEL-5235
You are welcome to try on your windows system with a SNAPSHOT version.
Make sure to restart your windows box beforehand so it has no file
locks from the start.
Also if you do a full unit tests of camel-core, eg from the source code
cd camel-core
mvn clean install
Then afte the testing you should be able to do a
mvn clean
To delete all the temporary files created, to make sure Windows dont
have any locks.
On Mon, Apr 30, 2012 at 8:58 PM, rdifrango <ro...@gmail.com> wrote:
> I wonder if using one the Marshalling frameworks would solve this?
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5676645.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: File Processor Not deleting the files
Posted by rdifrango <ro...@gmail.com>.
I wonder if using one the Marshalling frameworks would solve this?
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5676645.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Processor Not deleting the files
Posted by rdifrango <ro...@gmail.com>.
Thanks, in the meantime I'm going to see if I can get this running on a Unix
box.
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5676398.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Processor Not deleting the files
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Apr 30, 2012 at 3:27 PM, rdifrango <ro...@gmail.com> wrote:
> So, I converted this over to Java and it is exhibiting the same behavior.
> What did I do wrong:
>
> public class JavaSplitPerfProcessor {
> static class LineSplitter implements Processor {
> static final String format = "INSERT INTO EVT_PERF ( EVT_PERF_ID,
> CRLTN_ID, USER_ID, APPN_SYS_CD, HOST_NM, WEBLOGIC_INSTNC_NM,
> WEB_ANLYTCS_CRLTN_ID, LOGGER_CLASS_NM, EXEC_TIME ) values (
> EVT_PERF_ID_SEQ.NEXTVAL, ''{0}'', ''{1}'', ''{2}'', ''{3}'', ''{4}'',
> ''{5}'', ''{6}'', {7} )";
>
> @Override
> public void process(Exchange exchange) throws Exception {
> // Get the incoming line from the message exchange
> String line = exchange.getIn().getBody(String.class);
> // Remove the first part of the log message
> String[] lineSplit = line.split("\\[\\]\\: ");
> // Break it down into the individual components
> String[] split = lineSplit[1].split(" ");
> // Build the insert statement
> String x = MessageFormat
> .format(format, split[0], split[1], split[2], split[3],
> split[4], split[5], split[7], split[10]);
> // Replace the incoming message with the insert statement
> exchange.getIn().setBody(x);
> }
> }
>
> public static void main(String[] args) throws Exception {
> // Build the connection pool
> BasicDataSource ds = new BasicDataSource();
> ds.setDriverClassName("oracle.jdbc.OracleDriver");
> ds.setUrl("jdbc:oracle:thin:@localhost:1521:xe");
> ds.setUsername("****");
> ds.setPassword("****");
> // Register it with Camel
> SimpleRegistry reg = new SimpleRegistry();
> reg.put("dataSource", ds);
>
> final LineSplitter lineSplitter = new LineSplitter();
>
> CamelContext context = new DefaultCamelContext(reg);
> context.addRoutes(new RouteBuilder() {
> public void configure() {
> from("file:perf?delete=true").split(body().tokenize("\n"))
> .streaming().process(lineSplitter)
> .to("jdbc:dataSource?resetAutoCommit=false").end();
> }
> });
>
> context.start();
> Thread.sleep(60000);
> context.stop();
> }
> }
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5675961.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Windows is always a bit tricky with its file system.
I suspect this line
String line = exchange.getIn().getBody(String.class);
Despite we close the buffers un the hood, the JDK may not close
wrapped buffers, which may lead
to the FileInputStream not being closed, and Windows not able to
delete the source file when the processing is done.
I got a XP box for testing. So let see what that old fella says.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: File Processor Not deleting the files
Posted by rdifrango <ro...@gmail.com>.
So, I converted this over to Java and it is exhibiting the same behavior.
What did I do wrong:
public class JavaSplitPerfProcessor {
static class LineSplitter implements Processor {
static final String format = "INSERT INTO EVT_PERF ( EVT_PERF_ID,
CRLTN_ID, USER_ID, APPN_SYS_CD, HOST_NM, WEBLOGIC_INSTNC_NM,
WEB_ANLYTCS_CRLTN_ID, LOGGER_CLASS_NM, EXEC_TIME ) values (
EVT_PERF_ID_SEQ.NEXTVAL, ''{0}'', ''{1}'', ''{2}'', ''{3}'', ''{4}'',
''{5}'', ''{6}'', {7} )";
@Override
public void process(Exchange exchange) throws Exception {
// Get the incoming line from the message exchange
String line = exchange.getIn().getBody(String.class);
// Remove the first part of the log message
String[] lineSplit = line.split("\\[\\]\\: ");
// Break it down into the individual components
String[] split = lineSplit[1].split(" ");
// Build the insert statement
String x = MessageFormat
.format(format, split[0], split[1], split[2], split[3],
split[4], split[5], split[7], split[10]);
// Replace the incoming message with the insert statement
exchange.getIn().setBody(x);
}
}
public static void main(String[] args) throws Exception {
// Build the connection pool
BasicDataSource ds = new BasicDataSource();
ds.setDriverClassName("oracle.jdbc.OracleDriver");
ds.setUrl("jdbc:oracle:thin:@localhost:1521:xe");
ds.setUsername("****");
ds.setPassword("****");
// Register it with Camel
SimpleRegistry reg = new SimpleRegistry();
reg.put("dataSource", ds);
final LineSplitter lineSplitter = new LineSplitter();
CamelContext context = new DefaultCamelContext(reg);
context.addRoutes(new RouteBuilder() {
public void configure() {
from("file:perf?delete=true").split(body().tokenize("\n"))
.streaming().process(lineSplitter)
.to("jdbc:dataSource?resetAutoCommit=false").end();
}
});
context.start();
Thread.sleep(60000);
context.stop();
}
}
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5675961.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Processor Not deleting the files
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Apr 27, 2012 at 5:30 PM, rdifrango <ro...@gmail.com> wrote:
> Yeah, in the Scala DSL, it doesn't seem to support that. Is that a bug or
> feature request?
>
Fell free to log a JIRA. And as we love contributions you are welcome
to work on a patch to get that into the Scala DSL.
http://camel.apache.org/contributing.html
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5670621.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: File Processor Not deleting the files
Posted by rdifrango <ro...@gmail.com>.
Yeah, in the Scala DSL, it doesn't seem to support that. Is that a bug or
feature request?
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5670621.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Processor Not deleting the files
Posted by Claus Ibsen <cl...@gmail.com>.
Yeah use the tokenize expression
In Java it would be
.split().tokenize("\n").streaming()
On Fri, Apr 27, 2012 at 5:13 PM, rdifrango <ro...@gmail.com> wrote:
> I guess this is implicitly touching the stream:
>
> split(_.getIn().getBody(classOf[String]).split("\n"))
>
> Could be the root?
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5670589.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: File Processor Not deleting the files
Posted by rdifrango <ro...@gmail.com>.
I guess this is implicitly touching the stream:
split(_.getIn().getBody(classOf[String]).split("\n"))
Could be the root?
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5670589.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Processor Not deleting the files
Posted by Claus Ibsen <cl...@gmail.com>.
What Camel version and OS are you using?
On Fri, Apr 27, 2012 at 3:32 PM, rdifrango <ro...@gmail.com> wrote:
> I have the following route:
>
> context.addRoutes(new RouteBuilder {
> // Read the file(s) from the directory
> "file:perf?delete=true" ==> {
> // Split the file up at each line
> split(_.getIn().getBody(classOf[String]).split("\n")) {
> // The further split up the processing across multiple parsers
> loadbalance roundrobin {
> // Reference to my internal components
> to("direct:x")
> to("direct:y")
> to("direct:z")
> }
> // Join back the results from aboce
> // then split it out again turning off commit level changes
> }.loadbalance roundrobin {
> to("jdbc:dataSource?resetAutoCommit=false")
> to("jdbc:dataSource?resetAutoCommit=false")
> to("jdbc:dataSource?resetAutoCommit=false")
> }
> }
>
> The behavior I'm see is that if I have multiple files, they aren't deleted
> once they are processed. Any reason why not?
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5670301.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: File Processor Not deleting the files
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Apr 27, 2012 at 5:01 PM, rdifrango <ro...@gmail.com> wrote:
> Camel version 2.9.2 on a Windows 7 Machine.
>
Okay be careful on Windows OS to ensure you always closes file handles
and streams when you acces the files.
For example if you use a InputStream then make sure you close it etc.
> --
> View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5670530.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: File Processor Not deleting the files
Posted by rdifrango <ro...@gmail.com>.
I'm not using any streams directly in my code. The complete code listing is
as follows:
// Build the connection pool
val ds = new BasicDataSource
ds.setDriverClassName("oracle.jdbc.OracleDriver")
ds.setUrl("jdbc:oracle:thin:@localhost:1521:xe")
ds.setUsername("******")
ds.setPassword("******")
// Register it with Camel
val reg = new SimpleRegistry
reg.put("dataSource", ds)
val format = "INSERT INTO EVT_PERF ( EVT_PERF_ID, CRLTN_ID, USER_ID,
APPN_SYS_CD, HOST_NM, WEBLOGIC_INSTNC_NM, WEB_ANLYTCS_CRLTN_ID,
LOGGER_CLASS_NM, EXEC_TIME ) values ( EVT_PERF_ID_SEQ.NEXTVAL, ''{0}'',
''{1}'', ''{2}'', ''{3}'', ''{4}'', ''{5}'', ''{6}'', {7} )"
val processor = (exchange: Exchange) => {
// Get the incoming line from the message exchange
val line = exchange.getIn.getBody(classOf[String])
// Remove the first part of the log message
val lineSplit = line.split("\\[\\]\\: ")
// Break it down into the individual components
val split = lineSplit(1).split(" ")
// Build the insert statement
val x = MessageFormat.format(format, split(0), split(1), split(2),
split(3), split(4), split(5), split(7), split(10))
// Replace the incoming message with the insert statement
exchange.getIn().setBody(x)
}
// Create the Camel Context
val context = new DefaultCamelContext(reg)
// Build the Route
context.addRoutes(new RouteBuilder {
// Read the file(s) from the directory
"file:perf?delete=true&idempotent=true" ==> {
// Split the file up at each line
split(_.getIn().getBody(classOf[String]).split("\n")) {
// The further split up the processing across multiple parsers
loadbalance roundrobin {
// Reference to my internal components
to("direct:x")
to("direct:y")
to("direct:z")
}
}.loadbalance roundrobin {
to("jdbc:dataSource?resetAutoCommit=false")
to("jdbc:dataSource?resetAutoCommit=false")
to("jdbc:dataSource?resetAutoCommit=false")
}
}
// Setup my internal processors with references to my custom component
"direct:x" process (processor)
"direct:y" process (processor)
"direct:z" process (processor)
})
// Start the camel process
context.start
// Wait for the process to execute
Thread.sleep(120000)
// Stop the camel process
context.stop
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5670575.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: File Processor Not deleting the files
Posted by rdifrango <ro...@gmail.com>.
Camel version 2.9.2 on a Windows 7 Machine.
--
View this message in context: http://camel.465427.n5.nabble.com/File-Processor-Not-deleting-the-files-tp5670301p5670530.html
Sent from the Camel - Users mailing list archive at Nabble.com.