You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by "John F. Berry" <bo...@yahoo.com.INVALID> on 2018/12/13 20:48:45 UTC
[Solved] XML file contains filename to be moved (with itself)..
what packages should I use?
Thank you Claus for your help..I got it running!
package org.mainegeneral.camel;
import org.apache.camel.Processor;
import org.apache.camel.Exchange;
import org.apache.camel.builder.RouteBuilder;
public class MyRouteBuilder extends RouteBuilder {
public void configure() {
from("file:src/data?noop=true&idempotent=false&include=.*\\.xml")
.setProperty("SourceFN").xpath("/TOUCHWORKS/DICTATION/SOURCE_FILENAME", String.class)
.log("Reading XML file: ${header.CamelFileName}")
.log("XML Source Filename = ${property.SourceFN}")
.to("bean:RouteBean?method=MoveFiles(${property.SourceFN},${header.CamelFileName})");
}
}
==========RouteBean.java==================
import java.io.*;
import java.nio.file.Files;
import java.nio.file.*;
import java.io.IOException;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
public class RouteBean{
public void MoveFiles(String SourceFN, String XMLFN) throws IOException {
DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
Date date = new Date();
String ArchiveDate = dateFormat.format(date);
File SourceFile = new File("\\\\MGH16557\\c$\\TWtoFUSvoice\\src\\data\\" + SourceFN);
Path ArchiveDir = Paths.get("\\\\MGH16557\\c$\\TWtoFUSvoice\\src\\data\\archive\\" + ArchiveDate);
if (Files.notExists(ArchiveDir)) { Files.createDirectories(ArchiveDir); }
if (SourceFile.exists()) {
Path tmpcopy = Files.copy
(Paths.get("\\\\MGH16557\\c$\\TWtoFUSvoice\\src\\data\\" + SourceFN),
Paths.get(ArchiveDir + "\\" + SourceFN));
Path tmpmove = Files.move
(Paths.get("\\\\MGH16557\\c$\\TWtoFUSvoice\\src\\data\\" + SourceFN),
Paths.get("\\\\MGH16557\\c$\\TWtoFUSvoice\\target\\messages\\others\\" + SourceFN));
Path tmpcopy1 = Files.copy
(Paths.get("\\\\MGH16557\\c$\\TWtoFUSvoice\\src\\data\\" + XMLFN),
Paths.get(ArchiveDir + "\\" + XMLFN));
Path tmpmove1 = Files.move
(Paths.get("\\\\MGH16557\\c$\\TWtoFUSvoice\\src\\data\\" + XMLFN),
Paths.get("\\\\MGH16557\\c$\\TWtoFUSvoice\\target\\messages\\others\\" + XMLFN));
}
}
}
On Thursday, December 13, 2018, 12:28:56 PM EST, John F. Berry <bo...@yahoo.com> wrote:
So I execute and all I get is this... it just sits here:
camel.MainApp.main()] FileEndpoint INFO Endpoint is configured with noop=true so forcing endpoint to be idempotent as well
camel.MainApp.main()] FileEndpoint INFO Using default memory based idempotent repository with cache max size: 1000
camel.MainApp.main()] DefaultCamelContext INFO Route: route1 started and consuming from: file://src/data?include=.*%5C.xml&noop=true
camel.MainApp.main()] DefaultCamelContext INFO Total 1 routes, of which 1 are started
camel.MainApp.main()] DefaultCamelContext INFO Apache Camel 2.23.0 (CamelContext: camel-1) started in 1.581 seconds
On Thursday, December 13, 2018, 12:14:11 PM EST, John F. Berry <bo...@yahoo.com> wrote:
OK.. I currently have it configured to read in the XML, pull the contained second filename, log it, but send it to a bean for it to process both file moves. When I execute.. I don't even get the first log statements....
package org.mainegeneral.camel;
import org.apache.camel.Processor;
import org.apache.camel.Exchange;
import org.apache.camel.builder.RouteBuilder;
public class MyRouteBuilder extends RouteBuilder {
public void configure() {
from("file:src/data?noop=true&include=.*\\.xml")
.setProperty("SourceFN").xpath("/TOUCHWORKS/DICTATION/SOURCE_FILENAME", String.class)
.log("Reading XML file: ${header.CamelFileName}")
.log("XML Source Filename = ${property.SourceFN}")
.to("bean:RouteBean?method=MoveFiles(${property.SourceFN},${header.CamelFileName})");
}
}
=======RouteBean.java=======
import java.io.*;
import java.nio.file.Files;
import java.nio.file.*;
import java.io.IOException;
public class RouteBean{
public void MoveFiles(String SourceFN, String XMLFN) throws IOException {
File SourceFile = new File("\\\\MGH16557\\c$\\TWtoFUSvoice\\src\\data\\" + SourceFN);
if (SourceFile.exists()) {
Path temp = Files.move
(Paths.get("\\\\MGH16557\\c$\\TWtoFUSvoice\\src\\data\\" + SourceFN),
Paths.get("\\\\MGH16557\\c$\\TWtoFUSvoice\\target\\messages\\others\\" + SourceFN));
Path temp1 = Files.move
(Paths.get("\\\\MGH16557\\c$\\TWtoFUSvoice\\src\\data\\" + XMLFN),
Paths.get("\\\\MGH16557\\c$\\TWtoFUSvoice\\target\\messages\\others\\" + XMLFN));
}
}
}
Shouldn't I see.. even if the bean blows up... my log entries? It was working before I took out the ".to" to be a bean instead of a file.
On Thursday, December 13, 2018, 10:18:41 AM EST, Claus Ibsen <cl...@gmail.com> wrote:
Hi
If you in the first route just solves to move files to another folder,
when you have both of them (eg the pairs you talked a bit about).
The 2nd route can then process these files, knowing that both of them
are available.
Camel does have support for "done files" but they are based on a
simpler pattern, where its based on file name, eg foo.xml and foo.done
etc.
So you cant use that with a more complex pattern where the file name
is dynamic and also needs to be extracted via xpath.
You can also just move that other file yourself, so when you have the
file name extracted via xpath, then you can use a bit of java code to
check if it exists, and if so then move this file, and let Camel move
the xml file afterwards.
Or you can move both of them manually (and leave noop=true) so Camel
wont attempt to move the file also.
On Thu, Dec 13, 2018 at 3:33 PM John F. Berry
<bo...@yahoo.com.invalid> wrote:
>
> Yes.. this was my thinking, but with either scenario, I need to comingle exchanges to know what the first route is reading to make the decision in the second route.. In the previous e-mail you suggested I can extract the xpath in another java instance outside the route.. but I need that to know which file the route is currently consuming. So I can successfully do all these things.. each in their own "bubble".. but I lose it in the coordination between them.
> On Thursday, December 13, 2018 09:20:59 AM EST, Claus Ibsen <cl...@gmail.com> wrote:
>
> Hi
>
> An alternative is maybe to use 2 routes, where the first route checks
> if that 2nd file exists with a <choice> and then move the files to
> another folder, where the 2nd route pickup and process that file,
> knowing that the other file exists.
> On Thu, Dec 13, 2018 at 3:20 PM Claus Ibsen <cl...@gmail.com> wrote:
> >
> > Hi
> >
> > You can use a bit of Java code to read the file content into memory, and then use xpath to extract that other file name.
> > Camel has some helper APIs for that in IOHelper#loadTexxt and XPathBuilder.
> >
> > Here is some snippet from an unit test
> >
> > @Test
> > public void testNotUsingExchangeResultType() throws Exception {
> > String xml = "<xml><a>1</a><a>2</a></xml>";
> >
> > // will evaluate as NodeSet
> > XPathBuilder xpb = new XPathBuilder("/xml/a/text()");
> > assertEquals("12", xpb.evaluate(context, xml, String.class));
> >
> > xpb.setResultType(String.class);
> > assertEquals("1", xpb.evaluate(context, xml));
> > }
> > On Thu, Dec 13, 2018 at 2:58 PM John F. Berry <bo...@yahoo.com.invalid> wrote:
> > >
> > > Thank you Claus.. it's great to hear from you again..
> > > I did see an example close to this on stackoverflow.. someone referenced a bean as a filter to check for an extra file to exist, but in that example the extra file's name that needed to exist was static.The "chicken or the egg" question about that is.. I need to use the xpath in the current consumed XML to determine what file should exist for the route to continue. I haven't seen to many examples of parameter passing of the internal workings of a Camel route to beans and the route accepting data from to use in <choice>. I did think I needed to pass this off to another java execution, thus the reason I'm pulling my results to the log and so forth, but I still thought I needed the facilites of the CamelContext to run first... or is the filter not really so close on incept of an inbound file as I think? Is the inbound "message" (file in this case) really "consumed" as in loaded/headers made, etc. before the filter instruction applies?
> > > Thanks for the heads up on noop.. I did know I had to address that once things were in place.. I put it there to test "once throughs" and was going to adjust that later in testing.
> > >
> > >
> > > On Thursday, December 13, 2018 02:43:00 AM EST, Claus Ibsen <cl...@gmail.com> wrote:
> > >
> > > Hi
> > >
> > > Just use Java - You can use Java File API to check for the existence
> > > of a file ;)
> > >
> > > You can also use a file filter to only pickup the file where you check
> > > for the existence of that other file.
> > > You can then write a bit of Java code as the filter and configure it
> > > on the file endpoint. In the filter you grab the file context, do the
> > > xpath to extract the name, and check for the file exists.
> > >
> > > Also mind that noop=true will leave the file as-is after processing
> > > it, but will not pickup it again, unless you configure the idempotent
> > > to false also. Which you would need in cases while you may
> > > wait for that other file.
> > >
> > > So the better alternative is the file filter, and then move the files
> > > after it has been processed, so you wont process the file again in the
> > > future.
> > > On Thu, Dec 13, 2018 at 1:58 AM John F. Berry
> > > <bo...@yahoo.com.invalid> wrote:
> > > >
> > > > OK.. not hearing any pre-build advise.. I started building.
> > > > I can successfully extract the XML field value that is the name of the work file out of the consumed file of the route.
> > > > What I need to do is only move both (the file I'm consuming and it's work file).. if the work file shows up..
> > > >
> > > > Here's what I got so far....
> > > >
> > > >
> > > > package org.mainegeneral.camel;
> > > > import org.apache.camel.Processor;
> > > > import org.apache.camel.Exchange;
> > > > import org.apache.camel.builder.RouteBuilder;
> > > > public class MyRouteBuilder extends RouteBuilder {
> > > > public void configure() {
> > > > from("file:src/data?noop=true&include=.*\\.xml")
> > > > .setProperty("SourceFN").xpath("/SYSTEMJOB/WORKDETAIL/SOURCE_FILENAME", String.class)
> > > > .process(new Processor() {
> > > > public void process(Exchange exchange) throws Exception {
> > > > }
> > > > } )
> > > > .log("Reading XML file: ${header.CamelFileName}")
> > > > .log("XML Source Filename = ${property.SourceFN}")
> > > > .to("file:target/messages/others");
> > > > }
> > > > }
> > > >
> > > >
> > > > Now I will need to not move the XML I'm consuming if the file named in SOURCE_FILENAME doesn't yet exist, and go on to the other XML files in the directory and come back to check this one.
> > > > I was going to use a choice to perform both those moves in.. but struggling to check for the named SOURCE_FILENAME's existance.
> > > >
> > > > Suggestions?
> > > >
> > > > Thanks!
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Monday, December 10, 2018, 3:10:09 PM EST, John F. Berry <bo...@yahoo.com.INVALID> wrote:
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Figured I'd try something new and ask for advice here first before hitting a brick wall and asking after I hit that dead end...I need to poll for XML files dumping into a specific NFS directory, read in the XML, find the "work" file generated by the task that generated the XML, and move both files together. The XML file is a custom small app specific job detail sort of file, with the exported work contained in a file named between the <SOURCE_FILENAME> tag.I could just regex the file, instead of parsing XML "officially", then use a choice and when satisfied, move both the file I'm reading, and the result from the regex. I don't really need the 50 or so other job statistic tags contained in the XML. I plan to use Java DSL with a blueprint or some Karaf supported type to run as a bundle.
> > > > Since these xml files are generated at invocation, the "work" file might not show up for a few minutes. In that time, I'm hoping to round-robin any other XML files to find completed "pairs" in this landing directory. Sound simple enough? Any gotchas that people can think of?
> > > > Thanks!
> > >
> > >
> > >
> > > --
> > > Claus Ibsen
> > > -----------------
> > > http://davsclaus.com @davsclaus
> > > Camel in Action 2: https://www.manning.com/ibsen2
> >
> >
> >
> > --
> > Claus Ibsen
> > -----------------
> > http://davsclaus.com @davsclaus
> > Camel in Action 2: https://www.manning.com/ibsen2
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2