You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Licia <al...@gmail.com> on 2014/06/13 11:19:19 UTC
Problem with ETL example
Hi everyone,
I'm all new to Camel and after getting documentation, I went directly to the
ETL example because it really looks like what I'm trying to do.
So I just did mvn compile, which worked (everything did in the directory
where the POM is).
And then I tried mvn camel:run and I get a problem : it looks like the same
message get displayed over and over again.
"*2014-06-13 11:07:52,094 [.CustomerEntity] INFO Tracer
-
ID-ITEM-93866-58463-1402650414339-0-78 >>> (route2)
from(jpa://org.apache.camel
.example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=false&delay=30
00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
Headers:{CamelEntityManager
=org.apache.openjpa.persistence.EntityManagerImpl@5d84d74b,
breadcrumbId=ID-ITEM
-93866-58463-1402650414339-0-77},
BodyType:org.apache.camel.example.etl.Customer
Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<customer id="1">
<userName>james</userName>
<firstName>James</firstName>
<surname>Strachan</surname>
<city>London</city>
</customer>*
2014-06-13 11:07:52,096 [.CustomerEntity] INFO Tracer
-
ID-ITEM-93866-58463-1402650414339-0-78 >>> (route2)
setHeader[CamelFileName] --
> file://target/customers <<< Pattern:InOnly,
> Headers:{breadcrumbId=ID-ITEM-9386
6-58463-1402650414339-0-77, CamelFileName=james.xml,
CamelEntityManager=org.apac
he.openjpa.persistence.EntityManagerImpl@5d84d74b},
BodyType:org.apache.camel.ex
ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-8"
standalone="
yes"?>
<customer id="1">
<userName>james</userName>
<firstName>James</firstName>
<surname>Strachan</surname>
<city>London</city>
</customer>
2014-06-13 11:07:52,136 [.CustomerEntity] INFO Tracer
-
ID-ITEM-93866-58463-1402650414339-0-80 >>> (route2)
from(jpa://org.apache.camel
.example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=false&delay=30
00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
Headers:{CamelEntityManager
=org.apache.openjpa.persistence.EntityManagerImpl@5d84d74b,
breadcrumbId=ID-ITEM
-93866-58463-1402650414339-0-79},
BodyType:org.apache.camel.example.etl.Customer
Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<customer id="2">
<userName>hiram</userName>
<firstName>Hiram</firstName>
<surname>Chirino</surname>
<city>Tampa</city>
</customer>
2014-06-13 11:07:52,137 [.CustomerEntity] INFO Tracer
-
ID-ITEM-93866-58463-1402650414339-0-80 >>> (route2)
setHeader[CamelFileName] --
> file://target/customers <<< Pattern:InOnly,
> Headers:{CamelEntityManager=org.ap
ache.openjpa.persistence.EntityManagerImpl@5d84d74b,
breadcrumbId=ID-ITEM-93866-
58463-1402650414339-0-79, CamelFileName=hiram.xml},
BodyType:org.apache.camel.ex
ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-8"
standalone="
yes"?>
<customer id="2">
<userName>hiram</userName>
<firstName>Hiram</firstName>
<surname>Chirino</surname>
<city>Tampa</city>
</customer>
60513 camel TRACE [Camel (camel) thread #1 -
jpa://org.apache.camel.example.e
tl.CustomerEntity] openjpa.jdbc.SQL - <t 53213167, conn 227180630> executing
pre
pstmnt 882159474
SELECT t0.id, t0.city, t0.firstName, t0.phone, t0.street, t0.surname,
t0.userName, t0.zip
FROM customer t0
60514 camel TRACE [Camel (camel) thread #1 -
jpa://org.apache.camel.example.e
tl.CustomerEntity] openjpa.jdbc.SQL - <t 53213167, conn 227180630> [0 ms]
spent
*2014-06-13 11:07:55,165 [.CustomerEntity] INFO Tracer
-
ID-ITEM-93866-58463-1402650414339-0-82 >>> (route2)
from(jpa://org.apache.camel
.example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=false&delay=30
00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
Headers:{breadcrumbId=ID-IT
EM-93866-58463-1402650414339-0-81,
CamelEntityManager=org.apache.openjpa.persist
ence.EntityManagerImpl@5d84d74b},
BodyType:org.apache.camel.example.etl.Customer
Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<customer id="1">
<userName>james</userName>
<firstName>James</firstName>
<surname>Strachan</surname>
<city>London</city>
</customer>*
2014-06-13 11:07:55,167 [.CustomerEntity] INFO Tracer
-
ID-ITEM-93866-58463-1402650414339-0-82 >>> (route2)
setHeader[CamelFileName] --
> file://target/customers <<< Pattern:InOnly,
> Headers:{breadcrumbId=ID-ITEM-9386
6-58463-1402650414339-0-81, CamelFileName=james.xml,
CamelEntityManager=org.apac
he.openjpa.persistence.EntityManagerImpl@5d84d74b},
BodyType:org.apache.camel.ex
ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-8"
standalone="
yes"?>
<customer id="1">
<userName>james</userName>
<firstName>James</firstName>
<surname>Strachan</surname>
<city>London</city>
</customer>
2014-06-13 11:07:55,196 [.CustomerEntity] INFO Tracer
-
ID-ITEM-93866-58463-1402650414339-0-84 >>> (route2)
from(jpa://org.apache.camel
.example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=false&delay=30
00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
Headers:{breadcrumbId=ID-IT
EM-93866-58463-1402650414339-0-83,
CamelEntityManager=org.apache.openjpa.persist
ence.EntityManagerImpl@5d84d74b},
BodyType:org.apache.camel.example.etl.Customer
Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<customer id="2">
<userName>hiram</userName>
<firstName>Hiram</firstName>
<surname>Chirino</surname>
<city>Tampa</city>
</customer>
2014-06-13 11:07:55,198 [.CustomerEntity] INFO Tracer
-
ID-ITEM-93866-58463-1402650414339-0-84 >>> (route2)
setHeader[CamelFileName] --
> file://target/customers <<< Pattern:InOnly,
> Headers:{CamelEntityManager=org.ap
ache.openjpa.persistence.EntityManagerImpl@5d84d74b,
CamelFileName=hiram.xml, br
eadcrumbId=ID-ITEM-93866-58463-1402650414339-0-83},
BodyType:org.apache.camel.ex
ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-8"
standalone="
yes"?>
<customer id="2">
<userName>hiram</userName>
<firstName>Hiram</firstName>
<surname>Chirino</surname>
<city>Tampa</city>
</customer>
"
The two bold parts are the exact same ones.
Plus I'm really confused about what the example is supposed to do. Isn't it
supposed to create the java classes ? Or am I completely wrong ?
Else, isn't it supposed to create the persons in my database ? How do I see
if it has been done ?
Thanks a lot for your answers.
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-with-ETL-example-tp5752256.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: AW: Problem with ETL example
Posted by Babak Vahdat <ba...@swissonline.ch>.
In addition to what Jan has already said, maybe you want to go through the
following blog post to get a better understanding about this example:
http://saltnlight5.blogspot.ch/2013/08/getting-started-with-apache-camel-using.html
Babak
Licia wrote
> Thanks a lot ! It works now, I mean nothing get to be print on the
> console.
>
> But what I'm wondering is the following.
>
> I don't really understand how the Main class works. It just goes :
>
> public class Main extends org.apache.camel.spring.Main {
> public static void main(String... args) throws Exception {
> new Main().run(args);
> }
> }
>
> But it doesn't "launch" any method really. What is this run ? What is it
> doing ?
> Then I understand that the CustomerEntity.java describes the schema of the
> base and the PersonDocument uses JAXB to express what XML elements of the
> hiram.xml and james.xml correspond to a root, an element, an attribute
> etc.
> I also get the fact that the thing that converts the PersonDocument into
> CustomerEntity is the CustomerTransformer.
>
> And I understant the "general meaning" of routes even if here I don't
> really understand the details.
>
> But what I really don't get is how all the instructions are executed. In
> what order, who asks them to, etc.
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-with-ETL-example-tp5752256p5752278.html
Sent from the Camel - Users mailing list archive at Nabble.com.
AW: AW: Problem with ETL example
Posted by "Jan Matèrne (jhm)" <ap...@materne.de>.
This main class invoked the run(String[]) method defined in the superclass
org.apache.camel.spring.Main. Without reading that I guess it defines/loads
the routes
and starts the camel context.
Reading the files is done by the file-component ...
Jan
> -----Ursprüngliche Nachricht-----
> Von: Licia [mailto:alicia.parisse@gmail.com]
> Gesendet: Freitag, 13. Juni 2014 15:18
> An: users@camel.apache.org
> Betreff: Re: AW: Problem with ETL example
>
> Thanks a lot ! It works now, I mean nothing get to be print on the
> console.
>
> But what I'm wondering is the following.
>
> I don't really understand how the Main class works. It just goes :
>
> public class Main extends org.apache.camel.spring.Main {
> public static void main(String... args) throws Exception {
> new Main().run(args);
> }
> }
>
> But it doesn't "launch" any method really. What is this run ? What is
> it doing ?
> Then I understand that the CustomerEntity.java describes the schema of
> the base and the PersonDocument uses JAXB to express what XML elements
> of the hiram.xml and james.xml correspond to a root, an element, an
> attribute etc.
> I also get the fact that the thing that converts the PersonDocument
> into CustomerEntity is the CustomerTransformer.
>
> And I understant the "general meaning" of routes even if here I don't
> really understand the details.
>
> But what I really don't get is how all the instructions are executed.
> In what order, who asks them to, etc.
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Problem-with-ETL-example-
> tp5752256p5752270.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Problem with ETL example
Posted by Christoph Emmersberger <ce...@gmail.com>.
You’ll probably understand this a bit better when looking at the class “org.apache.camel.spring.Main".
Best,
Christoph
On 13 Jun 2014, at 15:17, Licia <al...@gmail.com> wrote:
> Thanks a lot ! It works now, I mean nothing get to be print on the console.
>
> But what I'm wondering is the following.
>
> I don't really understand how the Main class works. It just goes :
>
> public class Main extends org.apache.camel.spring.Main {
> public static void main(String... args) throws Exception {
> new Main().run(args);
> }
> }
>
> But it doesn't "launch" any method really. What is this run ? What is it
> doing ?
> Then I understand that the CustomerEntity.java describes the schema of the
> base and the PersonDocument uses JAXB to express what XML elements of the
> hiram.xml and james.xml correspond to a root, an element, an attribute etc.
> I also get the fact that the thing that converts the PersonDocument into
> CustomerEntity is the CustomerTransformer.
>
> And I understant the "general meaning" of routes even if here I don't really
> understand the details.
>
> But what I really don't get is how all the instructions are executed. In
> what order, who asks them to, etc.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Problem-with-ETL-example-tp5752256p5752270.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: AW: Problem with ETL example
Posted by Licia <al...@gmail.com>.
Thanks a lot ! It works now, I mean nothing get to be print on the console.
But what I'm wondering is the following.
I don't really understand how the Main class works. It just goes :
public class Main extends org.apache.camel.spring.Main {
public static void main(String... args) throws Exception {
new Main().run(args);
}
}
But it doesn't "launch" any method really. What is this run ? What is it
doing ?
Then I understand that the CustomerEntity.java describes the schema of the
base and the PersonDocument uses JAXB to express what XML elements of the
hiram.xml and james.xml correspond to a root, an element, an attribute etc.
I also get the fact that the thing that converts the PersonDocument into
CustomerEntity is the CustomerTransformer.
And I understant the "general meaning" of routes even if here I don't really
understand the details.
But what I really don't get is how all the instructions are executed. In
what order, who asks them to, etc.
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-with-ETL-example-tp5752256p5752270.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: AW: Problem with ETL example
Posted by Babak Vahdat <ba...@swissonline.ch>.
This is part of how Apache OpenJPA has been setup for this example, so if you
don't want to see those logs then just tweak the logging configuration a
bit:
https://github.com/apache/camel/blob/master/examples/camel-example-etl/src/main/resources/META-INF/persistence.xml#L27
E.g. set all the levels there to "WARN" instead of "INFO" or "TRACE".
Also note that Camel tracing is enabled by this example, so you would need
to disable tracing as well if you don't want to see the Camel's own trace
logs:
https://github.com/apache/camel/blob/master/examples/camel-example-etl/src/main/resources/META-INF/spring/camel-context.xml#L23
Now try again and the console will become completely silent after the two
database rows of "James" & "Hiram" get successfully consumed.
Babak
Licia wrote
> First of, thank you for your answer.
>
> But the thing is the probleme is not "really" solved because after i
> compiled using your tip, the console is still running on and on again,
> but with a different type of message that I understand even less than
> before...
>
> And yes, I have already been on the ETL example tutorial but it didn't
> really help...
>
>
> 26626 camel TRACE [Camel (camel) thread #1 -
> jpa://org.apache.camel.example.e
> tl.CustomerEntity] openjpa.jdbc.SQL -
> <t 195557432, conn 1486709135>
> [0 ms] spen
> t
> 29630 camel TRACE [Camel (camel) thread #1 -
> jpa://org.apache.camel.example.e
> tl.CustomerEntity] openjpa.jdbc.SQL -
> <t 195557432, conn 1154071886>
> executing p
> repstmnt 2005085230
> SELECT t0.id, t0.city, t0.firstName, t0.phone, t0.street, t0.surname,
> t0.userName, t0.zip
> FROM customer t0
>
> 29631 camel TRACE [Camel (camel) thread #1 -
> jpa://org.apache.camel.example.e
> tl.CustomerEntity] openjpa.jdbc.SQL -
> <t 195557432, conn 1154071886>
> [1 ms] spen
> t
> 32637 camel TRACE [Camel (camel) thread #1 -
> jpa://org.apache.camel.example.e
> tl.CustomerEntity] openjpa.jdbc.SQL -
> <t 195557432, conn 1716758312>
> executing p
> repstmnt 404887544
> SELECT t0.id, t0.city, t0.firstName, t0.phone, t0.street, t0.surname,
> t0.userName, t0.zip
> FROM customer t0
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-with-ETL-example-tp5752256p5752268.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: AW: Problem with ETL example
Posted by Licia <al...@gmail.com>.
First of, thank you for your answer.
But the thing is the probleme is not "really" solved because after i
compiled using your tip, the console is still running on and on again, but
with a different type of message that I understand even less than before...
26626 camel TRACE [Camel (camel) thread #1 -
jpa://org.apache.camel.example.e
tl.CustomerEntity] openjpa.jdbc.SQL - <t 195557432, conn 1486709135> [0 ms]
spen
t
29630 camel TRACE [Camel (camel) thread #1 -
jpa://org.apache.camel.example.e
tl.CustomerEntity] openjpa.jdbc.SQL - <t 195557432, conn 1154071886>
executing p
repstmnt 2005085230
SELECT t0.id, t0.city, t0.firstName, t0.phone, t0.street, t0.surname,
t0.userName, t0.zip
FROM customer t0
29631 camel TRACE [Camel (camel) thread #1 -
jpa://org.apache.camel.example.e
tl.CustomerEntity] openjpa.jdbc.SQL - <t 195557432, conn 1154071886> [1 ms]
spen
t
32637 camel TRACE [Camel (camel) thread #1 -
jpa://org.apache.camel.example.e
tl.CustomerEntity] openjpa.jdbc.SQL - <t 195557432, conn 1716758312>
executing p
repstmnt 404887544
SELECT t0.id, t0.city, t0.firstName, t0.phone, t0.street, t0.surname,
t0.userName, t0.zip
FROM customer t0
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-with-ETL-example-tp5752256p5752264.html
Sent from the Camel - Users mailing list archive at Nabble.com.
AW: AW: Problem with ETL example
Posted by "Jan Matèrne (jhm)" <ap...@materne.de>.
Thanks for explanation. I updated the wiki ;)
Jan
> -----Ursprüngliche Nachricht-----
> Von: Babak Vahdat [mailto:babak.vahdat@swissonline.ch]
> Gesendet: Freitag, 13. Juni 2014 13:15
> An: users@camel.apache.org
> Betreff: Re: AW: Problem with ETL example
>
> No, the file consumer makes use of the memory based Idempotent Consumer
> here.
> No matter if you make use of noop true/false it will consume each file
> *only* once. Change noop to false or remove it and you will not see any
> change in the behavior of the second route. The first route will
> *always* consume *only* 2 times (row1.xml & row2.xml) no matter what
> the noop option value is.
>
> To verify this one can also make use of hawtio, so just run:
>
> ~/dev/workspace/camel/examples/camel-example-etl>mvn
> io.hawt:hawtio-maven-plugin:1.4.4:camel
>
> Then refresh you browser and jump to the following URL:
>
> http://localhost:8080/hawtio/#/jmx/attributes?tab=camel&nid=root-
> org.apache.camel-camel-routes
>
> And you should see something like Screen_Shot_2014-06-13_at_13.png
> <http://camel.465427.n5.nabble.com/file/n5752263/Screen_Shot_2014-06-
> 13_at_13.png>
>
> As you see the the first route has the “Completed #” column count equal
> to 2 and the number doesn’t increase anymore, however the second route
> counts up continuously.
>
> Now change consumerDelete option by the second route and run hawtio
> maven plugin again as above and verify the change in the behavior of
> the second route.
>
> Babak
>
>
> Jan Matèrne (jhm) wrote
> > from("file:src/data?noop=true")
> > .convertBodyTo(PersonDocument.class)
> > .to("jpa:org.apache.camel.example.etl.CustomerEntity");
> >
> > // the following will dump the database to files
> >
> >
> from("jpa:org.apache.camel.example.etl.CustomerEntity?consumeDelete=fa
> > lse&de
> > lay=3000&consumeLockEntity=false")
> > .setHeader(Exchange.FILE_NAME,
> el("${in.body.userName}.xml"))
> > .to("file:target/customers");
> >
> > I think it is because the first route doesnt move the processed files
> > to somewhere else ...
> > http://camel.apache.org/file2.html
> >
> >
> > Jan
> >
> >
> >> -----Ursprüngliche Nachricht-----
> >> Von: Babak Vahdat [mailto:
>
> > babak.vahdat@
>
> > ]
> >> Gesendet: Freitag, 13. Juni 2014 12:18
> >> An:
>
> > users@.apache
>
> >> Betreff: Re: Problem with ETL example
> >>
> >> Hi
> >>
> >> The reason for the same logs you see is because of the consumeDelete
> >> option being set to false:
> >>
> >> http://camel.apache.org/jpa.html#JPA-Options
> >>
> >> So that the JPA consumer polls the *same* 2 entities again and
> again.
> >> This is just for the demonstration purpose so that you can
> >> continuously see something is running.
> >>
> >> You can set that option to true or remove it (default is true) and
> >> check the change in the behavior here:
> >>
> >> https://github.com/apache/camel/blob/master/examples/camel-example-
> >> etl/src/main/java/org/apache/camel/example/etl/EtlRoutes.java#L36
> >>
> >> And if not already done, maybe you want to go through the
> >> documentation of this example as well:
> >>
> >> http://camel.apache.org/etl-example.html
> >>
> >> Babak
> >>
> >> Licia wrote
> >> > Hi everyone,
> >> >
> >> > I'm all new to Camel and after getting documentation, I went
> >> > directly to the ETL example because it really looks like what I'm
> >> > trying to
> >> do.
> >> >
> >> > So I just did mvn compile, which worked (everything did in the
> >> > directory where the POM is).
> >> > And then I tried mvn camel:run and I get a problem : it looks like
> >> the
> >> > same message get displayed over and over again.
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > "
> >> *
> >> > 2014-06-13 11:07:52,094 [.CustomerEntity] INFO Tracer
> >> > -
> >> > ID-ITEM-93866-58463-1402650414339-0-78 >>> (route2)
> >> > from(jpa://org.apache.camel
> >> >
> >>
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fal
> >> s
> >> > e&delay=30
> >> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> >> > Headers:{CamelEntityManager
> >> > =org.apache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> >> > breadcrumbId=ID-ITEM
> >> > -93866-58463-1402650414339-0-77},
> >> > BodyType:org.apache.camel.example.etl.Customer
> >> > Entity, Body:<?xml version="1.0" encoding="UTF-8"
> >> > standalone="yes"?>
> >> >
> > <customer id="1">
> >> >
> >> >
> > <userName>
> >> > james
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > James
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Strachan
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > London
> >> >
> > </city>
> >> >
> > </customer>
> >> *
> >> >
> >> > 2014-06-13 11:07:52,096 [.CustomerEntity] INFO Tracer
> >> > -
> >> > ID-ITEM-93866-58463-1402650414339-0-78 >>> (route2)
> >> > setHeader[CamelFileName] --
> >> >> file://target/customers <<< Pattern:InOnly,
> >> >> Headers:{breadcrumbId=ID-ITEM-9386
> >> > 6-58463-1402650414339-0-77, CamelFileName=james.xml,
> >> > CamelEntityManager=org.apac
> >> > he.openjpa.persistence.EntityManagerImpl@5d84d74b},
> >> > BodyType:org.apache.camel.ex
> >> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-
> 8"
> >> > standalone="
> >> > yes"?>
> >> >
> > <customer id="1">
> >> >
> >> >
> > <userName>
> >> > james
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > James
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Strachan
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > London
> >> >
> > </city>
> >> >
> > </customer>
> >> > 2014-06-13 11:07:52,136 [.CustomerEntity] INFO Tracer
> >> > -
> >> > ID-ITEM-93866-58463-1402650414339-0-80 >>> (route2)
> >> > from(jpa://org.apache.camel
> >> >
> >>
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fal
> >> s
> >> > e&delay=30
> >> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> >> > Headers:{CamelEntityManager
> >> > =org.apache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> >> > breadcrumbId=ID-ITEM
> >> > -93866-58463-1402650414339-0-79},
> >> > BodyType:org.apache.camel.example.etl.Customer
> >> > Entity, Body:<?xml version="1.0" encoding="UTF-8"
> >> > standalone="yes"?>
> >> >
> > <customer id="2">
> >> >
> >> >
> > <userName>
> >> > hiram
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > Hiram
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Chirino
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > Tampa
> >> >
> > </city>
> >> >
> > </customer>
> >> > 2014-06-13 11:07:52,137 [.CustomerEntity] INFO Tracer
> >> > -
> >> > ID-ITEM-93866-58463-1402650414339-0-80 >>> (route2)
> >> > setHeader[CamelFileName] --
> >> >> file://target/customers <<< Pattern:InOnly,
> >> >> Headers:{CamelEntityManager=org.ap
> >> > ache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> >> > breadcrumbId=ID-ITEM-93866-
> >> > 58463-1402650414339-0-79, CamelFileName=hiram.xml},
> >> > BodyType:org.apache.camel.ex ample.etl.CustomerEntity, Body:<?xml
> >> > version="1.0" encoding="UTF-8"
> >> > standalone="
> >> > yes"?>
> >> >
> > <customer id="2">
> >> >
> >> >
> > <userName>
> >> > hiram
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > Hiram
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Chirino
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > Tampa
> >> >
> > </city>
> >> >
> > </customer>
> >> > 60513 camel TRACE [Camel (camel) thread #1 -
> >> > jpa://org.apache.camel.example.e tl.CustomerEntity]
> >> > openjpa.jdbc.SQL
> >> -
> >> >
> > <t 53213167, conn 227180630>
> > executing pre pstmnt 882159474 SELECT
> >> > t0.id, t0.city, t0.firstName, t0.phone, t0.street, t0.surname,
> >> > t0.userName, t0.zip
> >> > FROM customer t0
> >> >
> >> > 60514 camel TRACE [Camel (camel) thread #1 -
> >> > jpa://org.apache.camel.example.e tl.CustomerEntity]
> >> > openjpa.jdbc.SQL
> >> -
> >> >
> > <t 53213167, conn 227180630>
> > [0 ms] spent
> >> *
> >> > 2014-06-13 11:07:55,165 [.CustomerEntity] INFO Tracer
> >> > -
> >> > ID-ITEM-93866-58463-1402650414339-0-82 >>> (route2)
> >> > from(jpa://org.apache.camel
> >> >
> >>
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fal
> >> s
> >> > e&delay=30
> >> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> >> > Headers:{breadcrumbId=ID-IT EM-93866-58463-1402650414339-0-81,
> >> > CamelEntityManager=org.apache.openjpa.persist
> >> > ence.EntityManagerImpl@5d84d74b},
> >> > BodyType:org.apache.camel.example.etl.Customer
> >> > Entity, Body:<?xml version="1.0" encoding="UTF-8"
> >> > standalone="yes"?>
> >> >
> > <customer id="1">
> >> >
> >> >
> > <userName>
> >> > james
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > James
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Strachan
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > London
> >> >
> > </city>
> >> >
> > </customer>
> >> *
> >> >
> >> > 2014-06-13 11:07:55,167 [.CustomerEntity] INFO Tracer
> >> > -
> >> > ID-ITEM-93866-58463-1402650414339-0-82 >>> (route2)
> >> > setHeader[CamelFileName] --
> >> >> file://target/customers <<< Pattern:InOnly,
> >> >> Headers:{breadcrumbId=ID-ITEM-9386
> >> > 6-58463-1402650414339-0-81, CamelFileName=james.xml,
> >> > CamelEntityManager=org.apac
> >> > he.openjpa.persistence.EntityManagerImpl@5d84d74b},
> >> > BodyType:org.apache.camel.ex
> >> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-
> 8"
> >> > standalone="
> >> > yes"?>
> >> >
> > <customer id="1">
> >> >
> >> >
> > <userName>
> >> > james
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > James
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Strachan
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > London
> >> >
> > </city>
> >> >
> > </customer>
> >> > 2014-06-13 11:07:55,196 [.CustomerEntity] INFO Tracer
> >> > -
> >> > ID-ITEM-93866-58463-1402650414339-0-84 >>> (route2)
> >> > from(jpa://org.apache.camel
> >> >
> >>
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fal
> >> s
> >> > e&delay=30
> >> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> >> > Headers:{breadcrumbId=ID-IT EM-93866-58463-1402650414339-0-83,
> >> > CamelEntityManager=org.apache.openjpa.persist
> >> > ence.EntityManagerImpl@5d84d74b},
> >> > BodyType:org.apache.camel.example.etl.Customer
> >> > Entity, Body:<?xml version="1.0" encoding="UTF-8"
> >> > standalone="yes"?>
> >> >
> > <customer id="2">
> >> >
> >> >
> > <userName>
> >> > hiram
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > Hiram
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Chirino
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > Tampa
> >> >
> > </city>
> >> >
> > </customer>
> >> > 2014-06-13 11:07:55,198 [.CustomerEntity] INFO Tracer
> >> > -
> >> > ID-ITEM-93866-58463-1402650414339-0-84 >>> (route2)
> >> > setHeader[CamelFileName] --
> >> >> file://target/customers <<< Pattern:InOnly,
> >> >> Headers:{CamelEntityManager=org.ap
> >> > ache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> >> > CamelFileName=hiram.xml, br
> >> > eadcrumbId=ID-ITEM-93866-58463-1402650414339-0-83},
> >> > BodyType:org.apache.camel.ex
> >> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-
> 8"
> >> > standalone="
> >> > yes"?>
> >> >
> > <customer id="2">
> >> >
> >> >
> > <userName>
> >> > hiram
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > Hiram
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Chirino
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > Tampa
> >> >
> > </city>
> >> >
> > </customer>
> >> > "
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > The two bold parts are the exact same ones.
> >> >
> >> > Plus I'm really confused about what the example is supposed to do.
> >> > Isn't it supposed to create the java classes ? Or am I completely
> >> wrong ?
> >> > Else, isn't it supposed to create the persons in my database ? How
> >> > do I see if it has been done ?
> >> >
> >> > Thanks a lot for your answers.
> >>
> >>
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >> http://camel.465427.n5.nabble.com/Problem-with-ETL-example-
> >> tp5752256p5752259.html
> >> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Problem-with-ETL-example-
> tp5752256p5752263.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: AW: Problem with ETL example
Posted by Babak Vahdat <ba...@swissonline.ch>.
No, the file consumer makes use of the memory based Idempotent Consumer here.
No matter if you make use of noop true/false it will consume each file
*only* once. Change noop to false or remove it and you will not see any
change in the behavior of the second route. The first route will *always*
consume *only* 2 times (row1.xml & row2.xml) no matter what the noop option
value is.
To verify this one can also make use of hawtio, so just run:
~/dev/workspace/camel/examples/camel-example-etl>mvn
io.hawt:hawtio-maven-plugin:1.4.4:camel
Then refresh you browser and jump to the following URL:
http://localhost:8080/hawtio/#/jmx/attributes?tab=camel&nid=root-org.apache.camel-camel-routes
And you should see something like Screen_Shot_2014-06-13_at_13.png
<http://camel.465427.n5.nabble.com/file/n5752263/Screen_Shot_2014-06-13_at_13.png>
As you see the the first route has the “Completed #” column count equal to 2
and the number doesn’t increase anymore, however the second route counts up
continuously.
Now change consumerDelete option by the second route and run hawtio maven
plugin again as above and verify the change in the behavior of the second
route.
Babak
Jan Matèrne (jhm) wrote
> from("file:src/data?noop=true")
> .convertBodyTo(PersonDocument.class)
> .to("jpa:org.apache.camel.example.etl.CustomerEntity");
>
> // the following will dump the database to files
>
> from("jpa:org.apache.camel.example.etl.CustomerEntity?consumeDelete=false&de
> lay=3000&consumeLockEntity=false")
> .setHeader(Exchange.FILE_NAME, el("${in.body.userName}.xml"))
> .to("file:target/customers");
>
> I think it is because the first route doesnt move the processed files to
> somewhere else ...
> http://camel.apache.org/file2.html
>
>
> Jan
>
>
>> -----Ursprüngliche Nachricht-----
>> Von: Babak Vahdat [mailto:
> babak.vahdat@
> ]
>> Gesendet: Freitag, 13. Juni 2014 12:18
>> An:
> users@.apache
>> Betreff: Re: Problem with ETL example
>>
>> Hi
>>
>> The reason for the same logs you see is because of the consumeDelete
>> option being set to false:
>>
>> http://camel.apache.org/jpa.html#JPA-Options
>>
>> So that the JPA consumer polls the *same* 2 entities again and again.
>> This is just for the demonstration purpose so that you can continuously
>> see something is running.
>>
>> You can set that option to true or remove it (default is true) and
>> check the change in the behavior here:
>>
>> https://github.com/apache/camel/blob/master/examples/camel-example-
>> etl/src/main/java/org/apache/camel/example/etl/EtlRoutes.java#L36
>>
>> And if not already done, maybe you want to go through the documentation
>> of this example as well:
>>
>> http://camel.apache.org/etl-example.html
>>
>> Babak
>>
>> Licia wrote
>> > Hi everyone,
>> >
>> > I'm all new to Camel and after getting documentation, I went directly
>> > to the ETL example because it really looks like what I'm trying to
>> do.
>> >
>> > So I just did mvn compile, which worked (everything did in the
>> > directory where the POM is).
>> > And then I tried mvn camel:run and I get a problem : it looks like
>> the
>> > same message get displayed over and over again.
>> >
>> >
>> >
>> >
>> >
>> > "
>> *
>> > 2014-06-13 11:07:52,094 [.CustomerEntity] INFO Tracer
>> > -
>> > ID-ITEM-93866-58463-1402650414339-0-78 >>> (route2)
>> > from(jpa://org.apache.camel
>> >
>> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fals
>> > e&delay=30
>> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
>> > Headers:{CamelEntityManager
>> > =org.apache.openjpa.persistence.EntityManagerImpl@5d84d74b,
>> > breadcrumbId=ID-ITEM
>> > -93866-58463-1402650414339-0-77},
>> > BodyType:org.apache.camel.example.etl.Customer
>> > Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
>> >
> <customer id="1">
>> >
>> >
> <userName>
>> > james
>> >
> </userName>
>> >
>> >
> <firstName>
>> > James
>> >
> </firstName>
>> >
>> >
> <surname>
>> > Strachan
>> >
> </surname>
>> >
>> >
> <city>
>> > London
>> >
> </city>
>> >
> </customer>
>> *
>> >
>> > 2014-06-13 11:07:52,096 [.CustomerEntity] INFO Tracer
>> > -
>> > ID-ITEM-93866-58463-1402650414339-0-78 >>> (route2)
>> > setHeader[CamelFileName] --
>> >> file://target/customers <<< Pattern:InOnly,
>> >> Headers:{breadcrumbId=ID-ITEM-9386
>> > 6-58463-1402650414339-0-77, CamelFileName=james.xml,
>> > CamelEntityManager=org.apac
>> > he.openjpa.persistence.EntityManagerImpl@5d84d74b},
>> > BodyType:org.apache.camel.ex
>> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-8"
>> > standalone="
>> > yes"?>
>> >
> <customer id="1">
>> >
>> >
> <userName>
>> > james
>> >
> </userName>
>> >
>> >
> <firstName>
>> > James
>> >
> </firstName>
>> >
>> >
> <surname>
>> > Strachan
>> >
> </surname>
>> >
>> >
> <city>
>> > London
>> >
> </city>
>> >
> </customer>
>> > 2014-06-13 11:07:52,136 [.CustomerEntity] INFO Tracer
>> > -
>> > ID-ITEM-93866-58463-1402650414339-0-80 >>> (route2)
>> > from(jpa://org.apache.camel
>> >
>> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fals
>> > e&delay=30
>> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
>> > Headers:{CamelEntityManager
>> > =org.apache.openjpa.persistence.EntityManagerImpl@5d84d74b,
>> > breadcrumbId=ID-ITEM
>> > -93866-58463-1402650414339-0-79},
>> > BodyType:org.apache.camel.example.etl.Customer
>> > Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
>> >
> <customer id="2">
>> >
>> >
> <userName>
>> > hiram
>> >
> </userName>
>> >
>> >
> <firstName>
>> > Hiram
>> >
> </firstName>
>> >
>> >
> <surname>
>> > Chirino
>> >
> </surname>
>> >
>> >
> <city>
>> > Tampa
>> >
> </city>
>> >
> </customer>
>> > 2014-06-13 11:07:52,137 [.CustomerEntity] INFO Tracer
>> > -
>> > ID-ITEM-93866-58463-1402650414339-0-80 >>> (route2)
>> > setHeader[CamelFileName] --
>> >> file://target/customers <<< Pattern:InOnly,
>> >> Headers:{CamelEntityManager=org.ap
>> > ache.openjpa.persistence.EntityManagerImpl@5d84d74b,
>> > breadcrumbId=ID-ITEM-93866-
>> > 58463-1402650414339-0-79, CamelFileName=hiram.xml},
>> > BodyType:org.apache.camel.ex ample.etl.CustomerEntity, Body:<?xml
>> > version="1.0" encoding="UTF-8"
>> > standalone="
>> > yes"?>
>> >
> <customer id="2">
>> >
>> >
> <userName>
>> > hiram
>> >
> </userName>
>> >
>> >
> <firstName>
>> > Hiram
>> >
> </firstName>
>> >
>> >
> <surname>
>> > Chirino
>> >
> </surname>
>> >
>> >
> <city>
>> > Tampa
>> >
> </city>
>> >
> </customer>
>> > 60513 camel TRACE [Camel (camel) thread #1 -
>> > jpa://org.apache.camel.example.e tl.CustomerEntity] openjpa.jdbc.SQL
>> -
>> >
> <t 53213167, conn 227180630>
> executing pre pstmnt 882159474 SELECT
>> > t0.id, t0.city, t0.firstName, t0.phone, t0.street, t0.surname,
>> > t0.userName, t0.zip
>> > FROM customer t0
>> >
>> > 60514 camel TRACE [Camel (camel) thread #1 -
>> > jpa://org.apache.camel.example.e tl.CustomerEntity] openjpa.jdbc.SQL
>> -
>> >
> <t 53213167, conn 227180630>
> [0 ms] spent
>> *
>> > 2014-06-13 11:07:55,165 [.CustomerEntity] INFO Tracer
>> > -
>> > ID-ITEM-93866-58463-1402650414339-0-82 >>> (route2)
>> > from(jpa://org.apache.camel
>> >
>> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fals
>> > e&delay=30
>> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
>> > Headers:{breadcrumbId=ID-IT EM-93866-58463-1402650414339-0-81,
>> > CamelEntityManager=org.apache.openjpa.persist
>> > ence.EntityManagerImpl@5d84d74b},
>> > BodyType:org.apache.camel.example.etl.Customer
>> > Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
>> >
> <customer id="1">
>> >
>> >
> <userName>
>> > james
>> >
> </userName>
>> >
>> >
> <firstName>
>> > James
>> >
> </firstName>
>> >
>> >
> <surname>
>> > Strachan
>> >
> </surname>
>> >
>> >
> <city>
>> > London
>> >
> </city>
>> >
> </customer>
>> *
>> >
>> > 2014-06-13 11:07:55,167 [.CustomerEntity] INFO Tracer
>> > -
>> > ID-ITEM-93866-58463-1402650414339-0-82 >>> (route2)
>> > setHeader[CamelFileName] --
>> >> file://target/customers <<< Pattern:InOnly,
>> >> Headers:{breadcrumbId=ID-ITEM-9386
>> > 6-58463-1402650414339-0-81, CamelFileName=james.xml,
>> > CamelEntityManager=org.apac
>> > he.openjpa.persistence.EntityManagerImpl@5d84d74b},
>> > BodyType:org.apache.camel.ex
>> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-8"
>> > standalone="
>> > yes"?>
>> >
> <customer id="1">
>> >
>> >
> <userName>
>> > james
>> >
> </userName>
>> >
>> >
> <firstName>
>> > James
>> >
> </firstName>
>> >
>> >
> <surname>
>> > Strachan
>> >
> </surname>
>> >
>> >
> <city>
>> > London
>> >
> </city>
>> >
> </customer>
>> > 2014-06-13 11:07:55,196 [.CustomerEntity] INFO Tracer
>> > -
>> > ID-ITEM-93866-58463-1402650414339-0-84 >>> (route2)
>> > from(jpa://org.apache.camel
>> >
>> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fals
>> > e&delay=30
>> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
>> > Headers:{breadcrumbId=ID-IT EM-93866-58463-1402650414339-0-83,
>> > CamelEntityManager=org.apache.openjpa.persist
>> > ence.EntityManagerImpl@5d84d74b},
>> > BodyType:org.apache.camel.example.etl.Customer
>> > Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
>> >
> <customer id="2">
>> >
>> >
> <userName>
>> > hiram
>> >
> </userName>
>> >
>> >
> <firstName>
>> > Hiram
>> >
> </firstName>
>> >
>> >
> <surname>
>> > Chirino
>> >
> </surname>
>> >
>> >
> <city>
>> > Tampa
>> >
> </city>
>> >
> </customer>
>> > 2014-06-13 11:07:55,198 [.CustomerEntity] INFO Tracer
>> > -
>> > ID-ITEM-93866-58463-1402650414339-0-84 >>> (route2)
>> > setHeader[CamelFileName] --
>> >> file://target/customers <<< Pattern:InOnly,
>> >> Headers:{CamelEntityManager=org.ap
>> > ache.openjpa.persistence.EntityManagerImpl@5d84d74b,
>> > CamelFileName=hiram.xml, br
>> > eadcrumbId=ID-ITEM-93866-58463-1402650414339-0-83},
>> > BodyType:org.apache.camel.ex
>> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-8"
>> > standalone="
>> > yes"?>
>> >
> <customer id="2">
>> >
>> >
> <userName>
>> > hiram
>> >
> </userName>
>> >
>> >
> <firstName>
>> > Hiram
>> >
> </firstName>
>> >
>> >
> <surname>
>> > Chirino
>> >
> </surname>
>> >
>> >
> <city>
>> > Tampa
>> >
> </city>
>> >
> </customer>
>> > "
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > The two bold parts are the exact same ones.
>> >
>> > Plus I'm really confused about what the example is supposed to do.
>> > Isn't it supposed to create the java classes ? Or am I completely
>> wrong ?
>> > Else, isn't it supposed to create the persons in my database ? How do
>> > I see if it has been done ?
>> >
>> > Thanks a lot for your answers.
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://camel.465427.n5.nabble.com/Problem-with-ETL-example-
>> tp5752256p5752259.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-with-ETL-example-tp5752256p5752263.html
Sent from the Camel - Users mailing list archive at Nabble.com.
AW: Problem with ETL example
Posted by "Jan Matèrne (jhm)" <ap...@materne.de>.
from("file:src/data?noop=true")
.convertBodyTo(PersonDocument.class)
.to("jpa:org.apache.camel.example.etl.CustomerEntity");
// the following will dump the database to files
from("jpa:org.apache.camel.example.etl.CustomerEntity?consumeDelete=false&de
lay=3000&consumeLockEntity=false")
.setHeader(Exchange.FILE_NAME, el("${in.body.userName}.xml"))
.to("file:target/customers");
I think it is because the first route doesnt move the processed files to
somewhere else ...
http://camel.apache.org/file2.html
Jan
> -----Ursprüngliche Nachricht-----
> Von: Babak Vahdat [mailto:babak.vahdat@swissonline.ch]
> Gesendet: Freitag, 13. Juni 2014 12:18
> An: users@camel.apache.org
> Betreff: Re: Problem with ETL example
>
> Hi
>
> The reason for the same logs you see is because of the consumeDelete
> option being set to false:
>
> http://camel.apache.org/jpa.html#JPA-Options
>
> So that the JPA consumer polls the *same* 2 entities again and again.
> This is just for the demonstration purpose so that you can continuously
> see something is running.
>
> You can set that option to true or remove it (default is true) and
> check the change in the behavior here:
>
> https://github.com/apache/camel/blob/master/examples/camel-example-
> etl/src/main/java/org/apache/camel/example/etl/EtlRoutes.java#L36
>
> And if not already done, maybe you want to go through the documentation
> of this example as well:
>
> http://camel.apache.org/etl-example.html
>
> Babak
>
> Licia wrote
> > Hi everyone,
> >
> > I'm all new to Camel and after getting documentation, I went directly
> > to the ETL example because it really looks like what I'm trying to
> do.
> >
> > So I just did mvn compile, which worked (everything did in the
> > directory where the POM is).
> > And then I tried mvn camel:run and I get a problem : it looks like
> the
> > same message get displayed over and over again.
> >
> >
> >
> >
> >
> > "
> *
> > 2014-06-13 11:07:52,094 [.CustomerEntity] INFO Tracer
> > -
> > ID-ITEM-93866-58463-1402650414339-0-78 >>> (route2)
> > from(jpa://org.apache.camel
> >
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fals
> > e&delay=30
> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> > Headers:{CamelEntityManager
> > =org.apache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> > breadcrumbId=ID-ITEM
> > -93866-58463-1402650414339-0-77},
> > BodyType:org.apache.camel.example.etl.Customer
> > Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> > <customer id="1">
> >
> > <userName>
> > james
> > </userName>
> >
> > <firstName>
> > James
> > </firstName>
> >
> > <surname>
> > Strachan
> > </surname>
> >
> > <city>
> > London
> > </city>
> > </customer>
> *
> >
> > 2014-06-13 11:07:52,096 [.CustomerEntity] INFO Tracer
> > -
> > ID-ITEM-93866-58463-1402650414339-0-78 >>> (route2)
> > setHeader[CamelFileName] --
> >> file://target/customers <<< Pattern:InOnly,
> >> Headers:{breadcrumbId=ID-ITEM-9386
> > 6-58463-1402650414339-0-77, CamelFileName=james.xml,
> > CamelEntityManager=org.apac
> > he.openjpa.persistence.EntityManagerImpl@5d84d74b},
> > BodyType:org.apache.camel.ex
> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-8"
> > standalone="
> > yes"?>
> > <customer id="1">
> >
> > <userName>
> > james
> > </userName>
> >
> > <firstName>
> > James
> > </firstName>
> >
> > <surname>
> > Strachan
> > </surname>
> >
> > <city>
> > London
> > </city>
> > </customer>
> > 2014-06-13 11:07:52,136 [.CustomerEntity] INFO Tracer
> > -
> > ID-ITEM-93866-58463-1402650414339-0-80 >>> (route2)
> > from(jpa://org.apache.camel
> >
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fals
> > e&delay=30
> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> > Headers:{CamelEntityManager
> > =org.apache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> > breadcrumbId=ID-ITEM
> > -93866-58463-1402650414339-0-79},
> > BodyType:org.apache.camel.example.etl.Customer
> > Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> > <customer id="2">
> >
> > <userName>
> > hiram
> > </userName>
> >
> > <firstName>
> > Hiram
> > </firstName>
> >
> > <surname>
> > Chirino
> > </surname>
> >
> > <city>
> > Tampa
> > </city>
> > </customer>
> > 2014-06-13 11:07:52,137 [.CustomerEntity] INFO Tracer
> > -
> > ID-ITEM-93866-58463-1402650414339-0-80 >>> (route2)
> > setHeader[CamelFileName] --
> >> file://target/customers <<< Pattern:InOnly,
> >> Headers:{CamelEntityManager=org.ap
> > ache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> > breadcrumbId=ID-ITEM-93866-
> > 58463-1402650414339-0-79, CamelFileName=hiram.xml},
> > BodyType:org.apache.camel.ex ample.etl.CustomerEntity, Body:<?xml
> > version="1.0" encoding="UTF-8"
> > standalone="
> > yes"?>
> > <customer id="2">
> >
> > <userName>
> > hiram
> > </userName>
> >
> > <firstName>
> > Hiram
> > </firstName>
> >
> > <surname>
> > Chirino
> > </surname>
> >
> > <city>
> > Tampa
> > </city>
> > </customer>
> > 60513 camel TRACE [Camel (camel) thread #1 -
> > jpa://org.apache.camel.example.e tl.CustomerEntity] openjpa.jdbc.SQL
> -
> > <t 53213167, conn 227180630> executing pre pstmnt 882159474 SELECT
> > t0.id, t0.city, t0.firstName, t0.phone, t0.street, t0.surname,
> > t0.userName, t0.zip
> > FROM customer t0
> >
> > 60514 camel TRACE [Camel (camel) thread #1 -
> > jpa://org.apache.camel.example.e tl.CustomerEntity] openjpa.jdbc.SQL
> -
> > <t 53213167, conn 227180630> [0 ms] spent
> *
> > 2014-06-13 11:07:55,165 [.CustomerEntity] INFO Tracer
> > -
> > ID-ITEM-93866-58463-1402650414339-0-82 >>> (route2)
> > from(jpa://org.apache.camel
> >
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fals
> > e&delay=30
> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> > Headers:{breadcrumbId=ID-IT EM-93866-58463-1402650414339-0-81,
> > CamelEntityManager=org.apache.openjpa.persist
> > ence.EntityManagerImpl@5d84d74b},
> > BodyType:org.apache.camel.example.etl.Customer
> > Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> > <customer id="1">
> >
> > <userName>
> > james
> > </userName>
> >
> > <firstName>
> > James
> > </firstName>
> >
> > <surname>
> > Strachan
> > </surname>
> >
> > <city>
> > London
> > </city>
> > </customer>
> *
> >
> > 2014-06-13 11:07:55,167 [.CustomerEntity] INFO Tracer
> > -
> > ID-ITEM-93866-58463-1402650414339-0-82 >>> (route2)
> > setHeader[CamelFileName] --
> >> file://target/customers <<< Pattern:InOnly,
> >> Headers:{breadcrumbId=ID-ITEM-9386
> > 6-58463-1402650414339-0-81, CamelFileName=james.xml,
> > CamelEntityManager=org.apac
> > he.openjpa.persistence.EntityManagerImpl@5d84d74b},
> > BodyType:org.apache.camel.ex
> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-8"
> > standalone="
> > yes"?>
> > <customer id="1">
> >
> > <userName>
> > james
> > </userName>
> >
> > <firstName>
> > James
> > </firstName>
> >
> > <surname>
> > Strachan
> > </surname>
> >
> > <city>
> > London
> > </city>
> > </customer>
> > 2014-06-13 11:07:55,196 [.CustomerEntity] INFO Tracer
> > -
> > ID-ITEM-93866-58463-1402650414339-0-84 >>> (route2)
> > from(jpa://org.apache.camel
> >
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fals
> > e&delay=30
> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> > Headers:{breadcrumbId=ID-IT EM-93866-58463-1402650414339-0-83,
> > CamelEntityManager=org.apache.openjpa.persist
> > ence.EntityManagerImpl@5d84d74b},
> > BodyType:org.apache.camel.example.etl.Customer
> > Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> > <customer id="2">
> >
> > <userName>
> > hiram
> > </userName>
> >
> > <firstName>
> > Hiram
> > </firstName>
> >
> > <surname>
> > Chirino
> > </surname>
> >
> > <city>
> > Tampa
> > </city>
> > </customer>
> > 2014-06-13 11:07:55,198 [.CustomerEntity] INFO Tracer
> > -
> > ID-ITEM-93866-58463-1402650414339-0-84 >>> (route2)
> > setHeader[CamelFileName] --
> >> file://target/customers <<< Pattern:InOnly,
> >> Headers:{CamelEntityManager=org.ap
> > ache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> > CamelFileName=hiram.xml, br
> > eadcrumbId=ID-ITEM-93866-58463-1402650414339-0-83},
> > BodyType:org.apache.camel.ex
> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-8"
> > standalone="
> > yes"?>
> > <customer id="2">
> >
> > <userName>
> > hiram
> > </userName>
> >
> > <firstName>
> > Hiram
> > </firstName>
> >
> > <surname>
> > Chirino
> > </surname>
> >
> > <city>
> > Tampa
> > </city>
> > </customer>
> > "
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > The two bold parts are the exact same ones.
> >
> > Plus I'm really confused about what the example is supposed to do.
> > Isn't it supposed to create the java classes ? Or am I completely
> wrong ?
> > Else, isn't it supposed to create the persons in my database ? How do
> > I see if it has been done ?
> >
> > Thanks a lot for your answers.
>
>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Problem-with-ETL-example-
> tp5752256p5752259.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Problem with ETL example
Posted by Babak Vahdat <ba...@swissonline.ch>.
Hi
The reason for the same logs you see is because of the consumeDelete option
being set to false:
http://camel.apache.org/jpa.html#JPA-Options
So that the JPA consumer polls the *same* 2 entities again and again. This
is just for the demonstration purpose so that you can continuously see
something is running.
You can set that option to true or remove it (default is true) and check the
change in the behavior here:
https://github.com/apache/camel/blob/master/examples/camel-example-etl/src/main/java/org/apache/camel/example/etl/EtlRoutes.java#L36
And if not already done, maybe you want to go through the documentation of
this example as well:
http://camel.apache.org/etl-example.html
Babak
Licia wrote
> Hi everyone,
>
> I'm all new to Camel and after getting documentation, I went directly to
> the ETL example because it really looks like what I'm trying to do.
>
> So I just did mvn compile, which worked (everything did in the directory
> where the POM is).
> And then I tried mvn camel:run and I get a problem : it looks like the
> same message get displayed over and over again.
>
>
>
>
>
> "
*
> 2014-06-13 11:07:52,094 [.CustomerEntity] INFO Tracer
> -
> ID-ITEM-93866-58463-1402650414339-0-78 >>> (route2)
> from(jpa://org.apache.camel
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=false&delay=30
> 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> Headers:{CamelEntityManager
> =org.apache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> breadcrumbId=ID-ITEM
> -93866-58463-1402650414339-0-77},
> BodyType:org.apache.camel.example.etl.Customer
> Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <customer id="1">
>
> <userName>
> james
> </userName>
>
> <firstName>
> James
> </firstName>
>
> <surname>
> Strachan
> </surname>
>
> <city>
> London
> </city>
> </customer>
*
>
> 2014-06-13 11:07:52,096 [.CustomerEntity] INFO Tracer
> -
> ID-ITEM-93866-58463-1402650414339-0-78 >>> (route2)
> setHeader[CamelFileName] --
>> file://target/customers <<< Pattern:InOnly,
>> Headers:{breadcrumbId=ID-ITEM-9386
> 6-58463-1402650414339-0-77, CamelFileName=james.xml,
> CamelEntityManager=org.apac
> he.openjpa.persistence.EntityManagerImpl@5d84d74b},
> BodyType:org.apache.camel.ex
> ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-8"
> standalone="
> yes"?>
> <customer id="1">
>
> <userName>
> james
> </userName>
>
> <firstName>
> James
> </firstName>
>
> <surname>
> Strachan
> </surname>
>
> <city>
> London
> </city>
> </customer>
> 2014-06-13 11:07:52,136 [.CustomerEntity] INFO Tracer
> -
> ID-ITEM-93866-58463-1402650414339-0-80 >>> (route2)
> from(jpa://org.apache.camel
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=false&delay=30
> 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> Headers:{CamelEntityManager
> =org.apache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> breadcrumbId=ID-ITEM
> -93866-58463-1402650414339-0-79},
> BodyType:org.apache.camel.example.etl.Customer
> Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <customer id="2">
>
> <userName>
> hiram
> </userName>
>
> <firstName>
> Hiram
> </firstName>
>
> <surname>
> Chirino
> </surname>
>
> <city>
> Tampa
> </city>
> </customer>
> 2014-06-13 11:07:52,137 [.CustomerEntity] INFO Tracer
> -
> ID-ITEM-93866-58463-1402650414339-0-80 >>> (route2)
> setHeader[CamelFileName] --
>> file://target/customers <<< Pattern:InOnly,
>> Headers:{CamelEntityManager=org.ap
> ache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> breadcrumbId=ID-ITEM-93866-
> 58463-1402650414339-0-79, CamelFileName=hiram.xml},
> BodyType:org.apache.camel.ex
> ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-8"
> standalone="
> yes"?>
> <customer id="2">
>
> <userName>
> hiram
> </userName>
>
> <firstName>
> Hiram
> </firstName>
>
> <surname>
> Chirino
> </surname>
>
> <city>
> Tampa
> </city>
> </customer>
> 60513 camel TRACE [Camel (camel) thread #1 -
> jpa://org.apache.camel.example.e
> tl.CustomerEntity] openjpa.jdbc.SQL -
> <t 53213167, conn 227180630>
> executing pre
> pstmnt 882159474
> SELECT t0.id, t0.city, t0.firstName, t0.phone, t0.street, t0.surname,
> t0.userName, t0.zip
> FROM customer t0
>
> 60514 camel TRACE [Camel (camel) thread #1 -
> jpa://org.apache.camel.example.e
> tl.CustomerEntity] openjpa.jdbc.SQL -
> <t 53213167, conn 227180630>
> [0 ms] spent
*
> 2014-06-13 11:07:55,165 [.CustomerEntity] INFO Tracer
> -
> ID-ITEM-93866-58463-1402650414339-0-82 >>> (route2)
> from(jpa://org.apache.camel
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=false&delay=30
> 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> Headers:{breadcrumbId=ID-IT
> EM-93866-58463-1402650414339-0-81,
> CamelEntityManager=org.apache.openjpa.persist
> ence.EntityManagerImpl@5d84d74b},
> BodyType:org.apache.camel.example.etl.Customer
> Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <customer id="1">
>
> <userName>
> james
> </userName>
>
> <firstName>
> James
> </firstName>
>
> <surname>
> Strachan
> </surname>
>
> <city>
> London
> </city>
> </customer>
*
>
> 2014-06-13 11:07:55,167 [.CustomerEntity] INFO Tracer
> -
> ID-ITEM-93866-58463-1402650414339-0-82 >>> (route2)
> setHeader[CamelFileName] --
>> file://target/customers <<< Pattern:InOnly,
>> Headers:{breadcrumbId=ID-ITEM-9386
> 6-58463-1402650414339-0-81, CamelFileName=james.xml,
> CamelEntityManager=org.apac
> he.openjpa.persistence.EntityManagerImpl@5d84d74b},
> BodyType:org.apache.camel.ex
> ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-8"
> standalone="
> yes"?>
> <customer id="1">
>
> <userName>
> james
> </userName>
>
> <firstName>
> James
> </firstName>
>
> <surname>
> Strachan
> </surname>
>
> <city>
> London
> </city>
> </customer>
> 2014-06-13 11:07:55,196 [.CustomerEntity] INFO Tracer
> -
> ID-ITEM-93866-58463-1402650414339-0-84 >>> (route2)
> from(jpa://org.apache.camel
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=false&delay=30
> 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> Headers:{breadcrumbId=ID-IT
> EM-93866-58463-1402650414339-0-83,
> CamelEntityManager=org.apache.openjpa.persist
> ence.EntityManagerImpl@5d84d74b},
> BodyType:org.apache.camel.example.etl.Customer
> Entity, Body:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <customer id="2">
>
> <userName>
> hiram
> </userName>
>
> <firstName>
> Hiram
> </firstName>
>
> <surname>
> Chirino
> </surname>
>
> <city>
> Tampa
> </city>
> </customer>
> 2014-06-13 11:07:55,198 [.CustomerEntity] INFO Tracer
> -
> ID-ITEM-93866-58463-1402650414339-0-84 >>> (route2)
> setHeader[CamelFileName] --
>> file://target/customers <<< Pattern:InOnly,
>> Headers:{CamelEntityManager=org.ap
> ache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> CamelFileName=hiram.xml, br
> eadcrumbId=ID-ITEM-93866-58463-1402650414339-0-83},
> BodyType:org.apache.camel.ex
> ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-8"
> standalone="
> yes"?>
> <customer id="2">
>
> <userName>
> hiram
> </userName>
>
> <firstName>
> Hiram
> </firstName>
>
> <surname>
> Chirino
> </surname>
>
> <city>
> Tampa
> </city>
> </customer>
> "
>
>
>
>
>
>
>
>
>
>
> The two bold parts are the exact same ones.
>
> Plus I'm really confused about what the example is supposed to do. Isn't
> it supposed to create the java classes ? Or am I completely wrong ?
> Else, isn't it supposed to create the persons in my database ? How do I
> see if it has been done ?
>
> Thanks a lot for your answers.
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-with-ETL-example-tp5752256p5752259.html
Sent from the Camel - Users mailing list archive at Nabble.com.