You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by GaryLeeMills <ga...@ge.com> on 2017/01/24 20:38:13 UTC
camel pollEnrich API not working properly Camel 2.15 leaves file
locks on aggregation uri path
Hello,
Can I ask you if there are any known issues with the pollEnrich API in that
it does not release the file locks ? it is always on the pollenrichment
path.
<pollEnrich strategyRef="navSeaAggregationStrategy"
uri="{{CMAS_LPD22_MPDE1A}}?fileName=CMAS_LPD22_MPDE1A_PG2.csv&delay=2000"
timeout="60000" />
It leaves a messy file system, and then any additional events do not work
until the locks are removed. ???? the PG2 is always on the pollEnrich path
as the appended file or data.
└── tobeprocessed
│ ├── LPD221A
│ │ ├── .camel
│ │ │ ├── CMAS_LPD22_MPDE1A_PG1.csv
│ │ │ └── CMAS_LPD22_MPDE1A_PG2.csv
│ │ ├── CMAS_LPD22_MPDE1A_PG2.csv
│ │ └── CMAS_LPD22_MPDE1A_PG2.csv.camelLock
│ ├── LPD221B
│ │ ├── .camel
│ │ │ ├── CMAS_LPD22_MPDE1B_PG1.csv
│ │ │ └── CMAS_LPD22_MPDE1B_PG2.csv
│ │ ├── CMAS_LPD22_MPDE1B_PG2.csv
│ │ └── CMAS_LPD22_MPDE1B_PG2.csv.camelLock
│ ├── LPD222A
│ │ ├── .camel
│ │ │ ├── CMAS_LPD22_MPDE2A_PG1.csv
│ │ │ └── CMAS_LPD22_MPDE2A_PG2.csv
│ │ ├── CMAS_LPD22_MPDE2A_PG2.csv
│ │ └── CMAS_LPD22_MPDE2A_PG2.csv.camelLock
│ ├── LPD222B
│ │ ├── .camel
│ │ │ ├── CMAS_LPD22_MPDE2B_PG1.csv
│ │ │ └── CMAS_LPD22_MPDE2B_PG2.csv
│ │ ├── CMAS_LPD22_MPDE2B_PG2.csv
│ │ └── CMAS_LPD22_MPDE2B_PG2.csv.camelLock
--
View this message in context: http://camel.465427.n5.nabble.com/camel-pollEnrich-API-not-working-properly-Camel-2-15-leaves-file-locks-on-aggregation-uri-path-tp5793038.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: camel pollEnrich API not working properly Camel 2.15 leaves
file locks on aggregation uri path
Posted by GaryLeeMills <ga...@ge.com>.
thank you everyone and esp Claus - please ignore the last quesiton, you guys
have some of the best documentation around. I should have looked before
posting, to make the producer not write over an existing file. :) Loving
Camel, my bad
--
View this message in context: http://camel.465427.n5.nabble.com/camel-pollEnrich-API-not-working-properly-Camel-2-15-leaves-file-locks-on-aggregation-uri-path-tp5793038p5793145.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: camel pollEnrich API not working properly Camel 2.15 leaves
file locks on aggregation uri path
Posted by souciance <so...@gmail.com>.
I guess if the files are not removed automatically you could at a custom
bean that removes them after an OnCompletion block.
On Fri, Jan 27, 2017 at 10:45 PM, GaryLeeMills [via Camel] <
ml-node+s465427n5793139h72@n5.nabble.com> wrote:
> Clause, again thank you for taking time to help out of your busy day :) I
> am excited to receive your new publication.
>
> I have not upgraded yet, will on Monday.
>
> I have tried removing the readlock however, it does leave me open to over
> writing existing files. the frequency of files received is random. this is
> thru the pollEnrich enrichment path.
>
> is there anyway to make the producer not overwrite a file? I see on the
> consumer but not the producer.
> because of the pollEnrichment path and dynamic names I simply convert to a
> static name, then process in the pollEnrich API. now removing the readlock
> and an unknown file frequency sometimes a file or two get overwritten.
>
> thanks
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://camel.465427.n5.nabble.com/camel-pollEnrich-API-not-
> working-properly-Camel-2-15-leaves-file-locks-on-aggregation-uri-path-
> tp5793038p5793139.html
> To start a new topic under Camel - Users, email
> ml-node+s465427n465428h31@n5.nabble.com
> To unsubscribe from Camel - Users, click here
> <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=465428&code=c291Y2lhbmNlLmVxZGFtLnJhc2h0aUBnbWFpbC5jb218NDY1NDI4fDE1MzI5MTE2NTY=>
> .
> NAML
> <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://camel.465427.n5.nabble.com/camel-pollEnrich-API-not-working-properly-Camel-2-15-leaves-file-locks-on-aggregation-uri-path-tp5793038p5793141.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: camel pollEnrich API not working properly Camel 2.15 leaves
file locks on aggregation uri path
Posted by GaryLeeMills <ga...@ge.com>.
Clause, again thank you for taking time to help out of your busy day :) I am
excited to receive your new publication.
I have not upgraded yet, will on Monday.
I have tried removing the readlock however, it does leave me open to over
writing existing files. the frequency of files received is random. this is
thru the pollEnrich enrichment path.
is there anyway to make the producer not overwrite a file? I see on the
consumer but not the producer.
because of the pollEnrichment path and dynamic names I simply convert to a
static name, then process in the pollEnrich API. now removing the readlock
and an unknown file frequency sometimes a file or two get overwritten.
thanks
--
View this message in context: http://camel.465427.n5.nabble.com/camel-pollEnrich-API-not-working-properly-Camel-2-15-leaves-file-locks-on-aggregation-uri-path-tp5793038p5793139.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: camel pollEnrich API not working properly Camel 2.15 leaves file
locks on aggregation uri path
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
It may also matter what os your use. For example windows is much more
prone to not being able to delete files from Java even though the JVM
was the owner of the file handle.
And you can also try to make a small example where you only do poll
enrich and then maybe a empty processor or something to avoid reading
the file to see if those lock files are still there or not.
On Thu, Jan 26, 2017 at 9:46 PM, GaryLeeMills <ga...@ge.com> wrote:
> thanks again Claus
>
> yeah, I've already looked at that and unfortunately that's not it. bummer,
> would've been an easy fix.
>
> yes, I really do not need the lock because am the only one reading it. I
> will give this a try.
>
> I am upgrading also, I'll do this first, since I am using JBoss Fuse OSGI
> container... I am moving to
> <jboss.fuse.bom.version>6.2.1.redhat-084</jboss.fuse.bom.version>
> this release and version. It was recommended by Apache Fuse support. I am
> not sure right now what camel core release is bundled here but, going to
> give it a try.
>
> thanks so much, I'll let you know the outcome!
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/camel-pollEnrich-API-not-working-properly-Camel-2-15-leaves-file-locks-on-aggregation-uri-path-tp5793038p5793092.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: camel pollEnrich API not working properly Camel 2.15 leaves
file locks on aggregation uri path
Posted by GaryLeeMills <ga...@ge.com>.
thanks again Claus
yeah, I've already looked at that and unfortunately that's not it. bummer,
would've been an easy fix.
yes, I really do not need the lock because am the only one reading it. I
will give this a try.
I am upgrading also, I'll do this first, since I am using JBoss Fuse OSGI
container... I am moving to
<jboss.fuse.bom.version>6.2.1.redhat-084</jboss.fuse.bom.version>
this release and version. It was recommended by Apache Fuse support. I am
not sure right now what camel core release is bundled here but, going to
give it a try.
thanks so much, I'll let you know the outcome!
--
View this message in context: http://camel.465427.n5.nabble.com/camel-pollEnrich-API-not-working-properly-Camel-2-15-leaves-file-locks-on-aggregation-uri-path-tp5793038p5793092.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: camel pollEnrich API not working properly Camel 2.15 leaves file
locks on aggregation uri path
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Maybe you do some custom processing or something where you dont close
the file / input stream etc.
Also do you need the read lock at all? If you are the only one reading
those files you can then turn that off.
Also test with newer Camel versions
On Tue, Jan 24, 2017 at 9:38 PM, GaryLeeMills <ga...@ge.com> wrote:
> Hello,
>
> Can I ask you if there are any known issues with the pollEnrich API in that
> it does not release the file locks ? it is always on the pollenrichment
> path.
>
> <pollEnrich strategyRef="navSeaAggregationStrategy"
> uri="{{CMAS_LPD22_MPDE1A}}?fileName=CMAS_LPD22_MPDE1A_PG2.csv&delay=2000"
> timeout="60000" />
>
> It leaves a messy file system, and then any additional events do not work
> until the locks are removed. ???? the PG2 is always on the pollEnrich path
> as the appended file or data.
>
> └── tobeprocessed
> │ ├── LPD221A
> │ │ ├── .camel
> │ │ │ ├── CMAS_LPD22_MPDE1A_PG1.csv
> │ │ │ └── CMAS_LPD22_MPDE1A_PG2.csv
> │ │ ├── CMAS_LPD22_MPDE1A_PG2.csv
> │ │ └── CMAS_LPD22_MPDE1A_PG2.csv.camelLock
> │ ├── LPD221B
> │ │ ├── .camel
> │ │ │ ├── CMAS_LPD22_MPDE1B_PG1.csv
> │ │ │ └── CMAS_LPD22_MPDE1B_PG2.csv
> │ │ ├── CMAS_LPD22_MPDE1B_PG2.csv
> │ │ └── CMAS_LPD22_MPDE1B_PG2.csv.camelLock
> │ ├── LPD222A
> │ │ ├── .camel
> │ │ │ ├── CMAS_LPD22_MPDE2A_PG1.csv
> │ │ │ └── CMAS_LPD22_MPDE2A_PG2.csv
> │ │ ├── CMAS_LPD22_MPDE2A_PG2.csv
> │ │ └── CMAS_LPD22_MPDE2A_PG2.csv.camelLock
> │ ├── LPD222B
> │ │ ├── .camel
> │ │ │ ├── CMAS_LPD22_MPDE2B_PG1.csv
> │ │ │ └── CMAS_LPD22_MPDE2B_PG2.csv
> │ │ ├── CMAS_LPD22_MPDE2B_PG2.csv
> │ │ └── CMAS_LPD22_MPDE2B_PG2.csv.camelLock
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/camel-pollEnrich-API-not-working-properly-Camel-2-15-leaves-file-locks-on-aggregation-uri-path-tp5793038.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: camel pollEnrich API not working properly Camel 2.15 leaves
file locks on aggregation uri path
Posted by GaryLeeMills <ga...@ge.com>.
I am using Camel 2.15
thank you
--
View this message in context: http://camel.465427.n5.nabble.com/camel-pollEnrich-API-not-working-properly-Camel-2-15-leaves-file-locks-on-aggregation-uri-path-tp5793038p5793057.html
Sent from the Camel - Users mailing list archive at Nabble.com.