You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by aurelienmasse <am...@financeactive.com> on 2011/08/22 11:22:11 UTC
inProgressRepository not cleaned in GenericFileEndpoint ?
Hi
I've got a use case where i need to enrich a message by polling a FTP server
so as to get files like "request_1.xml", "request_2.xml", etc.
This FTP server acts as a resource provider, and i want to be able to get
the appropriate file every time a message is handled (the request number is
held in a header of current exchange).
I'm doing the polling in a processor, as pollEnrich can't use headers from
current exchange.
So, i've configured the URI of the FTP server with idempotent=false (which
theoritically allows me to poll the same file multiple times).
But it appears that the name of the file is held in the in-memory
inProgressRepository of the endpoint and is never removed.
So the second time I try to get a request.xml file, it doesn't work :
[TRACE] remote.FtpConsumer Skipping as file is already in progress:
request_1.xml
A workaround is to remove the file name from the inProgressRepository by
myself after getting the file and closing the consumer.
Is there a way to automatically clean the inProgressRepository after file is
handled ? Am I missing something ?
Thanks.
--
View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4722713.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: inProgressRepository not cleaned in GenericFileEndpoint ?
Posted by aurelienmasse <am...@financeactive.com>.
Hello Claus
Ok i've found the problem, it appeared to be a simple mistake on my code.
It works perfectly now using the ConsumerTemplate.
Thanks a lot for your help and for the time you spent.
--
View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4725854.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: inProgressRepository not cleaned in GenericFileEndpoint ?
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Aug 22, 2011 at 2:07 PM, aurelienmasse <am...@financeactive.com> wrote:
> I'm using 2.8.0
>
I created an unit test on trunk, and it works fine there
http://svn.apache.org/viewvc?rev=1160565&view=rev
> --
> View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4723151.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: inProgressRepository not cleaned in GenericFileEndpoint ?
Posted by aurelienmasse <am...@financeactive.com>.
I'm using 2.8.0
--
View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4723151.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: inProgressRepository not cleaned in GenericFileEndpoint ?
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Aug 22, 2011 at 2:03 PM, aurelienmasse <am...@financeactive.com> wrote:
> Hello Claus !
>
> thanks for the quick answer.
>
> I've changed the code to use a consumerTemplate instead of a
> pollingConsumer, so as to use the doneUoW method. So the changed code is :
>
> String uri = buildFullUri(baseUrl, fileName);
>
> ConsumerTemplate template = exchange.getContext().createConsumerTemplate();
> Exchange result = template.receive(uri, DEFAULT_TIMEOUT);
>
> GenericFile body = result.getIn().getBody(GenericFile.class);
> template.doneUoW(exchange);
> template.stop();
>
> Unfortunately, it didn't change the issue, on the second time I poll the
> endpoint, I still got :
> [TRACE] remote.FtpConsumer Skipping as file is already in progress:
> request_1.xml
>
>
What version of Camel are you using?
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4723141.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: inProgressRepository not cleaned in GenericFileEndpoint ?
Posted by aurelienmasse <am...@financeactive.com>.
Hello Claus !
thanks for the quick answer.
I've changed the code to use a consumerTemplate instead of a
pollingConsumer, so as to use the doneUoW method. So the changed code is :
String uri = buildFullUri(baseUrl, fileName);
ConsumerTemplate template = exchange.getContext().createConsumerTemplate();
Exchange result = template.receive(uri, DEFAULT_TIMEOUT);
GenericFile body = result.getIn().getBody(GenericFile.class);
template.doneUoW(exchange);
template.stop();
Unfortunately, it didn't change the issue, on the second time I poll the
endpoint, I still got :
[TRACE] remote.FtpConsumer Skipping as file is already in progress:
request_1.xml
--
View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4723141.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: inProgressRepository not cleaned in GenericFileEndpoint ?
Posted by Claus Ibsen <cl...@gmail.com>.
Ah read the javadoc of the consumer template, the doneUoW
http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/ConsumerTemplate.html
We should actually add same details to the PollingConsumer API javadoc.
Since you return an Exchange you need to manually done its UoW when
you are done using it.
On Mon, Aug 22, 2011 at 12:20 PM, aurelienmasse
<am...@financeactive.com> wrote:
> I'm using the released version 2.8.0
>
>
> Below is a code snippet of what i'm currently doing :
>
> String uri = buildFullUri(baseUrl, fileName);
>
> GenericFileEndpoint endpoint = (GenericFileEndpoint)
> exchange.getContext().getEndpoint(uri);
>
> PollingConsumer consumer = endpoint.createPollingConsumer();
>
> consumer.start();
> Exchange result = consumer.receive(DEFAULT_TIMEOUT);
> consumer.stop();
> if (result == null) {
> throw new IllegalStateException("Requested resource [" + fileName + "]
> has not been found.");
> }
> GenericFile body = result.getIn().getBody(GenericFile.class);
>
> endpoint.getInProgressRepository().remove(body.getAbsoluteFilePath());
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4722880.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: inProgressRepository not cleaned in GenericFileEndpoint ?
Posted by aurelienmasse <am...@financeactive.com>.
I'm using the released version 2.8.0
Below is a code snippet of what i'm currently doing :
String uri = buildFullUri(baseUrl, fileName);
GenericFileEndpoint endpoint = (GenericFileEndpoint)
exchange.getContext().getEndpoint(uri);
PollingConsumer consumer = endpoint.createPollingConsumer();
consumer.start();
Exchange result = consumer.receive(DEFAULT_TIMEOUT);
consumer.stop();
if (result == null) {
throw new IllegalStateException("Requested resource [" + fileName + "]
has not been found.");
}
GenericFile body = result.getIn().getBody(GenericFile.class);
endpoint.getInProgressRepository().remove(body.getAbsoluteFilePath());
--
View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4722880.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: inProgressRepository not cleaned in GenericFileEndpoint ?
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Aug 22, 2011 at 11:22 AM, aurelienmasse
<am...@financeactive.com> wrote:
> Hi
>
> I've got a use case where i need to enrich a message by polling a FTP server
> so as to get files like "request_1.xml", "request_2.xml", etc.
> This FTP server acts as a resource provider, and i want to be able to get
> the appropriate file every time a message is handled (the request number is
> held in a header of current exchange).
>
> I'm doing the polling in a processor, as pollEnrich can't use headers from
> current exchange.
>
> So, i've configured the URI of the FTP server with idempotent=false (which
> theoritically allows me to poll the same file multiple times).
>
What version of Camel are you using?
And have you tried upgrading to the latest release and see if it now works?
> But it appears that the name of the file is held in the in-memory
> inProgressRepository of the endpoint and is never removed.
> So the second time I try to get a request.xml file, it doesn't work :
> [TRACE] remote.FtpConsumer Skipping as file is already in progress:
> request_1.xml
>
> A workaround is to remove the file name from the inProgressRepository by
> myself after getting the file and closing the consumer.
>
> Is there a way to automatically clean the inProgressRepository after file is
> handled ? Am I missing something ?
>
> Thanks.
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/inProgressRepository-not-cleaned-in-GenericFileEndpoint-tp4722713p4722713.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/