You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by "fss.coc" <fs...@gmail.com> on 2016/02/04 22:29:34 UTC
Leak of Memory using JdbcMessageIdRepository
I have an application that performs the Routes management Camel. Basically we
add / remove routes on the basis of a specific logic.
One of these routes possesses one idempotentConsumer. If I add and remove
this route JdbcMessageIdRepository the service remains in context.
So if I repeat the addition operation and removal of the route (often) I
have a memory leak.
<http://camel.465427.n5.nabble.com/file/n5777242/JdbcMessageIdRepository.png>
--
View this message in context: http://camel.465427.n5.nabble.com/Leak-of-Memory-using-JdbcMessageIdRepository-tp5777242.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Leak of Memory using JdbcMessageIdRepository
Posted by "fss.coc" <fs...@gmail.com>.
>From what I saw in commits the CAMEL-9569 the idempotent of the problem has
been corrected.
But there is other leaks. In the same example of class even manually
removing the idempotent there's the leak.
--
View this message in context: http://camel.465427.n5.nabble.com/Leak-of-Memory-using-JdbcMessageIdRepository-tp5777242p5777555.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Leak of Memory using JdbcMessageIdRepository
Posted by "fss.coc" <fs...@gmail.com>.
2.16.1
--
View this message in context: http://camel.465427.n5.nabble.com/Leak-of-Memory-using-JdbcMessageIdRepository-tp5777242p5777298.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Leak of Memory using JdbcMessageIdRepository
Posted by Claus Ibsen <cl...@gmail.com>.
What Camel version do you use?
On Fri, Feb 5, 2016 at 5:30 PM, fss.coc <fs...@gmail.com> wrote:
> Nice...
> While the bug is not resolved to the idempotent of the problem I will use
> the workaround that I said.
> But in relation to the producer, is there any workaround possible?
>
> Thank you.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Leak-of-Memory-using-JdbcMessageIdRepository-tp5777242p5777293.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: Leak of Memory using JdbcMessageIdRepository
Posted by "fss.coc" <fs...@gmail.com>.
Nice...
While the bug is not resolved to the idempotent of the problem I will use
the workaround that I said.
But in relation to the producer, is there any workaround possible?
Thank you.
--
View this message in context: http://camel.465427.n5.nabble.com/Leak-of-Memory-using-JdbcMessageIdRepository-tp5777242p5777293.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Leak of Memory using JdbcMessageIdRepository
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Thanks yeah its a bug I have logged a ticket
https://issues.apache.org/jira/browse/CAMEL-9569
On Fri, Feb 5, 2016 at 3:17 PM, fss.coc <fs...@gmail.com> wrote:
> Regarding the idempotent, using as an example the MemoryIdempotentRepository
> and had the same behavior.
> The new test therefore shows two problems: Services and Produces growing
> infinitely.
>
> LeakOfMemory.java
> <http://camel.465427.n5.nabble.com/file/n5777278/LeakOfMemory.java>
>
>
>
> For the service (MemoryIdempotentRepository or JdbcMessageIdRepository), my
> resolution was to save the instance of IdempotentRepository, and removal of
> the route (LifecycleStrategy), manually remove the service
>
> public void removeRoute(final String routeId) throws Exception {
> final RouteContext routeContext = routeContextByRouteId.remove(routeId);
> assert routeContext.routeId.equals(routeId);
> if (routeContext != null) {
> camelContext.removeService(routeContext.jdbcMessageIdRepository);
> }
> }
>
> @Override
> public void onRoutesRemove(final Collection<Route> routes) {
> for (final Route route : routes) {
> try {
> removeRoute(route.getId());
> } catch (final Exception e) {
> logger.error("Error on remove jdbcMessageIdRepository");
> }
> }
> }
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Leak-of-Memory-using-JdbcMessageIdRepository-tp5777242p5777278.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: Leak of Memory using JdbcMessageIdRepository
Posted by "fss.coc" <fs...@gmail.com>.
Regarding the idempotent, using as an example the MemoryIdempotentRepository
and had the same behavior.
The new test therefore shows two problems: Services and Produces growing
infinitely.
LeakOfMemory.java
<http://camel.465427.n5.nabble.com/file/n5777278/LeakOfMemory.java>
For the service (MemoryIdempotentRepository or JdbcMessageIdRepository), my
resolution was to save the instance of IdempotentRepository, and removal of
the route (LifecycleStrategy), manually remove the service
public void removeRoute(final String routeId) throws Exception {
final RouteContext routeContext = routeContextByRouteId.remove(routeId);
assert routeContext.routeId.equals(routeId);
if (routeContext != null) {
camelContext.removeService(routeContext.jdbcMessageIdRepository);
}
}
@Override
public void onRoutesRemove(final Collection<Route> routes) {
for (final Route route : routes) {
try {
removeRoute(route.getId());
} catch (final Exception e) {
logger.error("Error on remove jdbcMessageIdRepository");
}
}
}
--
View this message in context: http://camel.465427.n5.nabble.com/Leak-of-Memory-using-JdbcMessageIdRepository-tp5777242p5777278.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Leak of Memory using JdbcMessageIdRepository
Posted by "fss.coc" <fs...@gmail.com>.
I did a test class to present the problem.
In this class below has another leak in relation to producers.
LeakOfMemory.java
<http://camel.465427.n5.nabble.com/file/n5777276/LeakOfMemory.java>
In this class basically I have two routes, one of which is always removed
and added again infinitely.
Using JConsole one realizes that each adding a new producer is created and
removing it is not removed.
This producer is associated with the fragment
onException (Exception.class) .handled (true) .to ("direct: routeError");
<http://camel.465427.n5.nabble.com/file/n5777276/Jconsole.png>
The code snippet where I insert the idempotent of service is
final JdbcMessageIdRepository jdbcMessageIdRepository =
getIdempotentRepository(
route.getInComponent().getId());
// @formatter:off
from(fromEndpoint)
.process(new TypeConversionProcessor())
.routeId(route.getId())
.setProperty(CamelConstants.RECIPIENTS_TO_PROCESS, constant(new
HashSet()))
.setHeader(CamelConstants.ROUTE_ID_HEADER,
constant(route.getId()))
.setHeader(CamelConstants.ROUTE_HEADER, constant(route))
.setHeader(CamelConstants.ROUTE_STRUCTURE,
constant(route.getStructure().toString()))
.setHeader(CamelConstants.INBOUND_ENDPOINT_TYPE,
constant(route.getInComponent().getType()))
.setHeader(CamelConstants.INBOUND_EDITABILITY_ROUTE,
constant(route.getEditabilityRule()))
.setHeader(CamelConstants.IS_LOADBALANCED_HEADER,
constant(Boolean.valueOf(true)))
.setHeader(CamelConstants.PARENT_ENDPOINT_ID_HEADER,
constant(parentId))
.process(new SqlPreProcessor())
.process("contentTypePreProcessor")
.filter(new Predicate() {
@Override
public boolean matches(final Exchange exchange) {
return
Boolean.valueOf(exchange.getIn().getHeader(CamelConstants.ALLOW_INBOUND_MESSAGE).toString());
}
}).choice()
.when(header(CamelConstants.INBOUND_EDITABILITY_ROUTE).isNotNull())
.convertBodyTo(String.class)
.process("administratableDocumentProcessor")
.endChoice()
.when(header(CamelConstants.INBOUND_ENDPOINT_TYPE).isEqualTo(EndpointType.JDBC))
.idempotentConsumer(header(CamelConstants.ROW_ID_HEADER),
jdbcMessageIdRepository)
.to("seda:loadBalancePrepare")
.endChoice()
.when(header(CamelConstants.PARENT_ENDPOINT_ID_HEADER).isNotNull())
.filter(childFilter)
.aggregate(header(PARENT_ENDPOINT_ID_HEADER), new
ChildEndpointAggregator())
.completionSize(children).completionTimeout(timeout)
.process("childEndpointParameterOrganiser")
.process(new Processor() {
@Override
public void process(final Exchange exchange) throws Exception
{
childFilter.reset();
}
})
.to("seda:loadBalancePrepare")
.end()
.endChoice()
.otherwise()
.to("seda:loadBalancePrepare")
.end();
// @formatter:on
public JdbcMessageIdRepository getIdempotentRepository(final String
endpointId) {
final JdbcMessageIdRepository repository = new
JdbcMessageIdRepository(applicationDataSource, endpointId);
// Need to override default for SQL Server as doesn't allow repository
// to specify a value for timestamp columns
repository.setCreateString(
"CREATE TABLE CAMEL_MESSAGEPROCESSED (processorName VARCHAR(255),
messageId VARCHAR(100), createdAt VARCHAR(100))");
return repository;
}
--
View this message in context: http://camel.465427.n5.nabble.com/Leak-of-Memory-using-JdbcMessageIdRepository-tp5777242p5777276.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Leak of Memory using JdbcMessageIdRepository
Posted by Claus Ibsen <cl...@gmail.com>.
How have you configure the idempotent consumer and the jdbc repository?
On Thu, Feb 4, 2016 at 10:29 PM, fss.coc <fs...@gmail.com> wrote:
> I have an application that performs the Routes management Camel. Basically we
> add / remove routes on the basis of a specific logic.
>
> One of these routes possesses one idempotentConsumer. If I add and remove
> this route JdbcMessageIdRepository the service remains in context.
> So if I repeat the addition operation and removal of the route (often) I
> have a memory leak.
>
> <http://camel.465427.n5.nabble.com/file/n5777242/JdbcMessageIdRepository.png>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Leak-of-Memory-using-JdbcMessageIdRepository-tp5777242.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2