You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by dermoritz <ta...@hotmail.com> on 2015/10/06 10:52:01 UTC
Problem with custom aggregation/ custom correlation (data base
rows)
I am fetching data from data base (jdbc component) that contains 1:n
relations. That means that i have multiple entries with same id:
id1,childId1, childFieldValue1, <other fields of id1>
id1,childId2, childFieldValue2, <other fields of id1>
id2,childId3, childFieldValue1, <other fields of id2>
id2,childId4, childFieldValue2, <other fields of id2>
id2,childId5, childFieldValue3, <other fields of id2>
I want to aggregate them to
id1,[{childId1, childFieldValue1},{childId2, childFieldValue2}], <other
fields of id1>.
id2,[{childId3, childFieldValue1},{childId4, childFieldValue2}, {childId5,
childFieldValue3}], <other fields of id2>
The aggregation itself is not the problem but the correlation or completing
aggregation. My first try was to aggregate all ("constant(true)" as
correlation). With this i compared the ids of newExchange and oldExchange.
If the id is different i mark the oldExchange with special header
("complete"). The problem here is, that i have to return the oldExchange and
with this i lose the newExchange that starts a new group.
Don't know if i really need it but i thought about creating a custom
Predicate for correlation. But how?
So how would you solve this problem?
Thanks in advance
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-with-custom-aggregation-custom-correlation-data-base-rows-tp5772324.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Problem with custom aggregation/ custom correlation (data base
rows)
Posted by dermoritz <ta...@hotmail.com>.
I feared that - we just migrated to 2.14.3 (not easy in the current project).
So going to 1.16 (is it released yet?) is no option at the moment.
How would you achieve this with 2.14? Within aggregator or with custom
correlation predicate and aggregator?
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-with-custom-aggregation-custom-correlation-data-base-rows-tp5772324p5772329.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Problem with custom aggregation/ custom correlation (data base rows)
Posted by Claus Ibsen <cl...@gmail.com>.
In Camel 2.16 there is a preCompletion mode that makes this use-case
possible/easier.
On Tue, Oct 6, 2015 at 10:52 AM, dermoritz <ta...@hotmail.com> wrote:
> I am fetching data from data base (jdbc component) that contains 1:n
> relations. That means that i have multiple entries with same id:
>
> id1,childId1, childFieldValue1, <other fields of id1>
> id1,childId2, childFieldValue2, <other fields of id1>
> id2,childId3, childFieldValue1, <other fields of id2>
> id2,childId4, childFieldValue2, <other fields of id2>
> id2,childId5, childFieldValue3, <other fields of id2>
>
> I want to aggregate them to
>
> id1,[{childId1, childFieldValue1},{childId2, childFieldValue2}], <other
> fields of id1>.
> id2,[{childId3, childFieldValue1},{childId4, childFieldValue2}, {childId5,
> childFieldValue3}], <other fields of id2>
> The aggregation itself is not the problem but the correlation or completing
> aggregation. My first try was to aggregate all ("constant(true)" as
> correlation). With this i compared the ids of newExchange and oldExchange.
> If the id is different i mark the oldExchange with special header
> ("complete"). The problem here is, that i have to return the oldExchange and
> with this i lose the newExchange that starts a new group.
> Don't know if i really need it but i thought about creating a custom
> Predicate for correlation. But how?
>
> So how would you solve this problem?
>
> Thanks in advance
>
>
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Problem-with-custom-aggregation-custom-correlation-data-base-rows-tp5772324.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2nd edition:
https://www.manning.com/books/camel-in-action-second-edition
Re: Problem with custom aggregation/ custom correlation (data base
rows)
Posted by dermoritz <ta...@hotmail.com>.
Thanks to input from here
<http://camel.465427.n5.nabble.com/aggreagator-completion-based-on-correlation-key-td5772332.html>
I created a working solution. Please comment if you have suggestion
(especially about: is it possible to get rid of the processor thats sets
correlation header, is there a better way for completion logic, the timeout
is needed to complete last group):
Route:
private class Route extends RouteBuilder {
@Override
public void configure() throws Exception {
from( DIRECT_IN ).split().body().process( new HeaderProcessor()
)
.aggregate( header( "ORDER_ID" ), new PackAggregator(
PACK_COLUMNS ) )
.eagerCheckCompletion()
.completionPredicate( header(
PackAggregator.COMPLETION_HEADER ).isEqualTo( true ) )
.completionTimeout( 1000 )
.to( MOCK_OUT );
}
}
private class HeaderProcessor implements Processor {
private Object lastId = null;
@Override
public void process( Exchange exchange ) throws Exception {
Map<String, Object> in = exchange.getIn().getBody( Map.class );
Object id = in.get( PackAggregator.FRONTEND_ORDER_ID_KEY );
exchange.getIn().setHeader( "ORDER_ID", id );
// set completion header, will only work if
.eagerCheckCompletion() is set
if( lastId != null && !lastId.equals( id ) ) {
exchange.getIn().setHeader(
PackAggregator.COMPLETION_HEADER, true );
}
lastId = id;
}
}
The Aggregator itself is the boring part - so i omitted it.
--
View this message in context: http://camel.465427.n5.nabble.com/Problem-with-custom-aggregation-custom-correlation-data-base-rows-tp5772324p5772338.html
Sent from the Camel - Users mailing list archive at Nabble.com.