Apache Camel EIP to process batch of messages identifying those which can be filtered

[gilboy <jo...@gmail.com>]

Hi

My route calls a DB procedure which returns a large number of order objects. 

Before I send out the orders to a counterparty I want to examine the list of
orders and see if the same order ID appears as both a new order and a cancel
order, e.g. order 1001 returned as a new order and order 1001 returned as a
cancel order. If I find any instances of the above I want to remove the
order, i.e. not send 1001 out to counterparty.

I looked at using the aggregator EIP but don't think its a great fit. Just
wondering if there is another more suitable EIP that fits this use case

Thanks
Joe



--
View this message in context: http://camel.465427.n5.nabble.com/Apache-Camel-EIP-to-process-batch-of-messages-identifying-those-which-can-be-filtered-tp5777844.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Apache Camel EIP to process batch of messages identifying those which can be filtered

[Quinn Stevenson <qu...@pronoia-solutions.com>]

How about a simple splitter that removes the duplicates?  For example, a route like this
<camelContext xmlns="http://camel.apache.org/schema/spring">
    <route>
        <from uri="file://src/data?noop=true" />
        <split strategyRef="removeDupsStrategy">
            <tokenize token="\n" />
            <setHeader headerName="MY_KEY">
                <simple>${body}</simple>
            </setHeader>
        </split>
        <log message="complete: ${body}" />
    </route>
</camelContext>

Could use a Splitter like this
public class RemoveDuplicateSplitterStrategry implements AggregationStrategy{
    @Override
    public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
        Logger log = LoggerFactory.getLogger(this.getClass());

        if ( null == oldExchange ) {
            Map<String, String> messageList = new HashMap<>();
            messageList.put( newExchange.getIn().getHeader("MY_KEY", String.class), newExchange.getIn().getBody(String.class) );
            newExchange.getIn().setBody( messageList);
            return newExchange;
        }

        Map<String, String> messageList = oldExchange.getIn().getBody( Map.class) ;
        String key = newExchange.getIn().getHeader("MY_KEY", String.class);
        if (messageList.containsKey(key)) {
            String value = messageList.remove(key);
            log.warn( "Removing duplicate for {}: {}", key, value);
        } else {
            messageList.put(key, newExchange.getIn().getBody(String.class) );
        }

        return oldExchange;
    }
}

> On Feb 18, 2016, at 10:17 AM, gilboy <jo...@gmail.com> wrote:
> 
> Thanks for the quick feedback. 
> 
> If I use the Idempotent Consumer EIP this would mean that the 2nd order
> (Order cancel) will be filtered out. However, I would want the new order to
> be filtered also. However, that would be sent to the counterparty with the
> Idempotent Consumer EIP
> 
> Regards
> Joe
> 
> 
> 
> --
> View this message in context: http://camel.465427.n5.nabble.com/Apache-Camel-EIP-to-process-batch-of-messages-identifying-those-which-can-be-filtered-tp5777844p5777868.html
> Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Apache Camel EIP to process batch of messages identifying those which can be filtered

[gilboy <jo...@gmail.com>]

Thanks for the quick feedback. 

If I use the Idempotent Consumer EIP this would mean that the 2nd order
(Order cancel) will be filtered out. However, I would want the new order to
be filtered also. However, that would be sent to the counterparty with the
Idempotent Consumer EIP

Regards
Joe



--
View this message in context: http://camel.465427.n5.nabble.com/Apache-Camel-EIP-to-process-batch-of-messages-identifying-those-which-can-be-filtered-tp5777844p5777868.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Apache Camel EIP to process batch of messages identifying those which can be filtered

[Quinn Stevenson <qu...@pronoia-solutions.com>]

Sounds like a good use case for the Idempotent Consumer EIP 
http://camel.apache.org/idempotent-consumer.html <http://camel.apache.org/idempotent-consumer.html>



> On Feb 18, 2016, at 7:06 AM, gilboy <jo...@gmail.com> wrote:
> 
> Hi
> 
> My route calls a DB procedure which returns a large number of order objects. 
> 
> Before I send out the orders to a counterparty I want to examine the list of
> orders and see if the same order ID appears as both a new order and a cancel
> order, e.g. order 1001 returned as a new order and order 1001 returned as a
> cancel order. If I find any instances of the above I want to remove the
> order, i.e. not send 1001 out to counterparty.
> 
> I looked at using the aggregator EIP but don't think its a great fit. Just
> wondering if there is another more suitable EIP that fits this use case
> 
> Thanks
> Joe
> 
> 
> 
> --
> View this message in context: http://camel.465427.n5.nabble.com/Apache-Camel-EIP-to-process-batch-of-messages-identifying-those-which-can-be-filtered-tp5777844.html
> Sent from the Camel - Users mailing list archive at Nabble.com.