What is the best approach for large number of concurrent mappings?

[Raul <ro...@gmail.com>]

>
> Hi,
>
> I am trying to work out what the best approach would be for the following
> problem?
>
> Conditions:
>
>    1. You have a river of data flowing at a varying rate into a cache,
>    the cache holds the last N minuets of data.
>    2. The river of data has to be analyzed as fast as possible
>    3. You have over 10,000 different map reduce functions that has to
>    aggregate into at least 1,000,000 buckets
>
> Question:
> Is it a good idea to spin up 10k continuous mappers or is there a better
> way
>
> Thank you for assistance
>
>
>

Re: What is the best approach for large number of concurrent mappings?

[vkulichenko <va...@gmail.com>]

Hi,

Can you please clarify what you mean by continuous mapper here? Actually,
your use case sounds similar to one implemented in the WordCount example
provided with Ignite:
https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples/streaming/wordcount

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/What-is-the-best-approach-for-large-number-of-concurrent-mappings-tp11445p11481.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.