Connected Streams - Controlling Order of arrival on the two streams

[Sameer W <sa...@axiomine.com>]

Hi,

I am using connected streams to send rules coded as JavaScript functions on
one stream and event data on another stream. They are both keyed by the
device id. The rules are cached in the co-map operation until another rule
arrives to override existing rule.

Is there a way to ensure that the rules stream arrives before the event
data stream. I am assuming there is no guarantee for this and I cache the
event data is the rules have not yet arrived and process and clear the
cache when the rules arrive. The rules are expected to arrive before the
event data. I am only using this method as a precautionary measure in case
the rules arrive late for reasons unrelated to when they were sent.

Is there a way to handle this situation without caching the streams?


Thanks,
Sameer

Re: Connected Streams - Controlling Order of arrival on the two streams

[Aljoscha Krettek <al...@apache.org>]

Hi,
I'm afraid you guessed correctly that it is not possible to ensure that
rules arrive before events. I think the way you solved it (with buffering)
is the correct way to go about this.

Cheers,
Aljoscha

On Wed, 10 Aug 2016 at 01:31 Sameer W <sa...@axiomine.com> wrote:

> Hi,
>
> I am using connected streams to send rules coded as JavaScript functions
> on one stream and event data on another stream. They are both keyed by the
> device id. The rules are cached in the co-map operation until another rule
> arrives to override existing rule.
>
> Is there a way to ensure that the rules stream arrives before the event
> data stream. I am assuming there is no guarantee for this and I cache the
> event data is the rules have not yet arrived and process and clear the
> cache when the rules arrive. The rules are expected to arrive before the
> event data. I am only using this method as a precautionary measure in case
> the rules arrive late for reasons unrelated to when they were sent.
>
> Is there a way to handle this situation without caching the streams?
>
>
> Thanks,
> Sameer
>