Priority polling across multiple Kafka spouts

[aravind s <su...@gmail.com>]

Hi,

Is there a way where we could fetch records from multiple spouts at the
particular ratio and merge into a single stream for processing as such.
Some kind of priority processing from multiple spouts a single topology
listens to as such.

For example
I have a three Kafka streams p1, p2, p3 as such

For 100 slots of buffer

If there are records in p1 then fill the buffer
If the buffer space exists then fill the buffer from the records from p2
Else from p3.

Maybe specify as weights to each to each stream to be processed as such.

Please do let me know is there a way we could do it.

Regards,
Aravind S

Re: Priority polling across multiple Kafka spouts

[aravind s <su...@gmail.com>]

Hi,

Any suggestions on same it would be helpful

Regards
Aravind S

On Thu, Mar 15, 2018 at 7:34 PM, aravind s <su...@gmail.com> wrote:

> Hi,
>
> Is there a way where we could fetch records from multiple spouts at the
> particular ratio and merge into a single stream for processing as such.
> Some kind of priority processing from multiple spouts a single topology
> listens to as such.
>
> For example
> I have a three Kafka streams p1, p2, p3 as such
>
> For 100 slots of buffer
>
> If there are records in p1 then fill the buffer
> If the buffer space exists then fill the buffer from the records from p2
> Else from p3.
>
> Maybe specify as weights to each to each stream to be processed as such.
>
> Please do let me know is there a way we could do it.
>
> Regards,
> Aravind S
>
>
>
>