You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Hemalatha A <he...@googlemail.com> on 2016/09/04 11:12:27 UTC
How does chaining of Windowed Dstreams work?
Hello,
I have a set of Dstreams on which I'm performing some computation on each
Dstreams which is widowed on one other from the base stream based on the
order of window intervals. I want to find out the best Strem on which I
could window a particular stream on?
Suppose, I have a spark Dstream, with batch interval as 10sec and other
streams are windowed on base steams as below:
*Stream*
*Window*
*Sliding*
*Windowed On*
StreamA
30
10
Base Stream
StreamB
20
20
Base Stream
StreamC
90
20
?
Now, should I base the StreamC on StreamA since its window is multiple of
StreamA or base it on StreamB since it has a higher and same sliding
interval. Which would be a better choice?
Or is it the same as window on Base stream? How does it basically work?
--
Regards
Hemalatha
Re: How does chaining of Windowed Dstreams work?
Posted by Hemalatha A <he...@googlemail.com>.
Hello,
Can anyone please answer the below question and help me understand the
windowing operations.
On Sun, Sep 4, 2016 at 4:42 PM, Hemalatha A <
hemalatha.amrutha@googlemail.com> wrote:
> Hello,
>
> I have a set of Dstreams on which I'm performing some computation on each
> Dstreams which is widowed on one other from the base stream based on the
> order of window intervals. I want to find out the best Strem on which I
> could window a particular stream on?
>
> Suppose, I have a spark Dstream, with batch interval as 10sec and other
> streams are windowed on base steams as below:
>
> *Stream*
>
> *Window*
>
> *Sliding*
>
> *Windowed On*
>
> StreamA
>
> 30
>
> 10
>
> Base Stream
>
> StreamB
>
> 20
>
> 20
>
> Base Stream
>
> StreamC
>
> 90
>
> 20
>
> ?
>
>
>
> Now, should I base the StreamC on StreamA since its window is multiple of
> StreamA or base it on StreamB since it has a higher and same sliding
> interval. Which would be a better choice?
>
>
> Or is it the same as window on Base stream? How does it basically work?
>
>
> --
>
>
> Regards
> Hemalatha
>
--
Regards
Hemalatha