You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by dachuan <hd...@gmail.com> on 2013/11/01 15:04:22 UTC
any idea about the difference between windowDuration and
slideDuration ?
they are in streaming.dstream.WindowDStream.
thanks.
Re: any idea about the difference between windowDuration and
slideDuration ?
Posted by dachuan <hd...@gmail.com>.
I see. thanks.
On Fri, Nov 1, 2013 at 1:31 PM, Patrick Wendell <pw...@gmail.com> wrote:
> Ya the window duration is how large the window is, the slide duration
> is how much the window slides each time.
>
> So let's say the underlying stream has the following batches:
>
> 1 2 3 4 5 6 7 8 9
>
> Now lets say you have:
> window=3
> slide=1
>
> [1 2 3] 4 5 6 7 8 9
> 1 [2 3 4] 5 6 7 8 9
> 1 2 [3 4 5] 6 7 8 9
> 1 2 3 [4 5 6] 7 8 9
>
> window=3
> slide=2
>
> [1 2 3] 4 5 6 7 8 9
> 1 2 [3 4 5] 6 7 8 9
> 1 2 3 4 [5 6 7] 8 9
> 1 2 3 4 5 6 [7 8 9]
>
> window=3
> slide=3
>
> [1 2 3] 4 5 6 7 8 9
> 1 2 3 [4 5 6] 7 8 9
> 1 2 3 4 5 6 [7 8 9]
>
> On Fri, Nov 1, 2013 at 7:04 AM, dachuan <hd...@gmail.com> wrote:
> > they are in streaming.dstream.WindowDStream.
> >
> > thanks.
>
--
Dachuan Huang
Cellphone: 614-390-7234
2015 Neil Avenue
Ohio State University
Columbus, Ohio
U.S.A.
43210
Re: any idea about the difference between windowDuration and
slideDuration ?
Posted by Patrick Wendell <pw...@gmail.com>.
Ya the window duration is how large the window is, the slide duration
is how much the window slides each time.
So let's say the underlying stream has the following batches:
1 2 3 4 5 6 7 8 9
Now lets say you have:
window=3
slide=1
[1 2 3] 4 5 6 7 8 9
1 [2 3 4] 5 6 7 8 9
1 2 [3 4 5] 6 7 8 9
1 2 3 [4 5 6] 7 8 9
window=3
slide=2
[1 2 3] 4 5 6 7 8 9
1 2 [3 4 5] 6 7 8 9
1 2 3 4 [5 6 7] 8 9
1 2 3 4 5 6 [7 8 9]
window=3
slide=3
[1 2 3] 4 5 6 7 8 9
1 2 3 [4 5 6] 7 8 9
1 2 3 4 5 6 [7 8 9]
On Fri, Nov 1, 2013 at 7:04 AM, dachuan <hd...@gmail.com> wrote:
> they are in streaming.dstream.WindowDStream.
>
> thanks.