You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Dong Mo <mo...@gmail.com> on 2014/03/31 20:05:02 UTC

how spark dstream handles congestion?

Dear list,

I was wondering how Spark handles congestion when the upstream is
generating dstreams faster than downstream workers can handle?

Thanks
-Mo

Re: how spark dstream handles congestion?

Posted by Dong Mo <mo...@gmail.com>.

Thanks
-Mo


2014-03-31 13:16 GMT-05:00 Evgeny Shishkin <it...@gmail.com>:

>
> On 31 Mar 2014, at 21:05, Dong Mo <mo...@gmail.com> wrote:
>
> > Dear list,
> >
> > I was wondering how Spark handles congestion when the upstream is
> generating dstreams faster than downstream workers can handle?
>
> It will eventually OOM.
>
>

Re: how spark dstream handles congestion?

Posted by Evgeny Shishkin <it...@gmail.com>.

On 31 Mar 2014, at 21:05, Dong Mo <mo...@gmail.com> wrote:

> Dear list,
> 
> I was wondering how Spark handles congestion when the upstream is generating dstreams faster than downstream workers can handle?

It will eventually OOM.