You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Josh J <jo...@gmail.com> on 2014/11/11 23:41:52 UTC

concat two Dstreams

Hi,

Is it possible to concatenate or append two Dstreams together? I have an
incoming stream that I wish to combine with data that's generated by a
utility. I then need to process the combined Dstream.

Thanks,
Josh

Re: concat two Dstreams

Posted by Sean Owen <so...@cloudera.com>.

Concatenate? no. It doesn't make sense in this context to think about one
potentially infinite stream coming after another one. Do you just want the
union of batches from two streams? yes, just union(). You can union() with
non-streaming RDDs too.

On Tue, Nov 11, 2014 at 10:41 PM, Josh J <jo...@gmail.com> wrote:

> Hi,
>
> Is it possible to concatenate or append two Dstreams together? I have an
> incoming stream that I wish to combine with data that's generated by a
> utility. I then need to process the combined Dstream.
>
> Thanks,
> Josh
>

Re: concat two Dstreams

Posted by Josh J <jo...@gmail.com>.

I think it's just called "union"

On Tue, Nov 11, 2014 at 2:41 PM, Josh J <jo...@gmail.com> wrote:

> Hi,
>
> Is it possible to concatenate or append two Dstreams together? I have an
> incoming stream that I wish to combine with data that's generated by a
> utility. I then need to process the combined Dstream.
>
> Thanks,
> Josh
>