You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Artemis User <ar...@dtechspace.com> on 2020/12/22 14:57:59 UTC
Integrating multiple streaming sources
Is there anyway to integrate/fuse multiple streaming sources into a
single stream process? In other words, the current structured streaming
API dictates a single a streaming source and sink. We'd like to have a
stream process that interfaces with multiple stream sources, perform a
join and direct the result to a single sink. Is this possible? Thanks!
-- ND
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: Integrating multiple streaming sources
Posted by Artemis User <ar...@dtechspace.com>.
Hmm, looks like Spark 2.3+ does support stream-to-stream join. But the
online doc doesn't provide any examples. If anyone could provide some
concrete reference, I'd really appreciate. Thanks! -- ND
On 12/22/20 9:57 AM, Artemis User wrote:
> Is there anyway to integrate/fuse multiple streaming sources into a
> single stream process? In other words, the current structured
> streaming API dictates a single a streaming source and sink. We'd
> like to have a stream process that interfaces with multiple stream
> sources, perform a join and direct the result to a single sink. Is
> this possible? Thanks!
>
> -- ND
>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org