You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by PedroMrChaves <pe...@gmail.com> on 2017/07/06 14:17:11 UTC
Can AsyncFunction be applied to connected streams
Hello,
Is there a way to apply the AsyncFunction to connected streams like in a
CoFlatMap?
I would like to connect streams from different types and process one of them
based on the state
created by the other in an asynchronous fashion.
Regards,
Pedro Chaves
-----
Best Regards,
Pedro Chaves
--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Can-AsyncFunction-be-applied-to-connected-streams-tp14137.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Can AsyncFunction be applied to connected streams
Posted by Aljoscha Krettek <al...@apache.org>.
I think this would not necessarily be a problem. If the async operation is directly after the enrichment operation the enriched operations will be directly forwarded to the async operation. (With a copy step, that can be disabled by enabling object reuse at the StreamExecutionEnvironment)
Best,
Aljoscha
> On 7. Jul 2017, at 15:59, PedroMrChaves <pe...@gmail.com> wrote:
>
> Hello,
>
> I wanted to keep the data locally, if I associate the fetched metadata with
> eachevent (in an enrichment phase) it would considerably increase their size
> since the metadata that I need to process the event in the async I/O is to
> large.
>
> Regards,
> Pedro.
>
>
>
> -----
> Best Regards,
> Pedro Chaves
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Can-AsyncFunction-be-applied-to-connected-streams-tp14137p14148.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Can AsyncFunction be applied to connected streams
Posted by PedroMrChaves <pe...@gmail.com>.
Hello,
I wanted to keep the data locally, if I associate the fetched metadata with
eachevent (in an enrichment phase) it would considerably increase their size
since the metadata that I need to process the event in the async I/O is to
large.
Regards,
Pedro.
-----
Best Regards,
Pedro Chaves
--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Can-AsyncFunction-be-applied-to-connected-streams-tp14137p14148.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Can AsyncFunction be applied to connected streams
Posted by Aljoscha Krettek <al...@apache.org>.
Hi,
While it’s not possible to directly apply an Async I/O operator to a connected stream you can use a two-stage approach:
- Have a CoFlatMap that enriches the elements from the first stream based on the second stream
- Use an Async I/O operator right after that, on the enriched stream
Would that be an option for you?
Best,
Aljoscha
> On 6. Jul 2017, at 16:17, PedroMrChaves <pe...@gmail.com> wrote:
>
> Hello,
>
> Is there a way to apply the AsyncFunction to connected streams like in a
> CoFlatMap?
> I would like to connect streams from different types and process one of them
> based on the state
> created by the other in an asynchronous fashion.
>
> Regards,
> Pedro Chaves
>
>
>
> -----
> Best Regards,
> Pedro Chaves
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Can-AsyncFunction-be-applied-to-connected-streams-tp14137.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.