You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by dhurandar S <dh...@gmail.com> on 2020/12/02 21:31:41 UTC
Running Flink job as a rest
Can Flink job be running as Rest Server, Where Apache Flink job is
listening on a port (443). When a user calls this URL with payload,
data directly goes to the Apache Flink windowing function.
Right now Flink can ingest data from Kafka or Kinesis, but we have a use
case where we would like to push data to Flink, where Flink is listening on
a port
--
Thank you and regards,
Dhurandar
Re: Running Flink job as a rest
Posted by "Jaffe, Julian" <Ju...@activision.com>.
I can't vouch for it personally, but perhaps the Apache Bahir Netty Source for Flink could help you? It sounds like you want to use HTTPS, which this doesn't support directly, but the source might be a helpful starting point to adding the functionality you need.
On 12/3/20, 1:33 AM, "Chesnay Schepler" <ch...@apache.org> wrote:
What you are asking for is an HTTP(s) source. This currently does not
exist for Flink, so you would have to implement it yourself.
Additionally you would have to figure out the host on which the source
runs on yourself.
It may be easier to setup a separate HTTP(s) server that accepts data,
which you then query from the source.
On 12/2/2020 10:31 PM, dhurandar S wrote:
> Can Flink job be running as Rest Server, Where Apache Flink job is
> listening on a port (443). When a user calls this URL with payload,
> data directly goes to the Apache Flink windowing function.
>
> Right now Flink can ingest data from Kafka or Kinesis, but we have a use
> case where we would like to push data to Flink, where Flink is listening on
> a port
>
Re: Running Flink job as a rest
Posted by Chesnay Schepler <ch...@apache.org>.
What you are asking for is an HTTP(s) source. This currently does not
exist for Flink, so you would have to implement it yourself.
Additionally you would have to figure out the host on which the source
runs on yourself.
It may be easier to setup a separate HTTP(s) server that accepts data,
which you then query from the source.
On 12/2/2020 10:31 PM, dhurandar S wrote:
> Can Flink job be running as Rest Server, Where Apache Flink job is
> listening on a port (443). When a user calls this URL with payload,
> data directly goes to the Apache Flink windowing function.
>
> Right now Flink can ingest data from Kafka or Kinesis, but we have a use
> case where we would like to push data to Flink, where Flink is listening on
> a port
>
Re: Running Flink job as a rest
Posted by Chesnay Schepler <ch...@apache.org>.
What you are asking for is an HTTP(s) source. This currently does not
exist for Flink, so you would have to implement it yourself.
Additionally you would have to figure out the host on which the source
runs on yourself.
It may be easier to setup a separate HTTP(s) server that accepts data,
which you then query from the source.
On 12/2/2020 10:31 PM, dhurandar S wrote:
> Can Flink job be running as Rest Server, Where Apache Flink job is
> listening on a port (443). When a user calls this URL with payload,
> data directly goes to the Apache Flink windowing function.
>
> Right now Flink can ingest data from Kafka or Kinesis, but we have a use
> case where we would like to push data to Flink, where Flink is listening on
> a port
>
Re: Running Flink job as a rest
Posted by Yun Tang <my...@live.com>.
Hi Dhurandar,
I'm afraid that Flink's rest API cannot satisfy your request as it would not act as any source. One possible example could be SocketWindowWordCount [1] which listens data on a port from all taskmanagers with sources.
[1] https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/socket/SocketWindowWordCount.java
Best
Yun Tang
________________________________
From: dhurandar S <dh...@gmail.com>
Sent: Thursday, December 3, 2020 5:31
To: Flink Dev <de...@flink.apache.org>; user <us...@flink.apache.org>
Subject: Running Flink job as a rest
Can Flink job be running as Rest Server, Where Apache Flink job is listening on a port (443). When a user calls this URL with payload, data directly goes to the Apache Flink windowing function.
Right now Flink can ingest data from Kafka or Kinesis, but we have a use case where we would like to push data to Flink, where Flink is listening on a port
--
Thank you and regards,
Dhurandar
Re: Running Flink job as a rest
Posted by Yun Tang <my...@live.com>.
Hi Dhurandar,
I'm afraid that Flink's rest API cannot satisfy your request as it would not act as any source. One possible example could be SocketWindowWordCount [1] which listens data on a port from all taskmanagers with sources.
[1] https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/socket/SocketWindowWordCount.java
Best
Yun Tang
________________________________
From: dhurandar S <dh...@gmail.com>
Sent: Thursday, December 3, 2020 5:31
To: Flink Dev <de...@flink.apache.org>; user <us...@flink.apache.org>
Subject: Running Flink job as a rest
Can Flink job be running as Rest Server, Where Apache Flink job is listening on a port (443). When a user calls this URL with payload, data directly goes to the Apache Flink windowing function.
Right now Flink can ingest data from Kafka or Kinesis, but we have a use case where we would like to push data to Flink, where Flink is listening on a port
--
Thank you and regards,
Dhurandar