You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by "Bohinski, Kevin" <Ke...@comcast.com> on 2021/03/24 20:37:34 UTC
PyFlink DataStream Example Kafka/Kinesis?
Hi,
Is there an example kafka/kinesis source or sink for the PyFlink DataStream API?
Best,
kevin
Re: PyFlink DataStream Example Kafka/Kinesis?
Posted by "Bohinski, Kevin" <Ke...@comcast.com>.
Nevermind, found this for anyone else looking: https://github.com/apache/flink/blob/master/flink-end-to-end-tests/flink-python-test/python/datastream/data_stream_job.py
From: "Bohinski, Kevin" <Ke...@comcast.com>
Date: Wednesday, March 24, 2021 at 4:38 PM
To: user <us...@flink.apache.org>
Subject: PyFlink DataStream Example Kafka/Kinesis?
Hi,
Is there an example kafka/kinesis source or sink for the PyFlink DataStream API?
Best,
kevin
Re: [EXTERNAL] Re: PyFlink DataStream Example Kafka/Kinesis?
Posted by Shuiqiang Chen <ac...@gmail.com>.
Hi Kevin, Xinbin,
Hi Shuiqiang,
>
> Thanks for the quick response on creating the ticket for Kinesis
> Connector. Do you mind giving me the chance to try to implement the
> connector over the weekend?
>
> I am interested in contributing to Flink, and I think this can be a good
> starting point to me
>
> Best
> Bin
>
Sorry for the confusion. Xinbin had personal letter that could not be
displayed in this email thread that he is interested in solving this issue
https://issues.apache.org/jira/browse/FLINK-21966. So both of you can
participate in this contribution as you like.
Best,
Shuiqiang
Bohinski, Kevin <Ke...@comcast.com> 于2021年3月25日周四 下午12:00写道：
> Hi Shuiqiang,
>
>
>
> Thanks for letting me know. Feel free to send any beginner level
> contributions for this effort my way 😊 .
>
>
>
> Best,
>
> kevin
>
>
>
> *From: *Shuiqiang Chen <ac...@gmail.com>
> *Date: *Wednesday, March 24, 2021 at 10:31 PM
> *To: *"Bohinski, Kevin" <Ke...@comcast.com>
> *Cc: *user <us...@flink.apache.org>
> *Subject: *[EXTERNAL] Re: PyFlink DataStream Example Kafka/Kinesis?
>
>
>
> Hi Kevin,
>
>
>
> Kinesis connector is not supported yet in Python DataStream API. We will
> add it in the future.
>
>
>
> Best,
>
> Shuiqiang
>
>
>
> Bohinski, Kevin <Ke...@comcast.com> 于2021年3月25日周四 上午5:03写道：
>
> Is there a kinesis example?
>
>
>
> *From: *"Bohinski, Kevin" <Ke...@comcast.com>
> *Date: *Wednesday, March 24, 2021 at 4:40 PM
> *To: *"Bohinski, Kevin" <Ke...@comcast.com>
> *Subject: *Re: PyFlink DataStream Example Kafka/Kinesis?
>
>
>
> Nevermind, found this for anyone else looking:
> https://github.com/apache/flink/blob/master/flink-end-to-end-tests/flink-python-test/python/datastream/data_stream_job.py
> <https://urldefense.com/v3/__https:/github.com/apache/flink/blob/master/flink-end-to-end-tests/flink-python-test/python/datastream/data_stream_job.py__;!!CQl3mcHX2A!QFaB5DO1ZU_Sx8v59FCwTcqH2lAH3CrM8-jFD1xIuUN-LvDep2fAnOlpFTwBV7CSMLggLw$>
>
>
>
> *From: *"Bohinski, Kevin" <Ke...@comcast.com>
> *Date: *Wednesday, March 24, 2021 at 4:38 PM
> *To: *user <us...@flink.apache.org>
> *Subject: *PyFlink DataStream Example Kafka/Kinesis?
>
>
>
> Hi,
>
>
>
> Is there an example kafka/kinesis source or sink for the PyFlink
> DataStream API?
>
>
>
> Best,
>
> kevin
>
>
Re: [EXTERNAL] Re: PyFlink DataStream Example Kafka/Kinesis?
Posted by "Bohinski, Kevin" <Ke...@comcast.com>.
Hi Shuiqiang,
Thanks for letting me know. Feel free to send any beginner level contributions for this effort my way 😊 .
Best,
kevin
From: Shuiqiang Chen <ac...@gmail.com>
Date: Wednesday, March 24, 2021 at 10:31 PM
To: "Bohinski, Kevin" <Ke...@comcast.com>
Cc: user <us...@flink.apache.org>
Subject: [EXTERNAL] Re: PyFlink DataStream Example Kafka/Kinesis?
Hi Kevin,
Kinesis connector is not supported yet in Python DataStream API. We will add it in the future.
Best,
Shuiqiang
Bohinski, Kevin <Ke...@comcast.com>> 于2021年3月25日周四 上午5:03写道：
Is there a kinesis example?
From: "Bohinski, Kevin" <Ke...@comcast.com>>
Date: Wednesday, March 24, 2021 at 4:40 PM
To: "Bohinski, Kevin" <Ke...@comcast.com>>
Subject: Re: PyFlink DataStream Example Kafka/Kinesis?
Nevermind, found this for anyone else looking: https://github.com/apache/flink/blob/master/flink-end-to-end-tests/flink-python-test/python/datastream/data_stream_job.py<https://urldefense.com/v3/__https:/github.com/apache/flink/blob/master/flink-end-to-end-tests/flink-python-test/python/datastream/data_stream_job.py__;!!CQl3mcHX2A!QFaB5DO1ZU_Sx8v59FCwTcqH2lAH3CrM8-jFD1xIuUN-LvDep2fAnOlpFTwBV7CSMLggLw$>
From: "Bohinski, Kevin" <Ke...@comcast.com>>
Date: Wednesday, March 24, 2021 at 4:38 PM
To: user <us...@flink.apache.org>>
Subject: PyFlink DataStream Example Kafka/Kinesis?
Hi,
Is there an example kafka/kinesis source or sink for the PyFlink DataStream API?
Best,
kevin
Re: PyFlink DataStream Example Kafka/Kinesis?
Posted by Shuiqiang Chen <ac...@gmail.com>.
Sure, no problem. You can refer to the implementation of Kafka connector,
they are very much alike.
Xinbin Huang <bi...@gmail.com> 于2021年3月25日周四 上午10:55写道：
> Hi Shuiqiang,
>
> Thanks for the quick response on creating the ticket for Kinesis
> Connector. Do you mind giving me the chance to try to implement the
> connector over the weekend?
>
> I am interested in contributing to Flink, and I think this can be a good
> starting point to me
>
> Best
> Bin
>
> On Wed, Mar 24, 2021 at 7:49 PM Shuiqiang Chen <ac...@gmail.com>
> wrote:
>
>> I have just created the jira
>> https://issues.apache.org/jira/browse/FLINK-21966 and will finish it
>> soon.
>>
>> Best,
>> Shuiqiang
>>
>> Xinbin Huang <bi...@gmail.com> 于2021年3月25日周四 上午10:43写道：
>>
>>> Hi Shuiqiang,
>>>
>>> I am interested in the same feature. Do we have a ticket to track this
>>> right now?
>>>
>>> Best
>>> Bin
>>>
>>> On Wed, Mar 24, 2021 at 7:30 PM Shuiqiang Chen <ac...@gmail.com>
>>> wrote:
>>>
>>>> Hi Kevin,
>>>>
>>>> Kinesis connector is not supported yet in Python DataStream API. We
>>>> will add it in the future.
>>>>
>>>> Best,
>>>> Shuiqiang
>>>>
>>>> Bohinski, Kevin <Ke...@comcast.com> 于2021年3月25日周四 上午5:03写道：
>>>>
>>>>> Is there a kinesis example?
>>>>>
>>>>>
>>>>>
>>>>> *From: *"Bohinski, Kevin" <Ke...@comcast.com>
>>>>> *Date: *Wednesday, March 24, 2021 at 4:40 PM
>>>>> *To: *"Bohinski, Kevin" <Ke...@comcast.com>
>>>>> *Subject: *Re: PyFlink DataStream Example Kafka/Kinesis?
>>>>>
>>>>>
>>>>>
>>>>> Nevermind, found this for anyone else looking:
>>>>> https://github.com/apache/flink/blob/master/flink-end-to-end-tests/flink-python-test/python/datastream/data_stream_job.py
>>>>>
>>>>>
>>>>>
>>>>> *From: *"Bohinski, Kevin" <Ke...@comcast.com>
>>>>> *Date: *Wednesday, March 24, 2021 at 4:38 PM
>>>>> *To: *user <us...@flink.apache.org>
>>>>> *Subject: *PyFlink DataStream Example Kafka/Kinesis?
>>>>>
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>>
>>>>>
>>>>> Is there an example kafka/kinesis source or sink for the PyFlink
>>>>> DataStream API?
>>>>>
>>>>>
>>>>>
>>>>> Best,
>>>>>
>>>>> kevin
>>>>>
>>>>
Re: PyFlink DataStream Example Kafka/Kinesis?
Posted by Shuiqiang Chen <ac...@gmail.com>.
I have just created the jira
https://issues.apache.org/jira/browse/FLINK-21966 and will finish it soon.
Best,
Shuiqiang
Xinbin Huang <bi...@gmail.com> 于2021年3月25日周四 上午10:43写道：
> Hi Shuiqiang,
>
> I am interested in the same feature. Do we have a ticket to track this
> right now?
>
> Best
> Bin
>
> On Wed, Mar 24, 2021 at 7:30 PM Shuiqiang Chen <ac...@gmail.com>
> wrote:
>
>> Hi Kevin,
>>
>> Kinesis connector is not supported yet in Python DataStream API. We will
>> add it in the future.
>>
>> Best,
>> Shuiqiang
>>
>> Bohinski, Kevin <Ke...@comcast.com> 于2021年3月25日周四 上午5:03写道：
>>
>>> Is there a kinesis example?
>>>
>>>
>>>
>>> *From: *"Bohinski, Kevin" <Ke...@comcast.com>
>>> *Date: *Wednesday, March 24, 2021 at 4:40 PM
>>> *To: *"Bohinski, Kevin" <Ke...@comcast.com>
>>> *Subject: *Re: PyFlink DataStream Example Kafka/Kinesis?
>>>
>>>
>>>
>>> Nevermind, found this for anyone else looking:
>>> https://github.com/apache/flink/blob/master/flink-end-to-end-tests/flink-python-test/python/datastream/data_stream_job.py
>>>
>>>
>>>
>>> *From: *"Bohinski, Kevin" <Ke...@comcast.com>
>>> *Date: *Wednesday, March 24, 2021 at 4:38 PM
>>> *To: *user <us...@flink.apache.org>
>>> *Subject: *PyFlink DataStream Example Kafka/Kinesis?
>>>
>>>
>>>
>>> Hi,
>>>
>>>
>>>
>>> Is there an example kafka/kinesis source or sink for the PyFlink
>>> DataStream API?
>>>
>>>
>>>
>>> Best,
>>>
>>> kevin
>>>
>>
Re: PyFlink DataStream Example Kafka/Kinesis?
Posted by Shuiqiang Chen <ac...@gmail.com>.
Hi Kevin,
Kinesis connector is not supported yet in Python DataStream API. We will
add it in the future.
Best,
Shuiqiang
Bohinski, Kevin <Ke...@comcast.com> 于2021年3月25日周四 上午5:03写道：
> Is there a kinesis example?
>
>
>
> *From: *"Bohinski, Kevin" <Ke...@comcast.com>
> *Date: *Wednesday, March 24, 2021 at 4:40 PM
> *To: *"Bohinski, Kevin" <Ke...@comcast.com>
> *Subject: *Re: PyFlink DataStream Example Kafka/Kinesis?
>
>
>
> Nevermind, found this for anyone else looking:
> https://github.com/apache/flink/blob/master/flink-end-to-end-tests/flink-python-test/python/datastream/data_stream_job.py
>
>
>
> *From: *"Bohinski, Kevin" <Ke...@comcast.com>
> *Date: *Wednesday, March 24, 2021 at 4:38 PM
> *To: *user <us...@flink.apache.org>
> *Subject: *PyFlink DataStream Example Kafka/Kinesis?
>
>
>
> Hi,
>
>
>
> Is there an example kafka/kinesis source or sink for the PyFlink
> DataStream API?
>
>
>
> Best,
>
> kevin
>
Re: PyFlink DataStream Example Kafka/Kinesis?
Posted by "Bohinski, Kevin" <Ke...@comcast.com>.
Is there a kinesis example?
From: "Bohinski, Kevin" <Ke...@comcast.com>
Date: Wednesday, March 24, 2021 at 4:40 PM
To: "Bohinski, Kevin" <Ke...@comcast.com>
Subject: Re: PyFlink DataStream Example Kafka/Kinesis?
Nevermind, found this for anyone else looking: https://github.com/apache/flink/blob/master/flink-end-to-end-tests/flink-python-test/python/datastream/data_stream_job.py
From: "Bohinski, Kevin" <Ke...@comcast.com>
Date: Wednesday, March 24, 2021 at 4:38 PM
To: user <us...@flink.apache.org>
Subject: PyFlink DataStream Example Kafka/Kinesis?
Hi,
Is there an example kafka/kinesis source or sink for the PyFlink DataStream API?
Best,
kevin