You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Ananth Gundabattula <ag...@darwinium.com> on 2022/02/17 08:56:32 UTC
Pyflink with pulsar
Hello All,
I am trying to build a pyflink application and I currently have a pulsar instance that I need to connect and start streaming messages from.
I was wondering if there is any advice regarding pulsar as a source connector available via python ?
Alternately, Pulsar seems to have a kafka protocol handler (KOP) and was wondering if anyone has built a pyflink application streaming from pulsar using the Kafka protocol ( By using pyflink kafka consumer ) ? If yes, could you please share your experiences.
Regards,
Ananth
Re: Pyflink with pulsar
Posted by Dian Fu <di...@gmail.com>.
Hi Ananth,
It's already code freeze for 1.15 and you can refer to [1] for more details.
Regards,
Dian
[1] https://www.mail-archive.com/dev@flink.apache.org/msg54262.html
On Sun, Feb 20, 2022 at 1:51 AM Ananth Gundabattula <
agundabattula@darwinium.com> wrote:
> Thanks a lot Wong.
>
>
>
> I was wondering what the timelines are for the 1.15 release (to plan a
> things around our CI pipeline). The below information really helps and
> thanks for the contribution.
>
>
>
> Regards,
>
> Ananth
>
>
>
> *From: *Luning Wong <gf...@gmail.com>
> *Date: *Friday, 18 February 2022 at 6:12 pm
> *To: *user@flink.apache.org <us...@flink.apache.org>, Ananth Gundabattula <
> agundabattula@darwinium.com>
> *Subject: *Re: Pyflink with pulsar
>
> The Pulsar python source connector will be released in 1.15 version.
> if you want to use it right now, you could compile the master branch.
> When I completed the python connector code, I only tested the native
> pulsar protocol without KOP.
>
> Usage examples are in comments of the PulsarSource class and in test
> case of the FlinkPulsarTest class.
> You can find them in the following PR link.
> https://github.com/apache/flink/pull/18388
>
> Best regards,
>
> Wong
>
> Xingbo Huang <hx...@gmail.com> 于2022年2月18日周五 14:32写道：
> >
> >
> >
> > ---------- Forwarded message ---------
> > 发件人： Ananth Gundabattula <ag...@darwinium.com>
> > Date: 2022年2月17日周四 16:57
> > Subject: Pyflink with pulsar
> > To: user@flink.apache.org <us...@flink.apache.org>
> >
> >
> > Hello All,
> >
> > I am trying to build a pyflink application and I currently have a pulsar
> instance that I need to connect and start streaming messages from.
> >
> > I was wondering if there is any advice regarding pulsar as a source
> connector available via python ?
> >
> > Alternately, Pulsar seems to have a kafka protocol handler (KOP) and was
> wondering if anyone has built a pyflink application streaming from pulsar
> using the Kafka protocol ( By using pyflink kafka consumer ) ? If yes,
> could you please share your experiences.
> >
> >
> > Regards,
> > Ananth
> >
> >
>
Re: Pyflink with pulsar
Posted by Ananth Gundabattula <ag...@darwinium.com>.
Thanks a lot Wong.
I was wondering what the timelines are for the 1.15 release (to plan a things around our CI pipeline). The below information really helps and thanks for the contribution.
Regards,
Ananth
From: Luning Wong <gf...@gmail.com>
Date: Friday, 18 February 2022 at 6:12 pm
To: user@flink.apache.org <us...@flink.apache.org>, Ananth Gundabattula <ag...@darwinium.com>
Subject: Re: Pyflink with pulsar
The Pulsar python source connector will be released in 1.15 version.
if you want to use it right now, you could compile the master branch.
When I completed the python connector code, I only tested the native
pulsar protocol without KOP.
Usage examples are in comments of the PulsarSource class and in test
case of the FlinkPulsarTest class.
You can find them in the following PR link.
https://github.com/apache/flink/pull/18388
Best regards,
Wong
Xingbo Huang <hx...@gmail.com> 于2022年2月18日周五 14:32写道：
>
>
>
> ---------- Forwarded message ---------
> 发件人： Ananth Gundabattula <ag...@darwinium.com>
> Date: 2022年2月17日周四 16:57
> Subject: Pyflink with pulsar
> To: user@flink.apache.org <us...@flink.apache.org>
>
>
> Hello All,
>
> I am trying to build a pyflink application and I currently have a pulsar instance that I need to connect and start streaming messages from.
>
> I was wondering if there is any advice regarding pulsar as a source connector available via python ?
>
> Alternately, Pulsar seems to have a kafka protocol handler (KOP) and was wondering if anyone has built a pyflink application streaming from pulsar using the Kafka protocol ( By using pyflink kafka consumer ) ? If yes, could you please share your experiences.
>
>
> Regards,
> Ananth
>
>
Re: Pyflink with pulsar
Posted by Luning Wong <gf...@gmail.com>.
The Pulsar python source connector will be released in 1.15 version.
if you want to use it right now, you could compile the master branch.
When I completed the python connector code, I only tested the native
pulsar protocol without KOP.
Usage examples are in comments of the PulsarSource class and in test
case of the FlinkPulsarTest class.
You can find them in the following PR link.
https://github.com/apache/flink/pull/18388
Best regards,
Wong
Xingbo Huang <hx...@gmail.com> 于2022年2月18日周五 14:32写道：
>
>
>
> ---------- Forwarded message ---------
> 发件人： Ananth Gundabattula <ag...@darwinium.com>
> Date: 2022年2月17日周四 16:57
> Subject: Pyflink with pulsar
> To: user@flink.apache.org <us...@flink.apache.org>
>
>
> Hello All,
>
> I am trying to build a pyflink application and I currently have a pulsar instance that I need to connect and start streaming messages from.
>
> I was wondering if there is any advice regarding pulsar as a source connector available via python ?
>
> Alternately, Pulsar seems to have a kafka protocol handler (KOP) and was wondering if anyone has built a pyflink application streaming from pulsar using the Kafka protocol ( By using pyflink kafka consumer ) ? If yes, could you please share your experiences.
>
>
> Regards,
> Ananth
>
>