You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Martijn Visser <ma...@apache.org> on 2022/08/01 13:40:37 UTC
[DISCUSS] FLIP-254 Redis Streams connector
Hi everyone,
Together with Mohammad we would like to start a discussion thread on
FLIP-254 for a Redis Streams connector. This would also be a new connector
in its own external connector repository, being
https://www.github.com/apache/flink-connector-redis. This repository
already exists, because there was an open PR to add this to the Flink main
repository and they were planned to be one of the first movers to the
external connector repo. However, we never formally agreed that we want
this, therefore we're now creating this FLIP.
Looking forward to any comments or feedback. If not, I'll open a VOTE
thread in the next couple of days.
Best regards,
Martijn
Re: [DISCUSS] FLIP-254 Redis Streams connector
Posted by Sazzadul Hoque <sa...@gmail.com>.
Hi Konstantin and Martijn,
Sincere apologies for the late reply.
Being the newest data structure, Streams is still catching up with others.
But it obviously has its own unique usages which makes it the most suitable
Redis data structures for Flink especially as a *Source*.
We are always considering all possible data structures for sink. For
example, Joao Boto's PR
<https://github.com/apache/flink-connector-redis/pull/3> and my PR
<https://github.com/apache/flink-connector-redis/pull/5> both have classes
named StreamWriteRequest, StringWriteRequest, representing respective data
structures for sink purpose.
Best regards,
Sazzadul
On Fri, Sep 9, 2022 at 3:35 PM Martijn Visser <ma...@apache.org>
wrote:
> Hi Konstantin,
>
> I don't know, perhaps Mohammad knows.
>
> Best regards,
>
> Martijn
>
> Op wo 31 aug. 2022 om 14:31 schreef Konstantin Knauf <kn...@apache.org>:
>
>> Hi Martijn, Hi Mohammad,
>>
>> thanks for creating this FLIP.
>>
>> I am wondering, do you know how much Redis Streams are used compared to
>> other Redis data structures? At least as a sink and LookupTableSource
>> supporting regular Redis data structures seems quite useful.
>>
>> I suppose such connectors would live in the same repository. Would be good
>> to consider when naming the Redis Streams connector.
>>
>> Cheers,
>>
>> Konstantin
>>
>>
>>
>> Am Mi., 31. Aug. 2022 um 13:55 Uhr schrieb Martijn Visser <
>> martijnvisser@apache.org>:
>>
>> > Hi everyone,
>> >
>> > Last call for comments, else I'll open up a VOTE thread later this week.
>> >
>> > Best regards,
>> >
>> > Martijn
>> >
>> > Op ma 1 aug. 2022 om 15:40 schreef Martijn Visser <
>> > martijnvisser@apache.org
>> > >:
>> >
>> > > Hi everyone,
>> > >
>> > > Together with Mohammad we would like to start a discussion thread on
>> > > FLIP-254 for a Redis Streams connector. This would also be a new
>> > connector
>> > > in its own external connector repository, being
>> > > https://www.github.com/apache/flink-connector-redis. This repository
>> > > already exists, because there was an open PR to add this to the Flink
>> > main
>> > > repository and they were planned to be one of the first movers to the
>> > > external connector repo. However, we never formally agreed that we
>> want
>> > > this, therefore we're now creating this FLIP.
>> > >
>> > > Looking forward to any comments or feedback. If not, I'll open a VOTE
>> > > thread in the next couple of days.
>> > >
>> > > Best regards,
>> > >
>> > > Martijn
>> > >
>> >
>>
>>
>> --
>> https://twitter.com/snntrable
>> https://github.com/knaufk
>>
>
Re: [DISCUSS] FLIP-254 Redis Streams connector
Posted by Martijn Visser <ma...@apache.org>.
Hi Konstantin,
I don't know, perhaps Mohammad knows.
Best regards,
Martijn
Op wo 31 aug. 2022 om 14:31 schreef Konstantin Knauf <kn...@apache.org>:
> Hi Martijn, Hi Mohammad,
>
> thanks for creating this FLIP.
>
> I am wondering, do you know how much Redis Streams are used compared to
> other Redis data structures? At least as a sink and LookupTableSource
> supporting regular Redis data structures seems quite useful.
>
> I suppose such connectors would live in the same repository. Would be good
> to consider when naming the Redis Streams connector.
>
> Cheers,
>
> Konstantin
>
>
>
> Am Mi., 31. Aug. 2022 um 13:55 Uhr schrieb Martijn Visser <
> martijnvisser@apache.org>:
>
> > Hi everyone,
> >
> > Last call for comments, else I'll open up a VOTE thread later this week.
> >
> > Best regards,
> >
> > Martijn
> >
> > Op ma 1 aug. 2022 om 15:40 schreef Martijn Visser <
> > martijnvisser@apache.org
> > >:
> >
> > > Hi everyone,
> > >
> > > Together with Mohammad we would like to start a discussion thread on
> > > FLIP-254 for a Redis Streams connector. This would also be a new
> > connector
> > > in its own external connector repository, being
> > > https://www.github.com/apache/flink-connector-redis. This repository
> > > already exists, because there was an open PR to add this to the Flink
> > main
> > > repository and they were planned to be one of the first movers to the
> > > external connector repo. However, we never formally agreed that we want
> > > this, therefore we're now creating this FLIP.
> > >
> > > Looking forward to any comments or feedback. If not, I'll open a VOTE
> > > thread in the next couple of days.
> > >
> > > Best regards,
> > >
> > > Martijn
> > >
> >
>
>
> --
> https://twitter.com/snntrable
> https://github.com/knaufk
>
Re: [DISCUSS] FLIP-254 Redis Streams connector
Posted by Konstantin Knauf <kn...@apache.org>.
Hi Martijn, Hi Mohammad,
thanks for creating this FLIP.
I am wondering, do you know how much Redis Streams are used compared to
other Redis data structures? At least as a sink and LookupTableSource
supporting regular Redis data structures seems quite useful.
I suppose such connectors would live in the same repository. Would be good
to consider when naming the Redis Streams connector.
Cheers,
Konstantin
Am Mi., 31. Aug. 2022 um 13:55 Uhr schrieb Martijn Visser <
martijnvisser@apache.org>:
> Hi everyone,
>
> Last call for comments, else I'll open up a VOTE thread later this week.
>
> Best regards,
>
> Martijn
>
> Op ma 1 aug. 2022 om 15:40 schreef Martijn Visser <
> martijnvisser@apache.org
> >:
>
> > Hi everyone,
> >
> > Together with Mohammad we would like to start a discussion thread on
> > FLIP-254 for a Redis Streams connector. This would also be a new
> connector
> > in its own external connector repository, being
> > https://www.github.com/apache/flink-connector-redis. This repository
> > already exists, because there was an open PR to add this to the Flink
> main
> > repository and they were planned to be one of the first movers to the
> > external connector repo. However, we never formally agreed that we want
> > this, therefore we're now creating this FLIP.
> >
> > Looking forward to any comments or feedback. If not, I'll open a VOTE
> > thread in the next couple of days.
> >
> > Best regards,
> >
> > Martijn
> >
>
--
https://twitter.com/snntrable
https://github.com/knaufk
Re: [DISCUSS] FLIP-254 Redis Streams connector
Posted by Martijn Visser <ma...@apache.org>.
Hi everyone,
Last call for comments, else I'll open up a VOTE thread later this week.
Best regards,
Martijn
Op ma 1 aug. 2022 om 15:40 schreef Martijn Visser <martijnvisser@apache.org
>:
> Hi everyone,
>
> Together with Mohammad we would like to start a discussion thread on
> FLIP-254 for a Redis Streams connector. This would also be a new connector
> in its own external connector repository, being
> https://www.github.com/apache/flink-connector-redis. This repository
> already exists, because there was an open PR to add this to the Flink main
> repository and they were planned to be one of the first movers to the
> external connector repo. However, we never formally agreed that we want
> this, therefore we're now creating this FLIP.
>
> Looking forward to any comments or feedback. If not, I'll open a VOTE
> thread in the next couple of days.
>
> Best regards,
>
> Martijn
>