You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by GitBox <gi...@apache.org> on 2022/07/29 09:10:59 UTC
[GitHub] [flink-connector-redis] eskabetxe opened a new pull request, #3: [FLINK-15571] Add Redis sink
eskabetxe opened a new pull request, #3:
URL: https://github.com/apache/flink-connector-redis/pull/3
Add basic RedisSink based on AsyncSinkWriter
As Jedis dont have a async call, it was implemented.
Maybe better choice was to use lettuce or Redisson libs..
Need to be added a lot of commands, its basic to allow the discussion of implementation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-connector-redis-streams] eskabetxe commented on pull request #3: [FLINK-15571] Add Redis sink
Posted by GitBox <gi...@apache.org>.
eskabetxe commented on PR #3:
URL: https://github.com/apache/flink-connector-redis-streams/pull/3#issuecomment-1330710004
Hi @MartijnVisser , @sazzad16
I update the code to only have Redis Streams with the format of flink-connectors
if you could check this, then we move to other things like source, table etc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-connector-redis] sazzad16 commented on pull request #3: [FLINK-15571] Add Redis sink
Posted by GitBox <gi...@apache.org>.
sazzad16 commented on PR #3:
URL: https://github.com/apache/flink-connector-redis/pull/3#issuecomment-1200710378
@eskabetxe Let's discuss.
Would Slack be a good place for this? Or must we use mailing list?
@MartijnVisser
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-connector-redis] MartijnVisser commented on pull request #3: [FLINK-15571] Add Redis sink
Posted by GitBox <gi...@apache.org>.
MartijnVisser commented on PR #3:
URL: https://github.com/apache/flink-connector-redis/pull/3#issuecomment-1200983872
> Would Slack be a good place for this? Or must we use mailing list?
I think you can DM each other in the Flink Slack workspace to collaborate on this topic. If you end up that you can't agree on the correct implementation strategy, then it should be brought up for a discussion on the Dev mailing list (or reach out to the User mailing list if you're looking for feedback).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-connector-redis] MartijnVisser commented on pull request #3: [FLINK-15571] Add Redis sink
Posted by GitBox <gi...@apache.org>.
MartijnVisser commented on PR #3:
URL: https://github.com/apache/flink-connector-redis/pull/3#issuecomment-1199173047
@eskabetxe Thanks a lot for the PR! It might make more sense for @sazzad16 and/or @chayim to review, as they are more familiar with Redis.
If you rebase this PR, you should also get the CI build information so we can validate that it works as expected.
Last but not least: before we merge it, we should get the FLIP accepted. There's currently a draft FLIP open (see https://cwiki.apache.org/confluence/display/FLINK/FLIP-254%3A+Redis+Streams+Connector) and I'm waiting for feedback from @sazzad16 before we can bring it up for discussion and vote in the Dev mailing list. If you have any feedback on the FLIP, feel free to let me know too!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-connector-redis] sazzad16 commented on pull request #3: [FLINK-15571] Add Redis sink
Posted by GitBox <gi...@apache.org>.
sazzad16 commented on PR #3:
URL: https://github.com/apache/flink-connector-redis/pull/3#issuecomment-1200398367
Hi @eskabetxe, nice to meet you.
I see that the user would have to set (and so know and understand) exact Redis command which I feel not very intuitive from user's perspective.
Did you get to look at #2? It is based on Flink paradigm from some time ago but supports all of Sink, Source, TableSink, TableSource. It starts with [Redis Streams](https://redis.io/docs/manual/data-types/streams/) data structure which is a great fit for Flink. Not to mention other data structures can also be supported later.
Let me know what do you think about #2. Perhaps we can collaborate to update it to current Flink paradigm?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-connector-redis-streams] MartijnVisser commented on pull request #3: [FLINK-15571] Add Redis sink
Posted by GitBox <gi...@apache.org>.
MartijnVisser commented on PR #3:
URL: https://github.com/apache/flink-connector-redis-streams/pull/3#issuecomment-1327197048
@eskabetxe @sazzad16 Is there any update on this process? Also check out https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development for some more insights on how to develop an externalized connector
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-connector-redis] sazzad16 commented on pull request #3: [FLINK-15571] Add Redis sink
Posted by GitBox <gi...@apache.org>.
sazzad16 commented on PR #3:
URL: https://github.com/apache/flink-connector-redis/pull/3#issuecomment-1200398428
@MartijnVisser The FLIP is great!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-connector-redis] eskabetxe commented on pull request #3: [FLINK-15571] Add Redis sink
Posted by GitBox <gi...@apache.org>.
eskabetxe commented on PR #3:
URL: https://github.com/apache/flink-connector-redis/pull/3#issuecomment-1199180404
Hi @MartijnVisser rebase done..
I will check the FLIP
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-connector-redis] eskabetxe commented on pull request #3: [FLINK-15571] Add Redis sink
Posted by GitBox <gi...@apache.org>.
eskabetxe commented on PR #3:
URL: https://github.com/apache/flink-connector-redis/pull/3#issuecomment-1200411096
Hi @sazzad16,
Im not an expert on Redis Streams, but I think bot ways must be implemented..
For my knowledge, Redis Streams is used like (sorry for the analogy) if you are using Kafka no?
If you already have a Redis cluster using other commands like string, hash, list you cant migrate to streams without migration?
This is my use case, using Redis as a cache, I doubt I could change to stream for example.
I think for example in Sink, the use of stream or other commands could be implemented with config, what will change is SinkWriter implementation..
I would be happy to talk to you about it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-connector-redis-streams] eskabetxe commented on pull request #3: [FLINK-15571] Add Redis sink
Posted by GitBox <gi...@apache.org>.
eskabetxe commented on PR #3:
URL: https://github.com/apache/flink-connector-redis-streams/pull/3#issuecomment-1327224834
Hi @MartijnVisser, it's pretty much at a standstill.
As it is only for streams it does not fit in our use case and it is difficult to dedicate time to it.
Anyway, let's see if we can get some time so we can move forward with it since it's practically all programmed.
> @eskabetxe @sazzad16 Is there any update on this process? Also check out https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development for some more insights on how to develop an externalized connector
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org