You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/10/14 14:08:41 UTC
[GitHub] [flink] echauchot opened a new pull request, #21073: [FLINK-26822] Add Source for Cassandra connector
echauchot opened a new pull request, #21073:
URL: https://github.com/apache/flink/pull/21073
## What is the purpose of the change
Add a flink source for Cassandra database compliant with [FLIP-27](https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface)
## Brief change log
Implement everything needed for the source: enumerator, cassandra splits, source reader, split reader etc... + the related unit tests and integration tests.
Moved the annotated Pojo class used in Cassandra batch example to a common place (tests / casssandra utils) to be able to use it with both example and source tests
## Verifying this change
This change added tests and can be verified as follows:
- CassandraSourceITCase: integration test using the new source test framework and a Cassandra test container
- CassandraEnumeratorStateSerializerTest and CassandraSplitSerializerTest: for serialization tests
- CassandraQueryTest: tests for query generation and query sanity checks
- SplitsGeneratorTest: test for ring ranges creation (use for splits)
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): add connector base and connector test utils
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: adds a new `@Public(Evolving)` API
- The serializers: add some new ones
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? javadoc
R: @zentol
CC: @MartijnVisser
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] echauchot commented on pull request #21073: [FLINK-26822] Add Source for Cassandra connector
Posted by GitBox <gi...@apache.org>.
echauchot commented on PR #21073:
URL: https://github.com/apache/flink/pull/21073#issuecomment-1327500964
Closing, because this PR was moved to the new cassandra dedicated repository: https://github.com/apache/flink-connector-cassandra/pull/3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] MartijnVisser commented on pull request #21073: [FLINK-26822] Add Source for Cassandra connector
Posted by GitBox <gi...@apache.org>.
MartijnVisser commented on PR #21073:
URL: https://github.com/apache/flink/pull/21073#issuecomment-1285251674
> you mean help reviewing ?
Yes indeed
> I don't know Flink specifics on reviews but can a non-committer give LGTM and then a committer simply merge ?
It could, but the biggest thing that would probably happen is that the committer can verify if the review looks properly so the committers review can be quicker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] echauchot commented on pull request #21073: [FLINK-26822] Add Source for Cassandra connector
Posted by GitBox <gi...@apache.org>.
echauchot commented on PR #21073:
URL: https://github.com/apache/flink/pull/21073#issuecomment-1282412326
@zentol, I've fixed the architecture tests. Now everything passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] echauchot commented on pull request #21073: [FLINK-26822] Add Source for Cassandra connector
Posted by GitBox <gi...@apache.org>.
echauchot commented on PR #21073:
URL: https://github.com/apache/flink/pull/21073#issuecomment-1284059553
CC @mzuehlke as you offered help. I'm keeping you updated so that we share maintenance of this connector
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] echauchot commented on pull request #21073: [FLINK-26822] Add Source for Cassandra connector
Posted by GitBox <gi...@apache.org>.
echauchot commented on PR #21073:
URL: https://github.com/apache/flink/pull/21073#issuecomment-1286623220
> > I don't know Flink specifics on reviews but can a non-committer give LGTM and then a committer simply merge ?
>
> > It could [...]
>
> Correction: No. The committer _must_ check the PR. But as Martijn said it can speed things up.
Yes this is what I do on Beam project: even though the project accepts that a non-committer reviews, I never merge a PR if I have not reviewed all of it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] echauchot commented on pull request #21073: [FLINK-26822] Add Source for Cassandra connector
Posted by GitBox <gi...@apache.org>.
echauchot commented on PR #21073:
URL: https://github.com/apache/flink/pull/21073#issuecomment-1300590629
friendly ping: anyone to review this PR now that Flink 1.16 is released ? As discussed in the ML we could review here and migrate to a dedicated connector repo if needed before merging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] echauchot commented on pull request #21073: [FLINK-26822] Add Source for Cassandra connector
Posted by GitBox <gi...@apache.org>.
echauchot commented on PR #21073:
URL: https://github.com/apache/flink/pull/21073#issuecomment-1285129346
> It would be great if @mzuehlke could indeed help out on this due to limited bandwidth at this moment
you mean help reviewing ? I don't know Flink specifics on reviews but can a non-committer give LGTM and then a committer simply merge ?
Anyway I think, as discussed in the ticket that sharing the maintenance of this component is important to avoid the bus factor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] MartijnVisser commented on pull request #21073: [FLINK-26822] Add Source for Cassandra connector
Posted by GitBox <gi...@apache.org>.
MartijnVisser commented on PR #21073:
URL: https://github.com/apache/flink/pull/21073#issuecomment-1284077976
It would be great if @mzuehlke could indeed help out on this due to limited bandwidth at this moment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #21073: [FLINK-26822] Add Source for Cassandra connector
Posted by GitBox <gi...@apache.org>.
flinkbot commented on PR #21073:
URL: https://github.com/apache/flink/pull/21073#issuecomment-1279069831
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "39b9a1b91a23296082e30e64ffeba6a838805952",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "39b9a1b91a23296082e30e64ffeba6a838805952",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 39b9a1b91a23296082e30e64ffeba6a838805952 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] zentol commented on pull request #21073: [FLINK-26822] Add Source for Cassandra connector
Posted by GitBox <gi...@apache.org>.
zentol commented on PR #21073:
URL: https://github.com/apache/flink/pull/21073#issuecomment-1285294978
> I don't know Flink specifics on reviews but can a non-committer give LGTM and then a committer simply merge ?
Correction: No. The committer _must_ check the PR. But as Martijn said it can speed things up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] echauchot closed pull request #21073: [FLINK-26822] Add Source for Cassandra connector
Posted by GitBox <gi...@apache.org>.
echauchot closed pull request #21073: [FLINK-26822] Add Source for Cassandra connector
URL: https://github.com/apache/flink/pull/21073
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org