You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/10/27 04:29:59 UTC
[GitHub] [flink] twalthr opened a new pull request #13802: [FLINK-19793][connector-kafka] Harden KafkaTableITCase.testKafkaSourceSinkWithMetadata
twalthr opened a new pull request #13802:
URL: https://github.com/apache/flink/pull/13802
## What is the purpose of the change
Hardens the KafkaTableITCase by further improving the new test utilities and ignoring non-deterministic metadata columns.
## Brief change log
- Offer an unordered list of row deep equals with matcher
- Remove offset from test
## Verifying this change
This change added tests and can be verified as follows: `RowTest`
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: yes
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dawidwys commented on a change in pull request #13802: [FLINK-19793][connector-kafka] Harden KafkaTableITCase.testKafkaSourceSinkWithMetadata
Posted by GitBox <gi...@apache.org>.
dawidwys commented on a change in pull request #13802:
URL: https://github.com/apache/flink/pull/13802#discussion_r512578037
##########
File path: flink-core/src/main/java/org/apache/flink/types/Row.java
##########
@@ -294,15 +295,24 @@ public static boolean deepEquals(Row row1, Row row2) {
/**
* Compares two {@link List}s of {@link Row} for deep equality. This method supports all conversion
- * classes of the table ecosystem.
+ * classes of the table ecosystem. The top-level lists can be compared with or without order.
*
* <p>The current implementation of {@link Row#equals(Object)} is not able to compare all deeply
* nested row structures that might be created in the table ecosystem. For example, it does not
* support comparing arrays stored in the values of a map. We might update the {@link #equals(Object)}
* with this implementation in future versions.
*/
- public static boolean deepEquals(List<Row> l1, List<Row> l2) {
- return deepEqualsInternal(l1, l2);
+ public static boolean deepEquals(List<Row> l1, List<Row> l2, boolean ignoreOrder) {
Review comment:
nit: Shouldn't this be a method in some test utility? It's rather strictly tailored for a particular test. It does not cover all the possible cases (e.g. nested ordered/unordered comparison).
Overall I think it does not harm having the method, but I'd at least have a method with `ignoreOrder = false` by default. Personally though I'd prefer to have it only in the `TableTestMatchers`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] twalthr closed pull request #13802: [FLINK-19793][connector-kafka] Harden KafkaTableITCase.testKafkaSourceSinkWithMetadata
Posted by GitBox <gi...@apache.org>.
twalthr closed pull request #13802:
URL: https://github.com/apache/flink/pull/13802
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #13802: [FLINK-19793][connector-kafka] Harden KafkaTableITCase.testKafkaSourceSinkWithMetadata
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #13802:
URL: https://github.com/apache/flink/pull/13802#issuecomment-716973857
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 25256d07a44b5e60b2f699a73220d6ea4a5d44f1 (Tue Oct 27 04:32:20 UTC 2020)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] twalthr commented on a change in pull request #13802: [FLINK-19793][connector-kafka] Harden KafkaTableITCase.testKafkaSourceSinkWithMetadata
Posted by GitBox <gi...@apache.org>.
twalthr commented on a change in pull request #13802:
URL: https://github.com/apache/flink/pull/13802#discussion_r512633303
##########
File path: flink-core/src/main/java/org/apache/flink/types/Row.java
##########
@@ -294,15 +295,24 @@ public static boolean deepEquals(Row row1, Row row2) {
/**
* Compares two {@link List}s of {@link Row} for deep equality. This method supports all conversion
- * classes of the table ecosystem.
+ * classes of the table ecosystem. The top-level lists can be compared with or without order.
*
* <p>The current implementation of {@link Row#equals(Object)} is not able to compare all deeply
* nested row structures that might be created in the table ecosystem. For example, it does not
* support comparing arrays stored in the values of a map. We might update the {@link #equals(Object)}
* with this implementation in future versions.
*/
- public static boolean deepEquals(List<Row> l1, List<Row> l2) {
- return deepEqualsInternal(l1, l2);
+ public static boolean deepEquals(List<Row> l1, List<Row> l2, boolean ignoreOrder) {
Review comment:
I was also thinking about this, but I wanted to give users an official API that allows for simple testing for comparing rows. I will see what I can do.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13802: [FLINK-19793][connector-kafka] Harden KafkaTableITCase.testKafkaSourceSinkWithMetadata
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13802:
URL: https://github.com/apache/flink/pull/13802#issuecomment-717141560
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "eb49829719243d230d94f60ded2e9f04bb09bdf0",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8376",
"triggerID" : "eb49829719243d230d94f60ded2e9f04bb09bdf0",
"triggerType" : "PUSH"
}, {
"hash" : "c46d5768e663a5dd5f08fa54951a2ad55b6ad53e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c46d5768e663a5dd5f08fa54951a2ad55b6ad53e",
"triggerType" : "PUSH"
}, {
"hash" : "2b7cdee797a91857a1bfe54bf42ea2480ef0dae7",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8413",
"triggerID" : "2b7cdee797a91857a1bfe54bf42ea2480ef0dae7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* eb49829719243d230d94f60ded2e9f04bb09bdf0 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8376)
* c46d5768e663a5dd5f08fa54951a2ad55b6ad53e UNKNOWN
* 2b7cdee797a91857a1bfe54bf42ea2480ef0dae7 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8413)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13802: [FLINK-19793][connector-kafka] Harden KafkaTableITCase.testKafkaSourceSinkWithMetadata
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13802:
URL: https://github.com/apache/flink/pull/13802#issuecomment-717141560
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "eb49829719243d230d94f60ded2e9f04bb09bdf0",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8376",
"triggerID" : "eb49829719243d230d94f60ded2e9f04bb09bdf0",
"triggerType" : "PUSH"
}, {
"hash" : "c46d5768e663a5dd5f08fa54951a2ad55b6ad53e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c46d5768e663a5dd5f08fa54951a2ad55b6ad53e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* eb49829719243d230d94f60ded2e9f04bb09bdf0 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8376)
* c46d5768e663a5dd5f08fa54951a2ad55b6ad53e UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13802: [FLINK-19793][connector-kafka] Harden KafkaTableITCase.testKafkaSourceSinkWithMetadata
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13802:
URL: https://github.com/apache/flink/pull/13802#issuecomment-717141560
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "eb49829719243d230d94f60ded2e9f04bb09bdf0",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8376",
"triggerID" : "eb49829719243d230d94f60ded2e9f04bb09bdf0",
"triggerType" : "PUSH"
}, {
"hash" : "c46d5768e663a5dd5f08fa54951a2ad55b6ad53e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c46d5768e663a5dd5f08fa54951a2ad55b6ad53e",
"triggerType" : "PUSH"
}, {
"hash" : "2b7cdee797a91857a1bfe54bf42ea2480ef0dae7",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2b7cdee797a91857a1bfe54bf42ea2480ef0dae7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* eb49829719243d230d94f60ded2e9f04bb09bdf0 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8376)
* c46d5768e663a5dd5f08fa54951a2ad55b6ad53e UNKNOWN
* 2b7cdee797a91857a1bfe54bf42ea2480ef0dae7 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13802: [FLINK-19793][connector-kafka] Harden KafkaTableITCase.testKafkaSourceSinkWithMetadata
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13802:
URL: https://github.com/apache/flink/pull/13802#issuecomment-717141560
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "eb49829719243d230d94f60ded2e9f04bb09bdf0",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8376",
"triggerID" : "eb49829719243d230d94f60ded2e9f04bb09bdf0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* eb49829719243d230d94f60ded2e9f04bb09bdf0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8376)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #13802: [FLINK-19793][connector-kafka] Harden KafkaTableITCase.testKafkaSourceSinkWithMetadata
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #13802:
URL: https://github.com/apache/flink/pull/13802#issuecomment-717141560
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "eb49829719243d230d94f60ded2e9f04bb09bdf0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "eb49829719243d230d94f60ded2e9f04bb09bdf0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* eb49829719243d230d94f60ded2e9f04bb09bdf0 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org