You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/08/03 13:23:55 UTC
[GitHub] [flink] twalthr opened a new pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
twalthr opened a new pull request #13054:
URL: https://github.com/apache/flink/pull/13054
## What is the purpose of the change
This enables support for `ListView` and `MapView` in FLIP-65 aggregate functions. There are two major differences to the previous design:
1. We use the internal serialization format for checkpoints/savepoints of views. Data structure converters are used at the edges which means that we support all data types and conversion classes.
2. The data types of data views are expressed in a declarative way. We don't instantiate the accumulators anymore. Instead users can use annotations or override `getTypeInference`.
Old aggregate functions are still supported when using `TableEnvironment.registerFunction`.
## Brief change log
See commit messages.
## Verifying this change
This change is already covered by existing tests, such as: `AggregateITCase`
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: yes
- The serializers: yes
- The runtime per-record code paths (performance sensitive): yes
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? JavaDocs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dawidwys commented on a change in pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
dawidwys commented on a change in pull request #13054:
URL: https://github.com/apache/flink/pull/13054#discussion_r465018429
##########
File path: flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/extraction/DataTypeExtractor.java
##########
@@ -595,4 +605,36 @@ private DataType closestBridging(DataType dataType, @Nullable Class<?> clazz) {
}
return dataType;
}
+
+ /**
+ * Data type hints are allowed on top of {@link DataView}s. They are validated and mapped to the underlying
+ * collection by this method.
+ */
+ private DataType handleDataViewHints(DataType dataType, @Nullable Class<?> clazz) {
+ if (clazz == null || !DataView.class.isAssignableFrom(clazz)) {
+ return dataType;
+ }
+
+ // data type went through regular extraction logic
+ if (isCompositeType(dataType.getLogicalType())) {
+ return dataType;
Review comment:
out of curiosity: Is it possible to end up here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ab1e73abdc448ffe907a21726bcafedfb85196d0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128)
* abedb100f5f02dc3249f58f8fb06e6f16f7246fb Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
}, {
"hash" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193",
"triggerID" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"triggerType" : "PUSH"
}, {
"hash" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 47ba3e28fd0cb5b55bc0831e67ae42c52759b809 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193)
* 998748349fe7c016ba16e6b496b0c297467a14c9 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
}, {
"hash" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193",
"triggerID" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"triggerType" : "PUSH"
}, {
"hash" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5200",
"triggerID" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"triggerType" : "PUSH"
}, {
"hash" : "5ddbdc329473b79a0485cf84c162bf4c93ff91fe",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5208",
"triggerID" : "5ddbdc329473b79a0485cf84c162bf4c93ff91fe",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5ddbdc329473b79a0485cf84c162bf4c93ff91fe Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5208)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668021587
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit ab1e73abdc448ffe907a21726bcafedfb85196d0 (Mon Aug 03 13:25:50 UTC 2020)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
}, {
"hash" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193",
"triggerID" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"triggerType" : "PUSH"
}, {
"hash" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5200",
"triggerID" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"triggerType" : "PUSH"
}, {
"hash" : "5ddbdc329473b79a0485cf84c162bf4c93ff91fe",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5208",
"triggerID" : "5ddbdc329473b79a0485cf84c162bf4c93ff91fe",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 998748349fe7c016ba16e6b496b0c297467a14c9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5200)
* 5ddbdc329473b79a0485cf84c162bf4c93ff91fe Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5208)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* abedb100f5f02dc3249f58f8fb06e6f16f7246fb Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131)
* 61fb866575b128a4814bcd2e9b7de0b64e2e7dbd Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156)
* 72f7d9a720ae8e3bb484fc85c85870744ed375b1 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] twalthr closed pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
twalthr closed pull request #13054:
URL: https://github.com/apache/flink/pull/13054
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] twalthr commented on a change in pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
twalthr commented on a change in pull request #13054:
URL: https://github.com/apache/flink/pull/13054#discussion_r464988360
##########
File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/CodeGeneratorContext.scala
##########
@@ -671,28 +677,33 @@ class CodeGeneratorContext(val tableConfig: TableConfig) {
/**
* Adds a reusable [[DataStructureConverter]] to the member area of the generated class.
*
- * @param converter converter to be added
+ * @param dataType converter to be added
* @param classLoaderTerm term to access the [[ClassLoader]] for user-defined classes
*/
def addReusableConverter(
- converter: DataStructureConverter[_, _],
+ dataType: DataType,
classLoaderTerm: String = null)
: String = {
+ reusableConverters.get(dataType) match {
Review comment:
The mapping `DataTypeConverters` maps to a factory. But the factory considers nullability. This is why FLINK-18701 was discovered for example.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ab1e73abdc448ffe907a21726bcafedfb85196d0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* abedb100f5f02dc3249f58f8fb06e6f16f7246fb Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131)
* 61fb866575b128a4814bcd2e9b7de0b64e2e7dbd UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
}, {
"hash" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193",
"triggerID" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"triggerType" : "PUSH"
}, {
"hash" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5200",
"triggerID" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"triggerType" : "PUSH"
}, {
"hash" : "5ddbdc329473b79a0485cf84c162bf4c93ff91fe",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5208",
"triggerID" : "5ddbdc329473b79a0485cf84c162bf4c93ff91fe",
"triggerType" : "PUSH"
}, {
"hash" : "26381042613df4f94aa57feb72a50665836ad7d1",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5211",
"triggerID" : "26381042613df4f94aa57feb72a50665836ad7d1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5ddbdc329473b79a0485cf84c162bf4c93ff91fe Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5208)
* 26381042613df4f94aa57feb72a50665836ad7d1 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5211)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
}, {
"hash" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 72f7d9a720ae8e3bb484fc85c85870744ed375b1 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163)
* 47ba3e28fd0cb5b55bc0831e67ae42c52759b809 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 72f7d9a720ae8e3bb484fc85c85870744ed375b1 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dawidwys commented on a change in pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
dawidwys commented on a change in pull request #13054:
URL: https://github.com/apache/flink/pull/13054#discussion_r465038039
##########
File path: flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/dataview/DataView.java
##########
@@ -19,18 +19,20 @@
package org.apache.flink.table.api.dataview;
import org.apache.flink.annotation.PublicEvolving;
-
-import java.io.Serializable;
+import org.apache.flink.table.functions.ImperativeAggregateFunction;
/**
* A {@link DataView} is a collection type that can be used in the accumulator of an
- * {@link org.apache.flink.table.functions.AggregateFunction}.
+ * {@link ImperativeAggregateFunction}.
Review comment:
I found out the answer myself. I was not aware the `ImperativeAggregateFunction` is more generic. Both `AggregateFunction` & `TableAggregateFunction` extend from it.
Thought it is the other way around.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
}, {
"hash" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193",
"triggerID" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 47ba3e28fd0cb5b55bc0831e67ae42c52759b809 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* abedb100f5f02dc3249f58f8fb06e6f16f7246fb Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131)
* 61fb866575b128a4814bcd2e9b7de0b64e2e7dbd Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] twalthr commented on a change in pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
twalthr commented on a change in pull request #13054:
URL: https://github.com/apache/flink/pull/13054#discussion_r465825000
##########
File path: flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/dataview/ListView.java
##########
@@ -156,17 +173,49 @@ public boolean equals(Object o) {
if (this == o) {
return true;
}
- if (o == null || getClass() != o.getClass()) {
+ if (!(o instanceof ListView)) {
return false;
}
- ListView<?> listView = (ListView<?>) o;
- return Objects.equals(elementType, listView.elementType) &&
- Objects.equals(list, listView.list);
+ final ListView<?> listView = (ListView<?>) o;
+ return getList().equals(listView.getList());
}
@Override
public int hashCode() {
- return Objects.hash(elementType, list);
+ return Objects.hash(getList());
}
+ // --------------------------------------------------------------------------------------------
+ // Utilities
+ // --------------------------------------------------------------------------------------------
+
+ /**
+ * Utility method for creating a {@link DataType} of {@link ListView} explicitly.
+ */
+ public static DataType newListViewDataType(DataType elementDataType) {
+ return DataTypes.STRUCTURED(
+ ListView.class,
+ DataTypes.FIELD(
+ "list",
+ DataTypes.ARRAY(elementDataType).bridgedTo(List.class)));
+ }
+
+ // --------------------------------------------------------------------------------------------
+ // Legacy
+ // --------------------------------------------------------------------------------------------
+
+ @Deprecated
+ public transient TypeInformation<?> elementType;
Review comment:
No we need the transient. Otherwise it will be part of the ListView structured typed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
}, {
"hash" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193",
"triggerID" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"triggerType" : "PUSH"
}, {
"hash" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5200",
"triggerID" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 998748349fe7c016ba16e6b496b0c297467a14c9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5200)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ab1e73abdc448ffe907a21726bcafedfb85196d0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128)
* abedb100f5f02dc3249f58f8fb06e6f16f7246fb UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ab1e73abdc448ffe907a21726bcafedfb85196d0 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* abedb100f5f02dc3249f58f8fb06e6f16f7246fb Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131)
* 61fb866575b128a4814bcd2e9b7de0b64e2e7dbd Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156)
* 72f7d9a720ae8e3bb484fc85c85870744ed375b1 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
}, {
"hash" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193",
"triggerID" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 72f7d9a720ae8e3bb484fc85c85870744ed375b1 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163)
* 47ba3e28fd0cb5b55bc0831e67ae42c52759b809 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dawidwys commented on a change in pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
dawidwys commented on a change in pull request #13054:
URL: https://github.com/apache/flink/pull/13054#discussion_r464991883
##########
File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/agg/AggsHandlerCodeGenerator.scala
##########
@@ -1206,23 +1206,51 @@ object AggsHandlerCodeGenerator {
enableBackupDataView: Boolean): Unit = {
// add reusable dataviews to context
viewSpecs.foreach { spec =>
- val (viewTypeTerm, registerCall) = spec match {
- case ListViewSpec(_, _, _) => (className[StateListView[_, _]], "getStateListView")
- case MapViewSpec(_, _, _) => (className[StateMapView[_, _, _]], "getStateMapView")
+ val stateTerm = '"' + spec.getStateId + '"'
Review comment:
nit: that's not really a term, is it? Can we name it `stateId`?
##########
File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/agg/AggsHandlerCodeGenerator.scala
##########
@@ -1206,23 +1206,51 @@ object AggsHandlerCodeGenerator {
enableBackupDataView: Boolean): Unit = {
// add reusable dataviews to context
viewSpecs.foreach { spec =>
- val (viewTypeTerm, registerCall) = spec match {
- case ListViewSpec(_, _, _) => (className[StateListView[_, _]], "getStateListView")
- case MapViewSpec(_, _, _) => (className[StateMapView[_, _, _]], "getStateMapView")
+ val stateTerm = '"' + spec.getStateId + '"'
+ val (viewTypeTerm, stateStoreCall) = spec match {
+
+ case spec: ListViewSpec =>
+ val viewTypeTerm = className[StateListView[_, _]]
+ val elementSerializerTerm = toScala(spec.getElementSerializer) match {
+ case Some(serializer) =>
+ ctx.addReusableObject(serializer, "elementSerializer")
+ case None =>
+ throw new UnsupportedOperationException("Data type serialization not supported.")
+ }
+ val stateStoreCall =
+ s"getStateListView($stateTerm, $elementSerializerTerm)"
+ (viewTypeTerm, stateStoreCall)
+
+ case spec: MapViewSpec =>
+ val viewTypeTerm = className[StateMapView[_, _, _]]
+ val withNullKey = spec.containsNullKey()
+ val keySerializerTerm = toScala(spec.getKeySerializer) match {
+ case Some(serializer) =>
+ ctx.addReusableObject(serializer, "keySerializer")
+ case None =>
+ throw new UnsupportedOperationException("Data type serialization not supported.")
+ }
+ val valueSerializerTerm = toScala(spec.getValueSerializer) match {
+ case Some(serializer) =>
+ ctx.addReusableObject(serializer, "valueSerializer")
+ case None =>
+ throw new UnsupportedOperationException("Data type serialization not supported.")
+ }
Review comment:
nit: you can extract this to a helper method. We call it 3 times.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
}, {
"hash" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193",
"triggerID" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"triggerType" : "PUSH"
}, {
"hash" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5200",
"triggerID" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 47ba3e28fd0cb5b55bc0831e67ae42c52759b809 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193)
* 998748349fe7c016ba16e6b496b0c297467a14c9 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5200)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
}, {
"hash" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193",
"triggerID" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"triggerType" : "PUSH"
}, {
"hash" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5200",
"triggerID" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"triggerType" : "PUSH"
}, {
"hash" : "5ddbdc329473b79a0485cf84c162bf4c93ff91fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5208",
"triggerID" : "5ddbdc329473b79a0485cf84c162bf4c93ff91fe",
"triggerType" : "PUSH"
}, {
"hash" : "26381042613df4f94aa57feb72a50665836ad7d1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5211",
"triggerID" : "26381042613df4f94aa57feb72a50665836ad7d1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 26381042613df4f94aa57feb72a50665836ad7d1 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5211)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
}, {
"hash" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193",
"triggerID" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"triggerType" : "PUSH"
}, {
"hash" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5200",
"triggerID" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"triggerType" : "PUSH"
}, {
"hash" : "5ddbdc329473b79a0485cf84c162bf4c93ff91fe",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5ddbdc329473b79a0485cf84c162bf4c93ff91fe",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 998748349fe7c016ba16e6b496b0c297467a14c9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5200)
* 5ddbdc329473b79a0485cf84c162bf4c93ff91fe UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 61fb866575b128a4814bcd2e9b7de0b64e2e7dbd Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156)
* 72f7d9a720ae8e3bb484fc85c85870744ed375b1 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dawidwys commented on a change in pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
dawidwys commented on a change in pull request #13054:
URL: https://github.com/apache/flink/pull/13054#discussion_r465012717
##########
File path: flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/dataview/ListView.java
##########
@@ -156,17 +173,49 @@ public boolean equals(Object o) {
if (this == o) {
return true;
}
- if (o == null || getClass() != o.getClass()) {
+ if (!(o instanceof ListView)) {
return false;
}
- ListView<?> listView = (ListView<?>) o;
- return Objects.equals(elementType, listView.elementType) &&
- Objects.equals(list, listView.list);
+ final ListView<?> listView = (ListView<?>) o;
+ return getList().equals(listView.getList());
}
@Override
public int hashCode() {
- return Objects.hash(elementType, list);
+ return Objects.hash(getList());
}
+ // --------------------------------------------------------------------------------------------
+ // Utilities
+ // --------------------------------------------------------------------------------------------
+
+ /**
+ * Utility method for creating a {@link DataType} of {@link ListView} explicitly.
+ */
+ public static DataType newListViewDataType(DataType elementDataType) {
+ return DataTypes.STRUCTURED(
+ ListView.class,
+ DataTypes.FIELD(
+ "list",
+ DataTypes.ARRAY(elementDataType).bridgedTo(List.class)));
+ }
+
+ // --------------------------------------------------------------------------------------------
+ // Legacy
+ // --------------------------------------------------------------------------------------------
+
+ @Deprecated
+ public transient TypeInformation<?> elementType;
Review comment:
nit: `@Nullable`
nit: can we remove the `transient`? The whole `ListView is no longer serializable.
##########
File path: flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/dataview/DataView.java
##########
@@ -19,18 +19,20 @@
package org.apache.flink.table.api.dataview;
import org.apache.flink.annotation.PublicEvolving;
-
-import java.io.Serializable;
+import org.apache.flink.table.functions.ImperativeAggregateFunction;
/**
* A {@link DataView} is a collection type that can be used in the accumulator of an
- * {@link org.apache.flink.table.functions.AggregateFunction}.
+ * {@link ImperativeAggregateFunction}.
Review comment:
Is the `AggregateFunction` no longer valid here?
##########
File path: flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/dataview/MapView.java
##########
@@ -19,94 +19,104 @@
package org.apache.flink.table.api.dataview;
import org.apache.flink.annotation.PublicEvolving;
+import org.apache.flink.api.common.state.MapState;
import org.apache.flink.api.common.typeinfo.TypeInfo;
import org.apache.flink.api.common.typeinfo.TypeInformation;
+import org.apache.flink.table.annotation.DataTypeHint;
+import org.apache.flink.table.api.DataTypes;
import org.apache.flink.table.dataview.MapViewTypeInfoFactory;
+import org.apache.flink.table.functions.AggregateFunction;
+import org.apache.flink.table.functions.TableAggregateFunction;
+import org.apache.flink.table.types.DataType;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
+import java.util.Objects;
/**
- * A {@link MapView} provides Map functionality for accumulators used by user-defined aggregate
- * functions [[AggregateFunction]].
+ * A {@link DataView} that provides {@link Map}-like functionality in the accumulator of an {@link AggregateFunction}
+ * or {@link TableAggregateFunction} when large amounts of data are expected.
*
- * <p>A {@link MapView} can be backed by a Java HashMap or a state backend, depending on the
- * context in which the aggregation function is used.
+ * <p>A {@link MapView} can be backed by a Java {@link HashMap} or can leverage Flink's state backends
+ * depending on the context in which the aggregate function is used. In many unbounded data scenarios,
+ * the {@link MapView} delegates all calls to a {@link MapState} instead of the {@link HashMap}.
*
- * <p>At runtime {@link MapView} will be replaced by a state MapView which is backed by a
- * {@link org.apache.flink.api.common.state.MapState} instead of {@link HashMap} if it works
- * in streaming.
+ * <p>Note: Keys of a {@link MapView} must not be null. Nulls in values are supported. For heap-based
Review comment:
Is this correct that `Note: Keys of a {@link MapView} must not be null` Why do we need the `withNullableKeys` in `MapViewSpec` then?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dawidwys commented on a change in pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
dawidwys commented on a change in pull request #13054:
URL: https://github.com/apache/flink/pull/13054#discussion_r464999979
##########
File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/CodeGeneratorContext.scala
##########
@@ -671,28 +677,33 @@ class CodeGeneratorContext(val tableConfig: TableConfig) {
/**
* Adds a reusable [[DataStructureConverter]] to the member area of the generated class.
*
- * @param converter converter to be added
+ * @param dataType converter to be added
* @param classLoaderTerm term to access the [[ClassLoader]] for user-defined classes
*/
def addReusableConverter(
- converter: DataStructureConverter[_, _],
+ dataType: DataType,
classLoaderTerm: String = null)
: String = {
+ reusableConverters.get(dataType) match {
Review comment:
My bad. You are right.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
}, {
"hash" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5156",
"triggerID" : "61fb866575b128a4814bcd2e9b7de0b64e2e7dbd",
"triggerType" : "PUSH"
}, {
"hash" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5163",
"triggerID" : "72f7d9a720ae8e3bb484fc85c85870744ed375b1",
"triggerType" : "PUSH"
}, {
"hash" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5193",
"triggerID" : "47ba3e28fd0cb5b55bc0831e67ae42c52759b809",
"triggerType" : "PUSH"
}, {
"hash" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5200",
"triggerID" : "998748349fe7c016ba16e6b496b0c297467a14c9",
"triggerType" : "PUSH"
}, {
"hash" : "5ddbdc329473b79a0485cf84c162bf4c93ff91fe",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5208",
"triggerID" : "5ddbdc329473b79a0485cf84c162bf4c93ff91fe",
"triggerType" : "PUSH"
}, {
"hash" : "26381042613df4f94aa57feb72a50665836ad7d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "26381042613df4f94aa57feb72a50665836ad7d1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5ddbdc329473b79a0485cf84c162bf4c93ff91fe Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5208)
* 26381042613df4f94aa57feb72a50665836ad7d1 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] twalthr commented on a change in pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
twalthr commented on a change in pull request #13054:
URL: https://github.com/apache/flink/pull/13054#discussion_r465106506
##########
File path: flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/dataview/MapView.java
##########
@@ -19,94 +19,104 @@
package org.apache.flink.table.api.dataview;
import org.apache.flink.annotation.PublicEvolving;
+import org.apache.flink.api.common.state.MapState;
import org.apache.flink.api.common.typeinfo.TypeInfo;
import org.apache.flink.api.common.typeinfo.TypeInformation;
+import org.apache.flink.table.annotation.DataTypeHint;
+import org.apache.flink.table.api.DataTypes;
import org.apache.flink.table.dataview.MapViewTypeInfoFactory;
+import org.apache.flink.table.functions.AggregateFunction;
+import org.apache.flink.table.functions.TableAggregateFunction;
+import org.apache.flink.table.types.DataType;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
+import java.util.Objects;
/**
- * A {@link MapView} provides Map functionality for accumulators used by user-defined aggregate
- * functions [[AggregateFunction]].
+ * A {@link DataView} that provides {@link Map}-like functionality in the accumulator of an {@link AggregateFunction}
+ * or {@link TableAggregateFunction} when large amounts of data are expected.
*
- * <p>A {@link MapView} can be backed by a Java HashMap or a state backend, depending on the
- * context in which the aggregation function is used.
+ * <p>A {@link MapView} can be backed by a Java {@link HashMap} or can leverage Flink's state backends
+ * depending on the context in which the aggregate function is used. In many unbounded data scenarios,
+ * the {@link MapView} delegates all calls to a {@link MapState} instead of the {@link HashMap}.
*
- * <p>At runtime {@link MapView} will be replaced by a state MapView which is backed by a
- * {@link org.apache.flink.api.common.state.MapState} instead of {@link HashMap} if it works
- * in streaming.
+ * <p>Note: Keys of a {@link MapView} must not be null. Nulls in values are supported. For heap-based
Review comment:
The `DISTINCT` uses `MapView` internally (see `AggregateUtil#extractDistinctInformation`). For this, we need to support nullable keys. But this is not exposed to the user. Otherwise we would need a similar logic for the `ListView`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dawidwys commented on a change in pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
dawidwys commented on a change in pull request #13054:
URL: https://github.com/apache/flink/pull/13054#discussion_r465004837
##########
File path: flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/typeutils/ExternalSerializer.java
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.runtime.typeutils;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.api.common.typeutils.CompositeTypeSerializerSnapshot;
+import org.apache.flink.api.common.typeutils.TypeSerializer;
+import org.apache.flink.api.common.typeutils.TypeSerializerSnapshot;
+import org.apache.flink.api.java.typeutils.runtime.DataInputViewStream;
+import org.apache.flink.api.java.typeutils.runtime.DataOutputViewStream;
+import org.apache.flink.core.memory.DataInputView;
+import org.apache.flink.core.memory.DataOutputView;
+import org.apache.flink.table.data.conversion.DataStructureConverter;
+import org.apache.flink.table.data.conversion.DataStructureConverters;
+import org.apache.flink.table.types.DataType;
+import org.apache.flink.util.InstantiationUtil;
+
+import java.io.IOException;
+import java.util.Objects;
+
+/**
+ * A serializer that can serialize and deserialize all data structures defined by a {@link DataType}.
+ *
+ * <p>This class combines {@link DataStructureConverters} and {@link InternalSerializers} into one
+ * entity. The serialized binary format is always an internal binary format.
+ *
+ * <p>Serializing {@code null} in the top-level is not supported.
+ *
+ * @param <I> internal data structure
+ * @param <E> external data structure
+ */
+@Internal
+public final class ExternalSerializer<I, E> extends TypeSerializer<E> {
+
+ private final DataType dataType;
+
+ private final TypeSerializer<I> internalSerializer;
+
+ private transient DataStructureConverter<I, E> converter;
+
+ private ExternalSerializer(DataType dataType, TypeSerializer<I> internalSerializer) {
+ this.dataType = dataType;
+ this.internalSerializer = internalSerializer;
+ }
+
+ /**
+ * Creates an instance of a {@link ExternalSerializer} defined by the given {@link DataType}.
+ */
+ public static <I, E> ExternalSerializer<I, E> of(DataType dataType) {
+ return new ExternalSerializer<>(dataType, InternalSerializers.create(dataType.getLogicalType()));
+ }
+
+ @SuppressWarnings("unchecked")
+ private void checkConverterInitialized() {
+ if (converter == null) {
+ converter = (DataStructureConverter<I, E>) DataStructureConverters.getConverter(dataType);
+ converter.open(Thread.currentThread().getContextClassLoader());
+ }
+ }
+
+ @Override
+ public boolean isImmutableType() {
+ return internalSerializer.isImmutableType();
+ }
+
+ @Override
+ public TypeSerializer<E> duplicate() {
+ return new ExternalSerializer<>(dataType, internalSerializer.duplicate());
+ }
+
+ @Override
+ public E createInstance() {
+ checkConverterInitialized();
+ I instance = internalSerializer.createInstance();
Review comment:
nit: Should we move this into the try-catch section as well?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dawidwys commented on a change in pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
dawidwys commented on a change in pull request #13054:
URL: https://github.com/apache/flink/pull/13054#discussion_r465033095
##########
File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/plan/utils/AggregateUtil.scala
##########
@@ -424,16 +430,27 @@ object AggregateUtil extends Enumeration {
hasStateBackedDataViews: Boolean)
: AggregateInfo = {
- // TODO handle data views here
+ // extract data views and adapt the data views in the accumulator type
+ // if a view is backed by a state backend
+ val dataViewSpecs: Array[DataViewSpec] = if (hasStateBackedDataViews) {
+ DataViewUtils.extractDataViews(index, accumulatorDataType).toList.toArray
Review comment:
nit: I think a preferred way is to do: `
```suggestion
DataViewUtils.extractDataViews(index, accumulatorDataType).asScala.toArray
```
The `toList` does a conversion, whereas `asScala` creates only a wrapper.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] twalthr commented on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
twalthr commented on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-669035732
Thanks @dawidwys. I will merge this once the build is green.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13054:
URL: https://github.com/apache/flink/pull/13054#issuecomment-668030431
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5128",
"triggerID" : "ab1e73abdc448ffe907a21726bcafedfb85196d0",
"triggerType" : "PUSH"
}, {
"hash" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131",
"triggerID" : "abedb100f5f02dc3249f58f8fb06e6f16f7246fb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* abedb100f5f02dc3249f58f8fb06e6f16f7246fb Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5131)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] twalthr commented on a change in pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
twalthr commented on a change in pull request #13054:
URL: https://github.com/apache/flink/pull/13054#discussion_r465115776
##########
File path: flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/extraction/DataTypeExtractor.java
##########
@@ -595,4 +605,36 @@ private DataType closestBridging(DataType dataType, @Nullable Class<?> clazz) {
}
return dataType;
}
+
+ /**
+ * Data type hints are allowed on top of {@link DataView}s. They are validated and mapped to the underlying
+ * collection by this method.
+ */
+ private DataType handleDataViewHints(DataType dataType, @Nullable Class<?> clazz) {
+ if (clazz == null || !DataView.class.isAssignableFrom(clazz)) {
+ return dataType;
+ }
+
+ // data type went through regular extraction logic
+ if (isCompositeType(dataType.getLogicalType())) {
+ return dataType;
Review comment:
A regular
```
class Acc {
public MapView<String, Long> view;
}
```
should end up there. But I will verify again.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dawidwys commented on a change in pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
dawidwys commented on a change in pull request #13054:
URL: https://github.com/apache/flink/pull/13054#discussion_r464985112
##########
File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/CodeGeneratorContext.scala
##########
@@ -671,28 +677,33 @@ class CodeGeneratorContext(val tableConfig: TableConfig) {
/**
* Adds a reusable [[DataStructureConverter]] to the member area of the generated class.
*
- * @param converter converter to be added
+ * @param dataType converter to be added
* @param classLoaderTerm term to access the [[ClassLoader]] for user-defined classes
*/
def addReusableConverter(
- converter: DataStructureConverter[_, _],
+ dataType: DataType,
classLoaderTerm: String = null)
: String = {
+ reusableConverters.get(dataType) match {
Review comment:
nit: should we ignore the nullability? If I am not mistaken we use the same converters for both nullable and not null types. In `DataTypeConverters` we use (`LogicalTypeRoot`, `conversionClass`) as a composite key.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] twalthr commented on a change in pull request #13054: [FLINK-15803][table] Support DataViews in FLIP-65 aggregate functions
Posted by GitBox <gi...@apache.org>.
twalthr commented on a change in pull request #13054:
URL: https://github.com/apache/flink/pull/13054#discussion_r465530691
##########
File path: flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/extraction/DataTypeExtractor.java
##########
@@ -595,4 +605,36 @@ private DataType closestBridging(DataType dataType, @Nullable Class<?> clazz) {
}
return dataType;
}
+
+ /**
+ * Data type hints are allowed on top of {@link DataView}s. They are validated and mapped to the underlying
+ * collection by this method.
+ */
+ private DataType handleDataViewHints(DataType dataType, @Nullable Class<?> clazz) {
+ if (clazz == null || !DataView.class.isAssignableFrom(clazz)) {
+ return dataType;
+ }
+
+ // data type went through regular extraction logic
+ if (isCompositeType(dataType.getLogicalType())) {
+ return dataType;
Review comment:
This code path is tested by:
```
TestSpec
.forType(
"Data view with default extraction",
AccumulatorWithDefaultDataView.class)
.lookupExpects(Object.class)
.expectDataType(
DataTypes.STRUCTURED(
AccumulatorWithDefaultDataView.class,
DataTypes.FIELD(
"listView",
ListView.newListViewDataType(DataTypes.RAW(new GenericTypeInfo<>(Object.class))))
)
)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org