You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "twalthr (via GitHub)" <gi...@apache.org> on 2025/06/05 13:24:50 UTC
[PR] [FLINK-37862][table] Support inline structured types in SQL [flink]
twalthr opened a new pull request, #26638:
URL: https://github.com/apache/flink/pull/26638
<!--
*Thank you very much for contributing to Apache Flink - we are happy that you want to help us improve Flink. To help the community review your contribution in the best possible way, please go through the checklist below, which will get the contribution into a shape in which it can be best reviewed.*
*Please understand that we do not do this to make contributions to Flink a hassle. In order to uphold a high standard of quality for code contributions, while at the same time managing a large number of contributions, we need contributors to prepare the contributions well, and give reviewers enough contextual information for the review. Please also understand that contributions that do not follow this guide will take longer to review and thus typically be picked up with lower priority by the community.*
## Contribution Checklist
- Make sure that the pull request corresponds to a [JIRA issue](https://issues.apache.org/jira/projects/FLINK/issues). Exceptions are made for typos in JavaDoc or documentation files, which need no JIRA issue.
- Name the pull request in the form "[FLINK-XXXX] [component] Title of the pull request", where *FLINK-XXXX* should be replaced by the actual issue number. Skip *component* if you are unsure about which is the best component.
Typo fixes that have no associated JIRA issue should be named following this pattern: `[hotfix] [docs] Fix typo in event time introduction` or `[hotfix] [javadocs] Expand JavaDoc for PuncuatedWatermarkGenerator`.
- Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
- Make sure that the change passes the automated tests, i.e., `mvn clean verify` passes. You can set up Azure Pipelines CI to do that following [this guide](https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-Tutorial:SettingupAzurePipelinesforaforkoftheFlinkrepository).
- Each pull request should address only one issue, not mix up code from multiple issues.
- Each commit in the pull request has a meaningful commit message (including the JIRA id)
- Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
**(The sections below can be removed for hotfixes of typos)**
-->
## What is the purpose of the change
*(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them on each deployment (during recovery).)*
## Brief change log
*(for example:)*
- *The TaskInfo is stored in the blob store on job creation time as a persistent artifact*
- *Deployments RPC transmits only the blob storage reference*
- *TaskManagers retrieve the TaskInfo from the blob cache*
## Verifying this change
Please make sure both new and modified tests in this PR follow [the conventions for tests defined in our code quality guide](https://flink.apache.org/how-to-contribute/code-style-and-quality-common/#7-testing).
*(Please pick either of the following options)*
This change is a trivial rework / code cleanup without any test coverage.
*(or)*
This change is already covered by existing tests, such as *(please describe tests)*.
*(or)*
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end deployment with large payloads (100MB)*
- *Extended integration test for recovery after master (JobManager) failure*
- *Added test that validates that TaskInfo is transferred only once across recoveries*
- *Manually verified the change by running a 4 node cluster with 2 JobManagers and 4 TaskManagers, a stateful streaming program, and killing one JobManager and two TaskManagers during the execution, verifying that recovery happens correctly.*
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / no)
- The serializers: (yes / no / don't know)
- The runtime per-record code paths (performance sensitive): (yes / no / don't know)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
- The S3 file system connector: (yes / no / don't know)
## Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37862][table] Support inline structured types in SQL [flink]
Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin commented on code in PR #26638:
URL: https://github.com/apache/flink/pull/26638#discussion_r2132125616
##########
flink-table/flink-table-common/src/test/java/org/apache/flink/table/types/LogicalTypesTest.java:
##########
@@ -797,42 +831,46 @@ private StructuredType createHumanType(boolean useDifferentImplementation) {
useDifferentImplementation ? SpecialHuman.class : Human.class)
.attributes(
Collections.singletonList(
- new StructuredType.StructuredAttribute(
- "name", UDT_NAME_TYPE, "Description.")))
+ new StructuredAttribute("name", UDT_NAME_TYPE, "Description.")))
.description("Human type desc.")
.setFinal(false)
.setInstantiable(false)
.build();
}
- private StructuredType createUserType(boolean isRegistered, boolean isFinal) {
+ private StructuredType createUserType(
Review Comment:
should it be a builder with more selfexplainable methods?
Asking since now `createUserType(false, false, false)` without looking into implementation it is hard to what all those `false` mean
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37862][table] Support inline structured types in SQL [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #26638:
URL: https://github.com/apache/flink/pull/26638#discussion_r2132110036
##########
docs/content.zh/docs/dev/table/types.md:
##########
@@ -1301,35 +1349,27 @@ class User {
// enrich the extraction with forcing using RAW types
public @DataTypeHint("RAW") Class<?> modelClass;
}
-
-DataTypes.of(User.class);
```
-**Bridging to JVM Types**
-
-| Java Type | Input | Output | Remarks |
-|:-------------------------------------|:-----:|:------:|:----------------------------------------|
-|*class* | X | X | Originating class or subclasses (for input) or <br>superclasses (for output). *Default* |
-|`org.apache.flink.types.Row` | X | X | Represent the structured type as a row. |
-|`org.apache.flink.table.data.RowData` | X | X | Internal data structure. |
-
-{{< /tab >}}
-{{< tab "Scala" >}}
-```scala
-case class User(
-
- // extract fields automatically
- age: Int,
- name: String,
+Or via explicit declaration:
+```java
+DataTypes.STRUCTURED(Class, DataTypes.FIELD(n0, t0), DataTypes.FIELD(n1, t1), ...);
+DataTypes.STRUCTURED(String, DataTypes.FIELD(n0, t0), DataTypes.FIELD(n1, t1), ...);
Review Comment:
Ah now I get it! I just got confused with the `String` class in Java, and not an actual "String"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37862][table] Support inline structured types in SQL [flink]
Posted by "twalthr (via GitHub)" <gi...@apache.org>.
twalthr commented on code in PR #26638:
URL: https://github.com/apache/flink/pull/26638#discussion_r2132142649
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/StructuredFunctionsITCase.java:
##########
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.functions;
+
+import org.apache.flink.table.api.DataTypes;
+import org.apache.flink.table.functions.BuiltInFunctionDefinitions;
+import org.apache.flink.table.functions.ScalarFunction;
+import org.apache.flink.table.types.logical.StructuredType;
+
+import java.util.stream.Stream;
+
+/** Tests for functions dealing with {@link StructuredType}. */
+public class StructuredFunctionsITCase extends BuiltInFunctionTestBase {
+
+ @Override
+ Stream<TestSetSpec> getTestSetSpecs() {
+ return Stream.of(
+ TestSetSpec.forFunction(BuiltInFunctionDefinitions.EQUALS)
+ .onFieldsWithData(14, "Bob")
+ .andDataTypes(DataTypes.INT(), DataTypes.STRING())
+ .withFunction(Type1.Type1Constructor.class)
+ .withFunction(Type2.Type2Constructor.class)
+ // Same value from function
+ .testSqlResult(
+ "Type1Constructor(f0, f1) = Type1Constructor(14, 'Bob')",
+ true,
+ DataTypes.BOOLEAN())
+ // Same value from CAST
+ .testSqlResult(
+ "Type1Constructor(f0, f1) = CAST((14, 'Bob') AS "
+ + Type1.TYPE
+ + ")",
+ true,
+ DataTypes.BOOLEAN())
+ // Different value from function
+ .testSqlResult(
+ "Type1Constructor(f0, f1) = Type1Constructor(15, 'Alice')",
+ false,
+ DataTypes.BOOLEAN())
+ // Different value from CAST
+ .testSqlResult(
+ "Type1Constructor(f0, f1) = CAST((15, 'Alice') AS "
+ + Type1.TYPE
+ + ")",
+ false,
+ DataTypes.BOOLEAN())
+ // Different class name from function
+ .testSqlValidationError(
+ "Type1Constructor(f0, f1) = Type2Constructor(14, 'Bob')",
+ "Incompatible structured types")
+ // Different class name from CAST
+ .testSqlValidationError(
+ "Type1Constructor(f0, f1) = CAST((14, 'Bob') AS "
+ + Type2.TYPE
+ + ")",
+ "Incompatible structured types")
+ // Same class name but different fields
+ .testSqlValidationError(
+ "Type1Constructor(f0, f1) = CAST((14, 'Bob') AS STRUCTURED<'"
+ + Type1.class.getName()
+ + "', a BOOLEAN, b BOOLEAN>)",
+ "Cannot apply '=' to arguments"));
+ }
+
+ // --------------------------------------------------------------------------------------------
+ // Helpers
+ // --------------------------------------------------------------------------------------------
+
+ /** Structured type Type1. */
+ public static class Type1 {
+ public static final String TYPE =
+ "STRUCTURED<'" + Type1.class.getName() + "', a INT, b STRING>";
+
+ public Integer a;
Review Comment:
yes because there is a mismatch in extracted and explicit type. The function will return the extracted one. But in general it is up to the user to ensure that data type matches class. Usually, users should not declare the structured types manually but use reflective extraction by just using the POJO in a function signature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37862][table] Support inline structured types in SQL [flink]
Posted by "twalthr (via GitHub)" <gi...@apache.org>.
twalthr commented on code in PR #26638:
URL: https://github.com/apache/flink/pull/26638#discussion_r2132100150
##########
docs/content.zh/docs/dev/table/types.md:
##########
@@ -1281,15 +1293,51 @@ For some classes an annotation is required in order to map the class to a data t
to assign a fixed precision and scale for `java.math.BigDecimal`).
{{< /tab >}}
{{< tab "Python" >}}
+```python
+Not supported.
+```
{{< /tab >}}
{{< /tabs >}}
**Declaration**
{{< tabs "c5e5527b-b09d-4dc5-9549-8fd2bfc7cc2a" >}}
-{{< tab "Java" >}}
+{{< tab "Java/Scala" >}}
+Structured types are usually declared via their implementation classes:
+
```java
-class User {
+// A simple POJO that qualifies as a structured type.
Review Comment:
Description cannot be declaratively extracted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37862][table] Support inline structured types in SQL [flink]
Posted by "twalthr (via GitHub)" <gi...@apache.org>.
twalthr merged PR #26638:
URL: https://github.com/apache/flink/pull/26638
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37862][table] Support inline structured types in SQL [flink]
Posted by "twalthr (via GitHub)" <gi...@apache.org>.
twalthr commented on code in PR #26638:
URL: https://github.com/apache/flink/pull/26638#discussion_r2132126772
##########
flink-table/flink-sql-parser/src/main/java/org/apache/flink/table/calcite/ExtendedRelTypeFactory.java:
##########
@@ -33,4 +35,10 @@ public interface ExtendedRelTypeFactory extends RelDataTypeFactory {
/** Creates a RAW type such as {@code RAW('org.my.Class', 'sW3Djsds...')}. */
RelDataType createRawType(String className, String serializerString);
+
+ /**
+ * Creates a STRUCTURED type such as {@code STRUCTURED('org.my.Class', name STRING, age INT)}.
+ */
+ RelDataType createStructuredType(
+ String className, List<RelDataType> typeList, List<String> fieldNameList);
Review Comment:
Very thorough review! Yes, the comments don't end up in the planner. We do the same for ROW. This is historical legacy that bites us until today. These comments make comparisons and other operations unnecessary complicated. For now, we leave them out. However, a catalog can use them already for `DESCRIBE t`. From a type definition point of view the type is defined. Everything else we need to solve in a future PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37862][table] Support inline structured types in SQL [flink]
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #26638:
URL: https://github.com/apache/flink/pull/26638#issuecomment-2944347793
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "69434abb93f4968815acd6be039573508bc92085",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "69434abb93f4968815acd6be039573508bc92085",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 69434abb93f4968815acd6be039573508bc92085 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37862][table] Support inline structured types in SQL [flink]
Posted by "twalthr (via GitHub)" <gi...@apache.org>.
twalthr commented on code in PR #26638:
URL: https://github.com/apache/flink/pull/26638#discussion_r2132094866
##########
docs/content.zh/docs/dev/table/types.md:
##########
@@ -1281,15 +1293,51 @@ For some classes an annotation is required in order to map the class to a data t
to assign a fixed precision and scale for `java.math.BigDecimal`).
{{< /tab >}}
{{< tab "Python" >}}
+```python
+Not supported.
+```
{{< /tab >}}
{{< /tabs >}}
**Declaration**
{{< tabs "c5e5527b-b09d-4dc5-9549-8fd2bfc7cc2a" >}}
-{{< tab "Java" >}}
+{{< tab "Java/Scala" >}}
+Structured types are usually declared via their implementation classes:
+
```java
-class User {
+// A simple POJO that qualifies as a structured type.
+// Note: Without a fully assigning constructor, the order of fields will be alphabetical.
+// The final data type will be:
+// STRUCTURED<'com.myorg.Customer', active BOOLEAN, id INT NOT NULL, name STRING, properties MAP<STRING, STRING>>
+class Customer {
+ public int id;
+ public String name;
+ public Map<String, String> properties;
+ public boolean active;
+}
+
+// A POJO with a fully assigning constructor defining the field order.
+// The final data type will be:
+// STRUCTURED<'com.myorg.Customer', id INT NOT NULL, name STRING, properties MAP<STRING, STRING>, active BOOLEAN>
Review Comment:
exactly, primitive ints are mapped to NOT NULL during reflective extraction
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37862][table] Support inline structured types in SQL [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #26638:
URL: https://github.com/apache/flink/pull/26638#discussion_r2131553523
##########
docs/content.zh/docs/dev/table/types.md:
##########
@@ -1281,15 +1293,51 @@ For some classes an annotation is required in order to map the class to a data t
to assign a fixed precision and scale for `java.math.BigDecimal`).
{{< /tab >}}
{{< tab "Python" >}}
+```python
+Not supported.
+```
{{< /tab >}}
{{< /tabs >}}
**Declaration**
{{< tabs "c5e5527b-b09d-4dc5-9549-8fd2bfc7cc2a" >}}
-{{< tab "Java" >}}
+{{< tab "Java/Scala" >}}
+Structured types are usually declared via their implementation classes:
+
```java
-class User {
+// A simple POJO that qualifies as a structured type.
+// Note: Without a fully assigning constructor, the order of fields will be alphabetical.
+// The final data type will be:
+// STRUCTURED<'com.myorg.Customer', active BOOLEAN, id INT NOT NULL, name STRING, properties MAP<STRING, STRING>>
+class Customer {
+ public int id;
+ public String name;
+ public Map<String, String> properties;
+ public boolean active;
+}
+
+// A POJO with a fully assigning constructor defining the field order.
+// The final data type will be:
+// STRUCTURED<'com.myorg.Customer', id INT NOT NULL, name STRING, properties MAP<STRING, STRING>, active BOOLEAN>
Review Comment:
Question: if the field `id` was defined as `public Integer id` would the structured type be
```
STRUCTURED<'com.myorg.Customer', id INT, name STRING, properties MAP<STRING, STRING>, active BOOLEAN>
```
##########
docs/content.zh/docs/dev/table/types.md:
##########
@@ -1232,34 +1232,46 @@ equivalent to `ROW<myField INT, myOtherField BOOLEAN>`.
### User-Defined Data Types
-{{< tabs "udf" >}}
-{{< tab "Java/Scala" >}}
-<span class="label label-danger">Attention</span> User-defined data types are not fully supported yet. They are
-currently (as of Flink 1.11) only exposed as unregistered structured types in parameters and return types of functions.
+#### `STRUCTURED`
-A structured type is similar to an object in an object-oriented programming language. It contains
-zero, one or more attributes. Each attribute consists of a name and a type.
+Data type for a user-defined object.
-There are two kinds of structured types:
+Compared to `ROW`, which may also be considered a "struct-like" type, structured types are distinguishable even if they
+contain the same set of fields. For example, `Visit(amount DOUBLE)` is distinct from `Interaction(amount DOUBLE)` due
+its identifier.
-- Types that are stored in a catalog and are identified by a _catalog identifier_ (like `cat.db.MyType`). Those
-are equal to the SQL standard definition of structured types.
+Similar to classes in object-oriented programming languages, structured types are identified by a class name and contain
+zero, one or more attributes. Each attribute consists of a name and a type. A type cannot be defined in such a way that
+one of its attribute types (transitively) refers to itself.
-- Anonymously defined, unregistered types (usually reflectively extracted) that are identified by
-an _implementation class_ (like `com.myorg.model.MyType`). Those are useful when programmatically
-defining a table program. They enable reusing existing JVM classes without manually defining the
-schema of a data type again.
+Structured types are internally converted by the system into suitable data structures. Serialization and equality checks
+are managed by the system based on the logical type.
-#### Registered Structured Types
+{{< tabs "udt" >}}
+{{< tab "SQL" >}}
+```sql
+STRUCTURED<'c', n0 t0, n1 t1, ...>
+STRUCTURED<'c', n0 t0, n1 t1 'd1', ...>
+```
+The type can be declared using `STRUCTURED<'c', n0 t0 'd0', n1 t1 'd1', ...>` where `c` is the class name, `n` is the
+unique name of a field, `t` is the logical type of a field, `d` is the description of a field.
Review Comment:
```suggestion
unique name of a field, `t` is the logical type of a field, `d` is the optional description of a field.
```
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/StructuredFunctionsITCase.java:
##########
Review Comment:
Do we want to have IT tests for nested STRUCTURED type?
##########
docs/content.zh/docs/dev/table/types.md:
##########
@@ -1232,34 +1232,46 @@ equivalent to `ROW<myField INT, myOtherField BOOLEAN>`.
### User-Defined Data Types
-{{< tabs "udf" >}}
-{{< tab "Java/Scala" >}}
-<span class="label label-danger">Attention</span> User-defined data types are not fully supported yet. They are
-currently (as of Flink 1.11) only exposed as unregistered structured types in parameters and return types of functions.
+#### `STRUCTURED`
-A structured type is similar to an object in an object-oriented programming language. It contains
-zero, one or more attributes. Each attribute consists of a name and a type.
+Data type for a user-defined object.
-There are two kinds of structured types:
+Compared to `ROW`, which may also be considered a "struct-like" type, structured types are distinguishable even if they
+contain the same set of fields. For example, `Visit(amount DOUBLE)` is distinct from `Interaction(amount DOUBLE)` due
+its identifier.
-- Types that are stored in a catalog and are identified by a _catalog identifier_ (like `cat.db.MyType`). Those
-are equal to the SQL standard definition of structured types.
+Similar to classes in object-oriented programming languages, structured types are identified by a class name and contain
+zero, one or more attributes. Each attribute consists of a name and a type. A type cannot be defined in such a way that
Review Comment:
```suggestion
zero, one or more attributes. Each attribute has a name, a type, and an optional description. A type cannot be defined in such a way that
```
##########
docs/content.zh/docs/dev/table/types.md:
##########
@@ -1281,15 +1293,51 @@ For some classes an annotation is required in order to map the class to a data t
to assign a fixed precision and scale for `java.math.BigDecimal`).
{{< /tab >}}
{{< tab "Python" >}}
+```python
+Not supported.
+```
{{< /tab >}}
{{< /tabs >}}
**Declaration**
{{< tabs "c5e5527b-b09d-4dc5-9549-8fd2bfc7cc2a" >}}
-{{< tab "Java" >}}
+{{< tab "Java/Scala" >}}
+Structured types are usually declared via their implementation classes:
+
```java
-class User {
+// A simple POJO that qualifies as a structured type.
Review Comment:
I don't know how important that would be, but having a tiny example with the description doesn't hurt. WDYT?
##########
flink-table/flink-sql-parser/src/main/codegen/includes/parserImpls.ftl:
##########
@@ -2489,6 +2489,34 @@ SqlTypeNameSpec ExtendedSqlRowTypeName() :
}
}
+/**
+ * Parse inline structured type such as STRUCTURED<'className', name1 type1 'comment', name2 type2>.
+ */
+SqlTypeNameSpec SqlStructuredTypeName() :
+{
+ final SqlNode className;
+ final List<SqlIdentifier> fieldNames = new ArrayList<SqlIdentifier>();
+ final List<SqlDataTypeSpec> fieldTypes = new ArrayList<SqlDataTypeSpec>();
+ final List<SqlCharStringLiteral> comments = new ArrayList<SqlCharStringLiteral>();
Review Comment:
Maybe staying consistent here would be good. In the docs (also in some parts of the code), this field is named `description`:
```suggestion
final List<SqlCharStringLiteral> descriptions = new ArrayList<SqlCharStringLiteral>();
```
This should apply to all the `comment` names in this file
##########
flink-table/flink-sql-parser/src/main/java/org/apache/flink/table/calcite/ExtendedRelTypeFactory.java:
##########
@@ -33,4 +35,10 @@ public interface ExtendedRelTypeFactory extends RelDataTypeFactory {
/** Creates a RAW type such as {@code RAW('org.my.Class', 'sW3Djsds...')}. */
RelDataType createRawType(String className, String serializerString);
+
+ /**
+ * Creates a STRUCTURED type such as {@code STRUCTURED('org.my.Class', name STRING, age INT)}.
+ */
+ RelDataType createStructuredType(
+ String className, List<RelDataType> typeList, List<String> fieldNameList);
Review Comment:
Question: Why is the comment/description field not present here? I think this because we don't really need them, and they are just for clarity, written by the user?
```suggestion
/**
* Creates a STRUCTURED type such as {@code STRUCTURED('org.my.Class', name STRING 'full name of the person', age INT)}.
*/
RelDataType createStructuredType(
String className, List<RelDataType> typeList, List<String> fieldNameList, List<String> fieldCommentList);
```
##########
docs/content.zh/docs/dev/table/types.md:
##########
@@ -1301,35 +1349,27 @@ class User {
// enrich the extraction with forcing using RAW types
public @DataTypeHint("RAW") Class<?> modelClass;
}
-
-DataTypes.of(User.class);
```
-**Bridging to JVM Types**
-
-| Java Type | Input | Output | Remarks |
-|:-------------------------------------|:-----:|:------:|:----------------------------------------|
-|*class* | X | X | Originating class or subclasses (for input) or <br>superclasses (for output). *Default* |
-|`org.apache.flink.types.Row` | X | X | Represent the structured type as a row. |
-|`org.apache.flink.table.data.RowData` | X | X | Internal data structure. |
-
-{{< /tab >}}
-{{< tab "Scala" >}}
-```scala
-case class User(
-
- // extract fields automatically
- age: Int,
- name: String,
+Or via explicit declaration:
+```java
+DataTypes.STRUCTURED(Class, DataTypes.FIELD(n0, t0), DataTypes.FIELD(n1, t1), ...);
+DataTypes.STRUCTURED(String, DataTypes.FIELD(n0, t0), DataTypes.FIELD(n1, t1), ...);
Review Comment:
Shouldn't this be?
```suggestion
DataTypes.STRUCTURED(String.class, DataTypes.FIELD(n0, t0), DataTypes.FIELD(n1, t1), ...);
```
And what is the difference between this line and the one before it?
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/StructuredFunctionsITCase.java:
##########
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.functions;
+
+import org.apache.flink.table.api.DataTypes;
+import org.apache.flink.table.functions.BuiltInFunctionDefinitions;
+import org.apache.flink.table.functions.ScalarFunction;
+import org.apache.flink.table.types.logical.StructuredType;
+
+import java.util.stream.Stream;
+
+/** Tests for functions dealing with {@link StructuredType}. */
+public class StructuredFunctionsITCase extends BuiltInFunctionTestBase {
+
+ @Override
+ Stream<TestSetSpec> getTestSetSpecs() {
+ return Stream.of(
+ TestSetSpec.forFunction(BuiltInFunctionDefinitions.EQUALS)
+ .onFieldsWithData(14, "Bob")
+ .andDataTypes(DataTypes.INT(), DataTypes.STRING())
+ .withFunction(Type1.Type1Constructor.class)
+ .withFunction(Type2.Type2Constructor.class)
+ // Same value from function
+ .testSqlResult(
+ "Type1Constructor(f0, f1) = Type1Constructor(14, 'Bob')",
+ true,
+ DataTypes.BOOLEAN())
+ // Same value from CAST
+ .testSqlResult(
+ "Type1Constructor(f0, f1) = CAST((14, 'Bob') AS "
+ + Type1.TYPE
+ + ")",
+ true,
+ DataTypes.BOOLEAN())
+ // Different value from function
+ .testSqlResult(
+ "Type1Constructor(f0, f1) = Type1Constructor(15, 'Alice')",
+ false,
+ DataTypes.BOOLEAN())
+ // Different value from CAST
+ .testSqlResult(
+ "Type1Constructor(f0, f1) = CAST((15, 'Alice') AS "
+ + Type1.TYPE
+ + ")",
+ false,
+ DataTypes.BOOLEAN())
+ // Different class name from function
+ .testSqlValidationError(
+ "Type1Constructor(f0, f1) = Type2Constructor(14, 'Bob')",
+ "Incompatible structured types")
+ // Different class name from CAST
+ .testSqlValidationError(
+ "Type1Constructor(f0, f1) = CAST((14, 'Bob') AS "
+ + Type2.TYPE
+ + ")",
+ "Incompatible structured types")
+ // Same class name but different fields
+ .testSqlValidationError(
+ "Type1Constructor(f0, f1) = CAST((14, 'Bob') AS STRUCTURED<'"
+ + Type1.class.getName()
+ + "', a BOOLEAN, b BOOLEAN>)",
+ "Cannot apply '=' to arguments"));
+ }
+
+ // --------------------------------------------------------------------------------------------
+ // Helpers
+ // --------------------------------------------------------------------------------------------
+
+ /** Structured type Type1. */
+ public static class Type1 {
+ public static final String TYPE =
Review Comment:
nit: can be private
```suggestion
private static final String TYPE =
```
Same with Type2
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/StructuredFunctionsITCase.java:
##########
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.functions;
+
+import org.apache.flink.table.api.DataTypes;
+import org.apache.flink.table.functions.BuiltInFunctionDefinitions;
+import org.apache.flink.table.functions.ScalarFunction;
+import org.apache.flink.table.types.logical.StructuredType;
+
+import java.util.stream.Stream;
+
+/** Tests for functions dealing with {@link StructuredType}. */
+public class StructuredFunctionsITCase extends BuiltInFunctionTestBase {
+
+ @Override
+ Stream<TestSetSpec> getTestSetSpecs() {
+ return Stream.of(
+ TestSetSpec.forFunction(BuiltInFunctionDefinitions.EQUALS)
+ .onFieldsWithData(14, "Bob")
+ .andDataTypes(DataTypes.INT(), DataTypes.STRING())
+ .withFunction(Type1.Type1Constructor.class)
+ .withFunction(Type2.Type2Constructor.class)
+ // Same value from function
+ .testSqlResult(
+ "Type1Constructor(f0, f1) = Type1Constructor(14, 'Bob')",
+ true,
+ DataTypes.BOOLEAN())
+ // Same value from CAST
+ .testSqlResult(
+ "Type1Constructor(f0, f1) = CAST((14, 'Bob') AS "
+ + Type1.TYPE
+ + ")",
+ true,
+ DataTypes.BOOLEAN())
+ // Different value from function
+ .testSqlResult(
+ "Type1Constructor(f0, f1) = Type1Constructor(15, 'Alice')",
+ false,
+ DataTypes.BOOLEAN())
+ // Different value from CAST
+ .testSqlResult(
+ "Type1Constructor(f0, f1) = CAST((15, 'Alice') AS "
+ + Type1.TYPE
+ + ")",
+ false,
+ DataTypes.BOOLEAN())
+ // Different class name from function
+ .testSqlValidationError(
+ "Type1Constructor(f0, f1) = Type2Constructor(14, 'Bob')",
+ "Incompatible structured types")
+ // Different class name from CAST
+ .testSqlValidationError(
+ "Type1Constructor(f0, f1) = CAST((14, 'Bob') AS "
+ + Type2.TYPE
+ + ")",
+ "Incompatible structured types")
+ // Same class name but different fields
+ .testSqlValidationError(
+ "Type1Constructor(f0, f1) = CAST((14, 'Bob') AS STRUCTURED<'"
+ + Type1.class.getName()
+ + "', a BOOLEAN, b BOOLEAN>)",
+ "Cannot apply '=' to arguments"));
+ }
+
+ // --------------------------------------------------------------------------------------------
+ // Helpers
+ // --------------------------------------------------------------------------------------------
+
+ /** Structured type Type1. */
+ public static class Type1 {
+ public static final String TYPE =
+ "STRUCTURED<'" + Type1.class.getName() + "', a INT, b STRING>";
+
+ public Integer a;
Review Comment:
Question: Changing the type to `int` on both classes will cause the tests to fail. Why is that?
```suggestion
public int a;
```
##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/DataTypes.java:
##########
@@ -928,37 +928,88 @@ public static <T> UnresolvedDataType RAW(TypeInformation<T> typeInformation) {
}
/**
- * Data type of a user-defined object structured type. Structured types contain zero, one or
- * more attributes. Each attribute consists of a name and a type. A type cannot be defined so
- * that one of its attribute types (transitively) uses itself.
- *
- * <p>There are two kinds of structured types. Types that are stored in a catalog and are
- * identified by an {@link ObjectIdentifier} or anonymously defined, unregistered types (usually
- * reflectively extracted) that are identified by an implementation {@link Class}.
- *
- * <p>This method helps in manually constructing anonymous, unregistered types. This is useful
- * in cases where the reflective extraction using {@link DataTypes#of(Class)} is not applicable.
- * However, {@link DataTypes#of(Class)} is the recommended way of creating inline structured
- * types as it also considers {@link DataTypeHint}s.
- *
- * <p>Structured types are converted to internal data structures by the runtime. The given
- * implementation class is only used at the edges of the table ecosystem (e.g. when bridging to
- * a function or connector). Serialization and equality ({@code hashCode/equals}) are handled by
- * the runtime based on the logical type. An implementation class must offer a default
- * constructor with zero arguments or a full constructor that assigns all attributes.
- *
- * <p>Note: A caller of this method must make sure that the {@link
- * DataType#getConversionClass()} of the given fields matches with the attributes of the given
- * implementation class, otherwise an exception might be thrown during runtime.
+ * Data type of a user-defined object structured type. Structured types are identified by a
+ * class name and contain zero, one or more attributes. Each attribute consists of a name and a
+ * type. A type cannot be defined in such a way that one of its attribute types (transitively)
+ * refers to itself.
+ *
+ * <p>Compared to {@link #ROW(Field...)}, which may also be considered a "struct-like" type,
+ * structured types are distinguishable even if they contain the same set of fields. For
+ * example, "Visit(amount DOUBLE)" is distinct from "Interaction(amount DOUBLE)" due its
+ * identifier.
+ *
+ * <p>This method allows for manually constructing an inline structured type. This is useful in
+ * cases where the reflective extraction using {@link DataTypes#of(Class)} is not applicable.
+ * However, {@link DataTypes#of(Class)} is the recommended approach for creating inline
+ * structured types, as it also considers {@link DataTypeHint}s.
+ *
+ * <p>Structured types are internally converted by the system into suitable data structures.
+ * Serialization and equality checks (e.g. {@code hashCode/equals}) are managed by the system
+ * based on the logical type.
+ *
+ * <p>If an optional implementation class is provided, the system will convert a structured
+ * object to a JVM object at the edges of the table ecosystem (e.g. when bridging to a function
+ * or connector). The implementation class must provide either a zero-argument constructor or a
+ * full constructor that assigns all attributes. The class name does not need to be resolvable
+ * in the classpath; it may be used solely to distinguish between objects with identical
+ * attribute sets. However, in Table API and UDF calls, the system will attempt to resolve the
+ * class name to an actual implementation class. If resolution fails, {@link Row} is used as a
+ * fallback.
+ *
+ * <p>Note: The caller of this method must ensure that the {@link DataType#getConversionClass()}
+ * of each field matches the corresponding attribute in the implementation class. Otherwise, a
+ * runtime exception may be thrown.
*
* @see DataTypes#of(Class)
* @see StructuredType
*/
public static <T> DataType STRUCTURED(Class<T> implementationClass, Field... fields) {
// some basic validation of the class to prevent common mistakes
validateStructuredClass(implementationClass);
+ return buildStructuredType(StructuredType.newBuilder(implementationClass), fields);
+ }
- final StructuredType.Builder builder = StructuredType.newBuilder(implementationClass);
+ /**
+ * Data type of a user-defined object structured type. Structured types are identified by a
+ * class name and contain zero, one or more attributes. Each attribute consists of a name and a
+ * type. A type cannot be defined in such a way that one of its attribute types (transitively)
+ * refers to itself.
+ *
+ * <p>Compared to {@link #ROW(Field...)}, which may also be considered a "struct-like" type,
+ * structured types are distinguishable even if they contain the same set of fields. For
+ * example, "Visit(amount DOUBLE)" is distinct from "Interaction(amount DOUBLE)" due its
+ * identifier.
+ *
+ * <p>This method allows for manually constructing an inline structured type. This is useful in
+ * cases where the reflective extraction using {@link DataTypes#of(Class)} is not applicable.
+ * However, {@link DataTypes#of(Class)} is the recommended approach for creating inline
+ * structured types, as it also considers {@link DataTypeHint}s.
+ *
+ * <p>Structured types are internally converted by the system into suitable data structures.
+ * Serialization and equality checks (e.g. {@code hashCode/equals}) are managed by the system
+ * based on the logical type.
+ *
+ * <p>If an optional implementation class is provided, the system will convert a structured
+ * object to a JVM object at the edges of the table ecosystem (e.g. when bridging to a function
+ * or connector). The implementation class must provide either a zero-argument constructor or a
+ * full constructor that assigns all attributes. The class name does not need to be resolvable
+ * in the classpath; it may be used solely to distinguish between objects with identical
+ * attribute sets. However, in Table API and UDF calls, the system will attempt to resolve the
+ * class name to an actual implementation class. If resolution fails, {@link Row} is used as a
+ * fallback.
+ *
+ * <p>Note: The caller of this method must ensure that the {@link DataType#getConversionClass()}
+ * of each field matches the corresponding attribute in the implementation class. Otherwise, a
+ * runtime exception may be thrown.
+ *
+ * @see DataTypes#of(Class)
+ * @see StructuredType
+ */
+ public static <T> DataType STRUCTURED(String className, Field... fields) {
+ return buildStructuredType(StructuredType.newBuilder(className), fields);
Review Comment:
Do you think it would be a good idea to fail fast here with a simple validation like `className.isBlank() || className.isEmpty()`?
Or is it a valid statement to have
```sql
STRUCTURED<``, f0 t0>
STRUCTURED<` `, f0 t0>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37862][table] Support inline structured types in SQL [flink]
Posted by "twalthr (via GitHub)" <gi...@apache.org>.
twalthr commented on code in PR #26638:
URL: https://github.com/apache/flink/pull/26638#discussion_r2132107949
##########
docs/content.zh/docs/dev/table/types.md:
##########
@@ -1301,35 +1349,27 @@ class User {
// enrich the extraction with forcing using RAW types
public @DataTypeHint("RAW") Class<?> modelClass;
}
-
-DataTypes.of(User.class);
```
-**Bridging to JVM Types**
-
-| Java Type | Input | Output | Remarks |
-|:-------------------------------------|:-----:|:------:|:----------------------------------------|
-|*class* | X | X | Originating class or subclasses (for input) or <br>superclasses (for output). *Default* |
-|`org.apache.flink.types.Row` | X | X | Represent the structured type as a row. |
-|`org.apache.flink.table.data.RowData` | X | X | Internal data structure. |
-
-{{< /tab >}}
-{{< tab "Scala" >}}
-```scala
-case class User(
-
- // extract fields automatically
- age: Int,
- name: String,
+Or via explicit declaration:
+```java
+DataTypes.STRUCTURED(Class, DataTypes.FIELD(n0, t0), DataTypes.FIELD(n1, t1), ...);
+DataTypes.STRUCTURED(String, DataTypes.FIELD(n0, t0), DataTypes.FIELD(n1, t1), ...);
Review Comment:
I just wanted to highlight that there are two methods taking either Class or String. I reworked this part.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37862][table] Support inline structured types in SQL [flink]
Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin commented on code in PR #26638:
URL: https://github.com/apache/flink/pull/26638#discussion_r2132125616
##########
flink-table/flink-table-common/src/test/java/org/apache/flink/table/types/LogicalTypesTest.java:
##########
@@ -797,42 +831,46 @@ private StructuredType createHumanType(boolean useDifferentImplementation) {
useDifferentImplementation ? SpecialHuman.class : Human.class)
.attributes(
Collections.singletonList(
- new StructuredType.StructuredAttribute(
- "name", UDT_NAME_TYPE, "Description.")))
+ new StructuredAttribute("name", UDT_NAME_TYPE, "Description.")))
.description("Human type desc.")
.setFinal(false)
.setInstantiable(false)
.build();
}
- private StructuredType createUserType(boolean isRegistered, boolean isFinal) {
+ private StructuredType createUserType(
Review Comment:
should it be a builder with more selfexplainable methods?
Asking since now `createUserType(false, false, false)` without looking into implementation it is hard to say what all those `false` mean
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37862][table] Support inline structured types in SQL [flink]
Posted by "twalthr (via GitHub)" <gi...@apache.org>.
twalthr commented on code in PR #26638:
URL: https://github.com/apache/flink/pull/26638#discussion_r2132132754
##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/DataTypes.java:
##########
@@ -928,37 +928,88 @@ public static <T> UnresolvedDataType RAW(TypeInformation<T> typeInformation) {
}
/**
- * Data type of a user-defined object structured type. Structured types contain zero, one or
- * more attributes. Each attribute consists of a name and a type. A type cannot be defined so
- * that one of its attribute types (transitively) uses itself.
- *
- * <p>There are two kinds of structured types. Types that are stored in a catalog and are
- * identified by an {@link ObjectIdentifier} or anonymously defined, unregistered types (usually
- * reflectively extracted) that are identified by an implementation {@link Class}.
- *
- * <p>This method helps in manually constructing anonymous, unregistered types. This is useful
- * in cases where the reflective extraction using {@link DataTypes#of(Class)} is not applicable.
- * However, {@link DataTypes#of(Class)} is the recommended way of creating inline structured
- * types as it also considers {@link DataTypeHint}s.
- *
- * <p>Structured types are converted to internal data structures by the runtime. The given
- * implementation class is only used at the edges of the table ecosystem (e.g. when bridging to
- * a function or connector). Serialization and equality ({@code hashCode/equals}) are handled by
- * the runtime based on the logical type. An implementation class must offer a default
- * constructor with zero arguments or a full constructor that assigns all attributes.
- *
- * <p>Note: A caller of this method must make sure that the {@link
- * DataType#getConversionClass()} of the given fields matches with the attributes of the given
- * implementation class, otherwise an exception might be thrown during runtime.
+ * Data type of a user-defined object structured type. Structured types are identified by a
+ * class name and contain zero, one or more attributes. Each attribute consists of a name and a
+ * type. A type cannot be defined in such a way that one of its attribute types (transitively)
+ * refers to itself.
+ *
+ * <p>Compared to {@link #ROW(Field...)}, which may also be considered a "struct-like" type,
+ * structured types are distinguishable even if they contain the same set of fields. For
+ * example, "Visit(amount DOUBLE)" is distinct from "Interaction(amount DOUBLE)" due its
+ * identifier.
+ *
+ * <p>This method allows for manually constructing an inline structured type. This is useful in
+ * cases where the reflective extraction using {@link DataTypes#of(Class)} is not applicable.
+ * However, {@link DataTypes#of(Class)} is the recommended approach for creating inline
+ * structured types, as it also considers {@link DataTypeHint}s.
+ *
+ * <p>Structured types are internally converted by the system into suitable data structures.
+ * Serialization and equality checks (e.g. {@code hashCode/equals}) are managed by the system
+ * based on the logical type.
+ *
+ * <p>If an optional implementation class is provided, the system will convert a structured
+ * object to a JVM object at the edges of the table ecosystem (e.g. when bridging to a function
+ * or connector). The implementation class must provide either a zero-argument constructor or a
+ * full constructor that assigns all attributes. The class name does not need to be resolvable
+ * in the classpath; it may be used solely to distinguish between objects with identical
+ * attribute sets. However, in Table API and UDF calls, the system will attempt to resolve the
+ * class name to an actual implementation class. If resolution fails, {@link Row} is used as a
+ * fallback.
+ *
+ * <p>Note: The caller of this method must ensure that the {@link DataType#getConversionClass()}
+ * of each field matches the corresponding attribute in the implementation class. Otherwise, a
+ * runtime exception may be thrown.
*
* @see DataTypes#of(Class)
* @see StructuredType
*/
public static <T> DataType STRUCTURED(Class<T> implementationClass, Field... fields) {
// some basic validation of the class to prevent common mistakes
validateStructuredClass(implementationClass);
+ return buildStructuredType(StructuredType.newBuilder(implementationClass), fields);
+ }
- final StructuredType.Builder builder = StructuredType.newBuilder(implementationClass);
+ /**
+ * Data type of a user-defined object structured type. Structured types are identified by a
+ * class name and contain zero, one or more attributes. Each attribute consists of a name and a
+ * type. A type cannot be defined in such a way that one of its attribute types (transitively)
+ * refers to itself.
+ *
+ * <p>Compared to {@link #ROW(Field...)}, which may also be considered a "struct-like" type,
+ * structured types are distinguishable even if they contain the same set of fields. For
+ * example, "Visit(amount DOUBLE)" is distinct from "Interaction(amount DOUBLE)" due its
+ * identifier.
+ *
+ * <p>This method allows for manually constructing an inline structured type. This is useful in
+ * cases where the reflective extraction using {@link DataTypes#of(Class)} is not applicable.
+ * However, {@link DataTypes#of(Class)} is the recommended approach for creating inline
+ * structured types, as it also considers {@link DataTypeHint}s.
+ *
+ * <p>Structured types are internally converted by the system into suitable data structures.
+ * Serialization and equality checks (e.g. {@code hashCode/equals}) are managed by the system
+ * based on the logical type.
+ *
+ * <p>If an optional implementation class is provided, the system will convert a structured
+ * object to a JVM object at the edges of the table ecosystem (e.g. when bridging to a function
+ * or connector). The implementation class must provide either a zero-argument constructor or a
+ * full constructor that assigns all attributes. The class name does not need to be resolvable
+ * in the classpath; it may be used solely to distinguish between objects with identical
+ * attribute sets. However, in Table API and UDF calls, the system will attempt to resolve the
+ * class name to an actual implementation class. If resolution fails, {@link Row} is used as a
+ * fallback.
+ *
+ * <p>Note: The caller of this method must ensure that the {@link DataType#getConversionClass()}
+ * of each field matches the corresponding attribute in the implementation class. Otherwise, a
+ * runtime exception may be thrown.
+ *
+ * @see DataTypes#of(Class)
+ * @see StructuredType
+ */
+ public static <T> DataType STRUCTURED(String className, Field... fields) {
+ return buildStructuredType(StructuredType.newBuilder(className), fields);
Review Comment:
The builder is currently only used internally, via `DataTypes.STRUCTURED` and other locations. When instantiating the StructuredType we run `checkClassName` which will do the full validation. Additional checks just spam the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org