You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/10/28 02:43:05 UTC

[GitHub] [flink] liyubin117 opened a new pull request, #21182: [FLINK-29679][table] DESCRIBE statement shows column comment

liyubin117 opened a new pull request, #21182:
URL: https://github.com/apache/flink/pull/21182

   ## What is the purpose of the change
   
   DESCRIBE statement print column comment to make results user-friendly, such as help analysts write sql adaptive to corresponding business logics.
   
   ## Brief change log
   
   * use new schema framework to get comment.
   
   ## Verifying this change
   
   This change added tests and can be verified as follows:
   
   flink-table/flink-sql-client/src/test/resources/sql/table.q
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): no
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no
     - The S3 file system connector: no
   
   ## Documentation
   
     - Does this pull request introduce a new feature? yes
     - If yes, how is the feature documented? yes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] flinkbot commented on pull request #21182: [FLINK-29679][table] DESCRIBE statement shows column comment

Posted by GitBox <gi...@apache.org>.
flinkbot commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1294381431

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "0a7afa1d3cb5370c44b1bea729afa6577ae7d3c9",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "0a7afa1d3cb5370c44b1bea729afa6577ae7d3c9",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 0a7afa1d3cb5370c44b1bea729afa6577ae7d3c9 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on pull request #21182: [FLINK-29679][table] DESCRIBE statement shows column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1301843584

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1031324721


##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/typeutils/TimeIndicatorTypeInfo.java:
##########
@@ -62,13 +61,10 @@ protected TimeIndicatorTypeInfo(boolean isEventTime) {
         this.isEventTime = isEventTime;
     }
 
-    // this replaces the effective serializer by a LongSerializer
-    // it is a hacky but efficient solution to keep the object creation overhead low but still
-    // be compatible with the corresponding SqlTimestampTypeInfo
     @Override
     @SuppressWarnings("unchecked")
     public TypeSerializer<Timestamp> createSerializer(ExecutionConfig executionConfig) {
-        return (TypeSerializer) LongSerializer.INSTANCE;
+        return (TypeSerializer) LocalDateTimeSerializer.INSTANCE;

Review Comment:
   The problem has stucked me for two weeks, if not do so, some tests like `FunctionITCase#testAggregateFunction` would fail. In fact, `TimeIndicatorTypeInfo` as a deprecated class just provide a hacky solution and not cover the new framework, when deprecated `TableSchema` coverted from new `ResolvedSchema`, extra info like `rowtime` would retain, thus flink regard a timestamp watermark column as `TimeIndicatorTypeInfo` and use LongSerializer to serialize a LocalDateTime object then throw ClassCastException.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on pull request #21182: [FLINK-29679][table] DESCRIBE statement shows column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1319881055

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1044056409


##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/typeutils/TimeIndicatorTypeInfo.java:
##########
@@ -62,13 +61,10 @@ protected TimeIndicatorTypeInfo(boolean isEventTime) {
         this.isEventTime = isEventTime;
     }
 
-    // this replaces the effective serializer by a LongSerializer
-    // it is a hacky but efficient solution to keep the object creation overhead low but still
-    // be compatible with the corresponding SqlTimestampTypeInfo
     @Override
     @SuppressWarnings("unchecked")
     public TypeSerializer<Timestamp> createSerializer(ExecutionConfig executionConfig) {
-        return (TypeSerializer) LongSerializer.INSTANCE;
+        return (TypeSerializer) LocalDateTimeSerializer.INSTANCE;

Review Comment:
   I used to remove `rowtime` attribution during the  conversion from `ResolvedSchema` to `Schema`, but this will result in about fifty test cases trigger the Calcite exception as follow, this is caused by the column type difference between ddl and execution, we have made created table adapt to new schema framework, column type should be same during execution correspondingly.
   ```
   Cannot add expression of different type to set:
   set type is RecordType(INTEGER amount, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency, TIMESTAMP(3) *ROWTIME* rowtime, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" NOT NULL currency0, INTEGER rate, TIMESTAMP(3) rowtime0) NOT NULL
   expression type is RecordType(INTEGER amount, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency, TIMESTAMP(3) *ROWTIME* rowtime, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" NOT NULL currency0, INTEGER rate, TIMESTAMP(3) *ROWTIME* rowtime0) NOT NULL
   ```
   In the other hand, As we can see in the previous version `TimeIndicatorTypeInfo`, LongSerializer is just a temporary hacky way and should be enhanced.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1044056409


##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/typeutils/TimeIndicatorTypeInfo.java:
##########
@@ -62,13 +61,10 @@ protected TimeIndicatorTypeInfo(boolean isEventTime) {
         this.isEventTime = isEventTime;
     }
 
-    // this replaces the effective serializer by a LongSerializer
-    // it is a hacky but efficient solution to keep the object creation overhead low but still
-    // be compatible with the corresponding SqlTimestampTypeInfo
     @Override
     @SuppressWarnings("unchecked")
     public TypeSerializer<Timestamp> createSerializer(ExecutionConfig executionConfig) {
-        return (TypeSerializer) LongSerializer.INSTANCE;
+        return (TypeSerializer) LocalDateTimeSerializer.INSTANCE;

Review Comment:
   I used to remove `rowtime` attribution during the  conversion from `ResolvedSchema` to `Schema`, but this will result in about fifty test cases trigger the Calcite exception as follow, this is caused by the column type difference between ddl and execution, we have made created table adapt to new schema framework, column type should be same during execution correspondingly.
   ```
   Cannot add expression of different type to set:
   set type is RecordType(INTEGER amount, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency, TIMESTAMP(3) *ROWTIME* rowtime, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" NOT NULL currency0, INTEGER rate, TIMESTAMP(3) rowtime0) NOT NULL
   expression type is RecordType(INTEGER amount, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency, TIMESTAMP(3) *ROWTIME* rowtime, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" NOT NULL currency0, INTEGER rate, TIMESTAMP(3) *ROWTIME* rowtime0) NOT NULL
   ```
   As we can see in the previous version `TimeIndicatorTypeInfo`, LongSerializer is just a temporal hacky way and should be enhanced.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1044056409


##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/typeutils/TimeIndicatorTypeInfo.java:
##########
@@ -62,13 +61,10 @@ protected TimeIndicatorTypeInfo(boolean isEventTime) {
         this.isEventTime = isEventTime;
     }
 
-    // this replaces the effective serializer by a LongSerializer
-    // it is a hacky but efficient solution to keep the object creation overhead low but still
-    // be compatible with the corresponding SqlTimestampTypeInfo
     @Override
     @SuppressWarnings("unchecked")
     public TypeSerializer<Timestamp> createSerializer(ExecutionConfig executionConfig) {
-        return (TypeSerializer) LongSerializer.INSTANCE;
+        return (TypeSerializer) LocalDateTimeSerializer.INSTANCE;

Review Comment:
   I tried to remove `rowtime` attribution during the conversion from `ResolvedSchema` to `Schema`, but CI results show that about fifty test cases trigger the Calcite exception as follow, this is caused by the column type difference between ddl and execution, we have made created table adapt to new schema framework, column type should be same during execution correspondingly.
   ```
   Cannot add expression of different type to set:
   set type is RecordType(INTEGER amount, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency, TIMESTAMP(3) *ROWTIME* rowtime, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" NOT NULL currency0, INTEGER rate, TIMESTAMP(3) rowtime0) NOT NULL
   expression type is RecordType(INTEGER amount, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency, TIMESTAMP(3) *ROWTIME* rowtime, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" NOT NULL currency0, INTEGER rate, TIMESTAMP(3) *ROWTIME* rowtime0) NOT NULL
   ```
   In the other hand, As we can see in the previous version `TimeIndicatorTypeInfo`, LongSerializer is just a temporary hacky way and should be enhanced.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on pull request #21182: [FLINK-29679][table] DESCRIBE statement shows column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1294506824

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1031324721


##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/typeutils/TimeIndicatorTypeInfo.java:
##########
@@ -62,13 +61,10 @@ protected TimeIndicatorTypeInfo(boolean isEventTime) {
         this.isEventTime = isEventTime;
     }
 
-    // this replaces the effective serializer by a LongSerializer
-    // it is a hacky but efficient solution to keep the object creation overhead low but still
-    // be compatible with the corresponding SqlTimestampTypeInfo
     @Override
     @SuppressWarnings("unchecked")
     public TypeSerializer<Timestamp> createSerializer(ExecutionConfig executionConfig) {
-        return (TypeSerializer) LongSerializer.INSTANCE;
+        return (TypeSerializer) LocalDateTimeSerializer.INSTANCE;

Review Comment:
   Good question~ The problem has stucked me for two weeks, if not do so, some tests like `FunctionITCase#testAggregateFunction` would fail. In fact, `TimeIndicatorTypeInfo` as a deprecated class just provide a hacky solution and not cover the new schema framework, when deprecated `TableSchema` coverted from new `ResolvedSchema`, extra info like `rowtime` would retain, thus flink regard a timestamp watermark column as `TimeIndicatorTypeInfo` and use LongSerializer to serialize a LocalDateTime object then throw ClassCastException.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1341934897

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] JingsongLi merged pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
JingsongLi merged PR #21182:
URL: https://github.com/apache/flink/pull/21182


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on pull request #21182: [FLINK-29679][table] DESCRIBE statement shows column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1319998371

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1044056409


##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/typeutils/TimeIndicatorTypeInfo.java:
##########
@@ -62,13 +61,10 @@ protected TimeIndicatorTypeInfo(boolean isEventTime) {
         this.isEventTime = isEventTime;
     }
 
-    // this replaces the effective serializer by a LongSerializer
-    // it is a hacky but efficient solution to keep the object creation overhead low but still
-    // be compatible with the corresponding SqlTimestampTypeInfo
     @Override
     @SuppressWarnings("unchecked")
     public TypeSerializer<Timestamp> createSerializer(ExecutionConfig executionConfig) {
-        return (TypeSerializer) LongSerializer.INSTANCE;
+        return (TypeSerializer) LocalDateTimeSerializer.INSTANCE;

Review Comment:
   I used to remove `rowtime` attri during the  conversion from `ResolvedSchema` to `Schema`, but this will make about fifty test cases trigger the Calcite exception as follow:
   ```
   Cannot add expression of different type to set:
   set type is RecordType(INTEGER amount, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency, TIMESTAMP(3) *ROWTIME* rowtime, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" NOT NULL currency0, INTEGER rate, TIMESTAMP(3) rowtime0) NOT NULL
   expression type is RecordType(INTEGER amount, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency, TIMESTAMP(3) *ROWTIME* rowtime, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" NOT NULL currency0, INTEGER rate, TIMESTAMP(3) *ROWTIME* rowtime0) NOT NULL
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on pull request #21182: [FLINK-29679][table] DESCRIBE statement shows column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1298232241

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on pull request #21182: [FLINK-29679][table] DESCRIBE statement shows column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1297078814

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1031327623


##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java:
##########
@@ -1600,23 +1619,38 @@ private Object[][] buildTableColumns(ResolvedSchema schema) {
                                                             "PRI(%s)",
                                                             String.join(", ", columns))));
                         });
-
+        boolean nonComments = isSchemaNonColumnComments(schema);
         return schema.getColumns().stream()
                 .map(
                         (c) -> {
                             final LogicalType logicalType = c.getDataType().getLogicalType();
+                            if (nonComments) {
+                                return new Object[] {

Review Comment:
   done :)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] JingsongLi commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
JingsongLi commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1031181983


##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/TableSchema.java:
##########
@@ -346,6 +346,41 @@ public Schema toSchema() {
         return builder.build();
     }
 
+    /** Helps to migrate to the new {@link Schema} class, retain comments when needed. */
+    public Schema toSchema(Map<String, String> comments) {
+        final Schema.Builder builder = Schema.newBuilder();
+
+        columns.forEach(
+                column -> {
+                    if (column instanceof PhysicalColumn) {
+                        final PhysicalColumn c = (PhysicalColumn) column;
+                        builder.column(c.getName(), c.getType());
+                    } else if (column instanceof MetadataColumn) {
+                        final MetadataColumn c = (MetadataColumn) column;
+                        builder.columnByMetadata(
+                                c.getName(),
+                                c.getType(),
+                                c.getMetadataAlias().orElse(null),
+                                c.isVirtual());
+                    } else if (column instanceof ComputedColumn) {
+                        final ComputedColumn c = (ComputedColumn) column;
+                        builder.columnByExpression(c.getName(), c.getExpression());
+                    } else {
+                        throw new IllegalArgumentException("Unsupported column type: " + column);
+                    }
+                    builder.withComment(comments.get(column.getName()));

Review Comment:
   Can it be null?
   We can improve this method a little bit.



##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java:
##########
@@ -1600,23 +1619,38 @@ private Object[][] buildTableColumns(ResolvedSchema schema) {
                                                             "PRI(%s)",
                                                             String.join(", ", columns))));
                         });
-
+        boolean nonComments = isSchemaNonColumnComments(schema);
         return schema.getColumns().stream()
                 .map(
                         (c) -> {
                             final LogicalType logicalType = c.getDataType().getLogicalType();
+                            if (nonComments) {
+                                return new Object[] {

Review Comment:
   Can we just create `List<Object>`, with comments, we can add a comment.
   Then `List.toArray` to a object array.



##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java:
##########
@@ -1600,23 +1619,38 @@ private Object[][] buildTableColumns(ResolvedSchema schema) {
                                                             "PRI(%s)",
                                                             String.join(", ", columns))));
                         });
-
+        boolean nonComments = isSchemaNonColumnComments(schema);
         return schema.getColumns().stream()
                 .map(
                         (c) -> {
                             final LogicalType logicalType = c.getDataType().getLogicalType();
+                            if (nonComments) {
+                                return new Object[] {
+                                    c.getName(),
+                                    logicalType.copy(true).asSummaryString(),
+                                    logicalType.isNullable(),
+                                    fieldToPrimaryKey.getOrDefault(c.getName(), null),
+                                    c.explainExtras().orElse(null),
+                                    fieldToWatermark.getOrDefault(c.getName(), null)
+                                };
+                            }
                             return new Object[] {
                                 c.getName(),
                                 logicalType.copy(true).asSummaryString(),
                                 logicalType.isNullable(),
                                 fieldToPrimaryKey.getOrDefault(c.getName(), null),
                                 c.explainExtras().orElse(null),
-                                fieldToWatermark.getOrDefault(c.getName(), null)
+                                fieldToWatermark.getOrDefault(c.getName(), null),
+                                c.getComment().orElse(null)
                             };
                         })
                 .toArray(Object[][]::new);
     }
 
+    private boolean isSchemaNonColumnComments(ResolvedSchema schema) {
+        return schema.getColumns().stream().noneMatch(col -> col.getComment().isPresent());

Review Comment:
   Minor: `schema.getColumns().stream().map(Column::getComment).noneMatch(Optional::isPresent);`
   This is just a matter of personal code style.



##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/typeutils/TimeIndicatorTypeInfo.java:
##########
@@ -62,13 +61,10 @@ protected TimeIndicatorTypeInfo(boolean isEventTime) {
         this.isEventTime = isEventTime;
     }
 
-    // this replaces the effective serializer by a LongSerializer
-    // it is a hacky but efficient solution to keep the object creation overhead low but still
-    // be compatible with the corresponding SqlTimestampTypeInfo
     @Override
     @SuppressWarnings("unchecked")
     public TypeSerializer<Timestamp> createSerializer(ExecutionConfig executionConfig) {
-        return (TypeSerializer) LongSerializer.INSTANCE;
+        return (TypeSerializer) LocalDateTimeSerializer.INSTANCE;

Review Comment:
   Can you explain why we need to modify this? Does it have any other impact?



##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/TableSchema.java:
##########
@@ -346,6 +346,41 @@ public Schema toSchema() {
         return builder.build();
     }
 
+    /** Helps to migrate to the new {@link Schema} class, retain comments when needed. */
+    public Schema toSchema(Map<String, String> comments) {

Review Comment:
   I like this method~



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1044056409


##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/typeutils/TimeIndicatorTypeInfo.java:
##########
@@ -62,13 +61,10 @@ protected TimeIndicatorTypeInfo(boolean isEventTime) {
         this.isEventTime = isEventTime;
     }
 
-    // this replaces the effective serializer by a LongSerializer
-    // it is a hacky but efficient solution to keep the object creation overhead low but still
-    // be compatible with the corresponding SqlTimestampTypeInfo
     @Override
     @SuppressWarnings("unchecked")
     public TypeSerializer<Timestamp> createSerializer(ExecutionConfig executionConfig) {
-        return (TypeSerializer) LongSerializer.INSTANCE;
+        return (TypeSerializer) LocalDateTimeSerializer.INSTANCE;

Review Comment:
   I used to remove `rowtime` attri during the  conversion from `ResolvedSchema` to `Schema`, but this will result in about fifty test cases trigger the Calcite exception as follow, this is caused by the column type difference between ddl and execution, we have made created table adapt to new schema framework, column type should be same during execution correspondingly.
   ```
   Cannot add expression of different type to set:
   set type is RecordType(INTEGER amount, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency, TIMESTAMP(3) *ROWTIME* rowtime, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" NOT NULL currency0, INTEGER rate, TIMESTAMP(3) rowtime0) NOT NULL
   expression type is RecordType(INTEGER amount, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency, TIMESTAMP(3) *ROWTIME* rowtime, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" NOT NULL currency0, INTEGER rate, TIMESTAMP(3) *ROWTIME* rowtime0) NOT NULL
   ```
   As we can see in the previous version `TimeIndicatorTypeInfo`, LongSerializer is just a temporal hacky way and should be enhanced.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1044035987


##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java:
##########
@@ -1600,23 +1619,34 @@ private Object[][] buildTableColumns(ResolvedSchema schema) {
                                                             "PRI(%s)",
                                                             String.join(", ", columns))));
                         });
-
+        boolean nonComments = isSchemaNonColumnComments(schema);
         return schema.getColumns().stream()
                 .map(
                         (c) -> {
                             final LogicalType logicalType = c.getDataType().getLogicalType();
-                            return new Object[] {
-                                c.getName(),
-                                logicalType.copy(true).asSummaryString(),
-                                logicalType.isNullable(),
-                                fieldToPrimaryKey.getOrDefault(c.getName(), null),
-                                c.explainExtras().orElse(null),
-                                fieldToWatermark.getOrDefault(c.getName(), null)
-                            };
+                            final ArrayList<Object> result =
+                                    new ArrayList<>(
+                                            Arrays.asList(
+                                                    c.getName(),
+                                                    logicalType.copy(true).asSummaryString(),
+                                                    logicalType.isNullable(),
+                                                    fieldToPrimaryKey.getOrDefault(
+                                                            c.getName(), null),
+                                                    c.explainExtras().orElse(null),
+                                                    fieldToWatermark.getOrDefault(
+                                                            c.getName(), null)));
+                            if (!nonComments) {
+                                result.add(c.getComment().orElse(null));
+                            }
+                            return result.toArray();
                         })
                 .toArray(Object[][]::new);
     }
 
+    private boolean isSchemaNonColumnComments(ResolvedSchema schema) {

Review Comment:
   The solution is to make results compatible with former version, so we could make the results simpler, also don't need to modify plenty of test cases.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1044033884


##########
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/connectors/hive/HiveTableFactory.java:
##########
@@ -49,7 +47,6 @@ public List<String> supportedProperties() {
     @Override
     public TableSource createTableSource(TableSourceFactory.Context context) {
         CatalogTable table = checkNotNull(context.getTable());
-        Preconditions.checkArgument(table instanceof CatalogTableImpl);

Review Comment:
   Because the new created table is a instance of ResolvedCatalogTable which could provide `Schema`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1343775931

   @godfreyhe Thanks for your excelent review! I have refactored toSchema() as you figured.  
   Question: `can we improve the conversion logic of ResolvedSchema to TableSchema ?`
   Answer: I used to remove `rowtime` attri during the  conversion from `ResolvedSchema` to `Schema`, but this will make about fifty test cases trigger the Calcite exception as follow:
   ```
   Cannot add expression of different type to set:
   set type is RecordType(INTEGER amount, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency, TIMESTAMP(3) *ROWTIME* rowtime, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" NOT NULL currency0, INTEGER rate, TIMESTAMP(3) rowtime0) NOT NULL
   expression type is RecordType(INTEGER amount, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency, TIMESTAMP(3) *ROWTIME* rowtime, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" NOT NULL currency0, INTEGER rate, TIMESTAMP(3) *ROWTIME* rowtime0) NOT NULL
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1344441038

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on pull request #21182: [FLINK-29679][table] DESCRIBE statement shows column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1319700101

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] godfreyhe commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
godfreyhe commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1043322430


##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/typeutils/TimeIndicatorTypeInfo.java:
##########
@@ -62,13 +61,10 @@ protected TimeIndicatorTypeInfo(boolean isEventTime) {
         this.isEventTime = isEventTime;
     }
 
-    // this replaces the effective serializer by a LongSerializer
-    // it is a hacky but efficient solution to keep the object creation overhead low but still
-    // be compatible with the corresponding SqlTimestampTypeInfo
     @Override
     @SuppressWarnings("unchecked")
     public TypeSerializer<Timestamp> createSerializer(ExecutionConfig executionConfig) {
-        return (TypeSerializer) LongSerializer.INSTANCE;
+        return (TypeSerializer) LocalDateTimeSerializer.INSTANCE;

Review Comment:
   can we improve the conversion logic of `ResolvedSchema to TableSchema` ?



##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/TableSchema.java:
##########
@@ -346,6 +346,41 @@ public Schema toSchema() {
         return builder.build();
     }
 
+    /** Helps to migrate to the new {@link Schema} class, retain comments when needed. */
+    public Schema toSchema(Map<String, String> comments) {

Review Comment:
   please let `toSchema()` call `toSchema(Map<String, String> comments)`



##########
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/connectors/hive/HiveTableFactory.java:
##########
@@ -49,7 +47,6 @@ public List<String> supportedProperties() {
     @Override
     public TableSource createTableSource(TableSourceFactory.Context context) {
         CatalogTable table = checkNotNull(context.getTable());
-        Preconditions.checkArgument(table instanceof CatalogTableImpl);

Review Comment:
   why we need remove this check?



##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java:
##########
@@ -1600,23 +1619,34 @@ private Object[][] buildTableColumns(ResolvedSchema schema) {
                                                             "PRI(%s)",
                                                             String.join(", ", columns))));
                         });
-
+        boolean nonComments = isSchemaNonColumnComments(schema);
         return schema.getColumns().stream()
                 .map(
                         (c) -> {
                             final LogicalType logicalType = c.getDataType().getLogicalType();
-                            return new Object[] {
-                                c.getName(),
-                                logicalType.copy(true).asSummaryString(),
-                                logicalType.isNullable(),
-                                fieldToPrimaryKey.getOrDefault(c.getName(), null),
-                                c.explainExtras().orElse(null),
-                                fieldToWatermark.getOrDefault(c.getName(), null)
-                            };
+                            final ArrayList<Object> result =
+                                    new ArrayList<>(
+                                            Arrays.asList(
+                                                    c.getName(),
+                                                    logicalType.copy(true).asSummaryString(),
+                                                    logicalType.isNullable(),
+                                                    fieldToPrimaryKey.getOrDefault(
+                                                            c.getName(), null),
+                                                    c.explainExtras().orElse(null),
+                                                    fieldToWatermark.getOrDefault(
+                                                            c.getName(), null)));
+                            if (!nonComments) {
+                                result.add(c.getComment().orElse(null));
+                            }
+                            return result.toArray();
                         })
                 .toArray(Object[][]::new);
     }
 
+    private boolean isSchemaNonColumnComments(ResolvedSchema schema) {

Review Comment:
   another solution: we always print the comments whether the schema has comments



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1044052680


##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/TableSchema.java:
##########
@@ -346,6 +346,41 @@ public Schema toSchema() {
         return builder.build();
     }
 
+    /** Helps to migrate to the new {@link Schema} class, retain comments when needed. */
+    public Schema toSchema(Map<String, String> comments) {

Review Comment:
   Thanks, I have refactored toSchema() as you figured.  



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1044033884


##########
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/connectors/hive/HiveTableFactory.java:
##########
@@ -49,7 +47,6 @@ public List<String> supportedProperties() {
     @Override
     public TableSource createTableSource(TableSourceFactory.Context context) {
         CatalogTable table = checkNotNull(context.getTable());
-        Preconditions.checkArgument(table instanceof CatalogTableImpl);

Review Comment:
   Because the new created table is a instance of `ResolvedCatalogTable` which could provide `Schema`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1031324721


##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/typeutils/TimeIndicatorTypeInfo.java:
##########
@@ -62,13 +61,10 @@ protected TimeIndicatorTypeInfo(boolean isEventTime) {
         this.isEventTime = isEventTime;
     }
 
-    // this replaces the effective serializer by a LongSerializer
-    // it is a hacky but efficient solution to keep the object creation overhead low but still
-    // be compatible with the corresponding SqlTimestampTypeInfo
     @Override
     @SuppressWarnings("unchecked")
     public TypeSerializer<Timestamp> createSerializer(ExecutionConfig executionConfig) {
-        return (TypeSerializer) LongSerializer.INSTANCE;
+        return (TypeSerializer) LocalDateTimeSerializer.INSTANCE;

Review Comment:
   Good question~ The problem has stucked me for two weeks, if not do so, some tests like `FunctionITCase#testAggregateFunction` would fail. In fact, `TimeIndicatorTypeInfo` as a deprecated class just provide a hacky solution and not cover the new schema framework, when deprecated `TableSchema` coverted from new `ResolvedSchema`, extra info like `rowtime` would retain, thus flink regard a timestamp watermark column as `TimeIndicatorTypeInfo` and use LongSerializer to serialize a LocalDateTime object then throw ClassCastException, so we should enhance the serialization. Given every test case has passed, it has no impact except above scenario.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1044052680


##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/TableSchema.java:
##########
@@ -346,6 +346,41 @@ public Schema toSchema() {
         return builder.build();
     }
 
+    /** Helps to migrate to the new {@link Schema} class, retain comments when needed. */
+    public Schema toSchema(Map<String, String> comments) {

Review Comment:
   done :)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1044056409


##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/typeutils/TimeIndicatorTypeInfo.java:
##########
@@ -62,13 +61,10 @@ protected TimeIndicatorTypeInfo(boolean isEventTime) {
         this.isEventTime = isEventTime;
     }
 
-    // this replaces the effective serializer by a LongSerializer
-    // it is a hacky but efficient solution to keep the object creation overhead low but still
-    // be compatible with the corresponding SqlTimestampTypeInfo
     @Override
     @SuppressWarnings("unchecked")
     public TypeSerializer<Timestamp> createSerializer(ExecutionConfig executionConfig) {
-        return (TypeSerializer) LongSerializer.INSTANCE;
+        return (TypeSerializer) LocalDateTimeSerializer.INSTANCE;

Review Comment:
   I used to remove `rowtime` attri during the  conversion from `ResolvedSchema` to `Schema`, but this will make about fifty test cases trigger the Calcite exception as follow:
   ```
   Cannot add expression of different type to set:
   set type is RecordType(INTEGER amount, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency, TIMESTAMP(3) *ROWTIME* rowtime, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" NOT NULL currency0, INTEGER rate, TIMESTAMP(3) rowtime0) NOT NULL
   expression type is RecordType(INTEGER amount, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" currency, TIMESTAMP(3) *ROWTIME* rowtime, TIMESTAMP_LTZ(3) *PROCTIME* NOT NULL proctime, VARCHAR(2147483647) CHARACTER SET "UTF-16LE" NOT NULL currency0, INTEGER rate, TIMESTAMP(3) *ROWTIME* rowtime0) NOT NULL
   ```
   As we can see in the previous version `TimeIndicatorTypeInfo`, LongSerializer is just a temporal hacky way and should be enhanced.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on pull request #21182: [FLINK-29679][table] DESCRIBE statement shows column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1294397909

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on pull request #21182: [FLINK-29679][table] DESCRIBE statement shows column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1294406889

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on pull request #21182: [FLINK-29679][table] DESCRIBE statement shows column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1319663034

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on a diff in pull request #21182: [FLINK-29679][table] Migrate to new schema framework & show column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on code in PR #21182:
URL: https://github.com/apache/flink/pull/21182#discussion_r1031327314


##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/TableSchema.java:
##########
@@ -346,6 +346,41 @@ public Schema toSchema() {
         return builder.build();
     }
 
+    /** Helps to migrate to the new {@link Schema} class, retain comments when needed. */
+    public Schema toSchema(Map<String, String> comments) {
+        final Schema.Builder builder = Schema.newBuilder();
+
+        columns.forEach(
+                column -> {
+                    if (column instanceof PhysicalColumn) {
+                        final PhysicalColumn c = (PhysicalColumn) column;
+                        builder.column(c.getName(), c.getType());
+                    } else if (column instanceof MetadataColumn) {
+                        final MetadataColumn c = (MetadataColumn) column;
+                        builder.columnByMetadata(
+                                c.getName(),
+                                c.getType(),
+                                c.getMetadataAlias().orElse(null),
+                                c.isVirtual());
+                    } else if (column instanceof ComputedColumn) {
+                        final ComputedColumn c = (ComputedColumn) column;
+                        builder.columnByExpression(c.getName(), c.getExpression());
+                    } else {
+                        throw new IllegalArgumentException("Unsupported column type: " + column);
+                    }
+                    builder.withComment(comments.get(column.getName()));

Review Comment:
   Thanks for your kind reminds, we don't need to invoke `withComment` when comment is null, I have fixed it.



##########
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java:
##########
@@ -1600,23 +1619,38 @@ private Object[][] buildTableColumns(ResolvedSchema schema) {
                                                             "PRI(%s)",
                                                             String.join(", ", columns))));
                         });
-
+        boolean nonComments = isSchemaNonColumnComments(schema);
         return schema.getColumns().stream()
                 .map(
                         (c) -> {
                             final LogicalType logicalType = c.getDataType().getLogicalType();
+                            if (nonComments) {
+                                return new Object[] {
+                                    c.getName(),
+                                    logicalType.copy(true).asSummaryString(),
+                                    logicalType.isNullable(),
+                                    fieldToPrimaryKey.getOrDefault(c.getName(), null),
+                                    c.explainExtras().orElse(null),
+                                    fieldToWatermark.getOrDefault(c.getName(), null)
+                                };
+                            }
                             return new Object[] {
                                 c.getName(),
                                 logicalType.copy(true).asSummaryString(),
                                 logicalType.isNullable(),
                                 fieldToPrimaryKey.getOrDefault(c.getName(), null),
                                 c.explainExtras().orElse(null),
-                                fieldToWatermark.getOrDefault(c.getName(), null)
+                                fieldToWatermark.getOrDefault(c.getName(), null),
+                                c.getComment().orElse(null)
                             };
                         })
                 .toArray(Object[][]::new);
     }
 
+    private boolean isSchemaNonColumnComments(ResolvedSchema schema) {
+        return schema.getColumns().stream().noneMatch(col -> col.getComment().isPresent());

Review Comment:
   done :)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [flink] liyubin117 commented on pull request #21182: [FLINK-29679][table] DESCRIBE statement shows column comment

Posted by GitBox <gi...@apache.org>.
liyubin117 commented on PR #21182:
URL: https://github.com/apache/flink/pull/21182#issuecomment-1298131808

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org