[GitHub] [flink] matriv commented on a change in pull request #19236: [FLINK-26712][table-planner] Metadata keys should not conflict with physical columns

[GitBox <gi...@apache.org>]

matriv commented on a change in pull request #19236:
URL: https://github.com/apache/flink/pull/19236#discussion_r839425345



##########
File path: flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/connectors/DynamicSourceUtils.java
##########
@@ -71,6 +71,9 @@
 @Internal
 public final class DynamicSourceUtils {
 
+    // Ensures that physical and metadata columns don't collide.
+    public static final String METADATA_COLUMN_PREFIX = "$metadata$";

Review comment:
       minor, but maybe we can define this constant in a common place for both source and sink?

##########
File path: flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/FileSystemTableSource.java
##########
@@ -225,16 +222,27 @@ public ScanRuntimeProvider getScanRuntimeProvider(ScanContext scanContext) {
             List<ReadableFileInfo> metadata,
             List<String> partitionKeys) {
         if (!metadata.isEmpty() || !partitionKeys.isEmpty()) {
+            final List<String> producedFieldNames = DataType.getFieldNames(producedDataType);
+            final Map<String, FileInfoAccessor> metadataColumns =
+                    IntStream.range(0, metadata.size())
+                            .mapToObj(
+                                    i -> {
+                                        // Access metadata columns from behind because the

Review comment:
       from behind -> `backwards`, or `in reverse` ?

##########
File path: flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/plan/stream/sql/TableScanTest.scala
##########
@@ -99,6 +99,25 @@ class TableScanTest extends TableTestBase {
     util.verifyExecPlan("SELECT * FROM MetadataTable")
   }
 
+  @Test
+  def testDDLWithMetadataThatConflictsWithPhysicalColumn(): Unit = {

Review comment:
       +1




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org