You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Jark Wu (Jira)" <ji...@apache.org> on 2020/07/21 15:17:00 UTC
[jira] [Assigned] (FLINK-18665) Filesystem connector should use
TableSchema exclude computed columns
[ https://issues.apache.org/jira/browse/FLINK-18665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jark Wu reassigned FLINK-18665:
-------------------------------
Assignee: Leonard Xu
> Filesystem connector should use TableSchema exclude computed columns
> --------------------------------------------------------------------
>
> Key: FLINK-18665
> URL: https://issues.apache.org/jira/browse/FLINK-18665
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / FileSystem, Table SQL / Ecosystem
> Reporter: Jark Wu
> Assignee: Leonard Xu
> Priority: Major
> Fix For: 1.12.0, 1.11.2
>
>
> This is reported in http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/How-to-use-a-nested-column-for-CREATE-TABLE-PARTITIONED-BY-td36796.html
> {code}
> create table navi (
> a STRING,
> location ROW<lastUpdateTime BIGINT, transId STRING>
> ) with (
> 'connector' = 'filesystem',
> 'path' = 'east-out',
> 'format' = 'json'
> )
> CREATE TABLE output (
> `partition` AS location.transId
> ) PARTITIONED BY (`partition`)
> WITH (
> 'connector' = 'filesystem',
> 'path' = 'east-out',
> 'format' = 'json'
> ) LIKE navi (EXCLUDING ALL)
> tEnv.sqlQuery("SELECT type, location FROM navi").executeInsert("output")
> {code}
> It throws the following exception
> {code}
> Exception in thread "main" org.apache.flink.table.api.ValidationException: The field count of logical schema of the table does not match with the field count of physical schema
> . The logical schema: [STRING,ROW<`lastUpdateTime` BIGINT, `transId` STRING>]
> The physical schema: [STRING,ROW<`lastUpdateTime` BIGINT, `transId` STRING>,STRING].
> {code}
> The reason is that {{FileSystemTableFactory#createTableSource}} should use schema excluded computed column, not the original catalog table schema.
> [1]: https://github.com/apache/flink/blob/master/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/filesystem/FileSystemTableFactory.java#L78
--
This message was sent by Atlassian Jira
(v8.3.4#803005)