[jira] [Updated] (FLINK-17158) Watermark strategy property cannot be expressed in YAML

["Timo Walther (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-17158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Timo Walther updated FLINK-17158:
---------------------------------
    Affects Version/s: 1.10.0

> Watermark strategy property cannot be expressed in YAML
> -------------------------------------------------------
>
>                 Key: FLINK-17158
>                 URL: https://issues.apache.org/jira/browse/FLINK-17158
>             Project: Flink
>          Issue Type: Bug
>          Components: Table SQL / API, Table SQL / Ecosystem
>    Affects Versions: 1.10.0
>            Reporter: Timo Walther
>            Priority: Major
>
> If the watermark strategy needs to be serialized into string-based properties for catalog storage, it is serialized into the following string-based properties:
> {code}
> DescriptorProperties properties = new DescriptorProperties();
> 		properties.putTableSchema("schema", schema);
> 		Map<String, String> actual = properties.asMap();
> 		Map<String, String> expected = new HashMap<>();
> 		expected.put("schema.0.name", "f0");
> 		expected.put("schema.0.data-type", "BIGINT");
> 		expected.put("schema.1.name", "f1");
> 		expected.put("schema.1.data-type", "ROW<`q1` VARCHAR(2147483647), `q2` TIMESTAMP(9)>");
> 		expected.put("schema.2.name", "f2");
> 		expected.put("schema.2.data-type", "VARCHAR(2147483647)");
> 		expected.put("schema.3.name", "f3");
> 		expected.put("schema.3.data-type", "BIGINT");
> 		expected.put("schema.3.expr", "f0 + 1");
> 		expected.put("schema.4.name", "f4");
> 		expected.put("schema.4.data-type", "DECIMAL(10, 3)");
> 		expected.put("schema.watermark.0.rowtime", "f1.q2");
> 		expected.put("schema.watermark.0.strategy.expr", "`f1`.`q2` - INTERVAL '5' SECOND");
> 		expected.put("schema.watermark.0.strategy.data-type", "TIMESTAMP(3)");
> {code}
> However, this design does not follow a proper hierarchy as both an array and watermarks are stored under a common `schema.` key. Thus, it is impossible to express those properties in JSON or YAML file:
> {code}
> tables:
>   - name: PROD_RATES_HISTORY
>     type: source
>     update-mode: append
>     schema:
>     - name: RS_TIMESTAMP
>       data-type: TIMESTAMP(3)
>     - name: RS_SYMBOL
>       data-type: STRING
>     - name: DOUBLE
>       data-type: RS_RATE
>       ???watermark:
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)