You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/03/08 12:16:00 UTC
[jira] [Commented] (FLINK-8854) Mapping of
SchemaValidator.deriveFieldMapping() is incorrect.
[ https://issues.apache.org/jira/browse/FLINK-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16391163#comment-16391163 ]
ASF GitHub Bot commented on FLINK-8854:
---------------------------------------
GitHub user twalthr opened a pull request:
https://github.com/apache/flink/pull/5662
[FLINK-8854] [table] Fix schema mapping with time attributes
## What is the purpose of the change
This PR fixes the invalid field mapping and improves the mapping of time attributes in general.
## Brief change log
- `SchemaValidator.deriveFieldMapping()` and `SchemaValidator.deriveFormatFields()`
## Verifying this change
- Existing tests extended
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? JavaDocs
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/twalthr/flink FLINK-8854
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5662.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5662
----
commit 45c1ca57b65004fa973b663e441848ab228ade3e
Author: Timo Walther <tw...@...>
Date: 2018-03-08T10:51:38Z
[FLINK-8854] [table] Fix schema mapping with time attributes
----
> Mapping of SchemaValidator.deriveFieldMapping() is incorrect.
> -------------------------------------------------------------
>
> Key: FLINK-8854
> URL: https://issues.apache.org/jira/browse/FLINK-8854
> Project: Flink
> Issue Type: Bug
> Components: Table API & SQL
> Affects Versions: 1.5.0
> Reporter: Fabian Hueske
> Assignee: Timo Walther
> Priority: Blocker
> Fix For: 1.5.0
>
>
> The field mapping returned by {{SchemaValidator.deriveFieldMapping()}} is not correct.
> It should not only include all fields of the table schema, but also all fields of the format schema (mapped to themselves). Otherwise, it is not possible to use a timestamp extractor on a field that is not in table schema.
> For example this configuration would fail:
> {code}
> sources:
> - name: TaxiRides
> schema:
> - name: rideId
> type: LONG
> - name: rowTime
> type: TIMESTAMP
> rowtime:
> timestamps:
> type: "from-field"
> from: "rideTime"
> watermarks:
> type: "periodic-bounded"
> delay: "60000"
> connector:
> ....
> format:
> property-version: 1
> type: json
> schema: "ROW(rideId LONG, rideTime TIMESTAMP)"
> {code}
> because {{rideTime}} is not in the table schema.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)