You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "yuxiqian (via GitHub)" <gi...@apache.org> on 2024/04/28 06:49:31 UTC
[PR] [FLINK-35256][runtime] Fix transform node does not respect type nullability [flink-cdc]
yuxiqian opened a new pull request, #3272:
URL: https://github.com/apache/flink-cdc/pull/3272
This closes [FLINK-35256](https://issues.apache.org/jira/browse/FLINK-35256).
Flink CDC 3.1.0 brought transform feature, allowing column type / value transformation prior to data routing process. However after the transformation, column type marked as `NOT NULL` lost their annotation, causing some downstream sinks to fail since they require primary key to be NOT NULL.
Here's the minimum reproducible example about this problem:
```yaml
source:
type: mysql
...
sink:
type: starrocks
name: StarRocks Sink
...
pipeline:
name: Sync MySQL Database to StarRocks
parallelism: 4
transform:
source-table: reicigo.\.*
projection: ID, UPPER(ID) AS UPID
```
MySQL source table schema definition is as follows:
```
CREATE TABLE table1 (
ID VARCHAR(17) NOT NULL,
...
PRIMARY KEY (ID)
);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35256][runtime] Fix transform node does not respect type nullability [flink-cdc]
Posted by "PatrickRen (via GitHub)" <gi...@apache.org>.
PatrickRen commented on PR #3272:
URL: https://github.com/apache/flink-cdc/pull/3272#issuecomment-2081482756
@yuxiqian Could you also back port the patch onto `release-3.1`? Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35256][runtime] Fix transform node does not respect type nullability [flink-cdc]
Posted by "lvyanquan (via GitHub)" <gi...@apache.org>.
lvyanquan commented on PR #3272:
URL: https://github.com/apache/flink-cdc/pull/3272#issuecomment-2081442070
I've tested that if namespace of TableId is not set and `__namespace_name__` column will be `null` string, so it won't cause error if MetadataColumns are not null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35256][runtime] Fix transform node does not respect type nullability [flink-cdc]
Posted by "yuxiqian (via GitHub)" <gi...@apache.org>.
yuxiqian commented on PR #3272:
URL: https://github.com/apache/flink-cdc/pull/3272#issuecomment-2081485658
@PatrickRen Sure! Done in #3277.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-35256][runtime] Fix transform node does not respect type nullability [flink-cdc]
Posted by "PatrickRen (via GitHub)" <gi...@apache.org>.
PatrickRen merged PR #3272:
URL: https://github.com/apache/flink-cdc/pull/3272
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org