[jira] [Updated] (FLINK-25618) Data quality by apache flink

["tanjialiang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-25618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

tanjialiang updated FLINK-25618:
--------------------------------
    Description: 
This is discussing about how to support data quality through apache flink.

For example, I has a sql job, a table in this job has a column named phone, and the data of the column phone must match the pattern of telephone, if not match, i can choose drop it or ignored, and we can mark it in the metrics, so that user can monitor the data of quality in source and sink.

After this, user can kown about the data quality of the source and sink, it is useful for the downstream.

  was:
This is discussing about how to support data quality through apache flink.

For example, I has a sql job, a table in this job has a column name phone, and the data of the column phone must match the pattern of telephone, if not match, i can choose drop it or ignored, and we can mark it in the metrics, so that user can monitor the data of quality in source and sink.


> Data quality by apache flink
> ----------------------------
>
>                 Key: FLINK-25618
>                 URL: https://issues.apache.org/jira/browse/FLINK-25618
>             Project: Flink
>          Issue Type: New Feature
>            Reporter: tanjialiang
>            Priority: Not a Priority
>
> This is discussing about how to support data quality through apache flink.
> For example, I has a sql job, a table in this job has a column named phone, and the data of the column phone must match the pattern of telephone, if not match, i can choose drop it or ignored, and we can mark it in the metrics, so that user can monitor the data of quality in source and sink.
> After this, user can kown about the data quality of the source and sink, it is useful for the downstream.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)