Flink SQL: Custom exception handling External

[Chong Yun Long <yu...@gojek.com>]

Hi,

Is there any mechanism for handling of errors produced by Flink SQL?
It can be useful for various use cases:
1. Logging exceptions and the erroneous row to a kafka topic
2. Ignoring transient exceptions instead of throwing and failing the entire
job

If there are no such mechanisms may I propose something which looks like

WITH (
   'connector' = 'jdbc',
   'url' = 'jdbc:mysql://localhost:3306/mydatabase',
   'table-name' = 'users'
   'exception-handler' = <classpath_to_handler_class>);


-- 
Regards,

Yun Long

Re: Flink SQL: Custom exception handling External

[Caizhi Weng <ts...@gmail.com>]

Hi!

As far as I know JDBC does not have this error handling mechanism. Also
there are very few connectors / formats which support skipping
erroneous records (for example the csv format).

Which type of exception are you faced with? As JDBC connectors, unlike
message queue connectors, rarely (if ever?) suffer from records with bad
formats.

If you really have the need to handle exceptions you might want to
implement your own JDBC connector by extending the classes in Flink and
call each method within a try-catch block.

Chong Yun Long <yu...@gojek.com> 于2021年8月25日周三 下午4:46写道：

> Hi,
>
> Is there any mechanism for handling of errors produced by Flink SQL?
> It can be useful for various use cases:
> 1. Logging exceptions and the erroneous row to a kafka topic
> 2. Ignoring transient exceptions instead of throwing and failing the
> entire job
>
> If there are no such mechanisms may I propose something which looks like
>
> WITH (
>    'connector' = 'jdbc',
>    'url' = 'jdbc:mysql://localhost:3306/mydatabase',
>    'table-name' = 'users'
>    'exception-handler' = <classpath_to_handler_class>);
>
>
> --
> Regards,
>
> Yun Long
>