[jira] [Commented] (FLINK-20972) TwoPhaseCommitSinkFunction Output a large amount of EventData

["Yun Gao (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-20972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264765#comment-17264765 ] 

Yun Gao commented on FLINK-20972:
---------------------------------

Hi [~benjobs], which log is outputted a lot~? Do you refer to 

LOG.info(
 "{} - checkpoint {} complete, committing transaction {} from checkpoint {}",
 name(),
 checkpointId,
 pendingTransaction,
 pendingTransactionCheckpointId);

This one ?

> TwoPhaseCommitSinkFunction Output a large amount of EventData
> -------------------------------------------------------------
>
>                 Key: FLINK-20972
>                 URL: https://issues.apache.org/jira/browse/FLINK-20972
>             Project: Flink
>          Issue Type: Improvement
>          Components: API / DataStream
>    Affects Versions: 1.12.0
>         Environment: flink 1.4.0 +
>            Reporter: huajiewang
>            Priority: Minor
>              Labels: easyfix
>         Attachments: Jdbc2PCSinkFunction.scala
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> when TwoPhaseCommitSinkFunctionOutput tigger notifyCheckpointComplete,
> Maybe A large number of EventData will be output (log.info)
> ,which will cause IO bottleneck and disk waste
>  
>  
> Jdbc2PCSinkFunction 1/1 - checkpoint 4 complete, committing transaction TransactionHolder{handle=Transaction(b420c880a951403984f231dd7e33597b,
> ListBuffer(insert into table(field1,field2) value ('11','22') ... ... ), transactionStartTime=1610426158532} from checkpoint 4
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)