You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "Vijaya Bhaskar V (Jira)" <ji...@apache.org> on 2023/07/22 08:37:00 UTC

[jira] [Created] (FLINK-32645) Flink pulsar sink is having poor performance

Vijaya Bhaskar V created FLINK-32645:
----------------------------------------

             Summary: Flink pulsar sink is having poor performance
                 Key: FLINK-32645
                 URL: https://issues.apache.org/jira/browse/FLINK-32645
             Project: Flink
          Issue Type: Bug
          Components: Connectors / Pulsar
    Affects Versions: 1.16.2
         Environment: !Screenshot 2023-07-22 at 1.59.42 PM.png!!Screenshot 2023-07-22 at 2.03.53 PM.png!

 
            Reporter: Vijaya Bhaskar V
         Attachments: Screenshot 2023-07-22 at 1.59.42 PM.png, Screenshot 2023-07-22 at 2.03.53 PM.png

Found following issue with flink pulsar sink:

 

Flink pulsar sink is always waiting while enqueueing the message and making the task slot busy no matter how many free slots we provide. Attached the screen shot of the same

Just sending messages of less rate 8k msg/sec and stand alone flink job with discarding sink is able to receive full rate if 8K msg/sec

Where as pulsar sink was consuming only upto 2K msg/sec and the sink is always busy waiting. Snapshot of thread dump attached.

Also snap shot of flink stream graph attached

 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)