[jira] [Commented] (BEAM-8542) Add async write to AWS SNS IO & remove retry logic

["Kenneth Knowles (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-8542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16965468#comment-16965468 ] 

Kenneth Knowles commented on BEAM-8542:
---------------------------------------

[~jhalarua] any feedback? Are you the best person to review?

> Add async write to AWS SNS IO & remove retry logic
> --------------------------------------------------
>
>                 Key: BEAM-8542
>                 URL: https://issues.apache.org/jira/browse/BEAM-8542
>             Project: Beam
>          Issue Type: Improvement
>          Components: io-java-aws
>            Reporter: Ajo Thomas
>            Assignee: Ajo Thomas
>            Priority: Major
>
> - While working with SNS IO for one of my work-related projects, I found that the IO uses synchronous publishes during writes. I had a simple mock pipeline where I was reading from a kinesis stream and publishing it to SNS using Beam's SNS IO. For comparison, I also had a lamdba which did the same using asynchronous publishes but was about 5x faster. Changing the SNS IO to use async publishes would improve publish latencies.
>  - SNS IO also has some retry logic which isn't required as SNS clients can handle retries. The retry logic in the SNS client is user-configurable and therefore, an explicit retry logic in SNS IO is not required
> I have a working version of the IO with these changes, will create a PR linking this ticket to it once I get some feedback here.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)