You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Ajo Thomas (Jira)" <ji...@apache.org> on 2019/11/01 15:20:00 UTC
[jira] [Created] (BEAM-8542) Add async write to AWS SNS IO & remove
retry logic
Ajo Thomas created BEAM-8542:
--------------------------------
Summary: Add async write to AWS SNS IO & remove retry logic
Key: BEAM-8542
URL: https://issues.apache.org/jira/browse/BEAM-8542
Project: Beam
Issue Type: Improvement
Components: io-java-aws
Reporter: Ajo Thomas
Assignee: Ajo Thomas
- While working with SNS IO for one of my work-related projects, I found that the IO uses synchronous publishes during writes. I had a simple mock pipeline where I was reading from a kinesis stream and publishing it to SNS using Beam's SNS IO. For comparison, I also had a lamdba which did the same using asynchronous publishes but was about 5x faster. Changing the SNS IO to use async publishes would improve publish latencies.
- SNS IO also has some retry logic which isn't required as SNS clients can handle retries. The retry logic in the SNS client is user-configurable and therefore, an explicit retry logic in SNS IO is not required
--
This message was sent by Atlassian Jira
(v8.3.4#803005)