[jira] [Created] (NIFI-6724) PutSQS does not check for individual status in batch message

["Martin (Jira)" <ji...@apache.org>]

Martin created NIFI-6724:
----------------------------

             Summary: PutSQS does not check for individual status in batch message
                 Key: NIFI-6724
                 URL: https://issues.apache.org/jira/browse/NIFI-6724
             Project: Apache NiFi
          Issue Type: Bug
          Components: Extensions
    Affects Versions: 1.9.2
         Environment: Docker running in AWS
            Reporter: Martin


PutSQS does not check for batch errors. This could lead to PutSQS reporting success and dropping files even while individual files fails. 

As specified in the [AWSSimpleQueueService API Reference for SendMessageBatch|[https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessageBatch.html]]:
{quote}The result of sending each message is reported individually in the response. Because the batch request can result in a combination of successful and unsuccessful actions, *you should check for batch errors even when the call returns an HTTP status code of 200.*
{quote}
 

We learned this as we did not Base64-encode compressed files (gzip) before sending them to PutSQS. All the compressed files failed to queue in SQS, but SQS acknowledged the batch message and reported status 200 (success). Thus PutSQS also reported success and dropped the files, when they actually failed. 

 

I suggest a simple fix by implementing [SendMessageBatchResult|[https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/sqs/model/SendMessageBatchResult.html]] to check status on individual files. 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)