[GitHub] [airflow] dbarrundiag opened a new issue, #25138: SQSPublishOperator should allow sending messages to a FIFO Queue

[GitBox <gi...@apache.org>]

dbarrundiag opened a new issue, #25138:
URL: https://github.com/apache/airflow/issues/25138

   ### Apache Airflow Provider(s)
   
   amazon
   
   ### Versions of Apache Airflow Providers
   
   `apache-airflow-providers-amazon==4.1.0`
   
   ### Apache Airflow version
   
   2.2.2
   
   ### Operating System
   
   Amazon Linux 2
   
   ### Deployment
   
   MWAA
   
   ### Deployment details
   
   _No response_
   
   ### What happened
   
   The current state of the [SQSPublishOperator](https://github.com/apache/airflow/blob/main/airflow/providers/amazon/aws/operators/sqs.py) does not support FIFO queues because:
   From the Boto3 documentation, a FIFO queue requires a MessageGroupId parameter -> [SQS.Client.send_message](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sqs.html#SQS.Client.send_message)
   The [SQSHook](https://airflow.apache.org/docs/apache-airflow-providers-amazon/2.4.0/_modules/airflow/providers/amazon/aws/hooks/sqs.html#SQSHook) calls the boto3 client with just these parameters:
   ```
   .send_message(
               QueueUrl=queue_url,
               MessageBody=message_body,
               DelaySeconds=delay_seconds,
               MessageAttributes=message_attributes or {},
   
           )
   ```
   so if `queue_url` happens to be a FIFO queue, then that API call will fail and therefore the Airflow Operator will fail too.
   
   ### What you think should happen instead
   
   The `SQSPublishOperator` should support FIFO queues by accepting the `MessageGroupId` as a parameter.
   
   ### How to reproduce
   
   1. Setup a FIFO SQS queue
   2. Setup an Airflow operator:
   ```
   task1 = SQSPublishOperator(
   task_id = task1,
   sqs_queue = "url",
   message_content = "Message string",
   delay_seconds = 0,
   )
   ```
   
   ### Anything else
   
   N/A
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk closed issue #25138: SQSPublishOperator should allow sending messages to a FIFO Queue

[GitBox <gi...@apache.org>]

potiuk closed issue #25138: SQSPublishOperator should allow sending messages to a FIFO Queue
URL: https://github.com/apache/airflow/issues/25138


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #25138: SQSPublishOperator should allow sending messages to a FIFO Queue

[GitBox <gi...@apache.org>]

potiuk commented on issue #25138:
URL: https://github.com/apache/airflow/issues/25138#issuecomment-1188232560

   Assigned you


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org