[GitHub] [airflow] potiuk commented on pull request #24019: Add celery broker configuration example using Amazon SQS

[GitBox <gi...@apache.org>]

potiuk commented on PR #24019:
URL: https://github.com/apache/airflow/pull/24019#issuecomment-1194108239

   Hmm. I thought a bit about it. And following the dicsusion in https://github.com/apache/airflow/pull/25121 - I think we should not add "offficial documenation" to features that our users tested, but we have no automated tests for and do not know ourselves what might be potential risks, problems etc. I think this falls in the same camp.
   
   @BasPH @jedcunningham @feluelle  - WDYT?  
   
   I think @tsugumi-sys that a better approach would be that you (for example) write a short blog post about using SQS as a broker, and describe your experiences it - and then we can even add it to "Airlfow" publication on Medium and even link it from the Ecosystem page of Airflow. https://airflow.apache.org/ecosystem/ 
   
   I think adding it to the official docs in experimental phase when we do not have any tests (nor intention to add them) is IMHO something we should not do - even if we add "Experimental" - because "experimental" means that we are planning to "run" some experiment and turn it into a "full feature" (or drop) when we finish the experiment. This is what happens with:
   
   * MsSQL support as meta-datada DB
   * ARM64 support for Docker Image.
   
   In both cases we already have tests in CI and we actively plan to act and gather observations/. For example I think we might beinclined to drop MsSQL if the Survey results will not show significant uptake in MsSQL usage after it's been officially "experimentally" supported in 2.3 https://airflow.apache.org/blog/airflow-survey-2022/  shows a very little usage so far. 
   
   Adding information that SQS "works" in the official docs is some kind of statement we make to our users and they might have certain expectations if they follow it.
   
   I think I'd be only inclined to add "experimental" SQS support if we have some tests in our CI making sure that it works (BTW. We have such - simple but regularly running tests for RabbitMQ and Redis).
   
   Others - WDYT?
   
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org