[GitHub] [airflow] bhirsz edited a comment on issue #22437: Migrate Alibaba example DAGs to new design

[GitBox <gi...@apache.org>]

bhirsz edited a comment on issue #22437:
URL: https://github.com/apache/airflow/issues/22437#issuecomment-1078892808


   I took a quick look into your examples - there are short so they could also make good example (since our PR is bloated with all changes required). You don't have any system tests - which in old design were usually pytest classes that were invoking example dags. In a new design it's somehow similar (in a sense that we have a file, that works both as a system test and example).  
   From my POV action points include:
   1. Moving example dags under tests/system/providers/alibaba
   3. Create tests/system/providers/README.md explaining how to run your examples (if there is need for preconfigured environment, created account etc). 
   4. If your DAG creates any object, that shall be deleted - add trigger rule ALL_DONE to operators doing the removal (that's to ensure it actually happens). 
   5. Append watcher at the end of DAG:
   ```
   from tests.system.utils import get_test_run  # noqa: E402
   
   # Needed to run the example DAG with pytest (see: tests/system/README.md#run_via_pytest)
   test_run = get_test_run(dag)
   ```
   6. Append test method at the end of file:
   ```
   from tests.system.utils import get_test_run  # noqa: E402
   
   # Needed to run the example DAG with pytest (see: tests/system/README.md#run_via_pytest)
   test_run = get_test_run(dag)
   
   ```
   7. Update documentation pointing to your examples. Whenever you used tags like "# [START howto_operator_oss_bucket]" you need update path in rst includes to new one (under system/tests). For example here https://github.com/apache/airflow/blob/main/docs/apache-airflow-providers-alibaba/operators/oss.rst . 
   
   System tests should be runnable using following command (use Breeze to ensure you have all dependencies):
   ```
   pytest --system alibaba tests/system
   ```
   I don't know your package and what's it's doing so I don't know if there are extra steps required for preparing environment. It would be best to describe in a way that would allow other users easily run it and of course enable community to include your system tests in CI checks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org