[GitHub] [airflow] xinbinhuang commented on pull request #8618: Add AWS EMR System tests

[GitBox <gi...@apache.org>]

xinbinhuang commented on pull request #8618:
URL: https://github.com/apache/airflow/pull/8618#issuecomment-620947924


   > > I feel like I should add some docs somewhere to indicate how to provide credentials for the system test, but not sure where...
   > 
   > We have documentation about credentials in system tests:
   > https://github.com/apache/airflow/blob/master/TESTING.rst#forwarding-authentication-from-the-host.
   > You can still add a small note in the docstring for the file: tests/providers/amazon/aws/operators/test_emr_system.py
   > 
   > The best solution is to add a marker that will skip the tests if the file with credentials is not available. If you try to run tests for GCP and you do not have credentials then you will see the message below:
   > 
   > ```
   > ========================================================================================= short test summary info ==========================================================================================
   > SKIPPED [1] /Users/kamilbregula/devel/google-airflow/airflow/tests/conftest.py:308: The test requires credential file /files/airflow-breeze-config/keys/gcp_life_sciences.json: <TestCaseFunction test_run_example_dag_function>
   > ============================================================================================ 1 skipped in 1.06s ============================================================================================
   > ```
   
   @mik-laj  Thanks for the suggestion. I took a look at the marker you mention, it needs to copy the credentials in the `files/breeze_config/keys` folder. For AWS system test, I think we can defer the discussion on sourcing credentials later as right now, we are still trying to add more helpers functions for AWS system test (i.e. #8581). So for this PR, I think it will better to use the default credentials for now, and we can change it after we have standardized the workflow.
   
   What do you think?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org