You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2020/05/18 08:46:21 UTC
[airflow] tag master-nightly updated (1bdaca9 -> 2121f49)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to tag master-nightly
in repository https://gitbox.apache.org/repos/asf/airflow.git.
*** WARNING: tag master-nightly was modified! ***
from 1bdaca9 (commit)
to 2121f49 (commit)
from 1bdaca9 Remove WTforms from setup.py (#8522)
add e11a838 Optimize GitLab CI configuration (#8499)
add ffcbb22 Move some tests to quarantine (#8511)
add 912aa4b Added GoogleDisplayVideo360DownloadLineItemsOperator (#8174)
add 72ddc94 Pass location using parmamter in Dataflow integration (#8382)
add d883ff4 Add Simply Business to the documentation-list of "companies using Airflow" (#8516)
add 9eaacbd Development PR template target issue if exists (#8524)
add eb7255f [AIRFLOW-4357] Fix SVG tooltip positioning with custom scripting (#8269)
add fc5390d Add Zerodha to Airflow users list (#8532)
add 16903ba [AIRFLOW-8474]: Adding possibility to get job_id from Databricks run (#8475)
add b6434de [AIRFLOW-7111] Add generate_presigned_url method to S3Hook (#8441)
add 5a47e80 fix help message display for dags test subcommand (#8552)
add 1ea9fa7 Fix --forward-credentials flag in Breeze (#8554)
add 14b22e6 Add hook and operator for Google Cloud Life Sciences (#8481)
add 0de597f The CRON job now is working and triggers builds on DockerHub (#8549)
add 3f8f4ad Add test to guard against command arg help message regression (#8561)
add 37fdfa9 [AIRFLOW-6281] Create guide for GCS to GCS transfer operators (#8442)
add 7ea66a1 Add example DAG for ECSOperator (#8452)
add da368e6 This test (test_mark_success_on_success_callback) fails often (#8563)
add 4799d5d Retry initialization of environment in case it fails (#8555)
add 3c4ca62 Building backport packages is now done inside Breeze container (#8558)
add df19d9b Fixed SOURCE_BRANCH set by tag names (#8562)
add e8d0f8f Improve idempodency in CloudDataTransferServiceCreateJobOperator (#8430)
add 3237c7e [AIRFLOW-5850] Capture task logs in DockerSwarmOperator (#6552)
add a8bcc1a AIRFLOW-6062 Watch worker pods from all namespaces (#8546)
add 9d7dab4 Monkey patch greenlet celery pools (#8559)
add 9833ed4 File should be specified in docker compose before command (#8569)
add 2fa2c4c Add test_localtaskjob_maintain_heart_rate to quarantine (#8566)
add 74bc316 [AIRFLOW-4438] Add Gzip compression to S3_hook (#8571)
add 5a864f0 User-friendly error messages when the configuration is incorrect (#8463)
add d70303c Bash command for production image (#8579)
add 4103ad4 Add Local Filesystem Secret Backend (#8436)
add bcbd888 Fix json string escape in tree view (#8551)
add c1d4a23 Update documentation with clarification about v1-10-stable (#8588)
add a48d38d Pin Pylint to 2.4.4 (#8592)
add 6450834 [AIRFLOW-6796] Clean up DAG serializations based on last_updated (#7424)
add bbdec5e Remove WTforms from setup.py (#8590)
add f69201d Fix typo in error message - systems -> system (#8584)
add 649935e [AIRFLOW-8472]: `PATCH` for Databricks hook `_do_api_call` (#8473)
add 0a1de16 Stop DockerSwarmOperator from pulling Docker images (#8533)
add a8299d0 Add back-compat secrets to TestMovingCoreToContrib (#8593)
add 379a884 fix: aws hook should work without conn id (#8534)
add 0248097 Fix example for Local Filesystem Secrets Backend (#8597)
add 00bbf32 Fixed optimistions of non-py-code builds (#8601)
add 1291ded Chown should work now when building the documentation (#8600)
add c1fb282 Refactor BigQueryHook dataset operations (#8477)
add 992a24c Split and improve BigQuery example DAG (#8529)
add 6c6d661 Allow to define custom XCom class (#8560)
add 830a4e2 Pin Flask-Appbuilder to 2.3.2 (#8602)
add 522d591 Fix sphinx add_stylesheet deprecation warning (#8627)
add 249e80b Add http system test (#8591)
add a28c66f [AIRFLOW-4734] Upsert functionality for PostgresHook.insert_rows() (#8625)
add 4a1d71d Fix the process of requirements generations (#8648)
add b185b36 Reduce response payload size of /dag_stats and /task_stats (#8633)
add 4421f01 Improve template capabilities of EMR job and step operators (#8572)
add 6560f29 Enhanced documentation around Cluster Policy (#8661)
add 511d98e [AIRFLOW-4363] Fix JSON encoding error (#8287)
add ce50538 Add check for pre-2.0 style hostname_callable config value (#8637)
add 0a7b500 Fix displaying Executor Class Name in "Base Job" table (#8679)
add d92e848 Persist start/end date and duration for DummyOperator Task Instance (#8663)
add 0954140 Ensure "started"/"ended" in tooltips are not shown if job not started (#8667)
add 19ac45a Add support for fetching logs from running pods (#8626)
add 1100cea Remove _get_pretty_exception_message in PrestoHook
add 62796b9 Improve tutorial - Include all imports statements (#8670)
add dd6a7bc Group Google services in one section (#8623)
add ac59735 Refactor test_variable_command.py (#8535)
add bc45fa6 Add system test and docs for Facebook Ads operators (#8503)
add 0b598a2 Fix connection add/edit for spark (#8685)
add ffbbbfc Sort connection type list in add/edit page alphabetically (#8692)
add d8cb0b5 Support k8s auth method in Vault Secrets provider (#8640)
add 67caae0 Add system test for gcs_to_bigquery (#8556)
add aec768b [AIRFLOW-7008] Add perf kit with common used decorators/contexts (#7650)
add c3a46b9 Invalid output in test_variable assertion (#8698)
add 5ddc458 Change provider:GCP to provider:Google for Labeler Bot (#8697)
add caa60b1 Remove config side effects from tests (#8607)
add 923f423 Check consistency between the reference list and howto directory (#8690)
add b31ad51 Prevent clickable sorting on non sortable columns in TI view (#8681)
add 6600e47 Import Connection directly from multiprocessing.connection. (#8711)
add 2c92a29 Fix typo in Google Display & Video 360 guide
add 41b4c27 Carefully parse warning messages when building documentation (#8693)
add 8d6f1aa Support num_retries field in env var for GCP connection (#8700)
add c717d12 Add __repr__ for DagTag so tags display properly in /dagmodel/show (#8719)
add 487b5cc Add guide for Apache Spark operators (#8305)
add 520aeed Fix pickling failure when spawning processes (#8671)
add 25ee421 Support all RuntimeEnvironment parameters in DataflowTemplatedJobStartOperator (#8531)
add d923b5b Add jinja template test for AirflowVersion (#8505)
add e673413 Avoid loading executors in jobs (#7888)
add 3437beaa Optimize count query on /home (#8729)
add 336aa27 Correctly deserialize dagrun_timeout field on DAGs (#8735)
add 2e9ef45 Stop Stalebot on Github issues (#8738)
add fd6e057 Make loading plugins from entrypoint fault-tolerant (#8732)
add bd29ee3 Ensure test_logging_config.test_reload_module works in spawn mode. (#8741)
add d15839d Latest debian-buster release broke image build (#8758)
add ff5b701 Add google_api_to_s3_transfer example dags and system tests (#8581)
add 7c04604 Add google_api_to_s3_transfer docs howto link (#8761)
add 723c52c Add documentation for SpannerDeployInstanceOperator (#8750)
add 6e4f5fa [AIRFLOW-4568]The ExternalTaskSensor should be configurable to raise an Airflow Exception in case the poked external task reaches a disallowed state, such as f.i. failed (#8509)
add b7566e1 Add SQL query tracking for pytest (#8754)
add 58aefb2 Added SDFtoGCSOperator (#8740)
add b37ce29 Patch Pool.DEFAULT_POOL_NAME in BaseOperator (#8587)
add 2bd3e76 Support same short flags for `create user` as 1.10 did for `user_create` (#8763)
add 09770e4 Add WorldRemit as Airflow user (#8786)
add a091c1f fix typing errors reported by dmypy (#8773)
add 42c5975 Update example SingularityOperator DAG (#8790)
add 791d1a7 Backport packages are renamed to include backport in their name (#8767)
add 100f530 Fixed test-target command (#8795)
add db1b51d Make celery worker_prefetch_multiplier configurable (#8695)
add bc19778 [AIP-31] Implement XComArg to pass output from one operator to the next (#8652)
add 7506c73 Add default `conf` parameter to Spark JDBC Hook (#8787)
add 5e1c33a Fix docs on creating CustomOperator (#8678)
add 21cc7d7 Document default timeout value for SSHOperator (#8744)
add cd635dd [AIRFLOW-5906] Add authenticator parameter to snowflake_hook (#8642)
add c7788a6 Add imap_attachment_to_s3 example dag and system test (#8669)
add a715aa6 Correctly store non-default Nones in serialized tasks/dags (#8772)
add 280f1f0 Correctly restore upstream_task_ids when deserializing Operators (#8775)
add cbebed2 Allow passing backend_kwargs to AWS SSM client (#8802)
add 79ef8be Added Upload Multiple Entity Read Files to specified big query dataset (#8610)
add e1cc17e Remove old airflow logger causing side effects in tests (#8746)
add 9bb91ef Add comments to breeze scripts (#8797)
add 493b685 Add separate example DAGs and system tests for google cloud speech (#8778)
add bed1995 Avoid color info in response of /dag_stats & /task_stats (#8742)
add b59adab Support cron presets in date_range function (#7777)
add 5f3774a [AIRFLOW-6921] Fetch celery states in bulk (#7542)
add d5c4001 Useful help information in test-target and docker-compose commands (#8796)
add a6434a5 Fix bash command in performance test dag (#8812)
add 0c3db84 [AIRFLOW-7068] Create EC2 Hook, Operator and Sensor (#7731)
add 5ae76d8 Option to set end_date for performance testing dag. (#8817)
add 2ec0130 [AIRFLOW-4549] Allow skipped tasks to satisfy wait_for_downstream (#7735)
add 1fb9f07 Synchronize extras between airflow and providers (#8819)
add d590e5e Add option to propagate tags in ECSOperator (#8811)
add f410d64 Use fork when test relies on mock.patch in parent process. (#8794)
add 3ad4f96 [AIRFLOW-1156] BugFix: Unpausing a DAG with catchup=False creates an extra DAG run (#8776)
add 4375607 Fix typo. 'zobmies' => 'zombies'. (#8832)
add 78a48db Add support for non-default orientation in `dag show` command (#8834)
add 7533378 Access function to be pickled as attribute, not method, to avoid error. (#8823)
add 1d12c34 Refactor BigQuery check operators (#8813)
add 4b06fde Fix Flake8 errors (#8841)
add 6911dfe Fix template fields in Google operators (#8840)
add 01db738 Azure storage 0.37.0 is not installable any more (#8833)
add 578fc51 [AIRFLOW-4543] Update slack operator to support slackclient v2 (#5519)
add 7236862 [AIRFLOW-2310] Enable AWS Glue Job Integration (#6007)
add 8b54919 Refactor BigQuery hook methods to use python library (#8631)
add 7d69987 Remove duplicate code from perf_kit (#8843)
add e1e833b Update GoogleBaseHook to not follow 308 and use 60s timeout (#8816)
add 8a94d18 Fix Environment Variable in perf/scheduler_dag_execution_timing.py (#8847)
add ed3f513 Correctly pass sleep time from AWSAthenaOperator down to the hook. (#8845)
add f1dc2e0 The librabbitmq library stopped installing for python3.7 (#8853)
add c3af681 Convert tests/jobs/test_base_job.py to pytest (#8856)
add 81fb9d6 Add metric for monitoring email notification failures (#8771)
add 2878f17 Relax Flask-Appbuilder version to ~=2.3.4 (#8857)
add e61b9bb Add AWS EMR System tests (#8618)
add fc862a3 Do not create a separate process for one task in CeleryExecutor (#8855)
add 961c710 Make Custom XCom backend a subsection of XCom docs (#8869)
add fe42191 Don't use ProcessorAgent to test ProcessorManager (#8871)
add 4813b94 Create log file w/abs path so tests pass on MacOS (#8820)
add 35c523f Fix list formatting of plugins doc. (#8873)
add 85bbab2 Add EMR operators howto docs (#8863)
add f82ad45 Fix KubernetesPodOperator pod name length validation (#8829)
add 92585ca Added automated release notes generation for backport operators (#8807)
add 82de6f7 Spend less time waiting for DagFileProcessor processes to complete (#8814)
add a3a4bac JIRA and Github issues explanation (#8539)
add f4edd90 Speed up TestAwsLambdaHook by not actually running a function (#8882)
add 15273f0 Check for same task instead of Equality to detect Duplicate Tasks (#8828)
add a3a3411 Fix master failing on generating requirements (#8885)
add f3521fb Regenerate readme files for backport package release (#8886)
add f6d5917 Updated docs for experimental API /dags/<DAG_ID>/dag_runs (#8800)
add 707bb0c [AIRFLOW-6535] Add AirflowFailException to fail without any retry (#7133)
add a546a10 Add Snowflake system test (#8422)
add 8985df0 Monitor pods by labels instead of names (#6377)
add ff342fc Added SalesforceHook missing method to return only dataframe (#8565) (#8644)
add 12c5e5d Prepare release candidate for backport packages (#8891)
add 2121f49 Avoid failure on transient requirements in CI image (#8892)
No new revisions were added by this update.
Summary of changes:
.github/PULL_REQUEST_TEMPLATE.md | 1 +
.github/boring-cyborg.yml | 9 +-
.github/stale.yml | 2 +-
.github/workflows/ci.yml | 441 ++++---
.pre-commit-config.yaml | 16 +-
.rat-excludes | 1 +
BREEZE.rst | 123 +-
CI.rst | 130 +++
CONTRIBUTING.rst | 81 +-
Dockerfile.ci | 4 +-
IMAGES.rst | 4 +-
INSTALL | 15 +-
LOCAL_VIRTUALENV.rst | 2 +-
README.md | 3 +
TESTING.rst | 77 +-
UPDATING.md | 121 ++
airflow/api/common/experimental/mark_tasks.py | 38 +-
airflow/cli/cli_parser.py | 22 +-
airflow/cli/commands/celery_command.py | 10 +-
airflow/cli/commands/variable_command.py | 3 +-
airflow/config_templates/config.yml | 36 +-
airflow/config_templates/default_airflow.cfg | 26 +-
airflow/config_templates/default_celery.py | 2 +-
airflow/config_templates/default_test.cfg | 2 +-
airflow/configuration.py | 102 +-
airflow/contrib/operators/bigquery_operator.py | 8 +-
airflow/example_dags/example_complex.py | 2 +-
.../example_external_task_marker_dag.py | 14 +
airflow/example_dags/tutorial.py | 2 +-
airflow/exceptions.py | 45 +
airflow/executors/celery_executor.py | 239 ++--
airflow/executors/executor_loader.py | 13 +-
airflow/executors/kubernetes_executor.py | 45 +-
airflow/hooks/dbapi_hook.py | 54 +-
airflow/jobs/backfill_job.py | 17 +-
airflow/jobs/base_job.py | 3 +-
airflow/jobs/scheduler_job.py | 62 +-
airflow/kubernetes/pod_generator.py | 23 +
airflow/kubernetes/pod_launcher.py | 20 +-
airflow/models/baseoperator.py | 49 +-
airflow/models/connection.py | 1 +
airflow/models/dag.py | 5 +-
airflow/models/dagcode.py | 10 +-
airflow/models/serialized_dag.py | 26 +-
airflow/models/taskinstance.py | 116 +-
airflow/models/xcom.py | 20 +-
airflow/models/xcom_arg.py | 149 +++
airflow/operators/latest_only_operator.py | 4 +-
airflow/plugins_manager.py | 19 +-
.../amazon/PROVIDERS_CHANGES_2020.05.20.md | 87 ++
airflow/providers/amazon/README.md | 332 ++++++
.../amazon/aws/example_dags/example_ecs_fargate.py | 81 ++
.../example_emr_job_flow_automatic_steps.py | 24 +-
.../example_emr_job_flow_manual_steps.py | 24 +-
.../example_google_api_to_s3_transfer_advanced.py | 132 +++
.../example_google_api_to_s3_transfer_basic.py | 56 +
.../example_dags/example_imap_attachment_to_s3.py | 53 +
airflow/providers/amazon/aws/hooks/base_aws.py | 13 +-
airflow/providers/amazon/aws/hooks/ec2.py | 88 ++
airflow/providers/amazon/aws/hooks/glue.py | 178 +++
.../providers/amazon/aws/hooks/lambda_function.py | 2 +-
airflow/providers/amazon/aws/hooks/s3.py | 34 +-
airflow/providers/amazon/aws/operators/athena.py | 2 +-
.../amazon/aws/operators/ec2_start_instance.py | 72 ++
.../amazon/aws/operators/ec2_stop_instance.py | 72 ++
airflow/providers/amazon/aws/operators/ecs.py | 7 +-
.../amazon/aws/operators/emr_add_steps.py | 9 +-
.../amazon/aws/operators/emr_create_job_flow.py | 14 +-
airflow/providers/amazon/aws/operators/glue.py | 114 ++
.../amazon/aws/secrets/systems_manager.py | 13 +-
.../amazon/aws/sensors/ec2_instance_state.py | 70 ++
airflow/providers/amazon/aws/sensors/glue.py | 65 ++
.../cassandra/PROVIDERS_CHANGES_2020.05.20.md | 13 +
airflow/providers/apache/cassandra/README.md | 107 ++
.../apache/druid/PROVIDERS_CHANGES_2020.05.20.md | 14 +
airflow/providers/apache/druid/README.md | 127 ++
.../apache/hdfs/PROVIDERS_CHANGES_2020.05.20.md | 15 +
airflow/providers/apache/hdfs/README.md | 112 ++
.../apache/hive/PROVIDERS_CHANGES_2020.05.20.md | 21 +
airflow/providers/apache/hive/README.md | 161 +++
.../apache/livy/PROVIDERS_CHANGES_2020.05.20.md | 11 +
airflow/providers/apache/livy/README.md | 127 ++
.../apache/pig/PROVIDERS_CHANGES_2020.05.20.md | 15 +
airflow/providers/apache/pig/README.md | 102 ++
.../apache/pinot/PROVIDERS_CHANGES_2020.05.20.md | 13 +
airflow/providers/apache/pinot/README.md | 94 ++
.../apache/spark/PROVIDERS_CHANGES_2020.05.20.md | 19 +
airflow/providers/apache/spark/README.md | 117 ++
.../apache/spark/example_dags}/__init__.py | 0
.../apache/spark/example_dags/example_spark_dag.py | 76 ++
airflow/providers/apache/spark/hooks/spark_jdbc.py | 2 +-
.../apache/sqoop/PROVIDERS_CHANGES_2020.05.20.md | 12 +
airflow/providers/apache/sqoop/README.md | 99 ++
.../celery/PROVIDERS_CHANGES_2020.05.20.md | 10 +
airflow/providers/celery/README.md | 92 ++
.../cloudant/PROVIDERS_CHANGES_2020.05.20.md | 11 +
airflow/providers/cloudant/README.md | 91 ++
.../kubernetes/PROVIDERS_CHANGES_2020.05.20.md | 24 +
airflow/providers/cncf/kubernetes/README.md | 139 +++
.../cncf/kubernetes/operators/kubernetes_pod.py | 232 ++--
.../databricks/PROVIDERS_CHANGES_2020.05.20.md | 16 +
airflow/providers/databricks/README.md | 111 ++
airflow/providers/databricks/hooks/databricks.py | 14 +
.../datadog/PROVIDERS_CHANGES_2020.05.20.md | 12 +
airflow/providers/datadog/README.md | 105 ++
.../dingding/PROVIDERS_CHANGES_2020.05.20.md | 16 +
airflow/providers/dingding/README.md | 119 ++
.../discord/PROVIDERS_CHANGES_2020.05.20.md | 12 +
airflow/providers/discord/README.md | 115 ++
.../docker/PROVIDERS_CHANGES_2020.05.20.md | 22 +
airflow/providers/docker/README.md | 117 ++
airflow/providers/docker/operators/docker.py | 31 +-
airflow/providers/docker/operators/docker_swarm.py | 72 +-
.../elasticsearch/PROVIDERS_CHANGES_2020.05.20.md | 9 +
airflow/providers/elasticsearch/README.md | 82 ++
.../email/PROVIDERS_CHANGES_2020.05.20.md | 11 +
airflow/providers/email/README.md | 85 ++
.../exasol/PROVIDERS_CHANGES_2020.05.20.md | 9 +
airflow/providers/exasol/README.md | 103 ++
.../facebook/PROVIDERS_CHANGES_2020.05.20.md | 10 +
airflow/providers/facebook/README.md | 90 ++
airflow/providers/facebook/ads/hooks/ads.py | 15 +-
.../providers/ftp/PROVIDERS_CHANGES_2020.05.20.md | 13 +
airflow/providers/ftp/README.md | 101 ++
airflow/providers/google/BACKPORT_README.md | 474 --------
.../google/PROVIDERS_CHANGES_2020.05.20.md | 159 +++
airflow/providers/google/README.md | 672 +++++++++++
.../google/cloud/example_dags/example_bigquery.py | 341 ------
.../example_dags/example_bigquery_operations.py | 204 ++++
.../cloud/example_dags/example_bigquery_queries.py | 159 +++
.../example_dags/example_bigquery_to_bigquery.py | 70 ++
.../cloud/example_dags/example_bigquery_to_gcs.py | 73 ++
.../example_dags/example_bigquery_transfer.py | 82 ++
.../example_cloud_storage_transfer_service_aws.py | 7 +-
.../cloud/example_dags/example_datacatalog.py | 2 +-
.../google/cloud/example_dags/example_dataflow.py | 7 +-
.../example_dags/example_facebook_ads_to_gcs.py} | 46 +-
.../google/cloud/example_dags/example_gcs.py | 4 +-
...ple_gcs_to_bq.py => example_gcs_to_bigquery.py} | 37 +-
.../cloud/example_dags/example_gcs_to_gcs.py | 97 +-
.../cloud/example_dags/example_life_sciences.py | 100 ++
.../cloud/example_dags/example_presto_to_gcs.py | 2 +-
...example_speech.py => example_speech_to_text.py} | 51 +-
...example_speech.py => example_text_to_speech.py} | 57 +-
...ample_speech.py => example_translate_speech.py} | 46 +-
airflow/providers/google/cloud/hooks/bigquery.py | 1225 +++++++++++---------
.../cloud/hooks/cloud_storage_transfer_service.py | 80 +-
airflow/providers/google/cloud/hooks/dataflow.py | 161 +--
airflow/providers/google/cloud/hooks/functions.py | 2 +-
.../providers/google/cloud/hooks/life_sciences.py | 150 +++
airflow/providers/google/cloud/operators/automl.py | 2 +-
.../providers/google/cloud/operators/bigquery.py | 292 +++--
.../operators/cloud_storage_transfer_service.py | 13 +-
.../providers/google/cloud/operators/dataflow.py | 77 +-
.../operators/facebook_ads_to_gcs.py} | 4 +
.../google/cloud/operators/life_sciences.py | 74 ++
.../providers/google/cloud/operators/spanner.py | 4 +
.../providers/google/common/hooks/base_google.py | 39 +-
.../facebook_ads_to_gcs/example_dags/__init__.py | 17 -
.../example_dags/example_display_video.py | 98 +-
.../marketing_platform/hooks/display_video.py | 106 ++
.../marketing_platform/operators/display_video.py | 249 +++-
.../marketing_platform/sensors/display_video.py | 59 +-
.../providers/grpc/PROVIDERS_CHANGES_2020.05.20.md | 13 +
airflow/providers/grpc/README.md | 107 ++
.../hashicorp/PROVIDERS_CHANGES_2020.05.20.md | 16 +
airflow/providers/hashicorp/README.md | 112 ++
airflow/providers/hashicorp/secrets/vault.py | 20 +-
.../providers/http/PROVIDERS_CHANGES_2020.05.20.md | 20 +
airflow/providers/http/README.md | 120 ++
.../providers/http/example_dags/example_http.py | 34 +-
.../providers/imap/PROVIDERS_CHANGES_2020.05.20.md | 13 +
airflow/providers/imap/README.md | 99 ++
.../providers/jdbc/PROVIDERS_CHANGES_2020.05.20.md | 12 +
airflow/providers/jdbc/README.md | 106 ++
.../jenkins/PROVIDERS_CHANGES_2020.05.20.md | 17 +
airflow/providers/jenkins/README.md | 111 ++
.../providers/jira/PROVIDERS_CHANGES_2020.05.20.md | 12 +
airflow/providers/jira/README.md | 120 ++
.../azure/PROVIDERS_CHANGES_2020.05.20.md | 23 +
airflow/providers/microsoft/azure/README.md | 189 +++
.../mssql/PROVIDERS_CHANGES_2020.05.20.md | 12 +
airflow/providers/microsoft/mssql/README.md | 122 ++
.../winrm/PROVIDERS_CHANGES_2020.05.20.md | 14 +
airflow/providers/microsoft/winrm/README.md | 108 ++
.../mongo/PROVIDERS_CHANGES_2020.05.20.md | 11 +
airflow/providers/mongo/README.md | 105 ++
.../mysql/PROVIDERS_CHANGES_2020.05.20.md | 20 +
airflow/providers/mysql/README.md | 142 +++
.../providers/odbc/PROVIDERS_CHANGES_2020.05.20.md | 10 +
airflow/providers/odbc/README.md | 90 ++
.../openfaas/PROVIDERS_CHANGES_2020.05.20.md | 11 +
airflow/providers/openfaas/README.md | 84 ++
.../opsgenie/PROVIDERS_CHANGES_2020.05.20.md | 14 +
airflow/providers/opsgenie/README.md | 117 ++
.../oracle/PROVIDERS_CHANGES_2020.05.20.md | 12 +
airflow/providers/oracle/README.md | 107 ++
.../pagerduty/PROVIDERS_CHANGES_2020.05.20.md | 11 +
airflow/providers/pagerduty/README.md | 91 ++
.../papermill/PROVIDERS_CHANGES_2020.05.20.md | 14 +
airflow/providers/papermill/README.md | 96 ++
.../postgres/PROVIDERS_CHANGES_2020.05.20.md | 15 +
airflow/providers/postgres/README.md | 125 ++
airflow/providers/postgres/hooks/postgres.py | 54 +
.../presto/PROVIDERS_CHANGES_2020.05.20.md | 14 +
airflow/providers/presto/README.md | 94 ++
airflow/providers/presto/hooks/presto.py | 20 +-
.../qubole/PROVIDERS_CHANGES_2020.05.20.md | 22 +
airflow/providers/qubole/README.md | 134 +++
airflow/providers/qubole/operators/qubole_check.py | 11 +-
.../redis/PROVIDERS_CHANGES_2020.05.20.md | 12 +
airflow/providers/redis/README.md | 120 ++
.../salesforce/PROVIDERS_CHANGES_2020.05.20.md | 19 +
airflow/providers/salesforce/README.md | 133 +++
airflow/providers/salesforce/hooks/salesforce.py | 76 +-
.../samba/PROVIDERS_CHANGES_2020.05.20.md | 12 +
airflow/providers/samba/README.md | 92 ++
.../segment/PROVIDERS_CHANGES_2020.05.20.md | 12 +
airflow/providers/segment/README.md | 106 ++
.../providers/sftp/PROVIDERS_CHANGES_2020.05.20.md | 14 +
airflow/providers/sftp/README.md | 139 +++
.../singularity/PROVIDERS_CHANGES_2020.05.20.md | 11 +
airflow/providers/singularity/README.md | 92 ++
.../example_dags/example_singularity_operator.py | 59 +-
.../slack/PROVIDERS_CHANGES_2020.05.20.md | 14 +
airflow/providers/slack/README.md | 127 ++
airflow/providers/slack/hooks/slack.py | 81 +-
airflow/providers/slack/operators/slack.py | 35 +-
airflow/providers/snowflake/BACKPORT_README.md | 47 +
.../snowflake/PROVIDERS_CHANGES_2020.05.20.md | 15 +
airflow/providers/snowflake/README.md | 111 ++
.../providers/snowflake/example_dags}/__init__.py | 0
.../snowflake/example_dags/example_snowflake.py | 81 ++
airflow/providers/snowflake/hooks/snowflake.py | 8 +-
.../snowflake/operators/s3_to_snowflake.py | 2 +-
airflow/providers/snowflake/operators/snowflake.py | 13 +-
.../sqlite/PROVIDERS_CHANGES_2020.05.20.md | 11 +
airflow/providers/sqlite/README.md | 98 ++
.../providers/ssh/PROVIDERS_CHANGES_2020.05.20.md | 14 +
airflow/providers/ssh/README.md | 110 ++
airflow/providers/ssh/operators/ssh.py | 2 +-
.../vertica/PROVIDERS_CHANGES_2020.05.20.md | 11 +
airflow/providers/vertica/README.md | 105 ++
.../yandex/PROVIDERS_CHANGES_2020.05.20.md | 13 +
airflow/providers/yandex/README.md | 113 ++
.../zendesk/PROVIDERS_CHANGES_2020.05.20.md | 12 +
airflow/providers/zendesk/README.md | 92 ++
airflow/secrets/__init__.py | 19 +-
airflow/secrets/local_filesystem.py | 266 +++++
airflow/sensors/external_task_sensor.py | 68 +-
airflow/serialization/serialized_objects.py | 29 +-
airflow/settings.py | 5 +-
airflow/stats.py | 35 +-
airflow/task/task_runner/__init__.py | 12 +-
airflow/utils/cli.py | 19 +-
airflow/utils/code_utils.py | 27 +
airflow/utils/dag_processing.py | 240 ++--
airflow/utils/dates.py | 2 +
airflow/utils/db.py | 13 +-
airflow/utils/dot_renderer.py | 3 +-
airflow/utils/email.py | 5 +-
airflow/utils/log/file_processor_handler.py | 13 +-
airflow/utils/log/file_task_handler.py | 24 +
airflow/utils/log/s3_task_handler.py | 2 +-
.../utils/mixins.py | 28 +-
airflow/utils/net.py | 20 +-
.../utils/platform.py | 39 +-
airflow/www/forms.py | 3 +-
airflow/www/static/js/connection_form.js | 4 +
airflow/www/static/js/task-instances.js | 9 +-
airflow/www/templates/airflow/dags.html | 51 +-
airflow/www/views.py | 47 +-
backport_packages/MANIFEST.in | 1 -
.../PROVIDERS_CHANGES_TEMPLATE.md.jinja2 | 22 +
.../PROVIDERS_CLASSES_TEMPLATE.md.jinja2 | 98 ++
.../PROVIDERS_README_TEMPLATE.md.jinja2 | 123 ++
backport_packages/README.md | 39 -
backport_packages/setup_backport_packages.py | 1073 ++++++++++++++---
breeze | 385 ++++--
breeze-complete | 29 +-
common/_image_variables.sh | 6 +-
dev/BACKPORT_PACKAGES.md | 422 +++++++
dev/send_email.py | 4 +-
dev/sign.sh | 9 +-
docs/autoapi_templates/index.rst | 15 +-
docs/build | 89 +-
docs/concepts.rst | 61 +
docs/conf.py | 6 +-
docs/exts/exampleinclude.py | 2 +-
docs/howto/connection/jdbc.rst | 46 +
docs/howto/connection/spark.rst | 43 +
docs/howto/custom-operator.rst | 6 +-
.../aws/_partials/prerequisite_tasks.rst} | 38 +-
docs/howto/operator/amazon/aws/ecs.rst | 57 +
docs/howto/operator/amazon/aws/emr.rst | 121 ++
.../amazon/aws/google_api_to_s3_transfer.rst | 141 +++
.../operator/amazon/aws/imap_attachment_to_s3.rst | 70 ++
docs/howto/operator/{ => apache}/index.rst | 26 +-
docs/howto/operator/apache/spark.rst | 104 ++
.../{external.rst => external_task_sensor.rst} | 5 +
.../operator/gcp/_partials/prerequisite_tasks.rst | 2 +-
docs/howto/operator/gcp/bigquery.rst | 40 +-
docs/howto/operator/gcp/display_video.rst | 87 +-
docs/howto/operator/gcp/facebook_ads_to_gcs.rst | 52 +
docs/howto/operator/gcp/gcs.rst | 6 +-
docs/howto/operator/gcp/gcs_to_gcs.rst | 172 ++-
docs/howto/operator/gcp/life_sciences.rst | 79 ++
docs/howto/operator/gcp/spanner.rst | 38 +
.../gcp/{speech.rst => speech_to_text.rst} | 60 +-
.../gcp/{speech.rst => text_to_speech.rst} | 58 +-
.../{translate-speech.rst => translate_speech.rst} | 26 +-
docs/howto/operator/http/http.rst | 77 ++
docs/howto/operator/{ => http}/index.rst | 30 +-
docs/howto/operator/index.rst | 4 +-
docs/howto/use-alternative-secrets-backend.rst | 99 +-
docs/howto/write-logs.rst | 2 +-
docs/installation.rst | 30 +-
docs/metrics.rst | 1 +
docs/operators-and-hooks-ref.rst | 353 ++++--
docs/plugins.rst | 7 +-
docs/rest-api-ref.rst | 2 +
.../ci/docker-compose/empty => empty}/.gitignore | 0
entrypoint.sh | 6 +-
images/CI.png | Bin 0 -> 243004 bytes
requirements/requirements-python3.6.txt | 160 +--
requirements/requirements-python3.7.txt | 157 +--
requirements/setup-3.6.md5 | 2 +-
requirements/setup-3.7.md5 | 2 +-
scripts/ci/_utils.sh | 292 +++--
scripts/ci/ci_build_dockerhub.sh | 36 +-
...tion_image.sh => ci_build_production_images.sh} | 0
...kport_packages.sh => ci_count_changed_files.sh} | 40 +-
.../ci/ci_free_space_on_ci.sh | 9 +
...ic_checks.sh => ci_generate_backport_readme.sh} | 9 +-
.../ci/ci_prepare_and_test_backport_packages.sh | 3 +-
...nt_tests.sh => ci_prepare_backport_packages.sh} | 9 +-
.../__init__.py => scripts/ci/ci_push_ci_image.sh | 9 +
...ction_image.sh => ci_push_production_images.sh} | 2 +-
scripts/ci/ci_run_airflow_testing.sh | 96 +-
...ll_static_checks.sh => ci_run_static_checks.sh} | 11 +-
scripts/ci/ci_test_backport_packages.sh | 2 -
scripts/ci/docker-compose/backend-sqlite.yml | 2 +-
scripts/ci/docker-compose/base.yml | 3 +-
scripts/ci/docker-compose/forward-credentials.yml | 1 -
scripts/ci/docker-compose/local.yml | 1 +
scripts/ci/in_container/_in_container_utils.sh | 2 +-
scripts/ci/in_container/check_environment.sh | 22 +-
scripts/ci/in_container/entrypoint_ci.sh | 10 +
scripts/ci/in_container/run_ci_tests.sh | 2 +-
.../in_container/run_generate_backport_readme.sh | 61 +
.../ci/in_container/run_generate_requirements.sh | 24 +-
.../run_prepare_backport_packages.sh} | 61 +-
.../in_container/run_test_package_installation.sh | 4 +-
scripts/ci/pre_commit_yaml_to_cfg.py | 4 +-
scripts/list-integrations.py | 2 +-
scripts/perf/dags/elastic_dag.py | 9 +-
scripts/perf/perf_kit/__init__.py | 122 ++
scripts/perf/perf_kit/memory.py | 82 ++
scripts/perf/perf_kit/python.py | 108 ++
scripts/perf/perf_kit/repeat_and_time.py | 148 +++
scripts/perf/perf_kit/sqlalchemy.py | 175 +++
scripts/perf/scheduler_dag_execution_timing.py | 2 +-
setup.py | 208 +++-
tests/bats/test_local_mounts.bats | 80 +-
tests/build_provider_packages_dependencies.py | 4 +-
tests/cli/commands/test_dag_command.py | 3 +
tests/cli/commands/test_task_command.py | 3 +
tests/cli/commands/test_variable_command.py | 150 ++-
tests/cli/commands/test_webserver_command.py | 2 +
tests/cli/test_cli_parser.py | 18 +
tests/conftest.py | 90 +-
tests/dags/test_issue_1225.py | 7 +-
.../dags/test_prev_dagrun_dep.py | 29 +-
tests/executors/test_celery_executor.py | 176 ++-
tests/executors/test_dask_executor.py | 45 +-
tests/executors/test_executor_loader.py | 2 +-
tests/executors/test_kubernetes_executor.py | 9 +-
tests/jobs/test_backfill_job.py | 2 +
tests/jobs/test_base_job.py | 91 +-
tests/jobs/test_local_task_job.py | 2 +
tests/jobs/test_scheduler_job.py | 182 ++-
tests/kubernetes/test_pod_launcher.py | 18 +
tests/models/test_cleartasks.py | 16 +-
tests/models/test_dag.py | 29 +-
tests/models/test_dagcode.py | 20 +-
tests/models/test_dagrun.py | 65 +-
tests/models/test_serialized_dag.py | 25 +-
tests/models/test_taskinstance.py | 115 +-
tests/models/test_xcom.py | 49 +
tests/models/test_xcom_arg.py | 157 +++
tests/plugins/test_plugins_manager.py | 29 +
.../amazon/aws/config_templates/args.json | 3 +
.../amazon/aws/config_templates/job.j2.json | 5 +
.../amazon/aws/config_templates/steps.j2.json | 10 +
.../amazon/aws/config_templates/steps.json | 14 +
tests/providers/amazon/aws/hooks/test_base_aws.py | 6 +
tests/providers/amazon/aws/hooks/test_ec2.py | 71 ++
tests/providers/amazon/aws/hooks/test_glue.py | 102 ++
.../amazon/aws/hooks/test_lambda_function.py | 54 +-
tests/providers/amazon/aws/hooks/test_s3.py | 23 +-
.../providers/amazon/aws/operators/test_athena.py | 7 +-
.../aws/operators/test_ec2_start_instance.py | 64 +
.../amazon/aws/operators/test_ec2_stop_instance.py | 64 +
tests/providers/amazon/aws/operators/test_ecs.py | 7 +-
.../amazon/aws/operators/test_emr_add_steps.py | 49 +
.../aws/operators/test_emr_create_job_flow.py | 52 +-
.../amazon/aws/operators/test_emr_system.py | 25 +-
tests/providers/amazon/aws/operators/test_glue.py | 54 +
.../test_google_api_to_s3_transfer_system.py | 65 ++
.../test_imap_attachment_to_s3_system.py} | 36 +-
.../amazon/aws/operators/test_s3_to_sftp.py | 6 +-
.../amazon/aws/secrets/test_systems_manager.py | 18 +
.../amazon/aws/sensors/test_ec2_instance_state.py | 128 ++
tests/providers/amazon/aws/sensors/test_glue.py | 62 +
.../providers/databricks/hooks/test_databricks.py | 33 +
tests/providers/docker/operators/test_docker.py | 5 +-
.../docker/operators/test_docker_swarm.py | 60 +-
tests/providers/facebook/ads/hooks/test_ads.py | 19 +-
.../providers/google/cloud/hooks/test_bigquery.py | 1219 ++++++++-----------
.../hooks/test_cloud_storage_transfer_service.py | 96 +-
.../providers/google/cloud/hooks/test_dataflow.py | 322 ++++-
.../google/cloud/hooks/test_life_sciences.py | 301 +++++
.../google/cloud/operators/test_bigquery.py | 9 +-
.../google/cloud/operators/test_bigquery_system.py | 11 +-
...stem.py => test_bigquery_to_bigquery_system.py} | 23 +-
...ry_system.py => test_bigquery_to_gcs_system.py} | 11 +-
.../google/cloud/operators/test_dataflow.py | 41 +-
.../operators/test_facebook_ads_to_gcs.py} | 6 +-
.../operators/test_facebook_ads_to_gcs_system.py | 80 ++
...ch_system.py => test_gcs_to_bigquery_system.py} | 25 +-
.../google/cloud/operators/test_life_sciences.py | 66 ++
...eech_system.py => test_life_sciences_system.py} | 32 +-
.../google/cloud/operators/test_mlengine_utils.py | 43 +-
...ech_system.py => test_speech_to_text_system.py} | 12 +-
...ech_system.py => test_text_to_speech_system.py} | 10 +-
...h_system.py => test_translate_speech_system.py} | 12 +-
tests/providers/google/cloud/sensors/test_gcs.py | 4 +-
.../google/cloud/utils/gcp_authenticator.py | 1 +
.../google/common/hooks/test_base_google.py | 64 +-
.../google/facebook_ads_to_gcs/__init__.py | 16 -
.../marketing_platform/hooks/test_display_video.py | 205 +++-
.../operators/test_display_video.py | 190 ++-
.../operators/test_display_video_system.py | 11 +-
.../sensors/test_display_video.py | 25 +-
tests/providers/hashicorp/secrets/test_vault.py | 21 +
.../providers/http/operators/test_http_system.py | 27 +-
.../microsoft/azure/operators/test_file_to_wasb.py | 4 +-
.../azure/operators/test_wasb_delete_blob.py | 4 +-
.../providers/microsoft/azure/sensors/test_wasb.py | 8 +-
tests/providers/postgres/hooks/test_postgres.py | 59 +
.../providers/salesforce/hooks/test_salesforce.py | 45 +
tests/providers/sftp/operators/test_sftp.py | 20 +-
tests/providers/slack/hooks/test_slack.py | 106 +-
tests/providers/slack/operators/test_slack.py | 47 -
tests/providers/snowflake/hooks/test_snowflake.py | 6 +-
.../snowflake/operators/test_snowflake_system.py | 70 ++
.../kubernetes/test_kubernetes_pod_operator.py | 210 +++-
tests/secrets/test_local_filesystem.py | 220 ++++
tests/secrets/test_secrets.py | 14 +
tests/security/test_kerberos.py | 35 +-
tests/sensors/test_base_sensor.py | 11 +-
tests/sensors/test_external_task_sensor.py | 67 +-
tests/sensors/test_sql_sensor.py | 8 +-
tests/sensors/test_timeout_sensor.py | 3 +
tests/serialization/test_dag_serialization.py | 168 ++-
tests/test_configuration.py | 50 +-
tests/test_core_to_contrib.py | 116 +-
tests/test_impersonation.py | 4 +
tests/test_logging_config.py | 92 +-
tests/test_order_setup.py | 30 +-
tests/test_project_structure.py | 9 -
tests/test_sqlalchemy_config.py | 3 +-
tests/test_stats.py | 14 +-
tests/test_utils/amazon_system_helpers.py | 94 ++
tests/test_utils/config.py | 20 +
tests/test_utils/mock_executor.py | 9 +-
tests/utils/log/test_s3_task_handler.py | 3 +-
tests/utils/test_dag_processing.py | 332 +++---
tests/utils/test_dates.py | 8 +
tests/utils/test_dot_renderer.py | 30 +
tests/utils/test_email.py | 4 +-
tests/utils/test_net.py | 51 +-
tests/utils/test_process_utils.py | 2 +
.../test_task_handler_with_custom_formatter.py | 4 +-
.../api/experimental/test_kerberos_endpoints.py | 23 +-
tests/www/test_views.py | 231 ++--
486 files changed, 25642 insertions(+), 6165 deletions(-)
create mode 100644 CI.rst
create mode 100644 airflow/models/xcom_arg.py
create mode 100644 airflow/providers/amazon/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/amazon/README.md
create mode 100644 airflow/providers/amazon/aws/example_dags/example_ecs_fargate.py
create mode 100644 airflow/providers/amazon/aws/example_dags/example_google_api_to_s3_transfer_advanced.py
create mode 100644 airflow/providers/amazon/aws/example_dags/example_google_api_to_s3_transfer_basic.py
create mode 100644 airflow/providers/amazon/aws/example_dags/example_imap_attachment_to_s3.py
create mode 100644 airflow/providers/amazon/aws/hooks/ec2.py
create mode 100644 airflow/providers/amazon/aws/hooks/glue.py
create mode 100644 airflow/providers/amazon/aws/operators/ec2_start_instance.py
create mode 100644 airflow/providers/amazon/aws/operators/ec2_stop_instance.py
create mode 100644 airflow/providers/amazon/aws/operators/glue.py
create mode 100644 airflow/providers/amazon/aws/sensors/ec2_instance_state.py
create mode 100644 airflow/providers/amazon/aws/sensors/glue.py
create mode 100644 airflow/providers/apache/cassandra/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/apache/cassandra/README.md
create mode 100644 airflow/providers/apache/druid/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/apache/druid/README.md
create mode 100644 airflow/providers/apache/hdfs/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/apache/hdfs/README.md
create mode 100644 airflow/providers/apache/hive/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/apache/hive/README.md
create mode 100644 airflow/providers/apache/livy/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/apache/livy/README.md
create mode 100644 airflow/providers/apache/pig/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/apache/pig/README.md
create mode 100644 airflow/providers/apache/pinot/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/apache/pinot/README.md
create mode 100644 airflow/providers/apache/spark/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/apache/spark/README.md
copy {tests/www/api/experimental => airflow/providers/apache/spark/example_dags}/__init__.py (100%)
create mode 100644 airflow/providers/apache/spark/example_dags/example_spark_dag.py
create mode 100644 airflow/providers/apache/sqoop/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/apache/sqoop/README.md
create mode 100644 airflow/providers/celery/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/celery/README.md
create mode 100644 airflow/providers/cloudant/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/cloudant/README.md
create mode 100644 airflow/providers/cncf/kubernetes/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/cncf/kubernetes/README.md
create mode 100644 airflow/providers/databricks/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/databricks/README.md
create mode 100644 airflow/providers/datadog/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/datadog/README.md
create mode 100644 airflow/providers/dingding/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/dingding/README.md
create mode 100644 airflow/providers/discord/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/discord/README.md
create mode 100644 airflow/providers/docker/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/docker/README.md
create mode 100644 airflow/providers/elasticsearch/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/elasticsearch/README.md
create mode 100644 airflow/providers/email/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/email/README.md
create mode 100644 airflow/providers/exasol/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/exasol/README.md
create mode 100644 airflow/providers/facebook/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/facebook/README.md
create mode 100644 airflow/providers/ftp/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/ftp/README.md
delete mode 100644 airflow/providers/google/BACKPORT_README.md
create mode 100644 airflow/providers/google/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/google/README.md
delete mode 100644 airflow/providers/google/cloud/example_dags/example_bigquery.py
create mode 100644 airflow/providers/google/cloud/example_dags/example_bigquery_operations.py
create mode 100644 airflow/providers/google/cloud/example_dags/example_bigquery_queries.py
create mode 100644 airflow/providers/google/cloud/example_dags/example_bigquery_to_bigquery.py
create mode 100644 airflow/providers/google/cloud/example_dags/example_bigquery_to_gcs.py
create mode 100644 airflow/providers/google/cloud/example_dags/example_bigquery_transfer.py
rename airflow/providers/google/{facebook_ads_to_gcs/example_dags/example_ads.py => cloud/example_dags/example_facebook_ads_to_gcs.py} (74%)
rename airflow/providers/google/cloud/example_dags/{example_gcs_to_bq.py => example_gcs_to_bigquery.py} (67%)
create mode 100644 airflow/providers/google/cloud/example_dags/example_life_sciences.py
copy airflow/providers/google/cloud/example_dags/{example_speech.py => example_speech_to_text.py} (61%)
copy airflow/providers/google/cloud/example_dags/{example_speech.py => example_text_to_speech.py} (51%)
rename airflow/providers/google/cloud/example_dags/{example_speech.py => example_translate_speech.py} (67%)
create mode 100644 airflow/providers/google/cloud/hooks/life_sciences.py
rename airflow/providers/google/{facebook_ads_to_gcs/operators/ads.py => cloud/operators/facebook_ads_to_gcs.py} (96%)
create mode 100644 airflow/providers/google/cloud/operators/life_sciences.py
delete mode 100644 airflow/providers/google/facebook_ads_to_gcs/example_dags/__init__.py
create mode 100644 airflow/providers/grpc/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/grpc/README.md
create mode 100644 airflow/providers/hashicorp/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/hashicorp/README.md
create mode 100644 airflow/providers/http/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/http/README.md
create mode 100644 airflow/providers/imap/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/imap/README.md
create mode 100644 airflow/providers/jdbc/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/jdbc/README.md
create mode 100644 airflow/providers/jenkins/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/jenkins/README.md
create mode 100644 airflow/providers/jira/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/jira/README.md
create mode 100644 airflow/providers/microsoft/azure/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/microsoft/azure/README.md
create mode 100644 airflow/providers/microsoft/mssql/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/microsoft/mssql/README.md
create mode 100644 airflow/providers/microsoft/winrm/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/microsoft/winrm/README.md
create mode 100644 airflow/providers/mongo/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/mongo/README.md
create mode 100644 airflow/providers/mysql/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/mysql/README.md
create mode 100644 airflow/providers/odbc/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/odbc/README.md
create mode 100644 airflow/providers/openfaas/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/openfaas/README.md
create mode 100644 airflow/providers/opsgenie/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/opsgenie/README.md
create mode 100644 airflow/providers/oracle/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/oracle/README.md
create mode 100644 airflow/providers/pagerduty/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/pagerduty/README.md
create mode 100644 airflow/providers/papermill/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/papermill/README.md
create mode 100644 airflow/providers/postgres/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/postgres/README.md
create mode 100644 airflow/providers/presto/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/presto/README.md
create mode 100644 airflow/providers/qubole/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/qubole/README.md
create mode 100644 airflow/providers/redis/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/redis/README.md
create mode 100644 airflow/providers/salesforce/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/salesforce/README.md
create mode 100644 airflow/providers/samba/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/samba/README.md
create mode 100644 airflow/providers/segment/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/segment/README.md
create mode 100644 airflow/providers/sftp/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/sftp/README.md
create mode 100644 airflow/providers/singularity/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/singularity/README.md
create mode 100644 airflow/providers/slack/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/slack/README.md
create mode 100644 airflow/providers/snowflake/BACKPORT_README.md
create mode 100644 airflow/providers/snowflake/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/snowflake/README.md
rename {tests/providers/google/facebook_ads_to_gcs/operators => airflow/providers/snowflake/example_dags}/__init__.py (100%)
create mode 100644 airflow/providers/snowflake/example_dags/example_snowflake.py
create mode 100644 airflow/providers/sqlite/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/sqlite/README.md
create mode 100644 airflow/providers/ssh/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/ssh/README.md
create mode 100644 airflow/providers/vertica/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/vertica/README.md
create mode 100644 airflow/providers/yandex/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/yandex/README.md
create mode 100644 airflow/providers/zendesk/PROVIDERS_CHANGES_2020.05.20.md
create mode 100644 airflow/providers/zendesk/README.md
create mode 100644 airflow/secrets/local_filesystem.py
copy scripts/ci/ci_run_static_checks_pylint_tests.sh => airflow/utils/mixins.py (58%)
mode change 100755 => 100644
copy scripts/ci/ci_run_all_static_checks.sh => airflow/utils/platform.py (52%)
mode change 100755 => 100644
create mode 100644 backport_packages/PROVIDERS_CHANGES_TEMPLATE.md.jinja2
create mode 100644 backport_packages/PROVIDERS_CLASSES_TEMPLATE.md.jinja2
create mode 100644 backport_packages/PROVIDERS_README_TEMPLATE.md.jinja2
delete mode 100644 backport_packages/README.md
create mode 100644 dev/BACKPORT_PACKAGES.md
create mode 100644 docs/howto/connection/jdbc.rst
create mode 100644 docs/howto/connection/spark.rst
copy docs/howto/operator/{index.rst => amazon/aws/_partials/prerequisite_tasks.rst} (62%)
create mode 100644 docs/howto/operator/amazon/aws/ecs.rst
create mode 100644 docs/howto/operator/amazon/aws/emr.rst
create mode 100644 docs/howto/operator/amazon/aws/google_api_to_s3_transfer.rst
create mode 100644 docs/howto/operator/amazon/aws/imap_attachment_to_s3.rst
copy docs/howto/operator/{ => apache}/index.rst (63%)
create mode 100644 docs/howto/operator/apache/spark.rst
rename docs/howto/operator/{external.rst => external_task_sensor.rst} (94%)
create mode 100644 docs/howto/operator/gcp/facebook_ads_to_gcs.rst
create mode 100644 docs/howto/operator/gcp/life_sciences.rst
copy docs/howto/operator/gcp/{speech.rst => speech_to_text.rst} (58%)
rename docs/howto/operator/gcp/{speech.rst => text_to_speech.rst} (60%)
rename docs/howto/operator/gcp/{translate-speech.rst => translate_speech.rst} (72%)
create mode 100644 docs/howto/operator/http/http.rst
copy docs/howto/operator/{ => http}/index.rst (64%)
copy {scripts/ci/docker-compose/empty => empty}/.gitignore (100%)
create mode 100644 images/CI.png
copy scripts/ci/{ci_build_production_image.sh => ci_build_production_images.sh} (100%)
copy scripts/ci/{ci_prepare_and_test_backport_packages.sh => ci_count_changed_files.sh} (53%)
rename airflow/providers/google/facebook_ads_to_gcs/operators/__init__.py => scripts/ci/ci_free_space_on_ci.sh (79%)
mode change 100644 => 100755
copy scripts/ci/{ci_run_all_static_checks.sh => ci_generate_backport_readme.sh} (86%)
copy scripts/ci/{ci_run_static_checks_pylint_tests.sh => ci_prepare_backport_packages.sh} (85%)
rename airflow/providers/google/facebook_ads_to_gcs/__init__.py => scripts/ci/ci_push_ci_image.sh (77%)
mode change 100644 => 100755
rename scripts/ci/{ci_build_production_image.sh => ci_push_production_images.sh} (98%)
copy scripts/ci/{ci_run_all_static_checks.sh => ci_run_static_checks.sh} (81%)
create mode 100755 scripts/ci/in_container/run_generate_backport_readme.sh
rename scripts/ci/{ci_prepare_packages.sh => in_container/run_prepare_backport_packages.sh} (73%)
create mode 100644 scripts/perf/perf_kit/__init__.py
create mode 100644 scripts/perf/perf_kit/memory.py
create mode 100644 scripts/perf/perf_kit/python.py
create mode 100644 scripts/perf/perf_kit/repeat_and_time.py
create mode 100644 scripts/perf/perf_kit/sqlalchemy.py
copy scripts/ci/ci_run_all_static_checks.sh => tests/dags/test_prev_dagrun_dep.py (50%)
mode change 100755 => 100644
create mode 100644 tests/models/test_xcom.py
create mode 100644 tests/models/test_xcom_arg.py
create mode 100644 tests/providers/amazon/aws/config_templates/args.json
create mode 100644 tests/providers/amazon/aws/config_templates/job.j2.json
create mode 100644 tests/providers/amazon/aws/config_templates/steps.j2.json
create mode 100644 tests/providers/amazon/aws/config_templates/steps.json
create mode 100644 tests/providers/amazon/aws/hooks/test_ec2.py
create mode 100644 tests/providers/amazon/aws/hooks/test_glue.py
create mode 100644 tests/providers/amazon/aws/operators/test_ec2_start_instance.py
create mode 100644 tests/providers/amazon/aws/operators/test_ec2_stop_instance.py
rename scripts/ci/ci_run_static_checks_pylint_tests.sh => tests/providers/amazon/aws/operators/test_emr_system.py (60%)
mode change 100755 => 100644
create mode 100644 tests/providers/amazon/aws/operators/test_glue.py
create mode 100644 tests/providers/amazon/aws/operators/test_google_api_to_s3_transfer_system.py
copy tests/providers/{google/cloud/operators/test_speech_system.py => amazon/aws/operators/test_imap_attachment_to_s3_system.py} (51%)
create mode 100644 tests/providers/amazon/aws/sensors/test_ec2_instance_state.py
create mode 100644 tests/providers/amazon/aws/sensors/test_glue.py
create mode 100644 tests/providers/google/cloud/hooks/test_life_sciences.py
copy tests/providers/google/cloud/operators/{test_bigquery_system.py => test_bigquery_to_bigquery_system.py} (65%)
copy tests/providers/google/cloud/operators/{test_bigquery_system.py => test_bigquery_to_gcs_system.py} (88%)
rename tests/providers/google/{facebook_ads_to_gcs/operators/test_ads.py => cloud/operators/test_facebook_ads_to_gcs.py} (90%)
create mode 100644 tests/providers/google/cloud/operators/test_facebook_ads_to_gcs_system.py
copy tests/providers/google/cloud/operators/{test_speech_system.py => test_gcs_to_bigquery_system.py} (61%)
create mode 100644 tests/providers/google/cloud/operators/test_life_sciences.py
copy tests/providers/google/cloud/operators/{test_speech_system.py => test_life_sciences_system.py} (60%)
copy tests/providers/google/cloud/operators/{test_speech_system.py => test_speech_to_text_system.py} (80%)
copy tests/providers/google/cloud/operators/{test_speech_system.py => test_text_to_speech_system.py} (84%)
rename tests/providers/google/cloud/operators/{test_speech_system.py => test_translate_speech_system.py} (80%)
delete mode 100644 tests/providers/google/facebook_ads_to_gcs/__init__.py
rename scripts/ci/ci_run_all_static_checks.sh => tests/providers/http/operators/test_http_system.py (63%)
mode change 100755 => 100644
create mode 100644 tests/providers/snowflake/operators/test_snowflake_system.py
create mode 100644 tests/secrets/test_local_filesystem.py
create mode 100644 tests/test_utils/amazon_system_helpers.py