You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ka...@apache.org on 2020/10/03 21:22:20 UTC
[airflow] branch v2-0-test updated (423a382 -> ff1ab97)
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a change to branch v2-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 423a382 SkipMixin: Add missing session.commit() and test (#10421)
add 1a14982 Fix typo in STATIC_CODE_CHECKS.rst (#11094)
add 35c4398 Avoid redundant SET conversion (#11091)
add a0374a5 Fix for pydocstyle D202 (#11096)
add 51181e8 Security upgrade lodash from 4.17.19 to 4.17.20 (#11095)
add 0edc3dd Introducing flags to skip example dags and default connections (#11099)
add daf8f31 Add template fields renderers for better UI rendering (#11061)
add ccfbc31 Fix sort-in-the-wild pre-commit on Mac (#11103)
add 8a34719 Fix typo in README (#11106)
add 04b8adf Add Opensignal to INTHEWILD.md (#11105)
add 7644c37 Revert "Introducing flags to skip example dags and default connections (#11099)" (#11110)
add b83507a Update initialize-database.rst (#11109)
add bcdd3bb Increasing type coverage FTP (#11107)
add e252a60 Adds timeout in CI/PROD waiting jobs (#11117)
add 620b098 Add Helm Chart linting (#11108)
add af3c677 README Doc: Link to Airflow directory in ASF Directory (#11137)
add e3f96ce Fix incorrect Usage of Optional[bool] (#11138)
add 51052aa Fix FROM directive in docs/production-deployment.rst (#11139)
add 45669be Increasing type coverage for salesforce provide (#11135)
add 68fa29b Added support for encrypted private keys in SSHHook (#11097)
add 7c0541b Fix error message when checking literalinclude in docs (#11140)
add 6970584 Upgrade to latest isort & pydocstyle (#11142)
add 37798f0 Do not silently allow the use of undefined variables in jinja2 templates (#11016)
add f4ec1f6 Move Backport Providers docs to our docsite (#11136)
add 1f0a785 Fix user in helm chart pgbouncer deployment (#11143)
add b40df1b Fixes celery deployments for Airflow 2.0 (#11129)
add ce6b257 Fix gitSync user in the helm Chart (#11127)
add 99accec Fix incorrect Usage of Optional[str] & Optional[int] (#11141)
add edf8033 Remove link to Dag Model view given the redundancy with DAG Details view (#11082)
add 33fe9a5 Make sure pgbouncer-exporter docker image is linux/amd64 (#11148)
add c65d4663 Update to latest version of pbgouncer-exporter (#11150)
add b92c60a Add new member to Polidea (#11153)
add ee90807 Massively speed up the query returned by TI.filter_for_tis (#11147)
add 08dfd8c Increase Type coverage for IMAP provider (#11154)
add 0161b5e Increasing type coverage for multiple provider (#11159)
add f16354b Optionally disables PIP cache from GitHub during the build (#11173)
add 2d831fb Update UPDATING.md (#11172)
add 0db7a30 New Breeze command start-airflow, it replaces the previous flag (#11157)
add 044b441 Conditional MySQL Client installation (#11174)
add cb52fb0 Add example DAG and system test for MySQLToGCSOperator (#10990)
add 54353f8 Increase type coverage for five different providers (#11170)
add 0ea3e61 Adds Kubernetes Service Account for the webserver (#11131)
add a888198c Allow overrides for pod_template_file (#11162)
add 4d2a787 Enables Kerberos sidecar support (#11130)
add e2dc706 Make kill log in DagFileProcessorProcess more informative (#11124)
add 6694eaa Show the location of the queries when the assert_queries_count fails. (#11186)
add 42f1da1 Improve Google Transfer header in documentation index file (#11166)
add 2ec1247 Fix typos in Dockerfile.ci (#11187)
add 4ff1290 Remove Unnecessary comprehension in 'any' builtin function (#11188)
add c9a34d2 Optionally tags image when building with Breeze (#11181)
add 68e0eb6 in_container bats pre-commit hook and updated bats-tests hook (#11179)
add 17c810e Fixes image tag readonly failure (#11194)
add ebd7150 More customizable build process for Docker images (#11176)
add 9860719 [AIRFLOW-5545] Fixes recursion in DAG cycle testing (#6175)
add 00ffedb Add amazon glacier to GCS transfer operator (#10947)
add 5093245 Strict type coverage for Oracle and Yandex provider (#11198)
add b6d5d1e Strict type checking for SSH (#11216)
add 427a4a8 Replace get accessible dag ids (#11027)
add 3ca11eb Kubernetes executor can adopt tasks from other schedulers (#10996)
add c74b3ac Optional import error tracebacks in web ui (#10663)
add 720912f Strict type check for multiple providers (#11229)
add 18f6cf1 Fix typo in command in CI.rst (#11233)
add e37dfc8 Add Python version to Breeze cmd (#11228)
add 0382f77 Use more meaningfull message for DagBag timeouts (#11235)
add 5220e4c Prepare Backport release 2020.09.07 (#11238)
add 24d0ecf Airflow 2.0 UI Overhaul/Refresh (#11195)
add 983e5a6 Restore description for provider packages. (#11239)
add 6d7c143 Small updates to provider preparation docs. (#11240)
add ca4238e Fixed month in backport packages to October (#11242)
add 7338912 Add task adoption to CeleryKubernetesExecutor (#11244)
add 9662626 Remove redundant parentheses (#11248)
add 0a0e1af Fix Broken Markdown links in Providers README TOC (#11249)
add e412566 Add option to bulk clear DAG Runs in Browse DAG Runs page (#11226)
add b7183de Update yamllint & isort pre-commit hooks (#11252)
add 4210618 Ensure target_dedicated_nodes or enable_auto_scale is set in AzureBatchOperator (#11251)
add 6d573e8 Add s3 key to template fields for s3/redshift transfer operators (#10890)
add ee81266 Add missing "example" tag on example DAG (#11253)
add 3db2e7c Breeze: Fix issue with pulling an image via ID (#11255)
add f697ff2 Move test tools from tests.utils to tests.test_utils (#10889)
add 7c66936 Add Github Code Scanning (#11211)
add aa3fedb Add Hacktoberfest topic to the repo (#11258)
add ff1ab97 Add operator link to access DAG triggered by TriggerDagRunOperator (#11254)
No new revisions were added by this update.
Summary of changes:
.asf.yaml | 1 +
.dockerignore | 2 +
.github/workflows/ci.yml | 2 +
.github/workflows/codeql-analysis.yml | 53 +
.pre-commit-config.yaml | 22 +-
BREEZE.rst | 313 +++--
CI.rst | 2 +-
CONTRIBUTING.rst | 2 +-
Dockerfile | 253 ++--
Dockerfile.ci | 138 ++-
IMAGES.rst | 123 +-
INTHEWILD.md | 3 +-
README.md | 90 +-
STATIC_CODE_CHECKS.rst | 6 +-
UPDATING.md | 16 +-
airflow/api_connexion/security.py | 1 -
airflow/cli/commands/dag_command.py | 6 +-
airflow/config_templates/config.yml | 17 +-
airflow/config_templates/default_airflow.cfg | 9 +-
airflow/example_dags/example_nested_branch_dag.py | 7 +-
airflow/example_dags/example_task_group.py | 2 +-
airflow/executors/base_executor.py | 2 +
airflow/executors/celery_kubernetes_executor.py | 24 +-
airflow/executors/kubernetes_executor.py | 192 ++-
airflow/jobs/scheduler_job.py | 6 +-
airflow/kubernetes/pod_generator.py | 4 +-
.../versions/952da73b5eff_add_dag_code_table.py | 2 +-
.../bef4f3d11e8b_drop_kuberesourceversion_and_.py | 106 ++
airflow/models/__init__.py | 5 -
airflow/models/baseoperator.py | 6 +-
airflow/models/connection.py | 2 +-
airflow/models/dag.py | 4 +-
airflow/models/dagbag.py | 21 +-
airflow/models/dagrun.py | 2 +-
airflow/models/kubernetes.py | 88 --
airflow/models/taskinstance.py | 68 +-
airflow/operators/bash.py | 1 +
airflow/operators/dagrun_operator.py | 22 +-
airflow/operators/sql.py | 2 +-
.../amazon/PROVIDERS_CHANGES_2020.10.05.md | 65 +
airflow/providers/amazon/README.md | 150 ++-
.../aws/example_dags/example_glacier_to_gcs.py | 72 ++
airflow/providers/amazon/aws/hooks/glacier.py | 73 ++
airflow/providers/amazon/aws/operators/glacier.py | 54 +
.../providers/amazon/aws/operators/s3_bucket.py | 2 +-
airflow/providers/amazon/aws/sensors/glacier.py | 99 ++
.../amazon/aws/transfers/glacier_to_gcs.py | 122 ++
.../providers/amazon/aws/transfers/mysql_to_s3.py | 4 +-
.../amazon/aws/transfers/redshift_to_s3.py | 2 +-
.../amazon/aws/transfers/s3_to_redshift.py | 2 +-
.../cassandra/PROVIDERS_CHANGES_2020.10.05.md | 17 +
airflow/providers/apache/cassandra/README.md | 32 +-
.../apache/druid/PROVIDERS_CHANGES_2020.10.05.md | 18 +
airflow/providers/apache/druid/README.md | 41 +-
.../apache/hdfs/PROVIDERS_CHANGES_2020.10.05.md | 18 +
airflow/providers/apache/hdfs/README.md | 33 +-
.../apache/hive/PROVIDERS_CHANGES_2020.10.05.md | 29 +
airflow/providers/apache/hive/README.md | 50 +-
airflow/providers/apache/hive/hooks/hive.py | 2 +-
.../apache/kylin/PROVIDERS_CHANGES_2020.10.05.md | 19 +
airflow/providers/apache/kylin/README.md | 109 ++
airflow/providers/apache/kylin/hooks/kylin.py | 2 +-
.../providers/apache/kylin/operators/kylin_cube.py | 4 +-
.../apache/livy/PROVIDERS_CHANGES_2020.10.05.md | 17 +
airflow/providers/apache/livy/README.md | 33 +-
.../apache/pig/PROVIDERS_CHANGES_2020.10.05.md | 18 +
airflow/providers/apache/pig/README.md | 34 +-
.../apache/pinot/PROVIDERS_CHANGES_2020.10.05.md | 14 +
airflow/providers/apache/pinot/README.md | 27 +-
airflow/providers/apache/pinot/hooks/pinot.py | 12 +-
.../apache/spark/PROVIDERS_CHANGES_2020.10.05.md | 24 +
airflow/providers/apache/spark/README.md | 40 +-
.../apache/sqoop/PROVIDERS_CHANGES_2020.10.05.md | 15 +
airflow/providers/apache/sqoop/README.md | 29 +-
.../celery/PROVIDERS_CHANGES_2020.10.05.md | 15 +
airflow/providers/celery/README.md | 30 +-
.../cloudant/PROVIDERS_CHANGES_2020.10.05.md | 13 +
airflow/providers/cloudant/README.md | 26 +-
.../kubernetes/PROVIDERS_CHANGES_2020.10.05.md | 62 +
airflow/providers/cncf/kubernetes/README.md | 179 +++
.../providers/cncf/kubernetes/hooks/kubernetes.py | 11 +-
.../cncf/kubernetes/operators/kubernetes_pod.py | 105 +-
.../cncf/kubernetes/sensors/spark_kubernetes.py | 8 +-
.../databricks/PROVIDERS_CHANGES_2020.10.05.md | 20 +
airflow/providers/databricks/README.md | 36 +-
airflow/providers/databricks/hooks/databricks.py | 32 +-
.../providers/databricks/operators/databricks.py | 63 +-
.../datadog/PROVIDERS_CHANGES_2020.10.05.md | 15 +
airflow/providers/datadog/README.md | 28 +-
.../dingding/PROVIDERS_CHANGES_2020.10.05.md | 17 +
airflow/providers/dingding/README.md | 33 +-
airflow/providers/dingding/hooks/dingding.py | 22 +-
airflow/providers/dingding/operators/dingding.py | 15 +-
.../discord/PROVIDERS_CHANGES_2020.10.05.md | 16 +
airflow/providers/discord/README.md | 32 +-
.../docker/PROVIDERS_CHANGES_2020.10.05.md | 21 +
airflow/providers/docker/README.md | 37 +-
airflow/providers/docker/operators/docker.py | 2 +-
.../elasticsearch/PROVIDERS_CHANGES_2020.10.05.md | 19 +
airflow/providers/elasticsearch/README.md | 34 +-
.../exasol/PROVIDERS_CHANGES_2020.10.05.md | 16 +
airflow/providers/exasol/README.md | 32 +-
airflow/providers/exasol/hooks/exasol.py | 24 +-
airflow/providers/exasol/operators/exasol.py | 8 +-
.../facebook/PROVIDERS_CHANGES_2020.10.05.md | 15 +
airflow/providers/facebook/README.md | 28 +-
.../providers/ftp/PROVIDERS_CHANGES_2020.10.05.md | 17 +
airflow/providers/ftp/README.md | 32 +-
airflow/providers/ftp/hooks/ftp.py | 11 +-
.../google/PROVIDERS_CHANGES_2020.10.05.md | 112 ++
airflow/providers/google/README.md | 203 +++-
.../cloud/example_dags/example_mysql_to_gcs.py} | 34 +-
.../providers/google/cloud/hooks/datacatalog.py | 2 +-
.../providers/google/cloud/operators/bigquery.py | 12 +-
.../google/cloud/operators/datacatalog.py | 3 +-
.../providers/google/cloud/operators/dataproc.py | 1 +
.../google/cloud/operators/stackdriver.py | 20 +-
.../providers/grpc/PROVIDERS_CHANGES_2020.10.05.md | 16 +
airflow/providers/grpc/README.md | 32 +-
.../hashicorp/PROVIDERS_CHANGES_2020.10.05.md | 18 +
airflow/providers/hashicorp/README.md | 37 +-
.../providers/http/PROVIDERS_CHANGES_2020.10.05.md | 21 +
airflow/providers/http/README.md | 37 +-
.../providers/imap/PROVIDERS_CHANGES_2020.10.05.md | 17 +
airflow/providers/imap/README.md | 32 +-
airflow/providers/imap/hooks/imap.py | 16 +-
airflow/providers/imap/sensors/imap_attachment.py | 4 +-
.../providers/jdbc/PROVIDERS_CHANGES_2020.10.05.md | 17 +
airflow/providers/jdbc/README.md | 33 +-
airflow/providers/jdbc/operators/jdbc.py | 2 +-
.../jenkins/PROVIDERS_CHANGES_2020.10.05.md | 17 +
airflow/providers/jenkins/README.md | 33 +-
.../providers/jira/PROVIDERS_CHANGES_2020.10.05.md | 16 +
airflow/providers/jira/README.md | 32 +-
.../azure/PROVIDERS_CHANGES_2020.10.05.md | 25 +
airflow/providers/microsoft/azure/README.md | 60 +-
.../microsoft/azure/operators/azure_batch.py | 6 +-
.../azure/transfers/oracle_to_azure_data_lake.py | 4 +-
.../mssql/PROVIDERS_CHANGES_2020.10.05.md | 14 +
airflow/providers/microsoft/mssql/README.md | 28 +-
.../winrm/PROVIDERS_CHANGES_2020.10.05.md | 15 +
airflow/providers/microsoft/winrm/README.md | 29 +-
.../mongo/PROVIDERS_CHANGES_2020.10.05.md | 15 +
airflow/providers/mongo/README.md | 28 +-
.../mysql/PROVIDERS_CHANGES_2020.10.05.md | 18 +
airflow/providers/mysql/README.md | 41 +-
.../providers/odbc/PROVIDERS_CHANGES_2020.10.05.md | 13 +
airflow/providers/odbc/README.md | 26 +-
airflow/providers/odbc/hooks/odbc.py | 22 +-
.../openfaas/PROVIDERS_CHANGES_2020.10.05.md | 13 +
airflow/providers/openfaas/README.md | 26 +-
.../opsgenie/PROVIDERS_CHANGES_2020.10.05.md | 15 +
airflow/providers/opsgenie/README.md | 29 +-
airflow/providers/opsgenie/hooks/opsgenie_alert.py | 11 +-
.../providers/opsgenie/operators/opsgenie_alert.py | 38 +-
.../oracle/PROVIDERS_CHANGES_2020.10.05.md | 17 +
airflow/providers/oracle/README.md | 38 +-
airflow/providers/oracle/hooks/oracle.py | 45 +-
airflow/providers/oracle/operators/oracle.py | 2 +-
.../providers/oracle/transfers/oracle_to_oracle.py | 19 +-
.../pagerduty/PROVIDERS_CHANGES_2020.10.05.md | 12 +
airflow/providers/pagerduty/README.md | 25 +-
.../plexus/PROVIDERS_CHANGES_2020.10.05.md | 10 +
airflow/providers/plexus/README.md | 100 ++
airflow/providers/plexus/hooks/plexus.py | 10 +-
airflow/providers/plexus/operators/job.py | 14 +-
.../postgres/PROVIDERS_CHANGES_2020.10.05.md | 16 +
airflow/providers/postgres/README.md | 32 +-
.../presto/PROVIDERS_CHANGES_2020.10.05.md | 14 +
airflow/providers/presto/README.md | 27 +-
airflow/providers/presto/hooks/presto.py | 46 +-
.../qubole/PROVIDERS_CHANGES_2020.10.05.md | 23 +
airflow/providers/qubole/README.md | 39 +-
airflow/providers/qubole/hooks/qubole.py | 55 +-
airflow/providers/qubole/hooks/qubole_check.py | 15 +-
airflow/providers/qubole/operators/qubole.py | 30 +-
airflow/providers/qubole/operators/qubole_check.py | 35 +-
airflow/providers/qubole/sensors/qubole.py | 12 +-
.../redis/PROVIDERS_CHANGES_2020.10.05.md | 19 +
airflow/providers/redis/README.md | 35 +-
.../salesforce/PROVIDERS_CHANGES_2020.10.05.md | 18 +
airflow/providers/salesforce/README.md | 34 +-
airflow/providers/salesforce/hooks/salesforce.py | 32 +-
airflow/providers/salesforce/hooks/tableau.py | 6 +-
.../operators/tableau_refresh_workbook.py | 2 +-
.../salesforce/sensors/tableau_job_status.py | 2 +-
.../samba/PROVIDERS_CHANGES_2020.10.05.md | 13 +
airflow/providers/samba/README.md | 26 +-
airflow/providers/samba/hooks/samba.py | 6 +-
.../segment/PROVIDERS_CHANGES_2020.10.05.md | 16 +
airflow/providers/segment/README.md | 32 +-
.../providers/sftp/PROVIDERS_CHANGES_2020.10.05.md | 18 +
airflow/providers/sftp/README.md | 34 +-
airflow/providers/sftp/operators/sftp.py | 7 +-
airflow/providers/sftp/sensors/sftp.py | 8 +-
.../singularity/PROVIDERS_CHANGES_2020.10.05.md | 17 +
airflow/providers/singularity/README.md | 33 +-
.../providers/singularity/operators/singularity.py | 28 +-
.../slack/PROVIDERS_CHANGES_2020.10.05.md | 23 +
airflow/providers/slack/README.md | 37 +-
airflow/providers/slack/hooks/slack.py | 4 +-
airflow/providers/slack/hooks/slack_webhook.py | 7 +-
airflow/providers/slack/operators/slack.py | 12 +-
airflow/providers/slack/operators/slack_webhook.py | 30 +-
.../snowflake/PROVIDERS_CHANGES_2020.10.05.md | 20 +
airflow/providers/snowflake/README.md | 41 +-
airflow/providers/snowflake/hooks/snowflake.py | 24 +-
airflow/providers/snowflake/operators/snowflake.py | 26 +-
.../snowflake/transfers/s3_to_snowflake.py | 21 +-
.../sqlite/PROVIDERS_CHANGES_2020.10.05.md | 17 +
airflow/providers/sqlite/README.md | 33 +-
.../providers/ssh/PROVIDERS_CHANGES_2020.10.05.md | 19 +
airflow/providers/ssh/README.md | 35 +-
airflow/providers/ssh/hooks/ssh.py | 39 +-
airflow/providers/ssh/operators/ssh.py | 25 +-
.../vertica/PROVIDERS_CHANGES_2020.10.05.md | 15 +
airflow/providers/vertica/README.md | 29 +-
.../yandex/PROVIDERS_CHANGES_2020.10.05.md | 18 +
airflow/providers/yandex/README.md | 34 +-
airflow/providers/yandex/hooks/yandex.py | 12 +-
.../providers/yandex/hooks/yandexcloud_dataproc.py | 2 +-
.../yandex/operators/yandexcloud_dataproc.py | 38 +-
.../zendesk/PROVIDERS_CHANGES_2020.10.05.md | 13 +
airflow/providers/zendesk/README.md | 26 +-
airflow/providers/zendesk/hooks/zendesk.py | 8 +-
airflow/serialization/serialized_objects.py | 3 +-
airflow/utils/dag_cycle_tester.py | 50 +-
airflow/utils/platform.py | 2 +-
airflow/www/extensions/init_appbuilder_links.py | 9 +-
airflow/www/extensions/init_jinja_globals.py | 15 +-
airflow/www/extensions/init_views.py | 1 -
airflow/www/package.json | 4 +-
airflow/www/security.py | 67 +-
airflow/www/static/css/bootstrap-theme.css | 1201 ++++---------------
airflow/www/static/css/graph.css | 12 +-
airflow/www/static/css/main.css | 128 +-
airflow/www/static/css/material-icons.css | 63 +
airflow/www/static/css/switch.css | 5 +-
airflow/www/static/css/tree.css | 6 +-
airflow/www/static/js/base.js | 106 +-
airflow/www/static/js/datetime-utils.js | 50 +-
airflow/www/static/js/graph.js | 20 +-
airflow/www/static/js/task-instances.js | 19 +-
airflow/www/templates/airflow/chart.html | 53 +-
airflow/www/templates/airflow/code.html | 28 +-
airflow/www/templates/airflow/confirm.html | 30 +-
airflow/www/templates/airflow/conn_create.html | 2 +-
airflow/www/templates/airflow/conn_edit.html | 2 +-
airflow/www/templates/airflow/dag.html | 234 ++--
airflow/www/templates/airflow/dag_code.html | 53 +-
airflow/www/templates/airflow/dag_details.html | 32 +-
airflow/www/templates/airflow/dags.html | 500 ++++----
airflow/www/templates/airflow/duration_chart.html | 61 +-
airflow/www/templates/airflow/gantt.html | 113 +-
airflow/www/templates/airflow/graph.html | 1238 ++++++++++----------
airflow/www/templates/airflow/master.html | 44 +-
airflow/www/templates/airflow/model_list.html | 129 +-
airflow/www/templates/airflow/task.html | 100 +-
airflow/www/templates/airflow/task_instance.html | 46 +-
airflow/www/templates/airflow/ti_code.html | 12 +-
airflow/www/templates/airflow/ti_log.html | 59 +-
airflow/www/templates/airflow/tree.html | 809 ++++++-------
airflow/www/templates/airflow/trigger.html | 32 +-
airflow/www/templates/airflow/variable_list.html | 2 +-
airflow/www/templates/airflow/version.html | 34 -
airflow/www/templates/airflow/xcom.html | 30 +-
airflow/www/templates/analytics/metarouter.html | 2 +-
airflow/www/templates/analytics/segment.html | 2 +-
airflow/www/templates/appbuilder/flash.html | 96 +-
airflow/www/templates/appbuilder/navbar.html | 26 +-
airflow/www/templates/appbuilder/navbar_menu.html | 53 +-
airflow/www/templates/appbuilder/navbar_right.html | 77 +-
airflow/www/utils.py | 16 +-
airflow/www/views.py | 117 +-
airflow/www/webpack.config.js | 104 +-
airflow/www/widgets.py | 2 +-
airflow/www/yarn.lock | 5 +
.../PROVIDERS_README_TEMPLATE.md.jinja2 | 2 +-
backport_packages/setup_backport_packages.py | 15 +-
breeze | 238 +++-
breeze-complete | 14 +-
chart/README.md | 17 +-
.../pgbouncer-exporter/build_and_push.sh | 42 +-
chart/templates/_helpers.yaml | 14 +
chart/templates/configmap.yaml | 4 +
chart/templates/flower/flower-deployment.yaml | 2 +-
.../templates/pgbouncer/pgbouncer-deployment.yaml | 2 +-
chart/templates/rbac/pod-launcher-role.yaml | 1 +
.../templates/webserver/webserver-deployment.yaml | 1 +
.../webserver/webserver-serviceaccount.yaml | 22 +-
chart/templates/workers/worker-deployment.yaml | 63 +-
chart/tests/git-sync-scheduler_test.yaml | 2 +
chart/tests/git-sync-webserver_test.yaml | 13 +-
chart/tests/git-sync-worker_test.yaml | 6 +
chart/tests/pod-template-file_test.yaml | 2 +
chart/values.yaml | 66 +-
dev/BACKPORT_PACKAGES.md | 8 +-
docs/backport-providers.rst | 109 ++
docs/build_docs.py | 3 +-
docs/howto/connection/ssh.rst | 1 +
docs/howto/custom-operator.rst | 29 +
docs/howto/initialize-database.rst | 5 +-
docs/howto/operator/amazon/aws/glacier.rst | 72 ++
docs/howto/operator/amazon/aws/index.rst | 1 +
.../amazon/aws/transfer/glacier_to_gcs.rst | 45 +
.../{google => amazon/aws}/transfer/index.rst | 4 +-
docs/howto/operator/google/transfer/index.rst | 2 +-
.../operator/google/transfer/mysql_to_gcs.rst | 58 +
docs/img/code.png | Bin 271371 -> 240101 bytes
docs/img/context.png | Bin 153823 -> 124467 bytes
docs/img/dags.png | Bin 258394 -> 338418 bytes
docs/img/duration.png | Bin 226323 -> 150876 bytes
docs/img/gantt.png | Bin 109644 -> 71033 bytes
docs/img/graph.png | Bin 134905 -> 118674 bytes
docs/img/tree.png | Bin 162041 -> 177465 bytes
docs/img/variable_hidden.png | Bin 121301 -> 55331 bytes
docs/index.rst | 1 +
docs/operators-and-hooks-ref.rst | 13 +-
docs/production-deployment.rst | 129 +-
docs/security/kerberos.rst | 2 +
docs/spelling_wordlist.txt | 2 +
kubernetes_tests/test_kubernetes_pod_operator.py | 95 +-
.../templates/metastore_browser/base.html | 44 +-
scripts/ci/docker-compose/base.yml | 2 +
scripts/ci/libraries/_build_images.sh | 73 +-
scripts/ci/libraries/_docker.env | 2 +
scripts/ci/libraries/_initialization.sh | 56 +-
.../pre_commit/pre_commit_helm_lint.sh} | 10 +-
...ild.sh => pre_commit_in_container_bats_test.sh} | 25 +-
.../ci/pre_commit/pre_commit_sort_in_the_wild.sh | 2 +-
scripts/ci/static_checks/bats_tests.sh | 19 +-
.../helm_lint.sh} | 17 +-
.../in_container_bats_tests.sh} | 28 +-
scripts/docker/install_mysql.sh | 58 +
scripts/in_container/check_environment.sh | 32 +-
scripts/in_container/entrypoint_ci.sh | 3 +
scripts/in_container/entrypoint_exec.sh | 3 +
scripts/in_container/run_tmux.sh | 53 +
.../in_container/run_tmux_welcome.sh | 16 +-
.../in_container/stop_tmux_airflow.sh | 14 +-
setup.py | 7 +-
.../test_in_container.bats} | 14 +-
tests/bats/test_empty_test.bats | 1 +
tests/conftest.py | 2 +-
tests/executors/test_celery_kubernetes_executor.py | 32 +
tests/executors/test_kubernetes_executor.py | 80 +-
tests/jobs/test_scheduler_job.py | 125 +-
tests/kubernetes/test_pod_generator.py | 4 +-
tests/models/test_baseoperator.py | 16 +-
tests/models/test_kubernetes.py | 56 -
tests/providers/amazon/aws/hooks/test_glacier.py | 128 ++
.../providers/amazon/aws/operators/test_glacier.py | 43 +
.../amazon/aws/operators/test_glacier_system.py | 30 +-
tests/providers/amazon/aws/sensors/test_glacier.py | 69 ++
.../amazon/aws/transfers/test_glacier_to_gcs.py | 66 ++
.../operators/test_cloud_build_system_helper.py | 2 +-
.../operators/test_cloud_sql_system_helper.py | 2 +-
.../cloud/operators/test_compute_system_helper.py | 2 +-
.../cloud/operators/test_gcs_system_helper.py | 2 +-
.../cloud/transfers/test_mysql_to_gcs_system.py | 81 ++
.../google/cloud/utils/gcp_authenticator.py | 2 +-
.../microsoft/azure/operators/test_azure_batch.py | 67 +-
tests/providers/ssh/hooks/test_ssh.py | 67 +-
tests/serialization/test_dag_serialization.py | 2 +
tests/test_project_structure.py | 1 -
tests/test_utils/amazon_system_helpers.py | 2 +-
tests/test_utils/asserts.py | 39 +-
tests/test_utils/gcp_system_helpers.py | 2 +-
.../logging_command_executor.py | 0
.../{utils => test_utils}/perf/dags/elastic_dag.py | 0
.../{utils => test_utils}/perf/dags/perf_dag_1.py | 0
.../{utils => test_utils}/perf/dags/perf_dag_2.py | 0
.../perf/dags/sql_perf_dag.py | 0
.../perf/perf_kit/__init__.py | 0
.../{utils => test_utils}/perf/perf_kit/memory.py | 0
.../{utils => test_utils}/perf/perf_kit/python.py | 0
.../perf/perf_kit/repeat_and_time.py | 0
.../perf/perf_kit/sqlalchemy.py | 0
.../perf/scheduler_dag_execution_timing.py | 0
.../perf/scheduler_ops_metrics.py | 0
tests/{utils => test_utils}/perf/sql_queries.py | 0
tests/test_utils/system_tests_class.py | 2 +-
.../{dag_cycle_tester.py => test_dag_cycle.py} | 53 +-
tests/www/test_security.py | 84 +-
tests/www/test_views.py | 69 +-
385 files changed, 9838 insertions(+), 5634 deletions(-)
create mode 100644 .github/workflows/codeql-analysis.yml
create mode 100644 airflow/migrations/versions/bef4f3d11e8b_drop_kuberesourceversion_and_.py
delete mode 100644 airflow/models/kubernetes.py
create mode 100644 airflow/providers/amazon/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/amazon/aws/example_dags/example_glacier_to_gcs.py
create mode 100644 airflow/providers/amazon/aws/hooks/glacier.py
create mode 100644 airflow/providers/amazon/aws/operators/glacier.py
create mode 100644 airflow/providers/amazon/aws/sensors/glacier.py
create mode 100644 airflow/providers/amazon/aws/transfers/glacier_to_gcs.py
create mode 100644 airflow/providers/apache/cassandra/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/apache/druid/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/apache/hdfs/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/apache/hive/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/apache/kylin/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/apache/kylin/README.md
create mode 100644 airflow/providers/apache/livy/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/apache/pig/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/apache/pinot/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/apache/spark/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/apache/sqoop/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/celery/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/cloudant/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/cncf/kubernetes/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/cncf/kubernetes/README.md
create mode 100644 airflow/providers/databricks/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/datadog/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/dingding/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/discord/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/docker/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/elasticsearch/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/exasol/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/facebook/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/ftp/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/google/PROVIDERS_CHANGES_2020.10.05.md
copy airflow/providers/{yandex/hooks/yandexcloud_dataproc.py => google/cloud/example_dags/example_mysql_to_gcs.py} (53%)
create mode 100644 airflow/providers/grpc/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/hashicorp/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/http/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/imap/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/jdbc/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/jenkins/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/jira/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/microsoft/azure/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/microsoft/mssql/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/microsoft/winrm/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/mongo/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/mysql/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/odbc/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/openfaas/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/opsgenie/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/oracle/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/pagerduty/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/plexus/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/plexus/README.md
create mode 100644 airflow/providers/postgres/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/presto/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/qubole/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/redis/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/salesforce/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/samba/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/segment/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/sftp/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/singularity/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/slack/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/snowflake/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/sqlite/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/ssh/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/vertica/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/yandex/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/providers/zendesk/PROVIDERS_CHANGES_2020.10.05.md
create mode 100644 airflow/www/static/css/material-icons.css
delete mode 100644 airflow/www/templates/airflow/version.html
copy scripts/in_container/entrypoint_exec.sh => chart/templates/webserver/webserver-serviceaccount.yaml (65%)
mode change 100755 => 100644
create mode 100644 docs/backport-providers.rst
create mode 100644 docs/howto/operator/amazon/aws/glacier.rst
create mode 100644 docs/howto/operator/amazon/aws/transfer/glacier_to_gcs.rst
copy docs/howto/operator/{google => amazon/aws}/transfer/index.rst (93%)
create mode 100644 docs/howto/operator/google/transfer/mysql_to_gcs.rst
copy scripts/{in_container/entrypoint_exec.sh => ci/pre_commit/pre_commit_helm_lint.sh} (75%)
copy scripts/ci/pre_commit/{pre_commit_sort_in_the_wild.sh => pre_commit_in_container_bats_test.sh} (64%)
copy scripts/ci/{pre_commit/pre_commit_sort_in_the_wild.sh => static_checks/helm_lint.sh} (77%)
copy scripts/ci/{pre_commit/pre_commit_sort_in_the_wild.sh => static_checks/in_container_bats_tests.sh} (61%)
mode change 100755 => 100644
create mode 100755 scripts/docker/install_mysql.sh
create mode 100755 scripts/in_container/run_tmux.sh
copy tests/bats/test_empty_test.bats => scripts/in_container/run_tmux_welcome.sh (78%)
mode change 100644 => 100755
copy tests/bats/test_empty_test.bats => scripts/in_container/stop_tmux_airflow.sh (82%)
mode change 100644 => 100755
copy tests/bats/{test_empty_test.bats => in_container/test_in_container.bats} (73%)
delete mode 100644 tests/models/test_kubernetes.py
create mode 100644 tests/providers/amazon/aws/hooks/test_glacier.py
create mode 100644 tests/providers/amazon/aws/operators/test_glacier.py
copy airflow/providers/yandex/hooks/yandexcloud_dataproc.py => tests/providers/amazon/aws/operators/test_glacier_system.py (58%)
create mode 100644 tests/providers/amazon/aws/sensors/test_glacier.py
create mode 100644 tests/providers/amazon/aws/transfers/test_glacier_to_gcs.py
create mode 100644 tests/providers/google/cloud/transfers/test_mysql_to_gcs_system.py
rename tests/{utils => test_utils}/logging_command_executor.py (100%)
rename tests/{utils => test_utils}/perf/dags/elastic_dag.py (100%)
rename tests/{utils => test_utils}/perf/dags/perf_dag_1.py (100%)
rename tests/{utils => test_utils}/perf/dags/perf_dag_2.py (100%)
rename tests/{utils => test_utils}/perf/dags/sql_perf_dag.py (100%)
rename tests/{utils => test_utils}/perf/perf_kit/__init__.py (100%)
rename tests/{utils => test_utils}/perf/perf_kit/memory.py (100%)
rename tests/{utils => test_utils}/perf/perf_kit/python.py (100%)
rename tests/{utils => test_utils}/perf/perf_kit/repeat_and_time.py (100%)
rename tests/{utils => test_utils}/perf/perf_kit/sqlalchemy.py (100%)
rename tests/{utils => test_utils}/perf/scheduler_dag_execution_timing.py (100%)
rename tests/{utils => test_utils}/perf/scheduler_ops_metrics.py (100%)
rename tests/{utils => test_utils}/perf/sql_queries.py (100%)
rename tests/utils/{dag_cycle_tester.py => test_dag_cycle.py} (75%)