You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ka...@apache.org on 2020/08/15 16:49:52 UTC
[airflow] branch v1-10-stable updated (7520da4 -> f487f2f)
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a change to branch v1-10-stable
in repository https://gitbox.apache.org/repos/asf/airflow.git.
omit 7520da4 Fixes name of pre-commit cache for multiple branches (#10299)
omit 69cc837 Fix task_instance_mutation_hook (#9910)
omit 17a9a5b [AIRFLOW-5391] Do not re-run skipped tasks when they are cleared (#7276)
add 43b75a0 Add Production Helm chart support (#8777)
add 26053b8 Fix typo in helm chart upgrade command for 2.0 (#9484)
add 16fa4a2 Remove non-existent chart value from readme (#9511)
add 9323e3a Fix typo of resultBackendConnection in chart README (#9537)
add 2bc9f61 Remove redundant airflowVersion from Helm Chart readme (#9592)
add cac19ca Fix broken link in chart/README.md (#9591)
add e7cf3b0 Switches to Helm Chart for Kubernetes tests (#9468)
add 35a9b82 Removes importlib usage - it's not needed (fails on Airflow 1.10) (#9613)
add 512f5f9 Update Breeze documentation (#9608)
add 5876261 Links Breeze documentation to new Breeze video (#9768)
add d44ad0b Fix is_terminal_support_colors functtion (#9734)
add df4b6f9 Fix typo in the word "Airflow" (#9772)
add 05cdc93 Fixes failing formatting of DAG file containing {} in docstring (#9779)
add 2f113fb Fix small errors in image building documentation (#9792)
add 90b66d0 Update README to add Py 3.8 in supported versions (#9804)
add 8f367db Python 3.8.4 release breaks our builds
add 8c8c9c7 Fixes sqlalchemy flask failure with python 3.8.4
add f46763e Fix failing docker image builds on python 2.7
add 91d1371 Add git sync option and unit tests for the Helm chart (#9371)
add b194879 Helm chart can now place arbitrary config settings in to airflow.cfg (#9816)
add 8ce2567 Bump k8s cache version
add cb9d65f Update Breeze documentation (#9608)
add 441921a Test are triggered now on more changes. (#9646)
add ee70afd The fix_ownership works independently of backend choice (#9664)
add b20362d Fix typo in pre_commit_breeze_cmd_line.sh (#9682)
add 92c79af Upgrade to latest pre-commit checks (#9686)
add 0b5f0fc Update some dependencies (#9684)
add e2e6853 Python base image version is retrieved in the right place (#9931)
add e6b017a Fix task_instance_mutation_hook (#9910)
add 179e930 [AIRFLOW-5391] Do not re-run skipped tasks when they are cleared (#7276)
add 0718977 Tests should also be triggered when there is just setup.py change (#9690)
add d61c33d The group of embedded DAGs should be root to be OpenShift compatible (#9794)
add 1d4782e Remove package.json and yarn.lock from the prod image (#9814)
add 1a41879 For now cloud tools are not needed in CI (#9818)
add 7ec2b3a Group CI scripts in subdirectories (#9653)
add f6c8f51 Reorganizing of CI tests (#9654)
add 6e290cf Added "all" to allowed breeze integrations and tried to clarify on fail (#9872)
add 5f93baf Constraint files are now maintained automatically (#9889)
add 25e0e26 Simplify pull request template (#9896)
add e33ffbe Tests are cancelled if any of faster checks fail (#9917)
add c72ce92 Pin github checkout action to v2 (#9938)
add 213500e Clean up tmp directory when exiting from breeze shell (#9930)
add 6d10a7f Pin google-cloud-container to <2 (#9901)
add 6b219e1 Breeze / KinD - support earlier k8s versions, fix recreate and kubectl versioning (#9905)
add 05ec21a Fix bug in executor_config when defining resources (#9935)
add bcd02dd Fixes PodMutationHook for backwards compatibility (#9903)
add bfa089d Add pre 1.10.11 Kubernetes Paths back with Deprecation Warning (#10067)
add 5c9ff4d Pin pymongo version to <3.11.0
add 6f8b0cc Set pytest version to be < 6.0.0 due to breaking changes (#10043)
add 70a7416 Pin Pyarrow < 1.0
add 06b06d7 Fix docstrings in BigQueryGetDataOperator (#10042)
add 05fc4b2 Allow to define custom XCom class (#8560)
add ff129f2 Pin fsspec<8.0.0 for Python <3.6 to fix Static Checks
add 88062f2 Fix more PodMutationHook issues for backwards compatibility (#10084)
add 37f08f2 Fix KubernetesPodOperator reattachment (#10230)
add 66d0210 Makes multi-namespace mode optional (#9570)
add 23788d6 Fixes name of pre-commit cache for multiple branches (#10299)
add e62d3cd Add Snowflake support to SQL operator and sensor (#9843)
add 21066c2 Avoid sharing session with RenderedTaskInstanceFields write and delete (#9993)
add 9031fbe [AIRFLOW-6931] Fixed migrations to find all dependencies for MSSQL (#9891)
add 1e10085 Fix regression in SQLThresholdCheckOperator (#9312)
add 277f33c show correct duration on graph view for running task (#8311) (#8675)
add ea32d0d [AIRFLOW-6706] Lazy load operator extra links (#7327) (#10318)
add a411383 Limit all google-cloud api to <2.0.0 (#10317)
add e83f79b Add Syntax Highlights to code-blocks in docs/best-practices.rst (#10258)
add 081263e Create separate section for Cron Presets (#10247)
add 2df8a65 Fix link for the Jinja Project in docs/tutorial.rst (#10245)
add 63db610 Improve heading on Email Configuration page (#10175)
add fd0756c Respect DAG Serialization setting when running sync_perm (#10321)
add b8d8834 Add __repr__ to SerializedDagModel (#9862)
add b9d2aaf Don't Update Serialized DAGs in DB if DAG didn't change (#9850)
add b95126b Update Serialized DAGs in Webserver when DAGs are Updated (#9851)
add d420398 Use Hash of Serialized DAG to determine DAG is changed or not (#10227)
add bce7d1c Update JS packages to latest versions (#9811) (#9921)
add 84e4bbe Fixes treatment of open slots in scheduler (#9316) (#9505)
add eca6c32 Remove chart from exported sources
add fb98e25 [AIRFLOW-5897] Allow setting -1 as pool slots value in webserver (#6550)
add 11d5d8d Fix airflow-webserver startup errors when using Kerberos Auth (#10047)
add 5ce0694 Handle IntegrityError while creating TIs (#10136)
add f51b490 Replace deprecated base classes used in bigquery_check_operator (#10272)
add 9ee28e2 Improve process terminating in scheduler_job (#8064)
add b2fbace [AIRFLOW-6843] Add delete_option_kwargs to delete_namespaced_pod (#7523)
add 7761e3f [AIRFLOW-4734] Upsert functionality for PostgresHook.insert_rows() (#8625)
add ee345bb Get Airflow configs with sensitive data from Secret Backends (#9645)
add 0ab6139 Fix clear future recursive when ExternalTaskMarker is used (#9515)
add 1b09eff Add ClusterPolicyViolation support to airflow local settings (#10282)
add c87ec44 Remove duplicate line from 1.10.10 CHANGELOG (#10289)
add 8c83a4c Set language on code-block on docs/howto/email-config.rst (#10238)
add 6aaee23 Allow `image` in `KubernetesPodOperator` to be templated (#10068)
add 7a8fa5b UI Graph View: Focus upstream / downstream task dependencies on mouseover (#9303)
add 0d7b387 Fix docstrings in BigQueryGetDataOperator (#10042)
add 5b69d10 Create a short-link for Airflow Slack Invites (#10034)
add 1d0c365 Add DateTimeSensor (#9697)
add 9efe9b3 Updated link to official documentation (#9629)
add a55c445 Add Qingping Hou to committers list (#9725)
add 4c0bffd Add new committers: Ry Walker & Leah Cole to project.rst (#9892)
add e61841a Bump version to 1.10.12
add cf53b1b Make Secret Backend docs clearer about Variable & Connection View (#8913)
add 004efff Fix typo in Task Lifecycle section (#9867)
add 9e8d566 Webserver: Sanitize values passed to origin param (#10334)
add f487f2f Add Changelog for 1.10.12
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (7520da4)
\
N -- N -- N refs/heads/v1-10-stable (f487f2f)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.dockerignore | 1 -
.gitattributes | 1 +
.github/PULL_REQUEST_TEMPLATE.md | 27 +-
.github/workflows/ci.yml | 347 +-
.gitignore | 2 -
.pre-commit-config.yaml | 40 +-
.rat-excludes | 1 +
BREEZE.rst | 1098 ++--
CHANGELOG.txt | 79 +-
CI.rst | 117 +-
CONTRIBUTING.rst | 71 +-
Dockerfile | 22 +-
Dockerfile.ci | 23 +-
IMAGES.rst | 88 +-
INSTALL | 7 +-
LOCAL_VIRTUALENV.rst | 13 +-
README.md | 44 +-
STATIC_CODE_CHECKS.rst | 15 +-
TESTING.rst | 75 +-
UPDATING.md | 10 +
airflow/bin/cli.py | 2 +-
airflow/config_templates/config.yml | 34 +
airflow/config_templates/default_airflow.cfg | 20 +
airflow/configuration.py | 94 +-
airflow/contrib/auth/backends/kerberos_auth.py | 4 +-
.../kubernetes_executor.py} | 3 +-
airflow/contrib/hooks/bigquery_hook.py | 2 +-
airflow/contrib/kubernetes/__init__.py | 2 -
.../contrib/kubernetes/kube_client.py | 14 +-
airflow/contrib/kubernetes/pod.py | 315 ++
.../contrib/kubernetes/pod_runtime_info_env.py | 14 +-
.../contrib/kubernetes/refresh_config.py | 16 +-
.../contrib/kubernetes/secret.py | 14 +-
.../contrib/kubernetes/volume.py | 14 +-
.../contrib/kubernetes/volume_mount.py | 20 +-
.../contrib/operators/bigquery_check_operator.py | 10 +-
airflow/contrib/operators/bigquery_get_data.py | 6 +-
.../contrib/operators/kubernetes_pod_operator.py | 48 +-
airflow/contrib/secrets/aws_secrets_manager.py | 21 +-
airflow/contrib/secrets/hashicorp_vault.py | 22 +-
airflow/exceptions.py | 4 +
airflow/executors/kubernetes_executor.py | 46 +-
airflow/hooks/dbapi_hook.py | 54 +-
airflow/hooks/postgres_hook.py | 54 +
airflow/jobs/scheduler_job.py | 36 +-
airflow/kubernetes/k8s_model.py | 16 +
airflow/kubernetes/pod.py | 68 +-
airflow/kubernetes/pod_generator.py | 109 +-
airflow/kubernetes/pod_launcher.py | 86 +-
airflow/kubernetes/pod_runtime_info_env.py | 2 +-
airflow/kubernetes/secret.py | 25 +-
airflow/kubernetes/volume.py | 17 +-
airflow/kubernetes/volume_mount.py | 4 +-
airflow/kubernetes/worker_configuration.py | 4 +
...867_change_datetime_to_datetime2_6_on_mssql_.py | 2 +-
...c3a5a_add_dag_hash_column_to_serialized_dag_.py | 44 +-
airflow/models/dag.py | 2 +-
airflow/models/dagbag.py | 47 +-
airflow/models/dagrun.py | 16 +-
airflow/models/serialized_dag.py | 46 +-
airflow/models/taskinstance.py | 4 +-
airflow/models/xcom.py | 19 +-
airflow/operators/__init__.py | 4 +-
airflow/operators/python_operator.py | 4 +-
airflow/operators/sql.py | 7 +-
airflow/plugins_manager.py | 29 +-
airflow/secrets/__init__.py | 43 +-
airflow/secrets/base_secrets.py | 16 +-
airflow/secrets/metastore.py | 5 +-
airflow/sensors/date_time_sensor.py | 76 +
airflow/sensors/sql_sensor.py | 2 +-
airflow/serialization/serialized_objects.py | 27 +-
airflow/settings.py | 5 +
airflow/utils/platform.py | 2 +-
airflow/version.py | 2 +-
airflow/www/utils.py | 13 +
airflow/www/views.py | 48 +-
airflow/www_rbac/package.json | 51 +-
airflow/www_rbac/static/js/task-instances.js | 22 +-
airflow/www_rbac/templates/airflow/graph.html | 32 +-
airflow/www_rbac/utils.py | 4 +-
airflow/www_rbac/views.py | 39 +-
airflow/www_rbac/webpack.config.js | 107 +-
airflow/www_rbac/yarn.lock | 5622 +++++++++++---------
breeze | 103 +-
breeze-complete | 19 +-
chart/.gitignore | 9 +
.../.helmignore | 30 +-
airflow/version.py => chart/Chart.yaml | 12 +-
chart/README.md | 308 ++
chart/requirements.lock | 6 +
airflow/version.py => chart/requirements.yaml | 9 +-
chart/templates/NOTES.txt | 29 +
chart/templates/_helpers.yaml | 355 ++
chart/templates/cleanup/cleanup-cronjob.yaml | 67 +
.../templates/cleanup/cleanup-serviceaccount.yaml | 24 +-
chart/templates/configmap.yaml | 57 +
chart/templates/create-user-job.yaml | 87 +
.../templates/dags-persistent-volume-claim.yaml | 31 +-
chart/templates/flower/flower-deployment.yaml | 102 +
chart/templates/flower/flower-networkpolicy.yaml | 51 +
.../templates/flower/flower-service.yaml | 36 +-
.../templates/limitrange.yaml | 28 +-
.../templates/pgbouncer/pgbouncer-deployment.yaml | 128 +
.../pgbouncer/pgbouncer-networkpolicy.yaml | 69 +
.../pgbouncer/pgbouncer-poddisruptionbudget.yaml | 34 +-
chart/templates/pgbouncer/pgbouncer-service.yaml | 56 +
.../templates/rbac/pod-cleanup-role.yaml | 34 +-
.../templates/rbac/pod-cleanup-rolebinding.yaml | 34 +-
chart/templates/rbac/pod-launcher-role.yaml | 58 +
chart/templates/rbac/pod-launcher-rolebinding.yaml | 51 +
chart/templates/redis/redis-networkpolicy.yaml | 63 +
.../templates/redis/redis-service.yaml | 36 +-
chart/templates/redis/redis-statefulset.yaml | 99 +
.../templates/resourcequota.yaml | 28 +-
.../templates/scheduler/scheduler-deployment.yaml | 211 +
.../scheduler/scheduler-networkpolicy.yaml | 55 +
.../scheduler/scheduler-poddisruptionbudget.yaml | 34 +-
.../templates/scheduler/scheduler-service.yaml | 36 +-
.../scheduler/scheduler-serviceaccount.yaml | 24 +-
.../templates/secrets/elasticsearch-secret.yaml | 22 +-
.../templates/secrets/fernetkey-secret.yaml | 27 +-
.../secrets/metadata-connection-secret.yaml | 42 +
.../templates/secrets/pgbouncer-config-secret.yaml | 23 +-
.../templates/secrets/pgbouncer-stats-secret.yaml | 22 +-
chart/templates/secrets/redis-secrets.yaml | 61 +
.../templates/secrets/registry-secret.yaml | 24 +-
.../secrets/result-backend-connection-secret.yaml | 37 +
chart/templates/statsd/statsd-deployment.yaml | 87 +
chart/templates/statsd/statsd-networkpolicy.yaml | 57 +
chart/templates/statsd/statsd-service.yaml | 56 +
.../templates/webserver/webserver-deployment.yaml | 158 +
.../webserver/webserver-networkpolicy.yaml | 51 +
.../templates/webserver/webserver-service.yaml | 34 +-
chart/templates/workers/worker-deployment.yaml | 179 +
chart/templates/workers/worker-kedaautoscaler.yaml | 47 +
chart/templates/workers/worker-networkpolicy.yaml | 53 +
.../templates/workers/worker-service.yaml | 36 +-
.../templates/workers/worker-serviceaccount.yaml | 24 +-
chart/tests/dags-persistent-volume-claim_test.yaml | 64 +
chart/tests/git-sync-scheduler_test.yaml | 135 +
chart/tests/git-sync-webserver_test.yaml | 66 +
chart/tests/git-sync-worker_test.yaml | 70 +
chart/values.yaml | 586 ++
common/_default_branch.sh | 1 +
docs/best-practices.rst | 6 +-
docs/concepts.rst | 47 +-
docs/conf.py | 2 +
docs/dag-run.rst | 7 +-
docs/dag-serialization.rst | 11 +-
docs/howto/email-config.rst | 6 +-
docs/howto/set-config.rst | 29 +-
docs/howto/use-alternative-secrets-backend.rst | 5 +
docs/installation.rst | 31 +-
docs/project.rst | 8 +-
docs/start_doc_server.sh | 4 +-
docs/tutorial.rst | 2 +-
hooks/build | 4 +-
hooks/push | 5 -
images/breeze/add_overlay.sh | 49 +
images/breeze/breeze.png | Bin 0 -> 492729 bytes
images/breeze/breeze_build_docs.png | Bin 0 -> 7730 bytes
images/breeze/breeze_build_images.png | Bin 0 -> 9975 bytes
images/breeze/breeze_build_images_prod.png | Bin 0 -> 12999 bytes
.../breeze_build_images_released_versions.png | Bin 0 -> 11361 bytes
images/breeze/breeze_cloud_tools.png | Bin 0 -> 7926 bytes
images/breeze/breeze_generate_requirements.png | Bin 0 -> 10410 bytes
images/breeze/breeze_initialize_virtualenv.png | Bin 0 -> 8356 bytes
images/breeze/breeze_installation.png | Bin 0 -> 7017 bytes
images/breeze/breeze_integrations.png | Bin 0 -> 8809 bytes
images/breeze/breeze_kubernetes_tests.png | Bin 0 -> 8592 bytes
images/breeze/breeze_running_tests.png | Bin 0 -> 8343 bytes
images/breeze/breeze_select_backend_python.png | Bin 0 -> 13114 bytes
images/breeze/breeze_static_checks.png | Bin 0 -> 8901 bytes
images/breeze/breeze_stop.png | Bin 0 -> 8489 bytes
images/breeze/breeze_using_exec.png | Bin 0 -> 8415 bytes
images/breeze/breeze_using_tmux.png | Bin 0 -> 8291 bytes
images/breeze/overlayed_breeze.png | Bin 0 -> 502317 bytes
images/breeze/overlayed_breeze_build_docs.png | Bin 0 -> 9198 bytes
images/breeze/overlayed_breeze_build_images.png | Bin 0 -> 11522 bytes
.../breeze/overlayed_breeze_build_images_prod.png | Bin 0 -> 14858 bytes
...layed_breeze_build_images_released_versions.png | Bin 0 -> 13213 bytes
images/breeze/overlayed_breeze_cloud_tools.png | Bin 0 -> 9075 bytes
.../overlayed_breeze_generate_requirements.png | Bin 0 -> 11928 bytes
.../overlayed_breeze_initialize_virtualenv.png | Bin 0 -> 10465 bytes
images/breeze/overlayed_breeze_installation.png | Bin 0 -> 8985 bytes
images/breeze/overlayed_breeze_integrations.png | Bin 0 -> 10993 bytes
.../breeze/overlayed_breeze_kubernetes_tests.png | Bin 0 -> 11060 bytes
images/breeze/overlayed_breeze_running_tests.png | Bin 0 -> 10590 bytes
.../overlayed_breeze_select_backend_python.png | Bin 0 -> 14527 bytes
images/breeze/overlayed_breeze_static_checks.png | Bin 0 -> 10246 bytes
images/breeze/overlayed_breeze_stop.png | Bin 0 -> 10013 bytes
images/breeze/overlayed_breeze_using_exec.png | Bin 0 -> 10087 bytes
images/breeze/overlayed_breeze_using_tmux.png | Bin 0 -> 10137 bytes
kubernetes_tests/test_kubernetes_executor.py | 40 +-
kubernetes_tests/test_kubernetes_pod_operator.py | 1 -
requirements/REMOVE.md | 22 +
requirements/requirements-python2.7.txt | 67 +-
requirements/requirements-python3.5.txt | 77 +-
requirements/requirements-python3.6.txt | 95 +-
requirements/requirements-python3.7.txt | 92 +-
requirements/requirements-python3.8.txt | 92 +-
requirements/setup-2.7.md5 | 2 +-
requirements/setup-3.5.md5 | 2 +-
requirements/setup-3.6.md5 | 2 +-
requirements/setup-3.7.md5 | 2 +-
requirements/setup-3.8.md5 | 2 +-
.../get_workflow_id.sh} | 24 +-
scripts/ci/ci_load_image_to_kind.sh | 34 -
.../ci_generate_constraints.sh} | 10 +-
scripts/ci/docker-compose/local.yml | 1 -
scripts/ci/{ => docs}/ci_docs.sh | 6 +-
scripts/ci/{ => images}/ci_build_dockerhub.sh | 4 +-
.../ci/{ => images}/ci_prepare_ci_image_on_ci.sh | 4 +-
.../ci/{ => images}/ci_prepare_prod_image_on_ci.sh | 4 +-
scripts/ci/{ => images}/ci_push_ci_image.sh | 6 +-
.../ci/{ => images}/ci_push_production_images.sh | 6 +-
.../ci/in_container/_in_container_script_init.sh | 4 +-
scripts/ci/in_container/_in_container_utils.sh | 12 +-
scripts/ci/in_container/entrypoint_ci.sh | 20 +-
...y_airflow_to_kubernetes.sh => run_clear_tmp.sh} | 4 +-
scripts/ci/{ => in_container}/run_cli_tool.sh | 0
.../run_generate_constraints.sh} | 34 +-
.../ci/in_container/run_generate_requirements.sh | 80 -
scripts/ci/in_container/run_system_tests.sh | 4 +-
scripts/ci/kubernetes/app/postgres.yaml | 94 -
.../kubernetes/app/templates/airflow.template.yaml | 207 -
.../app/templates/configmaps.template.yaml | 395 --
.../app/templates/init_git_sync.template.yaml | 36 -
scripts/ci/kubernetes/app/volumes.yaml | 87 -
.../ci_deploy_app_to_kubernetes.sh | 20 +-
.../ci_run_helm_testing.sh} | 15 +-
.../ci/{ => kubernetes}/ci_run_kubernetes_tests.sh | 15 +-
scripts/ci/kubernetes/docker/bootstrap.sh | 74 -
scripts/ci/kubernetes/kind-cluster-conf.yaml | 3 -
.../kubernetes/{app/secrets.yaml => volumes.yaml} | 29 +-
scripts/ci/{ => libraries}/_all_libs.sh | 29 +-
scripts/ci/libraries/_build_images.sh | 62 +-
scripts/ci/libraries/_initialization.sh | 51 +-
scripts/ci/libraries/_kind.sh | 411 +-
scripts/ci/libraries/_local_mounts.sh | 1 -
scripts/ci/libraries/_runs.sh | 8 +-
scripts/ci/{ => libraries}/_script_init.sh | 13 +-
scripts/ci/libraries/_verbosity.sh | 31 +
scripts/ci/minikdc.properties | 27 -
.../ci/{ => pre_commit}/pre_commit_bat_tests.sh | 4 +-
.../{ => pre_commit}/pre_commit_breeze_cmd_line.sh | 12 +-
.../pre_commit_check_integrations.sh | 14 +-
.../{ => pre_commit}/pre_commit_check_license.sh | 2 +-
.../ci/pre_commit/pre_commit_check_order_setup.py | 135 +
scripts/ci/{ => pre_commit}/pre_commit_ci_build.sh | 4 +-
scripts/ci/{ => pre_commit}/pre_commit_flake8.sh | 4 +-
.../ci/pre_commit/pre_commit_insert_extras.py | 4 +-
.../{ => pre_commit}/pre_commit_lint_dockerfile.sh | 4 +-
.../pre_commit_local_yml_mounts.sh | 8 +-
scripts/ci/{ => pre_commit}/pre_commit_mypy.sh | 4 +-
.../ci/{ => pre_commit}/pre_commit_yaml_to_cfg.py | 9 +-
scripts/ci/pre_commit_generate_requirements.sh | 24 -
scripts/ci/{ => static_checks}/ci_bat_tests.sh | 3 +
scripts/ci/{ => static_checks}/ci_check_license.sh | 6 +-
scripts/ci/{ => static_checks}/ci_flake8.sh | 8 +-
.../ci/{ => static_checks}/ci_lint_dockerfile.sh | 4 +-
scripts/ci/{ => static_checks}/ci_mypy.sh | 8 +-
.../ci/{ => static_checks}/ci_run_static_checks.sh | 11 +-
scripts/ci/{ => testing}/ci_run_airflow_testing.sh | 35 +-
.../ci_check_if_tests_should_be_run.sh} | 64 +-
.../{ci_fix_ownership.sh => tools/ci_clear_tmp.sh} | 16 +-
scripts/ci/{ => tools}/ci_count_changed_files.sh | 25 +-
scripts/ci/{ => tools}/ci_fix_ownership.sh | 18 +-
scripts/ci/{ => tools}/ci_free_space_on_ci.sh | 4 +-
setup.py | 38 +-
tests/bats/bats_utils.bash | 4 +-
tests/cluster_policies/__init__.py | 56 +
.../dags/test_missing_owner.py | 23 +-
.../dags/test_with_non_default_owner.py | 22 +-
tests/hooks/test_postgres_hook.py | 59 +
tests/jobs/test_scheduler_job.py | 144 +
tests/kubernetes/models/test_pod.py | 83 +
.../kubernetes/models/test_volume.py | 28 +-
tests/kubernetes/test_pod_generator.py | 338 +-
tests/kubernetes/test_pod_launcher.py | 153 +-
tests/kubernetes/test_worker_configuration.py | 22 +
tests/models/test_dagbag.py | 77 +
tests/models/test_dagrun.py | 19 +
tests/models/test_serialized_dag.py | 30 +
tests/models/test_xcom.py | 50 +
tests/operators/test_sql.py | 12 +-
.../kubernetes/operators/test_kubernetes_pod.py | 13 +
tests/sensors/test_date_time_sensor.py | 72 +
tests/sensors/test_external_task_sensor.py | 28 +-
tests/test_configuration.py | 50 +-
tests/test_core.py | 6 +-
tests/test_local_settings.py | 173 -
.../secrets => test_local_settings}/__init__.py | 0
tests/test_local_settings/test_local_settings.py | 461 ++
tests/test_order_setup.py | 134 -
tests/test_utils/asserts.py | 73 +
tests/www/test_utils.py | 28 +
tests/www/test_views.py | 23 +
tests/www_rbac/test_views.py | 16 +
300 files changed, 13198 insertions(+), 6040 deletions(-)
create mode 100644 .gitattributes
copy airflow/contrib/{kubernetes/__init__.py => executors/kubernetes_executor.py} (93%)
copy scripts/ci/ci_build_production_images.sh => airflow/contrib/kubernetes/kube_client.py (71%)
mode change 100755 => 100644
create mode 100644 airflow/contrib/kubernetes/pod.py
copy scripts/ci/ci_build_production_images.sh => airflow/contrib/kubernetes/pod_runtime_info_env.py (68%)
mode change 100755 => 100644
copy scripts/ci/ci_push_ci_image.sh => airflow/contrib/kubernetes/refresh_config.py (66%)
mode change 100755 => 100644
copy scripts/ci/ci_build_production_images.sh => airflow/contrib/kubernetes/secret.py (72%)
mode change 100755 => 100644
copy scripts/ci/ci_build_production_images.sh => airflow/contrib/kubernetes/volume.py (72%)
mode change 100755 => 100644
rename scripts/ci/pre_commit_update_extras.sh => airflow/contrib/kubernetes/volume_mount.py (70%)
mode change 100755 => 100644
rename scripts/ci/kubernetes/docker/airflow-test-env-init-db.sh => airflow/migrations/versions/da3f683c3a5a_add_dag_hash_column_to_serialized_dag_.py (53%)
mode change 100755 => 100644
create mode 100644 airflow/sensors/date_time_sensor.py
create mode 100644 chart/.gitignore
rename scripts/ci/ci_build_production_images.sh => chart/.helmignore (70%)
mode change 100755 => 100644
copy airflow/version.py => chart/Chart.yaml (80%)
create mode 100644 chart/README.md
create mode 100644 chart/requirements.lock
copy airflow/version.py => chart/requirements.yaml (87%)
create mode 100644 chart/templates/NOTES.txt
create mode 100644 chart/templates/_helpers.yaml
create mode 100644 chart/templates/cleanup/cleanup-cronjob.yaml
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/cleanup/cleanup-serviceaccount.yaml (63%)
create mode 100644 chart/templates/configmap.yaml
create mode 100644 chart/templates/create-user-job.yaml
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/dags-persistent-volume-claim.yaml (52%)
create mode 100644 chart/templates/flower/flower-deployment.yaml
create mode 100644 chart/templates/flower/flower-networkpolicy.yaml
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/flower/flower-service.yaml (52%)
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/limitrange.yaml (62%)
create mode 100644 chart/templates/pgbouncer/pgbouncer-deployment.yaml
create mode 100644 chart/templates/pgbouncer/pgbouncer-networkpolicy.yaml
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/pgbouncer/pgbouncer-poddisruptionbudget.yaml (52%)
create mode 100644 chart/templates/pgbouncer/pgbouncer-service.yaml
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/rbac/pod-cleanup-role.yaml (58%)
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/rbac/pod-cleanup-rolebinding.yaml (52%)
create mode 100644 chart/templates/rbac/pod-launcher-role.yaml
create mode 100644 chart/templates/rbac/pod-launcher-rolebinding.yaml
create mode 100644 chart/templates/redis/redis-networkpolicy.yaml
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/redis/redis-service.yaml (54%)
create mode 100644 chart/templates/redis/redis-statefulset.yaml
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/resourcequota.yaml (61%)
create mode 100644 chart/templates/scheduler/scheduler-deployment.yaml
create mode 100644 chart/templates/scheduler/scheduler-networkpolicy.yaml
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/scheduler/scheduler-poddisruptionbudget.yaml (54%)
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/scheduler/scheduler-service.yaml (51%)
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/scheduler/scheduler-serviceaccount.yaml (65%)
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/secrets/elasticsearch-secret.yaml (56%)
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/secrets/fernetkey-secret.yaml (56%)
create mode 100644 chart/templates/secrets/metadata-connection-secret.yaml
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/secrets/pgbouncer-config-secret.yaml (63%)
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/secrets/pgbouncer-stats-secret.yaml (59%)
create mode 100644 chart/templates/secrets/redis-secrets.yaml
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/secrets/registry-secret.yaml (62%)
create mode 100644 chart/templates/secrets/result-backend-connection-secret.yaml
create mode 100644 chart/templates/statsd/statsd-deployment.yaml
create mode 100644 chart/templates/statsd/statsd-networkpolicy.yaml
create mode 100644 chart/templates/statsd/statsd-service.yaml
create mode 100644 chart/templates/webserver/webserver-deployment.yaml
create mode 100644 chart/templates/webserver/webserver-networkpolicy.yaml
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/webserver/webserver-service.yaml (54%)
create mode 100644 chart/templates/workers/worker-deployment.yaml
create mode 100644 chart/templates/workers/worker-kedaautoscaler.yaml
create mode 100644 chart/templates/workers/worker-networkpolicy.yaml
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/workers/worker-service.yaml (53%)
copy scripts/ci/kubernetes/app/secrets.yaml => chart/templates/workers/worker-serviceaccount.yaml (65%)
create mode 100644 chart/tests/dags-persistent-volume-claim_test.yaml
create mode 100644 chart/tests/git-sync-scheduler_test.yaml
create mode 100644 chart/tests/git-sync-webserver_test.yaml
create mode 100644 chart/tests/git-sync-worker_test.yaml
create mode 100644 chart/values.yaml
create mode 100755 images/breeze/add_overlay.sh
create mode 100644 images/breeze/breeze.png
create mode 100644 images/breeze/breeze_build_docs.png
create mode 100644 images/breeze/breeze_build_images.png
create mode 100644 images/breeze/breeze_build_images_prod.png
create mode 100644 images/breeze/breeze_build_images_released_versions.png
create mode 100644 images/breeze/breeze_cloud_tools.png
create mode 100644 images/breeze/breeze_generate_requirements.png
create mode 100644 images/breeze/breeze_initialize_virtualenv.png
create mode 100644 images/breeze/breeze_installation.png
create mode 100644 images/breeze/breeze_integrations.png
create mode 100644 images/breeze/breeze_kubernetes_tests.png
create mode 100644 images/breeze/breeze_running_tests.png
create mode 100644 images/breeze/breeze_select_backend_python.png
create mode 100644 images/breeze/breeze_static_checks.png
create mode 100644 images/breeze/breeze_stop.png
create mode 100644 images/breeze/breeze_using_exec.png
create mode 100644 images/breeze/breeze_using_tmux.png
create mode 100644 images/breeze/overlayed_breeze.png
create mode 100644 images/breeze/overlayed_breeze_build_docs.png
create mode 100644 images/breeze/overlayed_breeze_build_images.png
create mode 100644 images/breeze/overlayed_breeze_build_images_prod.png
create mode 100644 images/breeze/overlayed_breeze_build_images_released_versions.png
create mode 100644 images/breeze/overlayed_breeze_cloud_tools.png
create mode 100644 images/breeze/overlayed_breeze_generate_requirements.png
create mode 100644 images/breeze/overlayed_breeze_initialize_virtualenv.png
create mode 100644 images/breeze/overlayed_breeze_installation.png
create mode 100644 images/breeze/overlayed_breeze_integrations.png
create mode 100644 images/breeze/overlayed_breeze_kubernetes_tests.png
create mode 100644 images/breeze/overlayed_breeze_running_tests.png
create mode 100644 images/breeze/overlayed_breeze_select_backend_python.png
create mode 100644 images/breeze/overlayed_breeze_static_checks.png
create mode 100644 images/breeze/overlayed_breeze_stop.png
create mode 100644 images/breeze/overlayed_breeze_using_exec.png
create mode 100644 images/breeze/overlayed_breeze_using_tmux.png
create mode 100644 requirements/REMOVE.md
rename scripts/ci/{kubernetes/docker/airflow-test-env-init-dags.sh => cancel/get_workflow_id.sh} (68%)
delete mode 100755 scripts/ci/ci_load_image_to_kind.sh
copy scripts/ci/{ci_generate_requirements.sh => constraints/ci_generate_constraints.sh} (83%)
rename scripts/ci/{ => docs}/ci_docs.sh (90%)
rename scripts/ci/{ => images}/ci_build_dockerhub.sh (95%)
rename scripts/ci/{ => images}/ci_prepare_ci_image_on_ci.sh (87%)
rename scripts/ci/{ => images}/ci_prepare_prod_image_on_ci.sh (87%)
rename scripts/ci/{ => images}/ci_push_ci_image.sh (87%)
rename scripts/ci/{ => images}/ci_push_production_images.sh (87%)
rename scripts/ci/in_container/{deploy_airflow_to_kubernetes.sh => run_clear_tmp.sh} (84%)
rename scripts/ci/{ => in_container}/run_cli_tool.sh (100%)
rename scripts/ci/{ci_perform_kind_cluster_operation.sh => in_container/run_generate_constraints.sh} (50%)
delete mode 100755 scripts/ci/in_container/run_generate_requirements.sh
delete mode 100644 scripts/ci/kubernetes/app/postgres.yaml
delete mode 100644 scripts/ci/kubernetes/app/templates/airflow.template.yaml
delete mode 100644 scripts/ci/kubernetes/app/templates/configmaps.template.yaml
delete mode 100644 scripts/ci/kubernetes/app/templates/init_git_sync.template.yaml
delete mode 100644 scripts/ci/kubernetes/app/volumes.yaml
rename scripts/ci/{ => kubernetes}/ci_deploy_app_to_kubernetes.sh (75%)
copy scripts/ci/{ci_generate_requirements.sh => kubernetes/ci_run_helm_testing.sh} (68%)
rename scripts/ci/{ => kubernetes}/ci_run_kubernetes_tests.sh (85%)
delete mode 100755 scripts/ci/kubernetes/docker/bootstrap.sh
rename scripts/ci/kubernetes/{app/secrets.yaml => volumes.yaml} (66%)
rename scripts/ci/{ => libraries}/_all_libs.sh (68%)
rename scripts/ci/{ => libraries}/_script_init.sh (73%)
delete mode 100644 scripts/ci/minikdc.properties
rename scripts/ci/{ => pre_commit}/pre_commit_bat_tests.sh (87%)
rename scripts/ci/{ => pre_commit}/pre_commit_breeze_cmd_line.sh (85%)
rename scripts/ci/{ => pre_commit}/pre_commit_check_integrations.sh (72%)
rename scripts/ci/{ => pre_commit}/pre_commit_check_license.sh (92%)
create mode 100755 scripts/ci/pre_commit/pre_commit_check_order_setup.py
rename scripts/ci/{ => pre_commit}/pre_commit_ci_build.sh (88%)
rename scripts/ci/{ => pre_commit}/pre_commit_flake8.sh (87%)
rename tests/insert_extras.py => scripts/ci/pre_commit/pre_commit_insert_extras.py (93%)
mode change 100644 => 100755
rename scripts/ci/{ => pre_commit}/pre_commit_lint_dockerfile.sh (84%)
rename scripts/ci/{ => pre_commit}/pre_commit_local_yml_mounts.sh (86%)
rename scripts/ci/{ => pre_commit}/pre_commit_mypy.sh (87%)
rename scripts/ci/{ => pre_commit}/pre_commit_yaml_to_cfg.py (94%)
delete mode 100755 scripts/ci/pre_commit_generate_requirements.sh
rename scripts/ci/{ => static_checks}/ci_bat_tests.sh (90%)
rename scripts/ci/{ => static_checks}/ci_check_license.sh (93%)
rename scripts/ci/{ => static_checks}/ci_flake8.sh (93%)
rename scripts/ci/{ => static_checks}/ci_lint_dockerfile.sh (92%)
rename scripts/ci/{ => static_checks}/ci_mypy.sh (89%)
rename scripts/ci/{ => static_checks}/ci_run_static_checks.sh (76%)
rename scripts/ci/{ => testing}/ci_run_airflow_testing.sh (81%)
copy scripts/ci/{in_container/run_system_tests.sh => tools/ci_check_if_tests_should_be_run.sh} (52%)
copy scripts/ci/{ci_fix_ownership.sh => tools/ci_clear_tmp.sh} (79%)
rename scripts/ci/{ => tools}/ci_count_changed_files.sh (68%)
rename scripts/ci/{ => tools}/ci_fix_ownership.sh (75%)
rename scripts/ci/{ => tools}/ci_free_space_on_ci.sh (87%)
create mode 100644 tests/cluster_policies/__init__.py
rename scripts/ci/ci_generate_requirements.sh => tests/dags/test_missing_owner.py (66%)
mode change 100755 => 100644
copy hooks/build => tests/dags/test_with_non_default_owner.py (65%)
mode change 100755 => 100644
copy hooks/build => tests/kubernetes/models/test_volume.py (54%)
mode change 100755 => 100644
create mode 100644 tests/models/test_xcom.py
create mode 100644 tests/sensors/test_date_time_sensor.py
delete mode 100644 tests/test_local_settings.py
copy tests/{contrib/secrets => test_local_settings}/__init__.py (100%)
create mode 100644 tests/test_local_settings/test_local_settings.py
delete mode 100755 tests/test_order_setup.py
create mode 100644 tests/test_utils/asserts.py