You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by as...@apache.org on 2020/12/07 17:27:02 UTC
[airflow] branch v2-0-test updated (5fafd98 -> 312a281)
This is an automated email from the ASF dual-hosted git repository.
ash pushed a change to branch v2-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 5fafd98 Replace foreign key constraints with foreign annotation (#12603)
add 9a74ee5 Add 1.10.13 to CI, Breeze and Docs (#12652)
add 456a1c5 Restructure the extras in setup.py and described them (#12548)
add 531e006 Typo Fix: Deprecated config force_log_out_after was not used (#12661)
add 41a699a Implement reading provider information from packages/sources (#12512)
add 6b3c6ad Update setup.py to get non-conflicting set of dependencies (#12636)
add fa8af2d Enable PIP check for both CI and PROD image (#12664)
add 704e724 Make migrations using kube_resource_version idempotent (#12670)
add 944bd4c Fix packages errors summary for docs build (#12658)
add 3b138d2 Remove "@" references from constraints generattion (#12671)
add 543d88b Add example dag and system tests for azure wasb and fileshare (#12673)
add de3b1e6 Move connection guides to provider documentation packages (#12653)
add 76bcd08 Added `@apply_defaults` decorator. (#12620)
add 1c500ee Temporarily disable PROD image check until Azure Blob is fixed (#12679)
add 08bc62b Validate JSON schema files with JSON Schema (#12682)
add e4ab453 Setup.cfg change triggers full build (#12684)
add 66b5525 The Pyarrow limitation in install_requires is not needed. (#12683)
add b858683 Adds providers information to `airflow info` command (#12687)
add 64f1475 Fix typos and added missing descriptions in provider.yaml schema (#12690)
add 0a1b434 Move production deployments tips to docs/production-deployment.rst (#12686)
add 850b74b Use rich to render info and cheat-sheet command (#12689)
add 7ef9aa7 Replace pkg_resources with importlib.metadata to avoid VersionConflict errors (#12694)
add 8291fab Ensure that tasks set to up_for_retry have an end date (#12675)
add 02d9434 Don't use time.time() or timezone.utcnow() for duration calculations (#12353)
add c9d1ea5 Refactor airflow plugins command (#12697)
add 2037303 Adds support for Connection/Hook discovery from providers (#12466)
add 7e426d3 Improve wording of selective checks comments (#12701)
add e57de8c Remove now-incorrect warning about pools and multiple schedulers (#12709)
add bb00f16 Refine the DB query logics in www.views.task_stats() (#12707)
add bd90136 Move operator guides to provider documentation packages (#12681)
add 5e13c37 Remove deprecated dagbag metrics (#12695)
add 4ac66cf Deprecate BaseHook.get_connections method (#10135) (#10192)
add dee304b Add Getir to in the wild! (#12719)
add 42f0a3d Move apache-airflow docs to subdirectory (#12715)
add e4cb0ef Output of installing remaining packages is shown also on success (#12723)
add ebc8fcf Improve verification of images with PIP check (#12718)
add 91c2203 Stronger language re: SQLite (#12727)
add 027fd74 Fix static checks - #12715 (#12729)
add ac3a8bf Allow switching xcom_pickling to JSON/Pickle (#12724)
add 0451d84 Pins PIP to 20.2.4 in our Dockerfiles (#12738)
add a02e0f7 User-friendly output of Breeze and CI scripts (#12735)
add a697c58 Fix chart jobs delete policy for improved idempotency (#12646)
add da2a7d6 Added Headout to INTHEWILD (#12734)
add ae0e8f4 Move config item 'worker_precheck' from section [core] to [celery] (#12746)
add cba8d62 Refactor list rendering in commands (#12704)
add dab783f Don't let webserver run with dangerous config (#12747)
add 101da21 Optimize subclasses of DummyOperator for Scheduling (#12745)
add 03fa6ed Order broken DAG messages in UI (#12749)
add 2947e09 SlackWebhookHook use password instead of extra (#12674)
add 0400ee3 Allow using _CMD / _SECRET to set `[webserver] secret_key` config (#12742)
add 67acdbd Remove store_serialized_dags from config (#12754)
add 386f6b2 Refactor and speed up "DAG:" prefix permissions migration (#12720)
add 8f48f12 Fix typo in airflow/serialization/serialized_objects.py (#12767)
add 56f82ba Change DEBUG color to green in coloured logger (#12784)
add b62abfb Handle ParserError when dag is triggered with invalid execution_date (#12618)
add be7d867 BugFix: Editing a DAG run or Task Instance on UI causes an Error (#12770)
add d9d6daf Fix the exception that the port is empty when using db shell (#12740)
add 28e83c3 Prevent unused scrollbars from appearing in FF on Linux (#12795)
add 6b339c7 Avoid log spam & have more meaningful log when pull image in DockerOperator (#12763)
add 4da94b5 Clean noqa labels wrongly handled by black linter (#12791)
add 292118e Add documentation on AIRFLOW__{SECTION}__{KEY}_SECRET config (#12797)
add fa4fc37 Artifacts in Github Action have a short retention period (#12793)
add abf5104 Convert state arguments to ExternalTaskSensor to list (#12794)
add 1bd98cd Improve error handling in cli and introduce consistency (#12764)
add 88aa174 Add SMTP timeout and retry limit for SMTP email backend. (#12801)
add 6878a7b Add more json-schama checks + display all errors (#12805)
add 2936c13 Get airflow version from importlib.metadata rather than hard-coding (#12786)
add 71d8745 Add expandable groups in the docs build log (#12799)
add 70b1cc2 Move secret backends guides to provider docs (#12798)
add 37afe55 Fix paths to images in README.md (#12756)
add c1cd504 Add 'headers' to template_fields in HttpSensor (#12809)
add e82cf0d Dagrun object doesn't exist in the TriggerDagRunOperator (#12819)
add c85f494 Updating documentation to specify sensitive config options (#12820)
add fbb8a4a Cleanup & improvements around scheduling (#12815)
add 252b047 Configuration.getsection copes with sections that only exist in user config (#12816)
add cd66450 Add Telegram hook and operator (#11850)
add 12ce5be Fix for empty Graph View when task does not have a DAG during relationship setting (#12829)
add 3ff5a35 Add paused column to `dags list` sub-command (#12830)
add 348ceb1 Add Qliro to INTHEWILD.md (#12833)
add 6150e26 Add `-o` as short form option for `--output` in CLI (#12831)
add 1dcd3e1 Add support for extra links coming from the providers (#12472)
add 1f4152b Fix docstring for models.Variable.get() (#12828)
add 12c1fe2 Increments number of providers detected and stop failing (#12841)
add e9b2ff5 Add notes about PIP 20.3 breaking Airflow installation (#12840)
add df9493c Add sensors section to describe different modes of sensors (#12803)
add 37b2679 Make `airflow --help` run five times quicker (#12836)
add 4a02e0a Don't emit first_task_scheduling_delay metric for only-once dags (#12835)
add 1257b5c Fix doc preview error in editor for google operators doc (#10962)
add 3f24d0e Updated UPGRADING TO 2.0 guide based on new releases (#12847)
add c960a9f Fix Commands for Publishing constraints (#12800)
add 82dfa5f Build normal/backport providers based on correct list (#12848)
add 537aba7 Add conditional version retrieval from setup. (#12853)
add 3d2ad30 Add missing crypto and s3 extras (#12850)
add 490a01b Quarantine test TestSchedulerJob.test_scheduler_task_start_date (#12860)
add a64997b Added reference to 2.0 CLI commands in 1.10.14 (#12864)
add 75d8ff9 Mark required fields in Forms as required (#12856)
add c045ff3 Store per-task TIDeps in serialized blob (#12858)
add ed1825c Production images on CI are now built from packages (#12685)
add 298c88a Fix command for verifying signature (#12869)
add a00f250 Change the format for sha512 sum for releases (#12867)
add 3663d15 Fix links to moved production-deployment file (#12854)
add a878959 Remove old option - git_password from sensitive_config_values (#12821)
add a70cace Don't show "Access deined" message on login page (#12846)
add 7cfd240 Add start_date and end_date to DagRun View in UI (#12871)
add 5d328a2 Show DAG serialization errors in the UI. (#12866)
add 4d44faa Adds airflow as viable docker command in official image (#12878)
add 818195f Remove first_task_scheduling_delay from Updating.md (#12885)
add 1dcef78 Clarifies version args for installing 1.10 in Docker (#12875)
add 2c70ac4 Update commands to verify singature and shasum for Providers (#12886)
add eed6576 Adjust inner/nested radius to remove unwanted visual gap (#12887)
add 4fb3121 Refactor `@provide_session` to do less "at runtime" (#12868)
add 01707d7 Improve support for special characters in DbApiHook.get_uri (#12775)
add 312a281 Bugfix: Entrypoint Import Exception masked by attribute error (#12862)
No new revisions were added by this update.
Summary of changes:
.dockerignore | 3 +-
.github/workflows/build-images-workflow-run.yml | 6 +-
.github/workflows/ci.yml | 111 ++++++++-
.github/workflows/delete_old_artifacts.yml | 31 ---
.../workflows/label_when_reviewed_workflow_run.yml | 24 +-
.github/workflows/scheduled_quarantined.yml | 3 +-
.pre-commit-config.yaml | 60 ++++-
BREEZE.rst | 252 +++++++++++++++-----
CI.rst | 14 +-
CONTRIBUTING.rst | 84 ++++++-
Dockerfile | 28 ++-
Dockerfile.ci | 24 +-
IMAGES.rst | 63 +++--
INSTALL | 23 +-
INTHEWILD.md | 3 +
LOCAL_VIRTUALENV.rst | 44 ++++
README.md | 37 +--
STATIC_CODE_CHECKS.rst | 4 +-
TESTING.rst | 54 ++++-
UPDATING.md | 59 +++++
UPGRADING_TO_2.0.md | 87 +++++--
airflow/api_connexion/schemas/dag_run_schema.py | 11 +-
airflow/cli/cli_parser.py | 66 +++++-
airflow/cli/commands/celery_command.py | 5 +-
airflow/cli/commands/cheat_sheet_command.py | 46 ++--
airflow/cli/commands/config_command.py | 7 +-
airflow/cli/commands/connection_command.py | 126 ++++------
airflow/cli/commands/dag_command.py | 105 ++++----
airflow/cli/commands/db_command.py | 6 +-
airflow/cli/commands/info_command.py | 221 ++++++++---------
airflow/cli/commands/kubernetes_command.py | 2 +-
airflow/cli/commands/plugins_command.py | 75 ++++--
airflow/cli/commands/pool_command.py | 55 +++--
airflow/cli/commands/provider_command.py | 85 +++++++
airflow/cli/commands/role_command.py | 13 +-
airflow/cli/commands/task_command.py | 70 +++---
airflow/cli/commands/user_command.py | 48 ++--
airflow/cli/commands/variable_command.py | 16 +-
airflow/cli/commands/webserver_command.py | 25 +-
airflow/cli/simple_table.py | 105 ++++++++
airflow/config_templates/config.yml | 43 ++--
airflow/config_templates/default_airflow.cfg | 13 +-
airflow/config_templates/default_test.cfg | 4 +-
airflow/configuration.py | 18 +-
airflow/exceptions.py | 4 +
airflow/hooks/base_hook.py | 11 +-
airflow/hooks/dbapi_hook.py | 3 +-
airflow/jobs/scheduler_job.py | 68 +++---
airflow/lineage/__init__.py | 3 +-
...7a1ff4_add_kubernetes_resource_checkpointing.py | 40 ++--
.../849da589634d_prefix_dag_permissions.py | 205 ++++++++++------
.../bef4f3d11e8b_drop_kuberesourceversion_and_.py | 21 +-
airflow/models/baseoperator.py | 30 ++-
airflow/models/connection.py | 81 +------
airflow/models/dag.py | 2 +-
airflow/models/dagbag.py | 70 +++---
airflow/models/dagrun.py | 86 +++----
airflow/models/taskinstance.py | 81 +++----
airflow/models/variable.py | 2 +-
airflow/models/xcom.py | 5 +-
airflow/operators/dagrun_operator.py | 2 +
airflow/operators/dummy_operator.py | 1 +
airflow/plugins_manager.py | 61 ++---
airflow/provider.yaml.schema.json | 34 ++-
airflow/providers/amazon/aws/hooks/base_aws.py | 2 +-
airflow/providers/amazon/aws/hooks/sagemaker.py | 6 +-
airflow/providers/amazon/aws/operators/datasync.py | 2 +-
airflow/providers/amazon/aws/operators/ecs.py | 2 +-
airflow/providers/amazon/aws/operators/glacier.py | 2 +-
airflow/providers/amazon/aws/sensors/glacier.py | 2 +-
.../amazon/aws/sensors/sagemaker_training.py | 2 +-
.../amazon/aws/transfers/glacier_to_gcs.py | 2 +-
.../amazon/aws/transfers/imap_attachment_to_s3.py | 2 +-
.../amazon/aws/transfers/s3_to_redshift.py | 2 +-
airflow/providers/amazon/provider.yaml | 16 +-
.../providers/apache/cassandra/hooks/cassandra.py | 6 +-
airflow/providers/apache/cassandra/provider.yaml | 5 +-
.../providers/apache/cassandra/sensors/record.py | 2 +-
.../providers/apache/cassandra/sensors/table.py | 2 +-
airflow/providers/apache/hive/hooks/hive.py | 7 +-
airflow/providers/apache/hive/provider.yaml | 4 +
.../providers/apache/kylin/operators/kylin_cube.py | 5 +-
airflow/providers/apache/pig/hooks/pig.py | 6 +-
airflow/providers/apache/pig/provider.yaml | 4 +
.../providers/apache/spark/operators/spark_jdbc.py | 2 +-
.../providers/apache/spark/operators/spark_sql.py | 2 +-
.../apache/spark/operators/spark_submit.py | 2 +-
airflow/providers/apache/spark/provider.yaml | 2 +-
airflow/providers/cloudant/hooks/cloudant.py | 6 +-
airflow/providers/cloudant/provider.yaml | 3 +
.../providers/cncf/kubernetes/hooks/kubernetes.py | 8 +-
.../cncf/kubernetes/operators/kubernetes_pod.py | 2 +-
airflow/providers/cncf/kubernetes/provider.yaml | 5 +-
.../providers/databricks/operators/databricks.py | 2 +-
airflow/providers/databricks/provider.yaml | 2 +-
airflow/providers/dingding/provider.yaml | 2 +-
airflow/providers/discord/README.md | 1 +
airflow/providers/docker/hooks/docker.py | 6 +-
airflow/providers/docker/operators/docker.py | 13 +-
airflow/providers/docker/provider.yaml | 3 +
.../providers/elasticsearch/hooks/elasticsearch.py | 1 +
airflow/providers/elasticsearch/provider.yaml | 3 +
airflow/providers/exasol/hooks/exasol.py | 1 +
airflow/providers/exasol/provider.yaml | 3 +
airflow/providers/google/cloud/hooks/bigquery.py | 4 +-
airflow/providers/google/cloud/hooks/cloud_sql.py | 7 +-
.../cloud/hooks/cloud_storage_transfer_service.py | 4 +-
.../providers/google/cloud/hooks/compute_ssh.py | 4 +
airflow/providers/google/cloud/hooks/dataprep.py | 6 +-
.../providers/google/cloud/operators/bigquery.py | 1 +
airflow/providers/google/provider.yaml | 11 +
airflow/providers/grpc/hooks/grpc.py | 6 +-
airflow/providers/grpc/provider.yaml | 3 +
airflow/providers/http/operators/http.py | 2 +-
airflow/providers/http/provider.yaml | 2 +-
airflow/providers/http/sensors/http.py | 4 +-
airflow/providers/imap/hooks/imap.py | 6 +-
airflow/providers/imap/provider.yaml | 3 +
airflow/providers/jdbc/hooks/jdbc.py | 1 +
airflow/providers/jdbc/operators/jdbc.py | 2 +-
airflow/providers/jdbc/provider.yaml | 5 +-
airflow/providers/jira/hooks/jira.py | 6 +-
airflow/providers/jira/provider.yaml | 3 +
.../azure/example_dags/example_file_to_wasb.py | 39 ++-
.../azure/example_dags/example_fileshare.py | 47 ++++
.../providers/microsoft/azure/hooks/azure_batch.py | 6 +-
.../microsoft/azure/hooks/azure_cosmos.py | 6 +-
.../microsoft/azure/hooks/azure_data_lake.py | 6 +-
airflow/providers/microsoft/azure/hooks/wasb.py | 18 +-
airflow/providers/microsoft/azure/provider.yaml | 10 +-
.../microsoft/azure/transfers/azure_blob_to_gcs.py | 2 +-
.../microsoft/azure/transfers/local_to_adls.py | 2 +-
airflow/providers/microsoft/mssql/hooks/mssql.py | 12 +-
airflow/providers/microsoft/mssql/provider.yaml | 3 +
airflow/providers/mongo/hooks/mongo.py | 4 +-
airflow/providers/mongo/provider.yaml | 3 +
airflow/providers/mysql/hooks/mysql.py | 1 +
airflow/providers/mysql/operators/mysql.py | 2 +-
airflow/providers/mysql/provider.yaml | 5 +-
airflow/providers/odbc/hooks/odbc.py | 3 +-
airflow/providers/odbc/provider.yaml | 3 +
airflow/providers/oracle/hooks/oracle.py | 1 +
airflow/providers/oracle/provider.yaml | 5 +
airflow/providers/papermill/provider.yaml | 2 +-
airflow/providers/postgres/hooks/postgres.py | 1 +
airflow/providers/postgres/provider.yaml | 3 +
airflow/providers/presto/hooks/presto.py | 1 +
airflow/providers/presto/provider.yaml | 3 +
airflow/providers/qubole/provider.yaml | 3 +
airflow/providers/redis/hooks/redis.py | 6 +-
airflow/providers/redis/provider.yaml | 3 +
airflow/providers/salesforce/hooks/tableau.py | 6 +-
airflow/providers/salesforce/provider.yaml | 3 +
airflow/providers/slack/hooks/slack_webhook.py | 18 +-
airflow/providers/snowflake/hooks/snowflake.py | 1 +
airflow/providers/snowflake/operators/snowflake.py | 2 +-
airflow/providers/snowflake/provider.yaml | 7 +-
.../snowflake/transfers/s3_to_snowflake.py | 2 +-
.../snowflake/transfers/snowflake_to_slack.py | 2 +-
airflow/providers/sqlite/hooks/sqlite.py | 1 +
airflow/providers/sqlite/provider.yaml | 3 +
airflow/providers/telegram/README.md | 59 +++++
.../providers/telegram}/__init__.py | 0
.../providers/telegram/example_dags}/__init__.py | 0
.../example_dags/example_telegram.py} | 49 ++--
.../providers/telegram/hooks}/__init__.py | 0
airflow/providers/telegram/hooks/telegram.py | 152 ++++++++++++
.../providers/telegram/operators}/__init__.py | 0
airflow/providers/telegram/operators/telegram.py | 84 +++++++
.../providers/{dingding => telegram}/provider.yaml | 23 +-
airflow/providers/vertica/hooks/vertica.py | 1 +
airflow/providers/vertica/provider.yaml | 3 +
airflow/providers/yandex/provider.yaml | 2 +-
airflow/providers_manager.py | 253 +++++++++++++++++---
airflow/secrets/base_secrets.py | 25 +-
airflow/secrets/local_filesystem.py | 25 +-
airflow/secrets/metastore.py | 24 +-
airflow/sensors/base_sensor_operator.py | 47 ++--
airflow/sensors/external_task_sensor.py | 18 +-
airflow/sensors/smart_sensor_operator.py | 17 +-
airflow/serialization/json_schema.py | 1 -
airflow/serialization/schema.json | 7 +
airflow/serialization/serialized_objects.py | 189 ++++++++++-----
airflow/settings.py | 6 +-
airflow/stats.py | 100 ++++++--
airflow/utils/cli.py | 33 ++-
airflow/utils/dag_processing.py | 22 +-
airflow/utils/email.py | 35 ++-
.../dummy_operator.py => utils/entry_points.py} | 32 +--
airflow/utils/log/colored_log.py | 2 +-
airflow/utils/log/file_task_handler.py | 8 +-
airflow/utils/orm_event_handlers.py | 4 +-
airflow/utils/session.py | 20 +-
airflow/version.py | 18 +-
airflow/www/forms.py | 67 +++++-
airflow/www/static/css/bootstrap-theme.css | 4 +-
airflow/www/static/css/main.css | 2 +-
airflow/www/templates/appbuilder/flash.html | 11 +-
airflow/www/templates/appbuilder/navbar_menu.html | 5 +-
airflow/www/utils.py | 4 +-
airflow/www/views.py | 65 +++--
airflow/www/widgets.py | 26 +-
breeze | 212 +++++++++++------
breeze-complete | 15 +-
chart/templates/create-user-job.yaml | 2 +-
chart/templates/migrate-database-job.yaml | 2 +-
dev/PROVIDER_PACKAGE_DETAILS.md | 24 +-
dev/README.md | 2 -
dev/README_RELEASE_AIRFLOW.md | 14 +-
dev/README_RELEASE_PROVIDER_PACKAGES.md | 34 +--
dev/provider_packages/MANIFEST_TEMPLATE.in.jinja2 | 2 -
.../PROVIDER_README_TEMPLATE.md.jinja2 | 8 +
dev/provider_packages/README.md | 23 +-
dev/provider_packages/SETUP_TEMPLATE.cfg.jinja2 | 4 +
dev/provider_packages/SETUP_TEMPLATE.py.jinja2 | 13 -
dev/provider_packages/build_source_package.sh | 6 +-
.../enter_breeze_provider_package_tests.sh | 2 -
.../get_provider_info_TEMPLATE.py.jinja2 | 6 +-
dev/provider_packages/prepare_provider_packages.py | 66 ++++--
dev/sign.sh | 2 +-
docker-context-files/README.md | 2 +-
docs/README.rst | 6 -
.../connections}/aws.rst | 0
.../img/aws-web-identity-federation-gcp.png | Bin
docs/apache-airflow-providers-amazon/index.rst | 8 +
.../operators}/_partials/prerequisite_tasks.rst | 4 +-
.../operators}/datasync.rst | 12 +-
.../operators}/ecs.rst | 2 +-
.../operators}/emr.rst | 6 +-
.../operators}/glacier.rst | 4 +-
.../operators}/google_api_to_s3_transfer.rst | 16 +-
.../operators}/imap_attachment_to_s3.rst | 4 +-
.../operators}/index.rst | 5 +-
.../operators}/s3_to_redshift.rst | 4 +-
.../operators}/transfer/glacier_to_gcs.rst | 2 +-
.../operators}/transfer/index.rst | 0
docs/apache-airflow-providers-amazon/redirects.txt | 1 +
.../secrets-backends/aws-secrets-manaager.rst} | 0
.../secrets-backends/aws-ssm-parameter-store.rst} | 0
.../secrets-backends}/index.rst | 5 +-
.../connections}/cassandra.rst | 0
.../index.rst | 7 +
.../operators.rst} | 6 +-
.../redirects.txt | 1 +
.../connections}/spark.rst | 0
.../index.rst | 7 +
.../operators.rst} | 8 +-
.../redirects.txt | 1 +
.../connections}/kubernetes.rst | 0
.../index.rst | 7 +
.../operators.rst} | 8 +-
.../redirects.txt | 1 +
docs/apache-airflow-providers-databricks/index.rst | 6 +
.../operators.rst} | 4 +-
docs/apache-airflow-providers-dingding/index.rst | 6 +
.../operators.rst} | 8 +-
.../operators/ads.rst | 2 +-
.../operators/cloud/automl.rst | 2 +-
.../operators/cloud/bigquery.rst | 2 +-
.../operators/cloud/bigquery_dts.rst | 2 +-
.../operators/cloud/bigtable.rst | 2 +-
.../operators/cloud/cloud_build.rst | 2 +-
.../operators/cloud/cloud_memorystore.rst | 2 +-
.../operators/cloud/cloud_sql.rst | 2 +-
.../cloud/cloud_storage_transfer_service.rst | 4 +-
.../operators/cloud/compute.rst | 2 +-
.../operators/cloud/data_loss_prevention.rst | 2 +-
.../operators/cloud/datacatalog.rst | 2 +-
.../operators/cloud/datafusion.rst | 2 +-
.../operators/cloud/dataprep.rst | 4 +-
.../operators/cloud/dataproc.rst | 2 +-
.../operators/cloud/datastore.rst | 2 +-
.../operators/cloud/functions.rst | 2 +-
.../operators/cloud/gcs.rst | 2 +-
.../operators/cloud/kubernetes_engine.rst | 2 +-
.../operators/cloud/life_sciences.rst | 2 +-
.../operators/cloud/natural_language.rst | 2 +-
.../operators/cloud/pubsub.rst | 2 +-
.../operators/cloud/spanner.rst | 2 +-
.../operators/cloud/speech_to_text.rst | 2 +-
.../operators/cloud/stackdriver.rst | 2 +-
.../operators/cloud/text_to_speech.rst | 2 +-
.../operators/cloud/translate.rst | 2 +-
.../operators/cloud/translate_speech.rst | 2 +-
.../operators/cloud/video_intelligence.rst | 19 +-
.../operators/cloud/vision.rst | 2 +-
.../operators/firebase/firestore.rst | 2 +-
.../operators/marketing_platform/analytics.rst | 2 +-
.../marketing_platform/campaign_manager.rst | 2 +-
.../operators/marketing_platform/display_video.rst | 2 +-
.../operators/marketing_platform/search_ads.rst | 2 +-
.../operators/suite/sheets.rst | 2 +-
.../operators/transfer/facebook_ads_to_gcs.rst | 2 +-
.../operators/transfer/gcs_to_gcs.rst | 2 +-
.../operators/transfer/gcs_to_gdrive.rst | 2 +-
.../operators/transfer/gcs_to_local.rst | 2 +-
.../operators/transfer/gcs_to_sftp.rst | 2 +-
.../operators/transfer/gcs_to_sheets.rst | 2 +-
.../operators/transfer/local_to_gcs.rst | 2 +-
.../operators/transfer/sftp_to_gcs.rst | 2 +-
.../operators/transfer/sheets_to_gcs.rst | 2 +-
docs/apache-airflow-providers-google/redirects.txt | 2 +
.../connections}/grpc.rst | 0
docs/apache-airflow-providers-grpc/index.rst | 6 +
docs/apache-airflow-providers-grpc/redirects.txt | 1 +
docs/apache-airflow-providers-hashicorp/index.rst | 6 +
.../redirects.txt | 1 +
.../secrets-backends/hashicorp-vault.rst} | 0
docs/apache-airflow-providers-http/index.rst | 6 +
.../operators.rst} | 15 +-
.../connections}/jdbc.rst | 0
docs/apache-airflow-providers-jdbc/index.rst | 12 +
.../operators.rst} | 6 +-
docs/apache-airflow-providers-jdbc/redirects.txt | 1 +
.../connections}/azure.rst | 0
.../index.rst | 8 +
.../operators}/_partials/prerequisite_tasks.rst | 4 +-
.../operators/azure_blob_to_gcs.rst} | 4 +-
.../operators}/index.rst | 4 +-
.../operators}/local_to_adls.rst | 4 +-
.../redirects.txt | 2 +
.../secrets-backends/azure-key-vault.rst} | 0
.../connections}/mysql.rst | 0
docs/apache-airflow-providers-mysql/index.rst | 7 +
.../operators.rst} | 6 +-
docs/apache-airflow-providers-mysql/redirects.txt | 1 +
.../connections}/odbc.rst | 0
docs/apache-airflow-providers-odbc/index.rst | 6 +
docs/apache-airflow-providers-odbc/redirects.txt | 1 +
.../connections}/oracle.rst | 0
docs/apache-airflow-providers-oracle/index.rst | 6 +
docs/apache-airflow-providers-papermill/index.rst | 6 +
.../operators.rst} | 2 +-
.../connections}/postgres.rst | 0
docs/apache-airflow-providers-postgres/index.rst | 6 +
.../redirects.txt | 1 +
.../connections}/salesforce.rst | 0
docs/apache-airflow-providers-salesforce/index.rst | 6 +
.../redirects.txt | 1 +
docs/apache-airflow-providers-snowflake/index.rst | 6 +
.../operators}/index.rst | 7 +-
.../operators/s3_to_snowflake.rst | 42 ++++
.../operators/snowflake.rst} | 45 ++--
.../operators/snowflake_to_slack.rst | 52 ++++
.../connections}/ssh.rst | 0
docs/apache-airflow-providers-ssh/index.rst | 6 +
.../index.rst | 12 +-
.../operators.rst | 59 +++++
.../connections}/yandexcloud.rst | 0
docs/apache-airflow-providers-yandex/index.rst | 7 +
.../operators.rst} | 0
docs/{ => apache-airflow}/backport-providers.rst | 0
docs/{ => apache-airflow}/best-practices.rst | 102 +-------
docs/{ => apache-airflow}/changelog.rst | 2 +-
.../cli-and-env-variables-ref.rst | 20 ++
docs/{ => apache-airflow}/concepts.rst | 102 ++++++--
docs/{ => apache-airflow}/configurations-ref.rst | 9 +
docs/{ => apache-airflow}/dag-run.rst | 0
docs/{ => apache-airflow}/dag-serialization.rst | 0
docs/{ => apache-airflow}/executor/celery.rst | 0
.../executor/celery_kubernetes.rst | 0
docs/{ => apache-airflow}/executor/dask.rst | 0
docs/{ => apache-airflow}/executor/debug.rst | 0
docs/{ => apache-airflow}/executor/index.rst | 0
docs/{ => apache-airflow}/executor/kubernetes.rst | 12 +-
docs/{ => apache-airflow}/executor/local.rst | 0
docs/{ => apache-airflow}/executor/sequential.rst | 0
docs/{ => apache-airflow}/extra-packages-ref.rst | 174 +++++++++-----
docs/{ => apache-airflow}/faq.rst | 0
docs/{ => apache-airflow}/howto/add-dag-tags.rst | 0
.../howto/connection.rst} | 15 +-
.../{ => apache-airflow}/howto/custom-operator.rst | 2 +-
.../howto/customize-state-colors-ui.rst | 0
.../howto/define_extra_link.rst | 0
docs/{ => apache-airflow}/howto/email-config.rst | 0
docs/{ => apache-airflow}/howto/index.rst | 2 +-
.../howto/initialize-database.rst | 0
docs/{ => apache-airflow}/howto/operator/bash.rst | 4 +-
.../howto/operator/external_task_sensor.rst | 4 +-
docs/{ => apache-airflow}/howto/operator/index.rst | 14 +-
.../{ => apache-airflow}/howto/operator/python.rst | 6 +-
.../howto/run-behind-proxy.rst | 0
.../howto/run-with-systemd.rst | 0
.../howto/run-with-upstart.rst | 0
docs/{ => apache-airflow}/howto/set-config.rst | 2 +-
.../{ => apache-airflow}/howto/use-test-config.rst | 0
docs/{ => apache-airflow}/howto/variable.rst | 0
docs/{ => apache-airflow}/img/add-dag-tags.png | Bin
docs/{ => apache-airflow}/img/add-role.png | Bin
docs/{ => apache-airflow}/img/airflow.gif | Bin
docs/{ => apache-airflow}/img/apache.jpg | Bin
docs/{ => apache-airflow}/img/arch-diag-basic.png | Bin
.../img/arch-diag-kubernetes.png | Bin
.../img/arch-diag-kubernetes2.png | Bin
.../{ => apache-airflow}/img/arch-diag-logging.png | Bin
.../img/aws-web-identity-federation-gcp.png | Bin
docs/{ => apache-airflow}/img/branch_note.png | Bin
.../img/branch_with_trigger.png | Bin
.../img/branch_without_trigger.png | Bin
.../img/change-ui-colors/dags-page-new.png | Bin
.../img/change-ui-colors/dags-page-old.png | Bin
.../img/change-ui-colors/graph-view-new.png | Bin
.../img/change-ui-colors/graph-view-old.png | Bin
.../img/change-ui-colors/tree-view-new.png | Bin
.../img/change-ui-colors/tree-view-old.png | Bin
docs/{ => apache-airflow}/img/cli_completion.gif | Bin
docs/{ => apache-airflow}/img/code.png | Bin
.../{ => apache-airflow}/img/connection_create.png | Bin
docs/{ => apache-airflow}/img/connection_edit.png | Bin
docs/{ => apache-airflow}/img/connections.png | Bin
docs/{ => apache-airflow}/img/context.png | Bin
.../{ => apache-airflow}/img/dag_serialization.png | Bin
docs/{ => apache-airflow}/img/dags.png | Bin
docs/{ => apache-airflow}/img/duration.png | Bin
.../img/example_passing_conf.png | Bin
docs/{ => apache-airflow}/img/gantt.png | Bin
docs/{ => apache-airflow}/img/graph.png | Bin
docs/{ => apache-airflow}/img/k8s-0-worker.jpeg | Bin
docs/{ => apache-airflow}/img/k8s-3-worker.jpeg | Bin
docs/{ => apache-airflow}/img/k8s-5-worker.jpeg | Bin
docs/{ => apache-airflow}/img/k8s-failed-pod.png | Bin
docs/{ => apache-airflow}/img/k8s-happy-path.png | Bin
.../img/latest_only_with_trigger.png | Bin
.../img/logos/airflow_64x64_emoji_transparent.png | Bin
.../img/logos/airflow_dark_bg.png | Bin
.../img/logos/airflow_transparent.png | Bin
.../img/logos/airflow_white_bg.png | Bin
docs/{ => apache-airflow}/img/logos/wordmark_1.png | Bin
docs/{ => apache-airflow}/img/logos/wordmark_1.svg | 0
docs/{ => apache-airflow}/img/logos/wordmark_2.png | Bin
docs/{ => apache-airflow}/img/logos/wordmark_2.svg | 0
docs/{ => apache-airflow}/img/nested_branching.png | Bin
docs/{ => apache-airflow}/img/new-role.png | Bin
.../img/operator_extra_link.png | Bin
.../img/run_task_on_celery_executor.png | Bin
.../img/run_task_on_celery_executor.puml | 0
docs/{ => apache-airflow}/img/scheduler_loop.jpg | Bin
.../img/smart_sensor_architecture.png | Bin
.../img/smart_sensor_single_task_execute_flow.png | Bin
docs/{ => apache-airflow}/img/subdag_after.png | Bin
docs/{ => apache-airflow}/img/subdag_before.png | Bin
docs/{ => apache-airflow}/img/subdag_zoom.png | Bin
docs/{ => apache-airflow}/img/task_group.gif | Bin
.../img/task_lifecycle_diagram.png | Bin
.../img/task_manual_vs_scheduled.png | Bin
docs/{ => apache-airflow}/img/task_stages.png | Bin
docs/{ => apache-airflow}/img/tree.png | Bin
.../img/ui-timezone-chooser.png | Bin
docs/{ => apache-airflow}/img/usage_cli_export.png | Bin
docs/{ => apache-airflow}/img/usage_cli_imgcat.png | Bin
docs/{ => apache-airflow}/img/variable_hidden.png | Bin
docs/{ => apache-airflow}/index.rst | 4 +-
docs/{ => apache-airflow}/installation.rst | 33 ++-
docs/{ => apache-airflow}/integration.rst | 0
docs/{ => apache-airflow}/kubernetes.rst | 2 +-
docs/{ => apache-airflow}/license.rst | 0
docs/{ => apache-airflow}/lineage.rst | 0
.../logging-monitoring/check-health.rst | 0
.../logging-monitoring/errors.rst | 0
.../logging-monitoring/index.rst | 0
.../logging-monitoring/logging-architecture.rst | 0
.../logging-monitoring/logging-tasks.rst | 0
.../logging-monitoring/metrics.rst | 2 +-
.../logging-monitoring/tracking-user-activity.rst | 0
docs/{ => apache-airflow}/macros-ref.rst | 0
docs/{ => apache-airflow}/modules_management.rst | 68 ++++--
.../operators-and-hooks-ref.rst | 0
docs/{ => apache-airflow}/plugins.rst | 0
docs/{ => apache-airflow}/privacy_notice.rst | 0
.../{ => apache-airflow}/production-deployment.rst | 202 ++++++++++++----
docs/{ => apache-airflow}/project.rst | 0
.../python-api-ref.rst} | 14 +-
docs/{ => apache-airflow}/redirects.txt | 1 +
docs/{ => apache-airflow}/rest-api-ref.rst | 0
docs/{ => apache-airflow}/scheduler.rst | 0
.../security/access-control.rst | 6 +-
docs/{ => apache-airflow}/security/api.rst | 0
docs/{ => apache-airflow}/security/flower.rst | 0
docs/{ => apache-airflow}/security/index.rst | 2 +-
docs/{ => apache-airflow}/security/kerberos.rst | 0
.../security/secrets/fernet.rst | 0
.../security/secrets/index.rst | 0
.../security/secrets/secrets-backend/index.rst | 2 +-
.../local-filesystem-secrets-backend.rst | 0
docs/{ => apache-airflow}/security/webserver.rst | 0
docs/{ => apache-airflow}/security/workload.rst | 0
docs/{ => apache-airflow}/smart-sensor.rst | 0
docs/{ => apache-airflow}/stable-rest-api-ref.rst | 0
docs/{ => apache-airflow}/start.rst | 9 +
.../{ => apache-airflow}/static/exampleinclude.css | 0
docs/{ => apache-airflow}/static/jira-links.js | 0
docs/{ => apache-airflow}/templates/footer.html | 0
docs/{ => apache-airflow}/timezone.rst | 0
docs/{ => apache-airflow}/tutorial.rst | 16 +-
.../{ => apache-airflow}/tutorial_taskflow_api.rst | 18 +-
docs/{ => apache-airflow}/ui.rst | 0
docs/{ => apache-airflow}/usage-cli.rst | 37 +++
docs/build_docs.py | 56 +++--
docs/conf.py | 28 +--
docs/exts/docs_build/errors.py | 2 +-
docs/exts/docs_build/lint_checks.py | 1 -
docs/exts/redirects.py | 4 +-
docs/howto/operator/amazon/aws/index.rst | 33 ---
docs/howto/operator/snowflake.rst | 124 ----------
.../google-cloud-secret-manager-backend.rst | 207 ----------------
docs/spelling_wordlist.txt | 4 +
.../ci_build_airflow_package.sh} | 13 +-
scripts/ci/docker-compose/base.yml | 3 +-
scripts/ci/docker-compose/files.yml | 1 +
scripts/ci/docker-compose/local-prod.yml | 1 -
scripts/ci/docker-compose/local.yml | 1 -
scripts/ci/dockerfiles/presto/entrypoint.sh | 7 +-
scripts/ci/images/ci_prepare_ci_image_on_ci.sh | 1 -
scripts/ci/images/ci_prepare_prod_image_on_ci.sh | 112 +++++----
scripts/ci/images/ci_verify_ci_image.sh | 55 +++++
scripts/ci/images/ci_verify_prod_image.sh | 95 ++++++++
scripts/ci/images/ci_wait_for_all_ci_images.sh | 36 +--
scripts/ci/images/ci_wait_for_all_prod_images.sh | 38 +--
.../ci_wait_for_ci_image.sh} | 26 +-
.../ci_wait_for_prod_image.sh} | 19 +-
scripts/ci/kubernetes/ci_run_kubernetes_tests.sh | 2 +-
scripts/ci/libraries/_all_libs.sh | 2 +
.../_build_airflow_packages.sh} | 36 +--
scripts/ci/libraries/_build_images.sh | 199 +++++++++++-----
scripts/ci/libraries/_docker.env | 3 +
scripts/ci/libraries/_initialization.sh | 66 ++++--
scripts/ci/libraries/_kind.sh | 30 +--
scripts/ci/libraries/_local_mounts.sh | 1 -
scripts/ci/libraries/_md5sum.sh | 1 -
scripts/ci/libraries/_parameters.sh | 17 +-
scripts/ci/libraries/_push_pull_remove_images.sh | 82 +++++--
scripts/ci/libraries/_repeats.sh | 13 +-
scripts/ci/libraries/_runs.sh | 18 +-
scripts/ci/libraries/_sanity_checks.sh | 68 +++---
.../ci/pre_commit/pre_commit_check_pre_commits.sh | 40 ++--
.../pre_commit_check_provider_yaml_files.py | 117 +++++----
.../pre_commit/pre_commit_check_providers_init.sh | 6 +-
.../pre_commit_check_setup_extra_packages_ref.py | 19 +-
scripts/ci/pre_commit/pre_commit_json_schema.py | 164 +++++++++++++
scripts/ci/pre_commit/pre_commit_mermaid.sh | 6 +-
...sh => ci_install_and_test_provider_packages.sh} | 11 +-
...er_readme.sh => ci_prepare_provider_readmes.sh} | 0
scripts/ci/selective_ci_checks.sh | 15 +-
scripts/ci/static_checks/check_license.sh | 15 +-
scripts/ci/testing/ci_run_airflow_testing.sh | 5 +-
scripts/in_container/_in_container_script_init.sh | 6 +-
scripts/in_container/_in_container_utils.sh | 263 ++++++++++++++++-----
scripts/in_container/check_environment.sh | 9 +-
scripts/in_container/configure_environment.sh | 3 -
scripts/in_container/entrypoint_ci.sh | 62 ++++-
scripts/in_container/entrypoint_exec.sh | 3 +
scripts/in_container/prod/entrypoint_prod.sh | 22 +-
scripts/in_container/run_ci_tests.sh | 83 ++++---
scripts/in_container/run_cli_tool.sh | 20 +-
scripts/in_container/run_generate_constraints.sh | 1 +
.../run_install_and_test_provider_packages.sh | 154 ++++++++++++
.../in_container/run_prepare_provider_packages.sh | 61 +++--
.../in_container/run_prepare_provider_readme.sh | 9 +-
scripts/in_container/run_system_tests.sh | 2 +
.../run_test_package_import_all_classes.sh | 107 ---------
scripts/in_container/run_test_package_install.sh | 72 ------
setup.cfg | 7 +-
setup.py | 87 +++++--
tests/api_connexion/schemas/test_dag_run_schema.py | 7 +
tests/bats/breeze/test_breeze_complete.bats | 8 +
tests/bats/breeze/test_breeze_params.bats | 17 +-
tests/bats/in_container/test_in_container.bats | 2 +
tests/cli/commands/test_celery_command.py | 6 +-
tests/cli/commands/test_cheat_sheet_command.py | 26 +-
tests/cli/commands/test_config_command.py | 12 +-
tests/cli/commands/test_connection_command.py | 45 ++--
tests/cli/commands/test_dag_command.py | 17 +-
tests/cli/commands/test_db_command.py | 45 +++-
tests/cli/commands/test_info_command.py | 46 ++--
tests/cli/commands/test_plugins_command.py | 16 +-
tests/cli/commands/test_pool_command.py | 5 +-
tests/cli/commands/test_role_command.py | 2 +-
tests/cli/commands/test_task_command.py | 43 ++--
tests/cli/commands/test_user_command.py | 2 +-
tests/cli/commands/test_webserver_command.py | 18 +-
tests/core/test_configuration.py | 18 +-
tests/core/test_providers_manager.py | 62 ++++-
tests/dags/test_only_dummy_tasks.py | 12 +-
tests/hooks/test_dbapi_hook.py | 13 +
tests/jobs/test_scheduler_job.py | 1 +
tests/models/test_connection.py | 14 --
tests/models/test_dagbag.py | 41 +++-
tests/models/test_dagrun.py | 75 +++---
tests/models/test_taskinstance.py | 14 ++
tests/models/test_xcom.py | 30 +++
tests/operators/test_dagrun_operator.py | 30 +++
tests/plugins/test_plugins_manager.py | 80 ++++---
tests/providers/amazon/aws/hooks/test_sagemaker.py | 2 +-
tests/providers/apache/hive/hooks/test_hive.py | 2 +-
.../google/cloud/operators/test_cloud_sql.py | 16 +-
tests/providers/google/suite/hooks/test_drive.py | 8 +-
.../microsoft/azure/hooks/test_fileshare_system.py | 32 +--
tests/providers/microsoft/azure/hooks/test_wasb.py | 19 +-
.../microsoft/azure/log/test_wasb_task_handler.py | 17 ++
.../azure/transfers/test_file_to_wasb_system.py | 48 ++++
tests/providers/slack/hooks/test_slack_webhook.py | 19 ++
.../telegram}/__init__.py | 0
.../telegram/hooks}/__init__.py | 0
tests/providers/telegram/hooks/test_telegram.py | 192 +++++++++++++++
.../telegram/operators}/__init__.py | 0
.../providers/telegram/operators/test_telegram.py | 149 ++++++++++++
tests/secrets/test_secrets.py | 24 +-
tests/sensors/test_base_sensor.py | 16 +-
tests/serialization/test_dag_serialization.py | 28 +++
tests/test_utils/azure_system_helpers.py | 35 +++
tests/utils/test_email.py | 127 ++++++++--
tests/utils/test_task_group.py | 17 ++
tests/www/test_views.py | 46 +---
613 files changed, 7445 insertions(+), 3692 deletions(-)
delete mode 100644 .github/workflows/delete_old_artifacts.yml
create mode 100644 airflow/cli/commands/provider_command.py
create mode 100644 airflow/cli/simple_table.py
copy tests/dags/test_only_dummy_tasks.py => airflow/providers/microsoft/azure/example_dags/example_file_to_wasb.py (54%)
create mode 100644 airflow/providers/microsoft/azure/example_dags/example_fileshare.py
create mode 100644 airflow/providers/telegram/README.md
copy {tests/www/api/experimental => airflow/providers/telegram}/__init__.py (100%)
copy {tests/www/api/experimental => airflow/providers/telegram/example_dags}/__init__.py (100%)
copy airflow/providers/{databricks/provider.yaml => telegram/example_dags/example_telegram.py} (58%)
copy {tests/www/api/experimental => airflow/providers/telegram/hooks}/__init__.py (100%)
create mode 100644 airflow/providers/telegram/hooks/telegram.py
copy {tests/www/api/experimental => airflow/providers/telegram/operators}/__init__.py (100%)
create mode 100644 airflow/providers/telegram/operators/telegram.py
copy airflow/providers/{dingding => telegram}/provider.yaml (68%)
copy airflow/{operators/dummy_operator.py => utils/entry_points.py} (52%)
copy airflow/version.py => dev/provider_packages/get_provider_info_TEMPLATE.py.jinja2 (93%)
rename docs/{howto/connection => apache-airflow-providers-amazon/connections}/aws.rst (100%)
copy docs/{ => apache-airflow-providers-amazon}/img/aws-web-identity-federation-gcp.png (100%)
rename docs/{howto/operator/amazon/aws => apache-airflow-providers-amazon/operators}/_partials/prerequisite_tasks.rst (90%)
rename docs/{howto/operator/amazon/aws => apache-airflow-providers-amazon/operators}/datasync.rst (94%)
rename docs/{howto/operator/amazon/aws => apache-airflow-providers-amazon/operators}/ecs.rst (94%)
rename docs/{howto/operator/amazon/aws => apache-airflow-providers-amazon/operators}/emr.rst (93%)
rename docs/{howto/operator/amazon/aws => apache-airflow-providers-amazon/operators}/glacier.rst (93%)
rename docs/{howto/operator/amazon/aws => apache-airflow-providers-amazon/operators}/google_api_to_s3_transfer.rst (85%)
rename docs/{howto/operator/amazon/aws => apache-airflow-providers-amazon/operators}/imap_attachment_to_s3.rst (92%)
rename docs/{howto/operator/microsoft/transfer => apache-airflow-providers-amazon/operators}/index.rst (93%)
rename docs/{howto/operator/amazon/aws => apache-airflow-providers-amazon/operators}/s3_to_redshift.rst (93%)
rename docs/{howto/operator/amazon/aws => apache-airflow-providers-amazon/operators}/transfer/glacier_to_gcs.rst (94%)
rename docs/{howto/operator/amazon/aws => apache-airflow-providers-amazon/operators}/transfer/index.rst (100%)
create mode 100644 docs/apache-airflow-providers-amazon/redirects.txt
rename docs/{security/secrets/secrets-backend/aws-secrets-manaager-backend.rst => apache-airflow-providers-amazon/secrets-backends/aws-secrets-manaager.rst} (100%)
rename docs/{security/secrets/secrets-backend/aws-ssm-parameter-store-secrets-backend.rst => apache-airflow-providers-amazon/secrets-backends/aws-ssm-parameter-store.rst} (100%)
copy docs/{howto/operator/apache => apache-airflow-providers-amazon/secrets-backends}/index.rst (96%)
rename docs/{howto/connection => apache-airflow-providers-apache-cassandra/connections}/cassandra.rst (100%)
rename docs/{howto/operator/apache/cassandra.rst => apache-airflow-providers-apache-cassandra/operators.rst} (92%)
create mode 100644 docs/apache-airflow-providers-apache-cassandra/redirects.txt
rename docs/{howto/connection => apache-airflow-providers-apache-spark/connections}/spark.rst (100%)
rename docs/{howto/operator/apache/spark.rst => apache-airflow-providers-apache-spark/operators.rst} (88%)
create mode 100644 docs/apache-airflow-providers-apache-spark/redirects.txt
rename docs/{howto/connection => apache-airflow-providers-cncf-kubernetes/connections}/kubernetes.rst (100%)
rename docs/{howto/operator/kubernetes.rst => apache-airflow-providers-cncf-kubernetes/operators.rst} (93%)
create mode 100644 docs/apache-airflow-providers-cncf-kubernetes/redirects.txt
rename docs/{howto/operator/databricks.rst => apache-airflow-providers-databricks/operators.rst} (94%)
rename docs/{howto/operator/dingding.rst => apache-airflow-providers-dingding/operators.rst} (90%)
create mode 100644 docs/apache-airflow-providers-google/redirects.txt
rename docs/{howto/connection => apache-airflow-providers-grpc/connections}/grpc.rst (100%)
create mode 100644 docs/apache-airflow-providers-grpc/redirects.txt
create mode 100644 docs/apache-airflow-providers-hashicorp/redirects.txt
rename docs/{security/secrets/secrets-backend/hashicorp-vault-secrets-backend.rst => apache-airflow-providers-hashicorp/secrets-backends/hashicorp-vault.rst} (100%)
rename docs/{howto/operator/http.rst => apache-airflow-providers-http/operators.rst} (86%)
rename docs/{howto/connection => apache-airflow-providers-jdbc/connections}/jdbc.rst (100%)
rename docs/{howto/operator/jdbc.rst => apache-airflow-providers-jdbc/operators.rst} (92%)
create mode 100644 docs/apache-airflow-providers-jdbc/redirects.txt
rename docs/{howto/connection => apache-airflow-providers-microsoft-azure/connections}/azure.rst (100%)
rename docs/{howto/operator/microsoft => apache-airflow-providers-microsoft-azure/operators}/_partials/prerequisite_tasks.rst (90%)
rename docs/{howto/operator/microsoft/transfer/blob_storage_to_gcs.rst => apache-airflow-providers-microsoft-azure/operators/azure_blob_to_gcs.rst} (94%)
rename docs/{howto/operator/microsoft => apache-airflow-providers-microsoft-azure/operators}/index.rst (97%)
rename docs/{howto/operator/microsoft/transfer => apache-airflow-providers-microsoft-azure/operators}/local_to_adls.rst (91%)
create mode 100644 docs/apache-airflow-providers-microsoft-azure/redirects.txt
rename docs/{security/secrets/secrets-backend/azure-key-vault-secrets-backend.rst => apache-airflow-providers-microsoft-azure/secrets-backends/azure-key-vault.rst} (100%)
rename docs/{howto/connection => apache-airflow-providers-mysql/connections}/mysql.rst (100%)
copy docs/{howto/operator/mysql.rst => apache-airflow-providers-mysql/operators.rst} (92%)
create mode 100644 docs/apache-airflow-providers-mysql/redirects.txt
rename docs/{howto/connection => apache-airflow-providers-odbc/connections}/odbc.rst (100%)
create mode 100644 docs/apache-airflow-providers-odbc/redirects.txt
rename docs/{howto/connection => apache-airflow-providers-oracle/connections}/oracle.rst (100%)
rename docs/{howto/operator/papermill.rst => apache-airflow-providers-papermill/operators.rst} (96%)
rename docs/{howto/connection => apache-airflow-providers-postgres/connections}/postgres.rst (100%)
create mode 100644 docs/apache-airflow-providers-postgres/redirects.txt
rename docs/{howto/connection => apache-airflow-providers-salesforce/connections}/salesforce.rst (100%)
create mode 100644 docs/apache-airflow-providers-salesforce/redirects.txt
rename docs/{howto/operator/apache => apache-airflow-providers-snowflake/operators}/index.rst (95%)
create mode 100644 docs/apache-airflow-providers-snowflake/operators/s3_to_snowflake.rst
rename docs/{howto/operator/mysql.rst => apache-airflow-providers-snowflake/operators/snowflake.rst} (50%)
create mode 100644 docs/apache-airflow-providers-snowflake/operators/snowflake_to_slack.rst
rename docs/{howto/connection => apache-airflow-providers-ssh/connections}/ssh.rst (100%)
copy docs/{apache-airflow-providers-dingding => apache-airflow-providers-telegram}/index.rst (83%)
create mode 100644 docs/apache-airflow-providers-telegram/operators.rst
rename docs/{howto/connection => apache-airflow-providers-yandex/connections}/yandexcloud.rst (100%)
rename docs/{howto/operator/yandexcloud.rst => apache-airflow-providers-yandex/operators.rst} (100%)
rename docs/{ => apache-airflow}/backport-providers.rst (100%)
rename docs/{ => apache-airflow}/best-practices.rst (67%)
rename docs/{ => apache-airflow}/changelog.rst (96%)
rename docs/{ => apache-airflow}/cli-and-env-variables-ref.rst (76%)
rename docs/{ => apache-airflow}/concepts.rst (92%)
rename docs/{ => apache-airflow}/configurations-ref.rst (86%)
rename docs/{ => apache-airflow}/dag-run.rst (100%)
rename docs/{ => apache-airflow}/dag-serialization.rst (100%)
rename docs/{ => apache-airflow}/executor/celery.rst (100%)
rename docs/{ => apache-airflow}/executor/celery_kubernetes.rst (100%)
rename docs/{ => apache-airflow}/executor/dask.rst (100%)
rename docs/{ => apache-airflow}/executor/debug.rst (100%)
rename docs/{ => apache-airflow}/executor/index.rst (100%)
rename docs/{ => apache-airflow}/executor/kubernetes.rst (95%)
rename docs/{ => apache-airflow}/executor/local.rst (100%)
rename docs/{ => apache-airflow}/executor/sequential.rst (100%)
rename docs/{ => apache-airflow}/extra-packages-ref.rst (84%)
rename docs/{ => apache-airflow}/faq.rst (100%)
rename docs/{ => apache-airflow}/howto/add-dag-tags.rst (100%)
rename docs/{howto/connection/index.rst => apache-airflow/howto/connection.rst} (98%)
rename docs/{ => apache-airflow}/howto/custom-operator.rst (99%)
rename docs/{ => apache-airflow}/howto/customize-state-colors-ui.rst (100%)
rename docs/{ => apache-airflow}/howto/define_extra_link.rst (100%)
rename docs/{ => apache-airflow}/howto/email-config.rst (100%)
rename docs/{ => apache-airflow}/howto/index.rst (98%)
rename docs/{ => apache-airflow}/howto/initialize-database.rst (100%)
rename docs/{ => apache-airflow}/howto/operator/bash.rst (96%)
rename docs/{ => apache-airflow}/howto/operator/external_task_sensor.rst (94%)
rename docs/{ => apache-airflow}/howto/operator/index.rst (83%)
rename docs/{ => apache-airflow}/howto/operator/python.rst (92%)
rename docs/{ => apache-airflow}/howto/run-behind-proxy.rst (100%)
rename docs/{ => apache-airflow}/howto/run-with-systemd.rst (100%)
rename docs/{ => apache-airflow}/howto/run-with-upstart.rst (100%)
rename docs/{ => apache-airflow}/howto/set-config.rst (98%)
rename docs/{ => apache-airflow}/howto/use-test-config.rst (100%)
rename docs/{ => apache-airflow}/howto/variable.rst (100%)
rename docs/{ => apache-airflow}/img/add-dag-tags.png (100%)
rename docs/{ => apache-airflow}/img/add-role.png (100%)
rename docs/{ => apache-airflow}/img/airflow.gif (100%)
rename docs/{ => apache-airflow}/img/apache.jpg (100%)
rename docs/{ => apache-airflow}/img/arch-diag-basic.png (100%)
rename docs/{ => apache-airflow}/img/arch-diag-kubernetes.png (100%)
rename docs/{ => apache-airflow}/img/arch-diag-kubernetes2.png (100%)
rename docs/{ => apache-airflow}/img/arch-diag-logging.png (100%)
rename docs/{ => apache-airflow}/img/aws-web-identity-federation-gcp.png (100%)
rename docs/{ => apache-airflow}/img/branch_note.png (100%)
rename docs/{ => apache-airflow}/img/branch_with_trigger.png (100%)
rename docs/{ => apache-airflow}/img/branch_without_trigger.png (100%)
rename docs/{ => apache-airflow}/img/change-ui-colors/dags-page-new.png (100%)
rename docs/{ => apache-airflow}/img/change-ui-colors/dags-page-old.png (100%)
rename docs/{ => apache-airflow}/img/change-ui-colors/graph-view-new.png (100%)
rename docs/{ => apache-airflow}/img/change-ui-colors/graph-view-old.png (100%)
rename docs/{ => apache-airflow}/img/change-ui-colors/tree-view-new.png (100%)
rename docs/{ => apache-airflow}/img/change-ui-colors/tree-view-old.png (100%)
rename docs/{ => apache-airflow}/img/cli_completion.gif (100%)
rename docs/{ => apache-airflow}/img/code.png (100%)
rename docs/{ => apache-airflow}/img/connection_create.png (100%)
rename docs/{ => apache-airflow}/img/connection_edit.png (100%)
rename docs/{ => apache-airflow}/img/connections.png (100%)
rename docs/{ => apache-airflow}/img/context.png (100%)
rename docs/{ => apache-airflow}/img/dag_serialization.png (100%)
rename docs/{ => apache-airflow}/img/dags.png (100%)
rename docs/{ => apache-airflow}/img/duration.png (100%)
rename docs/{ => apache-airflow}/img/example_passing_conf.png (100%)
rename docs/{ => apache-airflow}/img/gantt.png (100%)
rename docs/{ => apache-airflow}/img/graph.png (100%)
rename docs/{ => apache-airflow}/img/k8s-0-worker.jpeg (100%)
rename docs/{ => apache-airflow}/img/k8s-3-worker.jpeg (100%)
rename docs/{ => apache-airflow}/img/k8s-5-worker.jpeg (100%)
rename docs/{ => apache-airflow}/img/k8s-failed-pod.png (100%)
rename docs/{ => apache-airflow}/img/k8s-happy-path.png (100%)
rename docs/{ => apache-airflow}/img/latest_only_with_trigger.png (100%)
rename docs/{ => apache-airflow}/img/logos/airflow_64x64_emoji_transparent.png (100%)
rename docs/{ => apache-airflow}/img/logos/airflow_dark_bg.png (100%)
rename docs/{ => apache-airflow}/img/logos/airflow_transparent.png (100%)
rename docs/{ => apache-airflow}/img/logos/airflow_white_bg.png (100%)
rename docs/{ => apache-airflow}/img/logos/wordmark_1.png (100%)
rename docs/{ => apache-airflow}/img/logos/wordmark_1.svg (100%)
rename docs/{ => apache-airflow}/img/logos/wordmark_2.png (100%)
rename docs/{ => apache-airflow}/img/logos/wordmark_2.svg (100%)
rename docs/{ => apache-airflow}/img/nested_branching.png (100%)
rename docs/{ => apache-airflow}/img/new-role.png (100%)
rename docs/{ => apache-airflow}/img/operator_extra_link.png (100%)
rename docs/{ => apache-airflow}/img/run_task_on_celery_executor.png (100%)
rename docs/{ => apache-airflow}/img/run_task_on_celery_executor.puml (100%)
rename docs/{ => apache-airflow}/img/scheduler_loop.jpg (100%)
rename docs/{ => apache-airflow}/img/smart_sensor_architecture.png (100%)
rename docs/{ => apache-airflow}/img/smart_sensor_single_task_execute_flow.png (100%)
rename docs/{ => apache-airflow}/img/subdag_after.png (100%)
rename docs/{ => apache-airflow}/img/subdag_before.png (100%)
rename docs/{ => apache-airflow}/img/subdag_zoom.png (100%)
rename docs/{ => apache-airflow}/img/task_group.gif (100%)
rename docs/{ => apache-airflow}/img/task_lifecycle_diagram.png (100%)
rename docs/{ => apache-airflow}/img/task_manual_vs_scheduled.png (100%)
rename docs/{ => apache-airflow}/img/task_stages.png (100%)
rename docs/{ => apache-airflow}/img/tree.png (100%)
rename docs/{ => apache-airflow}/img/ui-timezone-chooser.png (100%)
rename docs/{ => apache-airflow}/img/usage_cli_export.png (100%)
rename docs/{ => apache-airflow}/img/usage_cli_imgcat.png (100%)
rename docs/{ => apache-airflow}/img/variable_hidden.png (100%)
rename docs/{ => apache-airflow}/index.rst (97%)
rename docs/{ => apache-airflow}/installation.rst (86%)
rename docs/{ => apache-airflow}/integration.rst (100%)
rename docs/{ => apache-airflow}/kubernetes.rst (94%)
rename docs/{ => apache-airflow}/license.rst (100%)
rename docs/{ => apache-airflow}/lineage.rst (100%)
rename docs/{ => apache-airflow}/logging-monitoring/check-health.rst (100%)
rename docs/{ => apache-airflow}/logging-monitoring/errors.rst (100%)
rename docs/{ => apache-airflow}/logging-monitoring/index.rst (100%)
rename docs/{ => apache-airflow}/logging-monitoring/logging-architecture.rst (100%)
rename docs/{ => apache-airflow}/logging-monitoring/logging-tasks.rst (100%)
rename docs/{ => apache-airflow}/logging-monitoring/metrics.rst (98%)
rename docs/{ => apache-airflow}/logging-monitoring/tracking-user-activity.rst (100%)
rename docs/{ => apache-airflow}/macros-ref.rst (100%)
rename docs/{ => apache-airflow}/modules_management.rst (79%)
rename docs/{ => apache-airflow}/operators-and-hooks-ref.rst (100%)
rename docs/{ => apache-airflow}/plugins.rst (100%)
rename docs/{ => apache-airflow}/privacy_notice.rst (100%)
rename docs/{ => apache-airflow}/production-deployment.rst (85%)
rename docs/{ => apache-airflow}/project.rst (100%)
rename docs/{autoapi_templates/index.rst => apache-airflow/python-api-ref.rst} (94%)
rename docs/{ => apache-airflow}/redirects.txt (99%)
rename docs/{ => apache-airflow}/rest-api-ref.rst (100%)
rename docs/{ => apache-airflow}/scheduler.rst (100%)
rename docs/{ => apache-airflow}/security/access-control.rst (99%)
rename docs/{ => apache-airflow}/security/api.rst (100%)
rename docs/{ => apache-airflow}/security/flower.rst (100%)
rename docs/{ => apache-airflow}/security/index.rst (95%)
rename docs/{ => apache-airflow}/security/kerberos.rst (100%)
rename docs/{ => apache-airflow}/security/secrets/fernet.rst (100%)
rename docs/{ => apache-airflow}/security/secrets/index.rst (100%)
rename docs/{ => apache-airflow}/security/secrets/secrets-backend/index.rst (96%)
rename docs/{ => apache-airflow}/security/secrets/secrets-backend/local-filesystem-secrets-backend.rst (100%)
rename docs/{ => apache-airflow}/security/webserver.rst (100%)
rename docs/{ => apache-airflow}/security/workload.rst (100%)
rename docs/{ => apache-airflow}/smart-sensor.rst (100%)
rename docs/{ => apache-airflow}/stable-rest-api-ref.rst (100%)
rename docs/{ => apache-airflow}/start.rst (90%)
rename docs/{ => apache-airflow}/static/exampleinclude.css (100%)
rename docs/{ => apache-airflow}/static/jira-links.js (100%)
rename docs/{ => apache-airflow}/templates/footer.html (100%)
rename docs/{ => apache-airflow}/timezone.rst (100%)
rename docs/{ => apache-airflow}/tutorial.rst (96%)
rename docs/{ => apache-airflow}/tutorial_taskflow_api.rst (89%)
rename docs/{ => apache-airflow}/ui.rst (100%)
rename docs/{ => apache-airflow}/usage-cli.rst (80%)
delete mode 100644 docs/howto/operator/amazon/aws/index.rst
delete mode 100644 docs/howto/operator/snowflake.rst
delete mode 100644 docs/security/secrets/secrets-backend/google-cloud-secret-manager-backend.rst
copy scripts/ci/{pre_commit/pre_commit_sync_version.sh => build_airflow/ci_build_airflow_package.sh} (79%)
create mode 100755 scripts/ci/images/ci_verify_ci_image.sh
create mode 100755 scripts/ci/images/ci_verify_prod_image.sh
rename scripts/ci/{provider_packages/ci_test_provider_packages_install.sh => images/ci_wait_for_ci_image.sh} (59%)
rename scripts/ci/{pre_commit/pre_commit_sync_version.sh => images/ci_wait_for_prod_image.sh} (67%)
rename scripts/ci/{provider_packages/ci_prepare_and_test_provider_packages.sh => libraries/_build_airflow_packages.sh} (54%)
mode change 100755 => 100644
create mode 100755 scripts/ci/pre_commit/pre_commit_json_schema.py
rename scripts/ci/provider_packages/{ci_test_provider_packages_import_all_classes.sh => ci_install_and_test_provider_packages.sh} (85%)
rename scripts/ci/provider_packages/{ci_prepare_provider_readme.sh => ci_prepare_provider_readmes.sh} (100%)
create mode 100755 scripts/in_container/run_install_and_test_provider_packages.sh
delete mode 100755 scripts/in_container/run_test_package_import_all_classes.sh
delete mode 100755 scripts/in_container/run_test_package_install.sh
copy airflow/operators/dummy_operator.py => tests/providers/microsoft/azure/hooks/test_fileshare_system.py (55%)
create mode 100644 tests/providers/microsoft/azure/transfers/test_file_to_wasb_system.py
copy tests/{www/api/experimental => providers/telegram}/__init__.py (100%)
copy tests/{www/api/experimental => providers/telegram/hooks}/__init__.py (100%)
create mode 100644 tests/providers/telegram/hooks/test_telegram.py
copy tests/{www/api/experimental => providers/telegram/operators}/__init__.py (100%)
create mode 100644 tests/providers/telegram/operators/test_telegram.py