You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ep...@apache.org on 2023/03/08 13:18:06 UTC
[airflow] 11/12: Switch to ruff for faster static checks (#28893)
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a commit to branch v2-5-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 14fdfc07acc9da01b0de4afa915cba8288d8d954
Author: Ash Berlin-Taylor <as...@apache.org>
AuthorDate: Thu Jan 12 21:40:23 2023 +0000
Switch to ruff for faster static checks (#28893)
Gone are:
- isort
- pyupgrade
- pydocstyle
- yesqa
- autoflake
- flake8
All replaced with [ruff](https://github.com/charliermarsh/ruff). A chunk
of the performance of ruff comes from the fact that it makes very good
use of multiple cores. And since most docker virtual machines are only
one or two core I have chosen to run it directly, not inside the breeze
docker container so we get the full benefit of speed.
* Work around namespace packages issue for providers
Ruff is currently detecting "google" as a the name of the current
package, so it thinks it goes in the "first" party import section
(cherry picked from commit ce858a5d719fb1dff85ad7e4747f0777404d1f56)
---
.flake8 | 8 --
.github/boring-cyborg.yml | 1 -
.github/workflows/ci.yml | 1 +
.pre-commit-config.yaml | 66 +++----------
.rat-excludes | 1 -
STATIC_CODE_CHECKS.rst | 12 +--
airflow/cli/commands/connection_command.py | 2 +-
airflow/compat/functools.pyi | 1 +
airflow/decorators/__init__.pyi | 13 ++-
airflow/example_dags/example_sensor_decorator.py | 1 +
airflow/example_dags/tutorial_taskflow_api.py | 1 +
airflow/hooks/dbapi.py | 6 +-
airflow/migrations/db_types.pyi | 1 +
airflow/providers/amazon/aws/hooks/emr.py | 10 +-
airflow/providers/amazon/aws/operators/sns.py | 2 +-
.../amazon/aws/transfers/dynamodb_to_s3.py | 4 +-
.../providers/cncf/kubernetes/utils/__init__.py | 2 +
.../providers/google/cloud/operators/dataproc.py | 2 +-
.../google/cloud/operators/kubernetes_engine.py | 2 +-
airflow/providers/microsoft/azure/hooks/wasb.py | 2 +-
airflow/providers/odbc/hooks/odbc.py | 2 +-
airflow/utils/context.pyi | 5 +-
airflow/utils/log/action_logger.py | 1 +
airflow/utils/process_utils.py | 4 +-
.../src/airflow_breeze/commands/main_command.py | 4 +-
dev/breeze/src/airflow_breeze/global_constants.py | 9 +-
dev/breeze/src/airflow_breeze/pre_commit_ids.py | 8 +-
.../pre_commit_ids_TEMPLATE.py.jinja2 | 1 +
docs/apache-airflow/img/airflow_erd.sha256 | 2 +-
docs/build_docs.py | 6 +-
docs/exts/provider_init_hack.py | 4 +-
docs/spelling_wordlist.txt | 1 +
images/breeze/output-commands-hash.txt | 2 +-
images/breeze/output_static-checks.svg | 58 ++++++-----
provider_packages/.flake8 | 1 -
pyproject.toml | 108 ++++++++++++++++++---
.../pre_commit_check_pre_commit_hooks.py | 10 +-
scripts/ci/pre_commit/pre_commit_flake8.py | 72 --------------
scripts/in_container/run_flake8.sh | 20 ----
setup.py | 8 +-
tests/api_connexion/endpoints/test_dag_endpoint.py | 15 ---
.../providers/google/suite/hooks/test_calendar.py | 3 +-
.../cncf/kubernetes/example_spark_kubernetes.py | 3 +-
.../google/cloud/bigtable/example_bigtable.py | 6 +-
tests/test_utils/get_all_tests.py | 4 +-
45 files changed, 205 insertions(+), 290 deletions(-)
diff --git a/.flake8 b/.flake8
deleted file mode 100644
index 14de564a32..0000000000
--- a/.flake8
+++ /dev/null
@@ -1,8 +0,0 @@
-[flake8]
-max-line-length = 110
-ignore = E203,E231,E731,W504,I001,W503
-exclude = .svn,CVS,.bzr,.hg,.git,__pycache__,.eggs,*.egg,node_modules
-format = ${cyan}%(path)s${reset}:${yellow_bold}%(row)d${reset}:${green_bold}%(col)d${reset}: ${red_bold}%(code)s${reset} %(text)s
-per-file-ignores =
- airflow/models/__init__.py:F401
- airflow/models/sqla_models.py:F401
diff --git a/.github/boring-cyborg.yml b/.github/boring-cyborg.yml
index 7a444cf253..de8a944d70 100644
--- a/.github/boring-cyborg.yml
+++ b/.github/boring-cyborg.yml
@@ -96,7 +96,6 @@ labelPRBasedOnFilePath:
- .asf.yaml
- .bash_completion
- .dockerignore
- - .flake8
- .hadolint.yaml
- .pre-commit-config.yaml
- .rat-excludes
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index ce5004c7d9..589582ae8a 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -583,6 +583,7 @@ jobs:
COLUMNS: "250"
SKIP_GROUP_OUTPUT: "true"
DEFAULT_BRANCH: ${{ needs.build-info.outputs.default-branch }}
+ RUFF_FORMAT: "github"
- name: "Fix ownership"
run: breeze ci fix-ownership
if: always()
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 93ca966d0b..07e5d18d58 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -156,11 +156,17 @@ repos:
additional_dependencies: ['pyyaml']
pass_filenames: false
require_serial: true
- - repo: https://github.com/PyCQA/isort
- rev: 5.11.2
- hooks:
- - id: isort
- name: Run isort to sort imports in Python files
+ - id: ruff
+ name: ruff
+ language: python
+ require_serial: true
+ pass_filenames: true
+ # Since ruff makes use of multiple cores we _purposefully_ don't run this in docker so it can use the
+ # host CPU to it's fullest
+ entry: ruff --fix --no-update-check --force-exclude
+ additional_dependencies: ['ruff>=0.0.219']
+ files: \.pyi?$
+ exclude: ^airflow/_vendor/
- repo: https://github.com/psf/black
rev: 22.12.0
hooks:
@@ -223,14 +229,6 @@ repos:
- "4"
files: ^chart/values\.schema\.json$|^chart/values_schema\.schema\.json$
pass_filenames: true
- # TODO: Bump to Python 3.8 when support for Python 3.7 is dropped in Airflow.
- - repo: https://github.com/asottile/pyupgrade
- rev: v3.3.1
- hooks:
- - id: pyupgrade
- name: Upgrade Python code automatically
- args: ["--py37-plus"]
- exclude: ^airflow/_vendor/
- repo: https://github.com/pre-commit/pygrep-hooks
rev: v1.9.0
hooks:
@@ -248,35 +246,6 @@ repos:
entry: yamllint -c yamllint-config.yml --strict
types: [yaml]
exclude: ^.*init_git_sync\.template\.yaml$|^.*airflow\.template\.yaml$|^chart/(?:templates|files)/.*\.yaml$|openapi/.*\.yaml$|^\.pre-commit-config\.yaml$|^airflow/_vendor/
- - repo: https://github.com/pycqa/pydocstyle
- rev: 6.1.1
- hooks:
- - id: pydocstyle
- name: Run pydocstyle
- args:
- - --convention=pep257
- - --add-ignore=D100,D102,D103,D104,D105,D107,D205,D400,D401
- exclude: |
- (?x)
- ^tests/.*\.py$|
- ^scripts/.*\.py$|
- ^dev|
- ^provider_packages|
- ^docker_tests|
- ^kubernetes_tests|
- .*example_dags/.*|
- ^chart/.*\.py$|
- ^airflow/_vendor/
- additional_dependencies: ['toml']
- - repo: https://github.com/asottile/yesqa
- rev: v1.4.0
- hooks:
- - id: yesqa
- name: Remove unnecessary noqa statements
- exclude: |
- (?x)
- ^airflow/_vendor/
- additional_dependencies: ['flake8>=4.0.1']
- repo: https://github.com/ikamensh/flynt
rev: '0.77'
hooks:
@@ -315,11 +284,6 @@ repos:
types: [file, text]
exclude: ^airflow/_vendor/|^clients/gen/go\.sh$|^\.gitmodules$
additional_dependencies: ['rich>=12.4.4']
- - id: static-check-autoflake
- name: Remove all unused code
- entry: autoflake --remove-all-unused-imports --ignore-init-module-imports --in-place
- language: python
- additional_dependencies: ['autoflake']
- id: lint-openapi
name: Lint OpenAPI using spectral
language: docker_image
@@ -891,14 +855,6 @@ repos:
exclude: ^docs/rtd-deprecation
require_serial: true
additional_dependencies: ['rich>=12.4.4', 'inputimeout']
- - id: run-flake8
- name: Run flake8
- language: python
- entry: ./scripts/ci/pre_commit/pre_commit_flake8.py
- files: \.py$
- pass_filenames: true
- exclude: ^airflow/_vendor/
- additional_dependencies: ['rich>=12.4.4', 'inputimeout']
- id: check-provider-yaml-valid
name: Validate provider.yaml files
pass_filenames: false
diff --git a/.rat-excludes b/.rat-excludes
index 1e16d61a67..138e8a0787 100644
--- a/.rat-excludes
+++ b/.rat-excludes
@@ -15,7 +15,6 @@
.codespellignorelines
.eslintrc
.eslintignore
-.flake8
.rat-excludes
.stylelintignore
.stylelintrc
diff --git a/STATIC_CODE_CHECKS.rst b/STATIC_CODE_CHECKS.rst
index 1b4732cd44..a31a29171e 100644
--- a/STATIC_CODE_CHECKS.rst
+++ b/STATIC_CODE_CHECKS.rst
@@ -260,8 +260,6 @@ require Breeze Docker image to be build locally.
| | * Add license for all md files | |
| | * Add license for all other files | |
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
-| isort | Run isort to sort imports in Python files | |
-+-----------------------------------------------------------+------------------------------------------------------------------+---------+
| lint-chart-schema | Lint chart/values.schema.json file | |
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
| lint-css | stylelint | |
@@ -286,17 +284,13 @@ require Breeze Docker image to be build locally.
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
| pretty-format-json | Format json files | |
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
-| pydocstyle | Run pydocstyle | |
-+-----------------------------------------------------------+------------------------------------------------------------------+---------+
| python-no-log-warn | Check if there are no deprecate log warn | |
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
-| pyupgrade | Upgrade Python code automatically | |
-+-----------------------------------------------------------+------------------------------------------------------------------+---------+
| replace-bad-characters | Replace bad characters | |
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
| rst-backticks | Check if RST files use double backticks for code | |
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
-| run-flake8 | Run flake8 | * |
+| ruff | ruff | |
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
| run-mypy | * Run mypy for dev | * |
| | * Run mypy for core | |
@@ -305,8 +299,6 @@ require Breeze Docker image to be build locally.
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
| run-shellcheck | Check Shell scripts syntax correctness | |
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
-| static-check-autoflake | Remove all unused code | |
-+-----------------------------------------------------------+------------------------------------------------------------------+---------+
| trailing-whitespace | Remove trailing whitespace at end of line | |
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
| ts-compile-and-lint-javascript | TS types generation and ESLint against current UI files | |
@@ -340,8 +332,6 @@ require Breeze Docker image to be build locally.
| update-version | Update version to the latest version in the documentation | |
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
| yamllint | Check YAML files with yamllint | |
-+-----------------------------------------------------------+------------------------------------------------------------------+---------+
-| yesqa | Remove unnecessary noqa statements | |
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
.. END AUTO-GENERATED STATIC CHECK LIST
diff --git a/airflow/cli/commands/connection_command.py b/airflow/cli/commands/connection_command.py
index c19490da93..9206ae2098 100644
--- a/airflow/cli/commands/connection_command.py
+++ b/airflow/cli/commands/connection_command.py
@@ -209,7 +209,7 @@ def connections_add(args):
if has_json and has_uri:
raise SystemExit("Cannot supply both conn-uri and conn-json")
- if has_type and not (args.conn_type in _get_connection_types()):
+ if has_type and args.conn_type not in _get_connection_types():
warnings.warn(f"The type provided to --conn-type is invalid: {args.conn_type}")
warnings.warn(
f"Supported --conn-types are:{_get_connection_types()}."
diff --git a/airflow/compat/functools.pyi b/airflow/compat/functools.pyi
index 8dabbd6004..32cbbaa431 100644
--- a/airflow/compat/functools.pyi
+++ b/airflow/compat/functools.pyi
@@ -18,6 +18,7 @@
# This stub exists to work around false linter errors due to python/mypy#10408.
# TODO: Remove this file after the upstream fix is available in our toolchain.
+from __future__ import annotations
from typing import Callable, TypeVar
diff --git a/airflow/decorators/__init__.pyi b/airflow/decorators/__init__.pyi
index b0edc7d2c2..c6a90139a8 100644
--- a/airflow/decorators/__init__.pyi
+++ b/airflow/decorators/__init__.pyi
@@ -18,9 +18,10 @@
# dynamically generated task decorators. Functions declared in this stub do not
# necessarily exist at run time. See "Creating Custom @task Decorators"
# documentation for more details.
+from __future__ import annotations
from datetime import timedelta
-from typing import Any, Callable, Iterable, Mapping, Union, overload
+from typing import Any, Callable, Iterable, Mapping, overload
from kubernetes.client import models as k8s
@@ -30,6 +31,7 @@ from airflow.decorators.external_python import external_python_task
from airflow.decorators.python import python_task
from airflow.decorators.python_virtualenv import virtualenv_task
from airflow.decorators.sensor import sensor_task
+from airflow.decorators.short_circuit import short_circuit_task
from airflow.decorators.task_group import task_group
from airflow.kubernetes.secret import Secret
from airflow.models.dag import dag
@@ -98,8 +100,8 @@ class TaskDecoratorCollection:
multiple_outputs: bool | None = None,
# 'python_callable', 'op_args' and 'op_kwargs' since they are filled by
# _PythonVirtualenvDecoratedOperator.
- requirements: Union[None, Iterable[str], str] = None,
- python_version: Union[None, str, int, float] = None,
+ requirements: None | Iterable[str] | str = None,
+ python_version: None | str | int | float = None,
use_dill: bool = False,
system_site_packages: bool = True,
templates_dict: Mapping[str, Any] | None = None,
@@ -263,7 +265,8 @@ class TaskDecoratorCollection:
None - No networking for this container
container:<name|id> - Use the network stack of another container specified via <name|id>
host - Use the host network stack. Incompatible with `port_bindings`
- '<network-name>|<network-id>' - Connects the container to user created network(using `docker network create` command)
+ '<network-name>|<network-id>' - Connects the container to user created network(using `docker
+ network create` command)
:param tls_ca_cert: Path to a PEM-encoded certificate authority
to secure the docker connection.
:param tls_client_cert: Path to the PEM-encoded certificate
@@ -448,6 +451,6 @@ class TaskDecoratorCollection:
:param max_wait: maximum wait interval between pokes, can be ``timedelta`` or ``float`` seconds
"""
@overload
- def sensor(self, python_callable: Optional[FParams, FReturn] = None) -> Task[FParams, FReturn]: ...
+ def sensor(self, python_callable: FParams | FReturn | None = None) -> Task[FParams, FReturn]: ...
task: TaskDecoratorCollection
diff --git a/airflow/example_dags/example_sensor_decorator.py b/airflow/example_dags/example_sensor_decorator.py
index 2197a6c53a..2ead792850 100644
--- a/airflow/example_dags/example_sensor_decorator.py
+++ b/airflow/example_dags/example_sensor_decorator.py
@@ -27,6 +27,7 @@ import pendulum
from airflow.decorators import dag, task
from airflow.sensors.base import PokeReturnValue
+
# [END import_module]
diff --git a/airflow/example_dags/tutorial_taskflow_api.py b/airflow/example_dags/tutorial_taskflow_api.py
index f41f729af8..27a28f4b79 100644
--- a/airflow/example_dags/tutorial_taskflow_api.py
+++ b/airflow/example_dags/tutorial_taskflow_api.py
@@ -25,6 +25,7 @@ import pendulum
from airflow.decorators import dag, task
+
# [END import_module]
diff --git a/airflow/hooks/dbapi.py b/airflow/hooks/dbapi.py
index cd4a39af8d..b4cd1be667 100644
--- a/airflow/hooks/dbapi.py
+++ b/airflow/hooks/dbapi.py
@@ -20,8 +20,10 @@ from __future__ import annotations
import warnings
from airflow.exceptions import RemovedInAirflow3Warning
-from airflow.providers.common.sql.hooks.sql import ConnectorProtocol # noqa
-from airflow.providers.common.sql.hooks.sql import DbApiHook # noqa
+from airflow.providers.common.sql.hooks.sql import (
+ ConnectorProtocol, # noqa
+ DbApiHook, # noqa
+)
warnings.warn(
"This module is deprecated. Please use `airflow.providers.common.sql.hooks.sql`.",
diff --git a/airflow/migrations/db_types.pyi b/airflow/migrations/db_types.pyi
index bdde6f9692..7fa9ff24b3 100644
--- a/airflow/migrations/db_types.pyi
+++ b/airflow/migrations/db_types.pyi
@@ -16,6 +16,7 @@
# specific language governing permissions and limitations
# under the License.
#
+from __future__ import annotations
import sqlalchemy as sa
diff --git a/airflow/providers/amazon/aws/hooks/emr.py b/airflow/providers/amazon/aws/hooks/emr.py
index 5423dd1af8..171a07263b 100644
--- a/airflow/providers/amazon/aws/hooks/emr.py
+++ b/airflow/providers/amazon/aws/hooks/emr.py
@@ -357,7 +357,7 @@ class EmrContainerHook(AwsBaseHook):
Submit a job to the EMR Containers API and return the job ID.
A job run is a unit of work, such as a Spark jar, PySpark script,
or SparkSQL query, that you submit to Amazon EMR on EKS.
- See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/emr-containers.html#EMRContainers.Client.start_job_run # noqa: E501
+ See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/emr-containers.html#EMRContainers.Client.start_job_run
:param name: The name of the job run.
:param execution_role_arn: The IAM role ARN associated with the job run.
@@ -369,7 +369,7 @@ class EmrContainerHook(AwsBaseHook):
Use this if you want to specify a unique ID to prevent two jobs from getting started.
:param tags: The tags assigned to job runs.
:return: Job ID
- """
+ """ # noqa: E501
params = {
"name": name,
"virtualClusterId": self.virtual_cluster_id,
@@ -422,10 +422,12 @@ class EmrContainerHook(AwsBaseHook):
def check_query_status(self, job_id: str) -> str | None:
"""
Fetch the status of submitted job run. Returns None or one of valid query states.
- See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/emr-containers.html#EMRContainers.Client.describe_job_run # noqa: E501
+
+ See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/emr-containers.html#EMRContainers.Client.describe_job_run
+
:param job_id: Id of submitted job run
:return: str
- """
+ """ # noqa: E501
try:
response = self.conn.describe_job_run(
virtualClusterId=self.virtual_cluster_id,
diff --git a/airflow/providers/amazon/aws/operators/sns.py b/airflow/providers/amazon/aws/operators/sns.py
index 99525c4cb6..2f5b9844bf 100644
--- a/airflow/providers/amazon/aws/operators/sns.py
+++ b/airflow/providers/amazon/aws/operators/sns.py
@@ -15,9 +15,9 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+"""Publish message to SNS queue"""
from __future__ import annotations
-"""Publish message to SNS queue"""
from typing import TYPE_CHECKING, Sequence
from airflow.models import BaseOperator
diff --git a/airflow/providers/amazon/aws/transfers/dynamodb_to_s3.py b/airflow/providers/amazon/aws/transfers/dynamodb_to_s3.py
index 155f5439a6..017c897778 100644
--- a/airflow/providers/amazon/aws/transfers/dynamodb_to_s3.py
+++ b/airflow/providers/amazon/aws/transfers/dynamodb_to_s3.py
@@ -69,7 +69,7 @@ class DynamoDBToS3Operator(BaseOperator):
:param dynamodb_table_name: Dynamodb table to replicate data from
:param s3_bucket_name: S3 bucket to replicate data to
:param file_size: Flush file to s3 if file size >= file_size
- :param dynamodb_scan_kwargs: kwargs pass to <https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#DynamoDB.Table.scan> # noqa: E501
+ :param dynamodb_scan_kwargs: kwargs pass to <https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#DynamoDB.Table.scan>
:param s3_key_prefix: Prefix of s3 object key
:param process_func: How we transforms a dynamodb item to bytes. By default we dump the json
:param aws_conn_id: The Airflow connection used for AWS credentials.
@@ -77,7 +77,7 @@ class DynamoDBToS3Operator(BaseOperator):
running Airflow in a distributed manner and aws_conn_id is None or
empty, then default boto3 configuration would be used (and must be
maintained on each worker node).
- """
+ """ # noqa: E501
template_fields: Sequence[str] = (
"s3_bucket_name",
diff --git a/airflow/providers/cncf/kubernetes/utils/__init__.py b/airflow/providers/cncf/kubernetes/utils/__init__.py
index 84e243c6db..69d825b440 100644
--- a/airflow/providers/cncf/kubernetes/utils/__init__.py
+++ b/airflow/providers/cncf/kubernetes/utils/__init__.py
@@ -14,4 +14,6 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+from __future__ import annotations
+
__all__ = ["xcom_sidecar", "pod_manager"]
diff --git a/airflow/providers/google/cloud/operators/dataproc.py b/airflow/providers/google/cloud/operators/dataproc.py
index 24ca7de401..3bb7684ed0 100644
--- a/airflow/providers/google/cloud/operators/dataproc.py
+++ b/airflow/providers/google/cloud/operators/dataproc.py
@@ -129,7 +129,7 @@ class ClusterGenerator:
``projects/[PROJECT_STORING_KEYS]/locations/[LOCATION]/keyRings/[KEY_RING_NAME]/cryptoKeys/[KEY_NAME]`` # noqa
:param enable_component_gateway: Provides access to the web interfaces of default and selected optional
components on the cluster.
- """
+ """ # noqa: E501
def __init__(
self,
diff --git a/airflow/providers/google/cloud/operators/kubernetes_engine.py b/airflow/providers/google/cloud/operators/kubernetes_engine.py
index 045935771c..6e057f6481 100644
--- a/airflow/providers/google/cloud/operators/kubernetes_engine.py
+++ b/airflow/providers/google/cloud/operators/kubernetes_engine.py
@@ -210,7 +210,7 @@ class GKECreateClusterOperator(BaseOperator):
def _check_input(self) -> None:
if (
not all([self.project_id, self.location, self.body])
- or (isinstance(self.body, dict) and not ("name" in self.body))
+ or (isinstance(self.body, dict) and "name" not in self.body)
or (
isinstance(self.body, dict)
and ("initial_node_count" not in self.body and "node_pools" not in self.body)
diff --git a/airflow/providers/microsoft/azure/hooks/wasb.py b/airflow/providers/microsoft/azure/hooks/wasb.py
index 27680a5b69..ff1914061c 100644
--- a/airflow/providers/microsoft/azure/hooks/wasb.py
+++ b/airflow/providers/microsoft/azure/hooks/wasb.py
@@ -442,7 +442,7 @@ class WasbHook(BaseHook):
self.log.info("Deleted container: %s", container_name)
except ResourceNotFoundError:
self.log.info("Unable to delete container %s (not found)", container_name)
- except: # noqa: E722
+ except:
self.log.info("Error deleting container: %s", container_name)
raise
diff --git a/airflow/providers/odbc/hooks/odbc.py b/airflow/providers/odbc/hooks/odbc.py
index 20e8e8864e..b1d754965e 100644
--- a/airflow/providers/odbc/hooks/odbc.py
+++ b/airflow/providers/odbc/hooks/odbc.py
@@ -145,7 +145,7 @@ class OdbcHook(DbApiHook):
extra_exclude = {"driver", "dsn", "connect_kwargs", "sqlalchemy_scheme"}
extra_params = {
- k: v for k, v in self.connection.extra_dejson.items() if not k.lower() in extra_exclude
+ k: v for k, v in self.connection.extra_dejson.items() if k.lower() not in extra_exclude
}
for k, v in extra_params.items():
conn_str += f"{k}={v};"
diff --git a/airflow/utils/context.pyi b/airflow/utils/context.pyi
index c7bab20c85..838162649a 100644
--- a/airflow/utils/context.pyi
+++ b/airflow/utils/context.pyi
@@ -24,8 +24,9 @@
# attributes are injected at runtime, and giving them a class would trigger
# undefined attribute errors from Mypy. Hopefully there will be a mechanism to
# declare "these are defined, but don't error if others are accessed" someday.
+from __future__ import annotations
-from typing import Any, Collection, Container, Iterable, Mapping, Union, overload
+from typing import Any, Collection, Container, Iterable, Mapping, overload
from pendulum import DateTime
@@ -61,7 +62,7 @@ class Context(TypedDict, total=False):
data_interval_start: DateTime
ds: str
ds_nodash: str
- exception: Union[KeyboardInterrupt, Exception, str, None]
+ exception: KeyboardInterrupt | Exception | str | None
execution_date: DateTime
expanded_ti_count: int | None
inlets: list
diff --git a/airflow/utils/log/action_logger.py b/airflow/utils/log/action_logger.py
index 1968604fe2..66e71c4653 100644
--- a/airflow/utils/log/action_logger.py
+++ b/airflow/utils/log/action_logger.py
@@ -15,6 +15,7 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+from __future__ import annotations
def action_event_from_permission(prefix: str, permission: str) -> str:
diff --git a/airflow/utils/process_utils.py b/airflow/utils/process_utils.py
index 98ff1e9147..6cbd18416e 100644
--- a/airflow/utils/process_utils.py
+++ b/airflow/utils/process_utils.py
@@ -30,9 +30,9 @@ import sys
from airflow.utils.platform import IS_WINDOWS
if not IS_WINDOWS:
- import tty
- import termios
import pty
+ import termios
+ import tty
from contextlib import contextmanager
from typing import Generator
diff --git a/dev/breeze/src/airflow_breeze/commands/main_command.py b/dev/breeze/src/airflow_breeze/commands/main_command.py
index 3761c4dabc..60a7598244 100644
--- a/dev/breeze/src/airflow_breeze/commands/main_command.py
+++ b/dev/breeze/src/airflow_breeze/commands/main_command.py
@@ -141,7 +141,7 @@ def check_for_python_emulation():
prompt="Are you REALLY sure you want to continue? (answer with y otherwise we exit in 20s)\n",
timeout=20,
)
- if not user_status.upper() in ["Y", "YES"]:
+ if user_status.upper() not in ["Y", "YES"]:
sys.exit(1)
except TimeoutOccurred:
get_console().print("\nNo answer, exiting...")
@@ -189,7 +189,7 @@ def check_for_rosetta_environment():
prompt="Are you REALLY sure you want to continue? (answer with y otherwise we exit in 20s)\n",
timeout=20,
)
- if not user_status.upper() in ["Y", "YES"]:
+ if user_status.upper() not in ["Y", "YES"]:
sys.exit(1)
except TimeoutOccurred:
get_console().print("\nNo answer, exiting...")
diff --git a/dev/breeze/src/airflow_breeze/global_constants.py b/dev/breeze/src/airflow_breeze/global_constants.py
index 6079f2d928..323efd1c19 100644
--- a/dev/breeze/src/airflow_breeze/global_constants.py
+++ b/dev/breeze/src/airflow_breeze/global_constants.py
@@ -14,17 +14,16 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-from __future__ import annotations
-
-import json
-from pathlib import Path
-
"""
Global constants that are used by all other Breeze components.
"""
+from __future__ import annotations
+
+import json
import platform
from enum import Enum
from functools import lru_cache
+from pathlib import Path
from airflow_breeze.utils.host_info_utils import Architecture
from airflow_breeze.utils.path_utils import AIRFLOW_SOURCES_ROOT
diff --git a/dev/breeze/src/airflow_breeze/pre_commit_ids.py b/dev/breeze/src/airflow_breeze/pre_commit_ids.py
index 851e68be39..3a95ec574f 100644
--- a/dev/breeze/src/airflow_breeze/pre_commit_ids.py
+++ b/dev/breeze/src/airflow_breeze/pre_commit_ids.py
@@ -20,6 +20,7 @@
#
# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
# `pre_commit_ids_TEMPLATE.py.jinja2` IN the `dev/breeze/src/airflow_breeze` DIRECTORY
+from __future__ import annotations
PRE_COMMIT_LIST = [
"all",
@@ -78,7 +79,6 @@ PRE_COMMIT_LIST = [
"flynt",
"identity",
"insert-license",
- "isort",
"lint-chart-schema",
"lint-css",
"lint-dockerfile",
@@ -88,15 +88,12 @@ PRE_COMMIT_LIST = [
"lint-openapi",
"mixed-line-ending",
"pretty-format-json",
- "pydocstyle",
"python-no-log-warn",
- "pyupgrade",
"replace-bad-characters",
"rst-backticks",
- "run-flake8",
+ "ruff",
"run-mypy",
"run-shellcheck",
- "static-check-autoflake",
"trailing-whitespace",
"ts-compile-and-lint-javascript",
"update-black-version",
@@ -114,5 +111,4 @@ PRE_COMMIT_LIST = [
"update-vendored-in-k8s-json-schema",
"update-version",
"yamllint",
- "yesqa",
]
diff --git a/dev/breeze/src/airflow_breeze/pre_commit_ids_TEMPLATE.py.jinja2 b/dev/breeze/src/airflow_breeze/pre_commit_ids_TEMPLATE.py.jinja2
index 12e32af0b9..714e731ece 100644
--- a/dev/breeze/src/airflow_breeze/pre_commit_ids_TEMPLATE.py.jinja2
+++ b/dev/breeze/src/airflow_breeze/pre_commit_ids_TEMPLATE.py.jinja2
@@ -20,5 +20,6 @@
#
# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
# `pre_commit_ids_TEMPLATE.py.jinja2` IN the `dev/breeze/src/airflow_breeze` DIRECTORY
+from __future__ import annotations
PRE_COMMIT_LIST= {{ PRE_COMMIT_IDS }}
diff --git a/docs/apache-airflow/img/airflow_erd.sha256 b/docs/apache-airflow/img/airflow_erd.sha256
index 5b0ac5fc48..2e63d0c0a0 100644
--- a/docs/apache-airflow/img/airflow_erd.sha256
+++ b/docs/apache-airflow/img/airflow_erd.sha256
@@ -1 +1 @@
-a9c9af1ba1a690ea1c77aba3458aff1ef6f7e776f759dae227641422ba6a5856
\ No newline at end of file
+edb1bcac449e2d38c4523cea6094e812da491a01c40cf9f79024d85e69977893
diff --git a/docs/build_docs.py b/docs/build_docs.py
index 273858be76..bc66fc793f 100755
--- a/docs/build_docs.py
+++ b/docs/build_docs.py
@@ -30,6 +30,9 @@ from collections import defaultdict
from itertools import filterfalse, tee
from typing import Callable, Iterable, NamedTuple, TypeVar
+from rich.console import Console
+from tabulate import tabulate
+
from docs.exts.docs_build import dev_index_generator, lint_checks
from docs.exts.docs_build.code_utils import CONSOLE_WIDTH, PROVIDER_INIT_FILE
from docs.exts.docs_build.docs_builder import DOCS_DIR, AirflowDocsBuilder, get_available_packages
@@ -39,9 +42,6 @@ from docs.exts.docs_build.github_action_utils import with_group
from docs.exts.docs_build.package_filter import process_package_filters
from docs.exts.docs_build.spelling_checks import SpellingError, display_spelling_error_summary
-from rich.console import Console
-from tabulate import tabulate
-
TEXT_RED = "\033[31m"
TEXT_RESET = "\033[0m"
diff --git a/docs/exts/provider_init_hack.py b/docs/exts/provider_init_hack.py
index be34d13b3a..fa082afc49 100644
--- a/docs/exts/provider_init_hack.py
+++ b/docs/exts/provider_init_hack.py
@@ -14,13 +14,13 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-from __future__ import annotations
-
"""
Bugs in sphinx-autoapi using metaclasses prevent us from upgrading to 1.3
which has implicit namespace support. Until that time, we make it look
like a real package for building docs
"""
+from __future__ import annotations
+
import os
from sphinx.application import Sphinx
diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt
index 8bd3aa4b75..916675ef3b 100644
--- a/docs/spelling_wordlist.txt
+++ b/docs/spelling_wordlist.txt
@@ -750,6 +750,7 @@ IRSA
isfile
ish
isn
+isort
iterable
iterables
iteratively
diff --git a/images/breeze/output-commands-hash.txt b/images/breeze/output-commands-hash.txt
index 3ac57898ce..d159616b63 100644
--- a/images/breeze/output-commands-hash.txt
+++ b/images/breeze/output-commands-hash.txt
@@ -55,7 +55,7 @@ setup:version:123b462a421884dc2320ffc5e54b2478
setup:f383b9236f6141f95276136ccd9217f5
shell:affbf6f7f469408d0af47f75c6a38f6c
start-airflow:109728919a0dd5c5ff5640ae86ba9e90
-static-checks:6c18cfc471ad4118a11fc84d41abb747
+static-checks:06708a5e0c50a6fc6cd18c2413431168
stop:e5aa686b4e53707ced4039d8414d5cd6
testing:docker-compose-tests:b86c044b24138af0659a05ed6331576c
testing:helm-tests:94a442e7f3f63b34c4831a84d165690a
diff --git a/images/breeze/output_static-checks.svg b/images/breeze/output_static-checks.svg
index 87a81e7c32..7934d746c9 100644
--- a/images/breeze/output_static-checks.svg
+++ b/images/breeze/output_static-checks.svg
@@ -1,4 +1,4 @@
-<svg class="rich-terminal" viewBox="0 0 1482 1367.6" xmlns="http://www.w3.org/2000/svg">
+<svg class="rich-terminal" viewBox="0 0 1482 1343.1999999999998" xmlns="http://www.w3.org/2000/svg">
<!-- Generated with Rich https://www.textualize.io -->
<style>
@@ -43,7 +43,7 @@
<defs>
<clipPath id="breeze-static-checks-clip-terminal">
- <rect x="0" y="0" width="1463.0" height="1316.6" />
+ <rect x="0" y="0" width="1463.0" height="1292.1999999999998" />
</clipPath>
<clipPath id="breeze-static-checks-line-0">
<rect x="0" y="1.5" width="1464" height="24.65"/>
@@ -201,12 +201,9 @@
<clipPath id="breeze-static-checks-line-51">
<rect x="0" y="1245.9" width="1464" height="24.65"/>
</clipPath>
-<clipPath id="breeze-static-checks-line-52">
- <rect x="0" y="1270.3" width="1464" height="24.65"/>
- </clipPath>
</defs>
- <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1" x="1" y="1" width="1480" height="1365.6" rx="8"/><text class="breeze-static-checks-title" fill="#c5c8c6" text-anchor="middle" x="740" y="27">Command: static-checks</text>
+ <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1" x="1" y="1" width="1480" height="1341.2" rx="8"/><text class="breeze-static-checks-title" fill="#c5c8c6" text-anchor="middle" x="740" y="27">Command: static-checks</text>
<g transform="translate(26,22)">
<circle cx="0" cy="0" r="7" fill="#ff5f57"/>
<circle cx="22" cy="0" r="7" fill="#febc2e"/>
@@ -245,31 +242,30 @@
</text><text class="breeze-static-checks-r5" x="0" y="654.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-26)">│</text><text class="breeze-static-checks-r7" x="366" y="654.4" textLength="1073.6" clip-path="url(#breeze-static-checks-line-26)">check-system-tests-tocs | check-xml | codespell | compile-www-assets |                  </text><text class="br [...]
</text><text class="breeze-static-checks-r5" x="0" y="678.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-27)">│</text><text class="breeze-static-checks-r7" x="366" y="678.8" textLength="1073.6" clip-path="url(#breeze-static-checks-line-27)">compile-www-assets-dev | create-missing-init-py-files-tests | debug-statements |        </text><text class="breeze-static-checks-r5" x="1451.8" y="678.8" textLength="12.2 [...]
</text><text class="breeze-static-checks-r5" x="0" y="703.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-28)">│</text><text class="breeze-static-checks-r7" x="366" y="703.2" textLength="1073.6" clip-path="url(#breeze-static-checks-line-28)">detect-private-key | doctoc | end-of-file-fixer | fix-encoding-pragma | flynt | identity</text><text class="breeze-static-checks-r5" x="1451.8" y="703.2" textLength="12.2" clip-path="ur [...]
-</text><text class="breeze-static-checks-r5" x="0" y="727.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-29)">│</text><text class="breeze-static-checks-r7" x="366" y="727.6" textLength="1073.6" clip-path="url(#breeze-static-checks-line-29)">| insert-license | isort | lint-chart-schema | lint-css | lint-dockerfile |             </text><text class="breeze-stati [...]
-</text><text class="breeze-static-checks-r5" x="0" y="752" textLength="12.2" clip-path="url(#breeze-static-checks-line-30)">│</text><text class="breeze-static-checks-r7" x="366" y="752" textLength="1073.6" clip-path="url(#breeze-static-checks-line-30)">lint-helm-chart | lint-json-schema | lint-markdown | lint-openapi | mixed-line-ending | </text><text class="breeze-static-checks-r5" x="1451.8" y="752" textLength="12.2" clip-path="url(#bre [...]
-</text><text class="breeze-static-checks-r5" x="0" y="776.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-31)">│</text><text class="breeze-static-checks-r7" x="366" y="776.4" textLength="1073.6" clip-path="url(#breeze-static-checks-line-31)">pretty-format-json | pydocstyle | python-no-log-warn | pyupgrade |                      </ [...]
-</text><text class="breeze-static-checks-r5" x="0" y="800.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-32)">│</text><text class="breeze-static-checks-r7" x="366" y="800.8" textLength="1073.6" clip-path="url(#breeze-static-checks-line-32)">replace-bad-characters | rst-backticks | run-flake8 | run-mypy | run-shellcheck |       </text><text class="breeze-static-checks-r5" x="1451.8" y="800.8" t [...]
-</text><text class="breeze-static-checks-r5" x="0" y="825.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-33)">│</text><text class="breeze-static-checks-r7" x="366" y="825.2" textLength="1073.6" clip-path="url(#breeze-static-checks-line-33)">static-check-autoflake | trailing-whitespace | ts-compile-and-lint-javascript |         </text><text class="breeze-static-checks-r5" x="1451.8" y="825.2" textLength= [...]
-</text><text class="breeze-static-checks-r5" x="0" y="849.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-34)">│</text><text class="breeze-static-checks-r7" x="366" y="849.6" textLength="1073.6" clip-path="url(#breeze-static-checks-line-34)">update-black-version | update-breeze-cmd-output | update-breeze-readme-config-hash |    </text><text class="breeze-static-checks-r5" x="1451.8" y="849.6" textLength="12.2" clip-path="url(#br [...]
-</text><text class="breeze-static-checks-r5" x="0" y="874" textLength="12.2" clip-path="url(#breeze-static-checks-line-35)">│</text><text class="breeze-static-checks-r7" x="366" y="874" textLength="1073.6" clip-path="url(#breeze-static-checks-line-35)">update-er-diagram | update-extras | update-in-the-wild-to-be-sorted |                   </text><text class="breeze-stat [...]
-</text><text class="breeze-static-checks-r5" x="0" y="898.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-36)">│</text><text class="breeze-static-checks-r7" x="366" y="898.4" textLength="1073.6" clip-path="url(#breeze-static-checks-line-36)">update-inlined-dockerfile-scripts | update-local-yml-file | update-migration-references </text><text class="breeze-static-checks-r5" x="1451.8" y="898.4" textLength="12.2" clip-path="url(#breeze-static-checks-l [...]
-</text><text class="breeze-static-checks-r5" x="0" y="922.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-37)">│</text><text class="breeze-static-checks-r7" x="366" y="922.8" textLength="1073.6" clip-path="url(#breeze-static-checks-line-37)">| update-providers-dependencies | update-spelling-wordlist-to-be-sorted |               </text><text class="breeze-static-checks-r5" x="1451 [...]
-</text><text class="breeze-static-checks-r5" x="0" y="947.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-38)">│</text><text class="breeze-static-checks-r7" x="366" y="947.2" textLength="1073.6" clip-path="url(#breeze-static-checks-line-38)">update-supported-versions | update-vendored-in-k8s-json-schema | update-version |       </text><text class="breeze-static-checks-r5" x="1451.8" y="947.2" textLength="12.2" cli [...]
-</text><text class="breeze-static-checks-r5" x="0" y="971.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-39)">│</text><text class="breeze-static-checks-r7" x="366" y="971.6" textLength="1073.6" clip-path="url(#breeze-static-checks-line-39)">yamllint | yesqa)                                   [...]
-</text><text class="breeze-static-checks-r5" x="0" y="996" textLength="12.2" clip-path="url(#breeze-static-checks-line-40)">│</text><text class="breeze-static-checks-r4" x="24.4" y="996" textLength="12.2" clip-path="url(#breeze-static-checks-line-40)">-</text><text class="breeze-static-checks-r4" x="36.6" y="996" textLength="61" clip-path="url(#breeze-static-checks-line-40)">-file</text><text class="breeze-static-checks-r6" x="317.2" y="996" textLength="24.4" clip-path="url(#breeze-stati [...]
-</text><text class="breeze-static-checks-r5" x="0" y="1020.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-41)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1020.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-41)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1020.4" textLength="48.8" clip-path="url(#breeze-static-checks-line-41)">-all</text><text class="breeze-static-checks-r4" x="85.4" y="1020.4" textLength="73.2" clip-path="url(# [...]
-</text><text class="breeze-static-checks-r5" x="0" y="1044.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-42)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1044.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-42)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1044.8" textLength="61" clip-path="url(#breeze-static-checks-line-42)">-show</text><text class="breeze-static-checks-r4" x="97.6" y="1044.8" textLength="195.2" clip-path="url(# [...]
-</text><text class="breeze-static-checks-r5" x="0" y="1069.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-43)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1069.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-43)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1069.2" textLength="61" clip-path="url(#breeze-static-checks-line-43)">-last</text><text class="breeze-static-checks-r4" x="97.6" y="1069.2" textLength="85.4" clip-path="url(#b [...]
-</text><text class="breeze-static-checks-r5" x="0" y="1093.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-44)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1093.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-44)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1093.6" textLength="85.4" clip-path="url(#breeze-static-checks-line-44)">-commit</text><text class="breeze-static-checks-r4" x="122" y="1093.6" textLength="48.8" clip-path="url [...]
-</text><text class="breeze-static-checks-r5" x="0" y="1118" textLength="12.2" clip-path="url(#breeze-static-checks-line-45)">│</text><text class="breeze-static-checks-r2" x="366" y="1118" textLength="292.8" clip-path="url(#breeze-static-checks-line-45)">Mutually exclusive with </text><text class="breeze-static-checks-r4" x="658.8" y="1118" textLength="12.2" clip-path="url(#breeze-static-checks-line-45)">-</text><text class="breeze-static-checks-r4" x="671" y="1118" textLen [...]
-</text><text class="breeze-static-checks-r5" x="0" y="1142.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-46)">│</text><text class="breeze-static-checks-r7" x="366" y="1142.4" textLength="1073.6" clip-path="url(#breeze-static-checks-line-46)">(TEXT)                                       [...]
-</text><text class="breeze-static-checks-r5" x="0" y="1166.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-47)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1166.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-47)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1166.8" textLength="85.4" clip-path="url(#breeze-static-checks-line-47)">-github</text><text class="breeze-static-checks-r4" x="122" y="1166.8" textLength="134.2" clip-path="ur [...]
-</text><text class="breeze-static-checks-r5" x="0" y="1191.2" textLength="1464" clip-path="url(#breeze-static-checks-line-48)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-static-checks-r2" x="1464" y="1191.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-48)">
-</text><text class="breeze-static-checks-r5" x="0" y="1215.6" textLength="24.4" clip-path="url(#breeze-static-checks-line-49)">╭─</text><text class="breeze-static-checks-r5" x="24.4" y="1215.6" textLength="195.2" clip-path="url(#breeze-static-checks-line-49)"> Common options </text><text class="breeze-static-checks-r5" x="219.6" y="1215.6" textLength="1220" clip-path="url(#breeze-static-checks-line-49)">────────────────────────────────────────────────────────────────────── [...]
-</text><text class="breeze-static-checks-r5" x="0" y="1240" textLength="12.2" clip-path="url(#breeze-static-checks-line-50)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1240" textLength="12.2" clip-path="url(#breeze-static-checks-line-50)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1240" textLength="97.6" clip-path="url(#breeze-static-checks-line-50)">-verbose</text><text class="breeze-static-checks-r6" x="158.6" y="1240" textLength="24.4" clip-path="url(#bre [...]
-</text><text class="breeze-static-checks-r5" x="0" y="1264.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-51)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1264.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-51)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1264.4" textLength="48.8" clip-path="url(#breeze-static-checks-line-51)">-dry</text><text class="breeze-static-checks-r4" x="85.4" y="1264.4" textLength="48.8" clip-path="url(# [...]
-</text><text class="breeze-static-checks-r5" x="0" y="1288.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-52)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1288.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-52)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1288.8" textLength="61" clip-path="url(#breeze-static-checks-line-52)">-help</text><text class="breeze-static-checks-r6" x="158.6" y="1288.8" textLength="24.4" clip-path="url(# [...]
-</text><text class="breeze-static-checks-r5" x="0" y="1313.2" textLength="1464" clip-path="url(#breeze-static-checks-line-53)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-static-checks-r2" x="1464" y="1313.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-53)">
+</text><text class="breeze-static-checks-r5" x="0" y="727.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-29)">│</text><text class="breeze-static-checks-r7" x="366" y="727.6" textLength="1073.6" clip-path="url(#breeze-static-checks-line-29)">| insert-license | lint-chart-schema | lint-css | lint-dockerfile | lint-helm-chart |   </text><text class="breeze-static-checks-r5" x="1451.8" y="727.6" textLength="12.2 [...]
+</text><text class="breeze-static-checks-r5" x="0" y="752" textLength="12.2" clip-path="url(#breeze-static-checks-line-30)">│</text><text class="breeze-static-checks-r7" x="366" y="752" textLength="1073.6" clip-path="url(#breeze-static-checks-line-30)">lint-json-schema | lint-markdown | lint-openapi | mixed-line-ending | pretty-format-json</text><text class="breeze-static-checks-r5" x="1451.8" y="752" textLength="12.2" clip-path="url(#breeze-static [...]
+</text><text class="breeze-static-checks-r5" x="0" y="776.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-31)">│</text><text class="breeze-static-checks-r7" x="366" y="776.4" textLength="1073.6" clip-path="url(#breeze-static-checks-line-31)">| python-no-log-warn | replace-bad-characters | rst-backticks | ruff | run-mypy |       </text><text class="breeze-static-checks-r5" x="1451.8" y="776 [...]
+</text><text class="breeze-static-checks-r5" x="0" y="800.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-32)">│</text><text class="breeze-static-checks-r7" x="366" y="800.8" textLength="1073.6" clip-path="url(#breeze-static-checks-line-32)">run-shellcheck | trailing-whitespace | ts-compile-and-lint-javascript |                 </text><text class="breeze-static-che [...]
+</text><text class="breeze-static-checks-r5" x="0" y="825.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-33)">│</text><text class="breeze-static-checks-r7" x="366" y="825.2" textLength="1073.6" clip-path="url(#breeze-static-checks-line-33)">update-black-version | update-breeze-cmd-output | update-breeze-readme-config-hash |    </text><text class="breeze-static-checks-r5" x="1451.8" y="825.2" textLength="12.2" clip-path="url(#br [...]
+</text><text class="breeze-static-checks-r5" x="0" y="849.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-34)">│</text><text class="breeze-static-checks-r7" x="366" y="849.6" textLength="1073.6" clip-path="url(#breeze-static-checks-line-34)">update-er-diagram | update-extras | update-in-the-wild-to-be-sorted |                   </text><text class="breeze- [...]
+</text><text class="breeze-static-checks-r5" x="0" y="874" textLength="12.2" clip-path="url(#breeze-static-checks-line-35)">│</text><text class="breeze-static-checks-r7" x="366" y="874" textLength="1073.6" clip-path="url(#breeze-static-checks-line-35)">update-inlined-dockerfile-scripts | update-local-yml-file | update-migration-references </text><text class="breeze-static-checks-r5" x="1451.8" y="874" textLength="12.2" clip-path="url(#breeze-static-checks-line-35 [...]
+</text><text class="breeze-static-checks-r5" x="0" y="898.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-36)">│</text><text class="breeze-static-checks-r7" x="366" y="898.4" textLength="1073.6" clip-path="url(#breeze-static-checks-line-36)">| update-providers-dependencies | update-spelling-wordlist-to-be-sorted |               </text><text class="breeze-static-checks-r5" x="1451 [...]
+</text><text class="breeze-static-checks-r5" x="0" y="922.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-37)">│</text><text class="breeze-static-checks-r7" x="366" y="922.8" textLength="1073.6" clip-path="url(#breeze-static-checks-line-37)">update-supported-versions | update-vendored-in-k8s-json-schema | update-version |       </text><text class="breeze-static-checks-r5" x="1451.8" y="922.8" textLength="12.2" cli [...]
+</text><text class="breeze-static-checks-r5" x="0" y="947.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-38)">│</text><text class="breeze-static-checks-r7" x="366" y="947.2" textLength="1073.6" clip-path="url(#breeze-static-checks-line-38)">yamllint)                                      [...]
+</text><text class="breeze-static-checks-r5" x="0" y="971.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-39)">│</text><text class="breeze-static-checks-r4" x="24.4" y="971.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-39)">-</text><text class="breeze-static-checks-r4" x="36.6" y="971.6" textLength="61" clip-path="url(#breeze-static-checks-line-39)">-file</text><text class="breeze-static-checks-r6" x="317.2" y="971.6" textLength="24.4" clip-path="url(#bree [...]
+</text><text class="breeze-static-checks-r5" x="0" y="996" textLength="12.2" clip-path="url(#breeze-static-checks-line-40)">│</text><text class="breeze-static-checks-r4" x="24.4" y="996" textLength="12.2" clip-path="url(#breeze-static-checks-line-40)">-</text><text class="breeze-static-checks-r4" x="36.6" y="996" textLength="48.8" clip-path="url(#breeze-static-checks-line-40)">-all</text><text class="breeze-static-checks-r4" x="85.4" y="996" textLength="73.2" clip-path="url(#breeze-stati [...]
+</text><text class="breeze-static-checks-r5" x="0" y="1020.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-41)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1020.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-41)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1020.4" textLength="61" clip-path="url(#breeze-static-checks-line-41)">-show</text><text class="breeze-static-checks-r4" x="97.6" y="1020.4" textLength="195.2" clip-path="url(# [...]
+</text><text class="breeze-static-checks-r5" x="0" y="1044.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-42)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1044.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-42)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1044.8" textLength="61" clip-path="url(#breeze-static-checks-line-42)">-last</text><text class="breeze-static-checks-r4" x="97.6" y="1044.8" textLength="85.4" clip-path="url(#b [...]
+</text><text class="breeze-static-checks-r5" x="0" y="1069.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-43)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1069.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-43)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1069.2" textLength="85.4" clip-path="url(#breeze-static-checks-line-43)">-commit</text><text class="breeze-static-checks-r4" x="122" y="1069.2" textLength="48.8" clip-path="url [...]
+</text><text class="breeze-static-checks-r5" x="0" y="1093.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-44)">│</text><text class="breeze-static-checks-r2" x="366" y="1093.6" textLength="292.8" clip-path="url(#breeze-static-checks-line-44)">Mutually exclusive with </text><text class="breeze-static-checks-r4" x="658.8" y="1093.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-44)">-</text><text class="breeze-static-checks-r4" x="671" y="1093.6" [...]
+</text><text class="breeze-static-checks-r5" x="0" y="1118" textLength="12.2" clip-path="url(#breeze-static-checks-line-45)">│</text><text class="breeze-static-checks-r7" x="366" y="1118" textLength="1073.6" clip-path="url(#breeze-static-checks-line-45)">(TEXT)                                       [...]
+</text><text class="breeze-static-checks-r5" x="0" y="1142.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-46)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1142.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-46)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1142.4" textLength="85.4" clip-path="url(#breeze-static-checks-line-46)">-github</text><text class="breeze-static-checks-r4" x="122" y="1142.4" textLength="134.2" clip-path="ur [...]
+</text><text class="breeze-static-checks-r5" x="0" y="1166.8" textLength="1464" clip-path="url(#breeze-static-checks-line-47)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-static-checks-r2" x="1464" y="1166.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-47)">
+</text><text class="breeze-static-checks-r5" x="0" y="1191.2" textLength="24.4" clip-path="url(#breeze-static-checks-line-48)">╭─</text><text class="breeze-static-checks-r5" x="24.4" y="1191.2" textLength="195.2" clip-path="url(#breeze-static-checks-line-48)"> Common options </text><text class="breeze-static-checks-r5" x="219.6" y="1191.2" textLength="1220" clip-path="url(#breeze-static-checks-line-48)">────────────────────────────────────────────────────────────────────── [...]
+</text><text class="breeze-static-checks-r5" x="0" y="1215.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-49)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1215.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-49)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1215.6" textLength="97.6" clip-path="url(#breeze-static-checks-line-49)">-verbose</text><text class="breeze-static-checks-r6" x="158.6" y="1215.6" textLength="24.4" clip-path=" [...]
+</text><text class="breeze-static-checks-r5" x="0" y="1240" textLength="12.2" clip-path="url(#breeze-static-checks-line-50)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1240" textLength="12.2" clip-path="url(#breeze-static-checks-line-50)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1240" textLength="48.8" clip-path="url(#breeze-static-checks-line-50)">-dry</text><text class="breeze-static-checks-r4" x="85.4" y="1240" textLength="48.8" clip-path="url(#breeze-s [...]
+</text><text class="breeze-static-checks-r5" x="0" y="1264.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-51)">│</text><text class="breeze-static-checks-r4" x="24.4" y="1264.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-51)">-</text><text class="breeze-static-checks-r4" x="36.6" y="1264.4" textLength="61" clip-path="url(#breeze-static-checks-line-51)">-help</text><text class="breeze-static-checks-r6" x="158.6" y="1264.4" textLength="24.4" clip-path="url(# [...]
+</text><text class="breeze-static-checks-r5" x="0" y="1288.8" textLength="1464" clip-path="url(#breeze-static-checks-line-52)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-static-checks-r2" x="1464" y="1288.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-52)">
</text>
</g>
</g>
diff --git a/provider_packages/.flake8 b/provider_packages/.flake8
deleted file mode 120000
index cb0568d647..0000000000
--- a/provider_packages/.flake8
+++ /dev/null
@@ -1 +0,0 @@
-../.flake8
\ No newline at end of file
diff --git a/pyproject.toml b/pyproject.toml
index 0f951ba318..722d50de53 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -25,16 +25,102 @@ target-version = ['py37', 'py38', 'py39', 'py310']
[build-system]
requires = ['setuptools==67.2.0']
build-backend = "setuptools.build_meta"
-[tool.isort]
-add_imports = ["from __future__ import annotations"]
-append_only = true
-line_length = 110
-combine_as_imports = true
-default_section = "THIRDPARTY"
-known_first_party = ["airflow", "airflow_breeze", "docker_tests", "docs", "kubernetes_tests", "tests"]
+
+[tool.ruff]
+typing-modules = ["airflow.typing_compat"]
+line-length = 110
+extend-exclude = [
+ ".eggs",
+ "airflow/_vendor/*",
+
+ # The files generated by stubgen aren't 100% valid syntax it turns out, and we don't ship them, so we can
+ # ignore them in ruff
+ "airflow/providers/common/sql/*/*.pyi"
+]
+
+# TODO: Bump to Python 3.8 when support for Python 3.7 is dropped in Airflow.
+target-version = "py37"
+
+extend-select = [
+ "I", # Missing required import (auto-fixable)
+ "UP", # Pyupgrade
+ "RUF100", # Unused noqa (auto-fixable)
+
+ # We ignore more pydocstyle than we enable, so be more selective at what we enable
+ "D101",
+ "D106",
+ "D2",
+ "D3",
+ # "D401", # Not enabled by ruff, but we don't want it
+ "D402",
+ "D403",
+ "D412",
+ "D419"
+]
+extend-ignore = [
+ "D203",
+ "D205",
+ "D212",
+ "D213",
+ "D214",
+ "D215",
+ "E731",
+]
+
+[tool.ruff.isort]
+known-first-party = ["airflow", "airflow_breeze", "docker_tests", "docs", "kubernetes_tests", "tests"]
+required-imports = ["from __future__ import annotations"]
+combine-as-imports = true
+
+# TODO: for now, https://github.com/charliermarsh/ruff/issues/1817
+known-third-party = [
+ "asana",
+ "atlassian",
+ "celery",
+ "cloudant",
+ "databricks",
+ "datadog",
+ "docker",
+ "elasticsearch",
+ "github",
+ "google",
+ "grpc",
+ "jenkins",
+ "mysql",
+ "neo4j",
+ "papermill",
+ "redis",
+ "sendgrid",
+ "snowflake",
+ "telegram",
+ "trino",
+]
+
+[tool.ruff.per-file-ignores]
+"airflow/models/__init__.py" = ["F401"]
+"airflow/models/sqla_models.py" = ["F401"]
+
+
# The test_python.py is needed because adding __future__.annotations breaks runtime checks that are
# needed for the test to work
-skip = ["build", ".tox", "venv", "tests/decorators/test_python.py"]
-lines_between_types = 0
-skip_glob = ["*.pyi"]
-profile = "black"
+"tests/decorators/test_python.py" = ["I002"]
+
+# Ignore pydoc style from these
+"*.pyi" = ["D"]
+"tests/*" = ["D"]
+"scripts/*" = ["D"]
+"dev/*" = ["D"]
+"docs/*" = ["D"]
+"provider_packages/*" = ["D"]
+"docker_tests/*" = ["D"]
+"kubernetes_tests/*" = ["D"]
+"*/example_dags/*" = ["D"]
+"chart/*" = ["D"]
+
+# All of the modules which have an extra license header (i.e. that we copy from another project) need to
+# ignore E402 -- module level import not at top level
+"airflow/api/auth/backend/kerberos_auth.py" = ["E402"]
+"airflow/security/kerberos.py" = ["E402"]
+"airflow/security/utils.py" = ["E402"]
+"tests/providers/elasticsearch/log/elasticmock/__init__.py" = ["E402"]
+"tests/providers/elasticsearch/log/elasticmock/utilities/__init__.py" = ["E402"]
diff --git a/scripts/ci/pre_commit/pre_commit_check_pre_commit_hooks.py b/scripts/ci/pre_commit/pre_commit_check_pre_commit_hooks.py
index 2cee66a4b2..d6e32a0937 100755
--- a/scripts/ci/pre_commit/pre_commit_check_pre_commit_hooks.py
+++ b/scripts/ci/pre_commit/pre_commit_check_pre_commit_hooks.py
@@ -119,10 +119,10 @@ def black_mode():
return Mode(
target_versions=target_versions,
- line_length=bool(config.get("line_length", Mode.line_length)),
- is_pyi=bool(config.get("is_pyi", Mode.is_pyi)),
- string_normalization=not bool(config.get("skip_string_normalization", not Mode.string_normalization)),
- preview=bool(config.get("preview", Mode.preview)),
+ line_length=config.get("line_length", Mode.line_length),
+ is_pyi=config.get("is_pyi", False),
+ string_normalization=not config.get("skip_string_normalization", False),
+ preview=config.get("preview", False),
)
@@ -170,7 +170,7 @@ def main():
parser = argparse.ArgumentParser()
parser.add_argument("--max-length", help="Max length for hook names")
args = parser.parse_args()
- max_length = int(args.max_length) or 70
+ max_length = int(args.max_length or 70)
content = yaml.safe_load(PRE_COMMIT_YAML_FILE.read_text())
errors, hooks, image_hooks = get_errors_and_hooks(content, max_length)
if errors:
diff --git a/scripts/ci/pre_commit/pre_commit_flake8.py b/scripts/ci/pre_commit/pre_commit_flake8.py
deleted file mode 100755
index 1a09c9f474..0000000000
--- a/scripts/ci/pre_commit/pre_commit_flake8.py
+++ /dev/null
@@ -1,72 +0,0 @@
-#!/usr/bin/env python
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-from __future__ import annotations
-
-import os
-import sys
-from pathlib import Path
-
-if __name__ not in ("__main__", "__mp_main__"):
- raise SystemExit(
- "This file is intended to be executed as an executable program. You cannot use it as a module."
- f"To run this script, run the ./{__file__} command"
- )
-
-AIRFLOW_SOURCES = Path(__file__).parents[3].resolve()
-GITHUB_REPOSITORY = os.environ.get("GITHUB_REPOSITORY", "apache/airflow")
-os.environ["SKIP_GROUP_OUTPUT"] = "true"
-
-if __name__ == "__main__":
- sys.path.insert(0, str(Path(__file__).parent.resolve())) # make sure common_precommit_utils is imported
- from common_precommit_utils import filter_out_providers_on_non_main_branch
-
- sys.path.insert(0, str(AIRFLOW_SOURCES / "dev" / "breeze" / "src"))
- from airflow_breeze.global_constants import MOUNT_SELECTED
- from airflow_breeze.utils.console import get_console
- from airflow_breeze.utils.docker_command_utils import get_extra_docker_flags
- from airflow_breeze.utils.run_utils import get_ci_image_for_pre_commits, run_command
-
- files_to_test = filter_out_providers_on_non_main_branch(sys.argv[1:])
- if not files_to_test:
- print("No files to tests. Quitting")
- sys.exit(0)
- airflow_image = get_ci_image_for_pre_commits()
- cmd_result = run_command(
- [
- "docker",
- "run",
- "-t",
- *get_extra_docker_flags(MOUNT_SELECTED),
- "-e",
- "SKIP_ENVIRONMENT_INITIALIZATION=true",
- "-e",
- "BACKEND=sqlite",
- "--pull",
- "never",
- airflow_image,
- "/opt/airflow/scripts/in_container/run_flake8.sh",
- *files_to_test,
- ],
- check=False,
- )
- if cmd_result.returncode != 0:
- get_console().print(
- "[warning]If you see strange stacktraces above, "
- "run `breeze ci-image build --python 3.7` and try again."
- )
- sys.exit(cmd_result.returncode)
diff --git a/scripts/in_container/run_flake8.sh b/scripts/in_container/run_flake8.sh
deleted file mode 100755
index f6c7baa3e1..0000000000
--- a/scripts/in_container/run_flake8.sh
+++ /dev/null
@@ -1,20 +0,0 @@
-#!/usr/bin/env bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-# shellcheck source=scripts/in_container/_in_container_script_init.sh
-. "$( dirname "${BASH_SOURCE[0]}" )/_in_container_script_init.sh"
-flake8 "$@"
diff --git a/setup.py b/setup.py
index 622d630c05..d9279da7b0 100644
--- a/setup.py
+++ b/setup.py
@@ -366,15 +366,8 @@ devel_only = [
"click>=8.0",
"coverage",
"filelock",
- "flake8>=3.9.0",
- "flake8-colors",
- "flake8-implicit-str-concat",
"gitpython",
"ipdb",
- # make sure that we are using stable sorting order from 5.* version (some changes were introduced
- # in 5.11.3. Black is not compatible yet, so we need to limit isort
- # we can remove the limit when black and isort agree on the order
- "isort==5.11.2",
"jira",
"jsondiff",
"mongomock",
@@ -399,6 +392,7 @@ devel_only = [
"pytest-httpx",
"requests_mock",
"rich-click>=1.5",
+ "ruff>=0.0.219",
"semver",
"time-machine",
"towncrier",
diff --git a/tests/api_connexion/endpoints/test_dag_endpoint.py b/tests/api_connexion/endpoints/test_dag_endpoint.py
index 3aa60abd33..a6ace057f4 100644
--- a/tests/api_connexion/endpoints/test_dag_endpoint.py
+++ b/tests/api_connexion/endpoints/test_dag_endpoint.py
@@ -1190,21 +1190,6 @@ class TestPatchDags(TestDagEndpoint):
"timetable_description": None,
"has_import_errors": False,
"pickle_id": None,
- "next_dagrun": None,
- "has_task_concurrency_limits": True,
- "next_dagrun_data_interval_start": None,
- "next_dagrun_data_interval_end": None,
- "max_active_runs": 16,
- "next_dagrun_create_after": None,
- "last_expired": None,
- "max_active_tasks": 16,
- "last_pickled": None,
- "default_view": None,
- "last_parsed_time": None,
- "scheduler_lock": None,
- "timetable_description": None,
- "has_import_errors": False,
- "pickle_id": None,
},
{
"dag_id": "TEST_DAG_DELETED_1",
diff --git a/tests/providers/google/suite/hooks/test_calendar.py b/tests/providers/google/suite/hooks/test_calendar.py
index f472b3da3f..6f51351715 100644
--- a/tests/providers/google/suite/hooks/test_calendar.py
+++ b/tests/providers/google/suite/hooks/test_calendar.py
@@ -15,11 +15,10 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-from __future__ import annotations
-
"""
Unit Tests for the Google Calendar Hook
"""
+from __future__ import annotations
import unittest
from unittest import mock
diff --git a/tests/system/providers/cncf/kubernetes/example_spark_kubernetes.py b/tests/system/providers/cncf/kubernetes/example_spark_kubernetes.py
index ba2e21ee23..03fd0a34e5 100644
--- a/tests/system/providers/cncf/kubernetes/example_spark_kubernetes.py
+++ b/tests/system/providers/cncf/kubernetes/example_spark_kubernetes.py
@@ -15,8 +15,6 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-from __future__ import annotations
-
"""
This is an example DAG which uses SparkKubernetesOperator and SparkKubernetesSensor.
In this example, we create two tasks which execute sequentially.
@@ -26,6 +24,7 @@ and the second task is to check the final state of the sparkApplication that sub
Spark-on-k8s operator is required to be already installed on Kubernetes
https://github.com/GoogleCloudPlatform/spark-on-k8s-operator
"""
+from __future__ import annotations
import os
from datetime import datetime, timedelta
diff --git a/tests/system/providers/google/cloud/bigtable/example_bigtable.py b/tests/system/providers/google/cloud/bigtable/example_bigtable.py
index 96a3c1c450..b105634118 100644
--- a/tests/system/providers/google/cloud/bigtable/example_bigtable.py
+++ b/tests/system/providers/google/cloud/bigtable/example_bigtable.py
@@ -29,7 +29,7 @@ This DAG relies on the following environment variables:
* CBT_INSTANCE_ID - desired ID of a Cloud Bigtable instance
* CBT_INSTANCE_DISPLAY_NAME - desired human-readable display name of the Instance
* CBT_INSTANCE_TYPE - type of the Instance, e.g. 1 for DEVELOPMENT
- See https://googleapis.github.io/google-cloud-python/latest/bigtable/instance.html#google.cloud.bigtable.instance.Instance # noqa E501
+ See https://googleapis.github.io/google-cloud-python/latest/bigtable/instance.html#google.cloud.bigtable.instance.Instance
* CBT_INSTANCE_LABELS - labels to add for the Instance
* CBT_CLUSTER_ID - desired ID of the main Cluster created for the Instance
* CBT_CLUSTER_ZONE - zone in which main Cluster will be created. e.g. europe-west1-b
@@ -37,10 +37,10 @@ This DAG relies on the following environment variables:
* CBT_CLUSTER_NODES - initial amount of nodes of the Cluster
* CBT_CLUSTER_NODES_UPDATED - amount of nodes for BigtableClusterUpdateOperator
* CBT_CLUSTER_STORAGE_TYPE - storage for the Cluster, e.g. 1 for SSD
- See https://googleapis.github.io/google-cloud-python/latest/bigtable/instance.html#google.cloud.bigtable.instance.Instance.cluster # noqa E501
+ See https://googleapis.github.io/google-cloud-python/latest/bigtable/instance.html#google.cloud.bigtable.instance.Instance.cluster
* CBT_TABLE_ID - desired ID of the Table
* CBT_POKE_INTERVAL - number of seconds between every attempt of Sensor check
-"""
+""" # noqa: E501
from __future__ import annotations
import os
diff --git a/tests/test_utils/get_all_tests.py b/tests/test_utils/get_all_tests.py
index ff2b6e4975..1ed04ed64d 100644
--- a/tests/test_utils/get_all_tests.py
+++ b/tests/test_utils/get_all_tests.py
@@ -16,11 +16,11 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-from __future__ import annotations
-
"""
Gets all tests cases from xunit file.
"""
+from __future__ import annotations
+
import sys
from xml.etree import ElementTree