You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ka...@apache.org on 2024/10/23 00:28:57 UTC
(airflow) branch main updated: Bump ``black`` to ``24.10.0`` (#43285)
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 07fdfd7ddb Bump ``black`` to ``24.10.0`` (#43285)
07fdfd7ddb is described below
commit 07fdfd7ddbd44a773cd30f46f88c2a83cfaf3de2
Author: Kaxil Naik <ka...@apache.org>
AuthorDate: Wed Oct 23 01:28:50 2024 +0100
Bump ``black`` to ``24.10.0`` (#43285)
https://pypi.org/project/black/#history
We were using a version from Oct 17, 2023
---
.pre-commit-config.yaml | 12 ++--
RELEASE_NOTES.rst | 77 ++++++++--------------
airflow/reproducible_build.yaml | 4 +-
contributing-docs/testing/helm_unit_tests.rst | 3 +-
contributing-docs/testing/unit_tests.rst | 33 ++++------
.../guides/developer.rst | 9 +--
docs/apache-airflow/core-concepts/dag-run.rst | 1 +
docs/apache-airflow/core-concepts/dags.rst | 1 +
8 files changed, 52 insertions(+), 88 deletions(-)
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 031f4e6e2e..0f0421d119 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -173,7 +173,7 @@ repos:
entry: ./scripts/ci/pre_commit/update_common_sql_api_stubs.py
language: python
files: ^scripts/ci/pre_commit/update_common_sql_api\.py|^providers/src/airflow/providers/common/sql/.*\.pyi?$
- additional_dependencies: ['rich>=12.4.4', 'mypy==1.9.0', 'black==23.10.0', 'jinja2']
+ additional_dependencies: ['rich>=12.4.4', 'mypy==1.9.0', 'black==24.10.0', 'jinja2']
pass_filenames: false
require_serial: true
- id: update-black-version
@@ -234,12 +234,12 @@ repos:
name: Run black on docs
args:
- --line-length=110
- - --target-version=py37
- - --target-version=py38
- --target-version=py39
- --target-version=py310
+ - --target-version=py311
+ - --target-version=py312
alias: blacken-docs
- additional_dependencies: [black==23.10.0]
+ additional_dependencies: [black==24.10.0]
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
@@ -898,7 +898,7 @@ repos:
- --max-length=53
language: python
files: ^\.pre-commit-config\.yaml$|^scripts/ci/pre_commit/check_pre_commit_hooks\.py$
- additional_dependencies: ['pyyaml', 'jinja2', 'black==23.10.0', 'tabulate', 'rich>=12.4.4']
+ additional_dependencies: ['pyyaml', 'jinja2', 'black==24.10.0', 'tabulate', 'rich>=12.4.4']
require_serial: true
pass_filenames: false
- id: check-integrations-list-consistent
@@ -906,7 +906,7 @@ repos:
entry: ./scripts/ci/pre_commit/check_integrations_list.py
language: python
files: ^scripts/ci/docker-compose/integration-.*\.yml$|^contributing-docs/testing/integration_tests.rst$
- additional_dependencies: ['black==23.10.0', 'tabulate', 'rich>=12.4.4', 'pyyaml']
+ additional_dependencies: ['black==24.10.0', 'tabulate', 'rich>=12.4.4', 'pyyaml']
require_serial: true
pass_filenames: false
- id: update-breeze-readme-config-hash
diff --git a/RELEASE_NOTES.rst b/RELEASE_NOTES.rst
index c2408bd63f..f44c8a0d13 100644
--- a/RELEASE_NOTES.rst
+++ b/RELEASE_NOTES.rst
@@ -3251,8 +3251,7 @@ And to mark a task as producing a dataset pass the dataset(s) to the ``outlets``
.. code-block:: python
@task(outlets=[dataset])
- def my_task():
- ...
+ def my_task(): ...
# Or for classic operators
@@ -3286,8 +3285,7 @@ Previously you had to assign a DAG to a module-level variable in order for Airfl
@dag
- def dag_maker():
- ...
+ def dag_maker(): ...
dag2 = dag_maker()
@@ -3302,8 +3300,7 @@ can become
@dag
- def dag_maker():
- ...
+ def dag_maker(): ...
dag_maker()
@@ -3634,13 +3631,11 @@ For example, in your ``custom_config.py``:
# before
- class YourCustomFormatter(logging.Formatter):
- ...
+ class YourCustomFormatter(logging.Formatter): ...
# after
- class YourCustomFormatter(TimezoneAware):
- ...
+ class YourCustomFormatter(TimezoneAware): ...
AIRFLOW_FORMATTER = LOGGING_CONFIG["formatters"]["airflow"]
@@ -6330,27 +6325,22 @@ The old syntax of passing ``context`` as a dictionary will continue to work with
.. code-block:: python
- def execution_date_fn(execution_date, ctx):
- ...
+ def execution_date_fn(execution_date, ctx): ...
``execution_date_fn`` can take in any number of keyword arguments available in the task context dictionary. The following forms of ``execution_date_fn`` are all supported:
.. code-block:: python
- def execution_date_fn(dt):
- ...
+ def execution_date_fn(dt): ...
- def execution_date_fn(execution_date):
- ...
+ def execution_date_fn(execution_date): ...
- def execution_date_fn(execution_date, ds_nodash):
- ...
+ def execution_date_fn(execution_date, ds_nodash): ...
- def execution_date_fn(execution_date, ds_nodash, dag):
- ...
+ def execution_date_fn(execution_date, ds_nodash, dag): ...
The default value for ``[webserver] cookie_samesite`` has been changed to ``Lax``
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
@@ -7077,8 +7067,7 @@ Previous signature:
external_trigger=False,
conf=None,
session=None,
- ):
- ...
+ ): ...
current:
@@ -7094,8 +7083,7 @@ current:
conf=None,
run_type=None,
session=None,
- ):
- ...
+ ): ...
If user provides ``run_id`` then the ``run_type`` will be derived from it by checking prefix, allowed types
: ``manual``\ , ``scheduled``\ , ``backfill`` (defined by ``airflow.utils.types.DagRunType``\ ).
@@ -7193,8 +7181,9 @@ can be replaced by the following code:
logger = logging.getLogger("custom-logger")
- with redirect_stdout(StreamLogWriter(logger, logging.INFO)), redirect_stderr(
- StreamLogWriter(logger, logging.WARN)
+ with (
+ redirect_stdout(StreamLogWriter(logger, logging.INFO)),
+ redirect_stderr(StreamLogWriter(logger, logging.WARN)),
):
print("I Love Airflow")
@@ -7223,8 +7212,7 @@ are deprecated and will be removed in future versions.
include_examples=conf.getboolean("core", "LOAD_EXAMPLES"),
safe_mode=conf.getboolean("core", "DAG_DISCOVERY_SAFE_MODE"),
store_serialized_dags=False,
- ):
- ...
+ ): ...
**current**\ :
@@ -7235,8 +7223,7 @@ are deprecated and will be removed in future versions.
include_examples=conf.getboolean("core", "LOAD_EXAMPLES"),
safe_mode=conf.getboolean("core", "DAG_DISCOVERY_SAFE_MODE"),
read_dags_from_db=False,
- ):
- ...
+ ): ...
If you were using positional arguments, it requires no change but if you were using keyword
arguments, please change ``store_serialized_dags`` to ``read_dags_from_db``.
@@ -8058,8 +8045,7 @@ Before:
dataset_id: str,
dataset_resource: dict,
# ...
- ):
- ...
+ ): ...
After:
@@ -8069,8 +8055,7 @@ After:
dataset_resource: dict,
dataset_id: Optional[str] = None,
# ...
- ):
- ...
+ ): ...
Changes in ``amazon`` provider package
""""""""""""""""""""""""""""""""""""""""""
@@ -10150,16 +10135,14 @@ Old signature:
.. code-block:: python
- def get_task_instances(self, session, start_date=None, end_date=None):
- ...
+ def get_task_instances(self, session, start_date=None, end_date=None): ...
New signature:
.. code-block:: python
@provide_session
- def get_task_instances(self, start_date=None, end_date=None, session=None):
- ...
+ def get_task_instances(self, start_date=None, end_date=None, session=None): ...
For ``DAG``
~~~~~~~~~~~~~~~
@@ -10168,16 +10151,14 @@ Old signature:
.. code-block:: python
- def get_task_instances(self, session, start_date=None, end_date=None, state=None):
- ...
+ def get_task_instances(self, session, start_date=None, end_date=None, state=None): ...
New signature:
.. code-block:: python
@provide_session
- def get_task_instances(self, start_date=None, end_date=None, state=None, session=None):
- ...
+ def get_task_instances(self, start_date=None, end_date=None, state=None, session=None): ...
In either case, it is necessary to rewrite calls to the ``get_task_instances`` method that currently provide the ``session`` positional argument. New calls to this method look like:
@@ -10658,15 +10639,13 @@ Old signature:
.. code-block:: python
- def create_transfer_job(self, description, schedule, transfer_spec, project_id=None):
- ...
+ def create_transfer_job(self, description, schedule, transfer_spec, project_id=None): ...
New signature:
.. code-block:: python
- def create_transfer_job(self, body):
- ...
+ def create_transfer_job(self, body): ...
It is necessary to rewrite calls to method. The new call looks like this:
@@ -10691,15 +10670,13 @@ Old signature:
.. code-block:: python
- def wait_for_transfer_job(self, job):
- ...
+ def wait_for_transfer_job(self, job): ...
New signature:
.. code-block:: python
- def wait_for_transfer_job(self, job, expected_statuses=(GcpTransferOperationStatus.SUCCESS,)):
- ...
+ def wait_for_transfer_job(self, job, expected_statuses=(GcpTransferOperationStatus.SUCCESS,)): ...
The behavior of ``wait_for_transfer_job`` has changed:
diff --git a/airflow/reproducible_build.yaml b/airflow/reproducible_build.yaml
index bbf930c9d5..e02435f29f 100644
--- a/airflow/reproducible_build.yaml
+++ b/airflow/reproducible_build.yaml
@@ -1,2 +1,2 @@
-release-notes-hash: 48b5906017b111b01aeb940a1ec904bd
-source-date-epoch: 1728409953
+release-notes-hash: c1049adaf66bd2d1cc6550b406594d24
+source-date-epoch: 1729642591
diff --git a/contributing-docs/testing/helm_unit_tests.rst b/contributing-docs/testing/helm_unit_tests.rst
index 266be65d81..9a733104b6 100644
--- a/contributing-docs/testing/helm_unit_tests.rst
+++ b/contributing-docs/testing/helm_unit_tests.rst
@@ -25,8 +25,7 @@ add them in ``helm_tests``.
.. code-block:: python
- class TestBaseChartTest:
- ...
+ class TestBaseChartTest: ...
To render the chart create a YAML string with the nested dictionary of options you wish to test. You can then
use our ``render_chart`` function to render the object of interest into a testable Python dictionary. Once the chart
diff --git a/contributing-docs/testing/unit_tests.rst b/contributing-docs/testing/unit_tests.rst
index 40f7566392..2c136b55eb 100644
--- a/contributing-docs/testing/unit_tests.rst
+++ b/contributing-docs/testing/unit_tests.rst
@@ -320,8 +320,7 @@ Method level:
@pytest.mark.db_test
- def test_add_tagging(self, sentry, task_instance):
- ...
+ def test_add_tagging(self, sentry, task_instance): ...
Class level:
@@ -332,8 +331,7 @@ Class level:
@pytest.mark.db_test
- class TestDatabricksHookAsyncAadTokenSpOutside:
- ...
+ class TestDatabricksHookAsyncAadTokenSpOutside: ...
Module level (at the top of the module):
@@ -437,8 +435,7 @@ The fix for that is to sort the parameters in ``parametrize``. For example inste
.. code-block:: python
@pytest.mark.parametrize("status", ALL_STATES)
- def test_method():
- ...
+ def test_method(): ...
do that:
@@ -447,8 +444,7 @@ do that:
.. code-block:: python
@pytest.mark.parametrize("status", sorted(ALL_STATES))
- def test_method():
- ...
+ def test_method(): ...
Similarly if your parameters are defined as result of utcnow() or other dynamic method - you should
avoid that, or assign unique IDs for those parametrized tests. Instead of this:
@@ -470,8 +466,7 @@ avoid that, or assign unique IDs for those parametrized tests. Instead of this:
),
],
)
- def test_end_date_gte_lte(url, expected_dag_run_ids):
- ...
+ def test_end_date_gte_lte(url, expected_dag_run_ids): ...
Do this:
@@ -494,8 +489,7 @@ Do this:
),
],
)
- def test_end_date_gte_lte(url, expected_dag_run_ids):
- ...
+ def test_end_date_gte_lte(url, expected_dag_run_ids): ...
@@ -558,8 +552,7 @@ the test is marked as DB test:
),
],
)
- def test_from_json(self, input, request_class):
- ...
+ def test_from_json(self, input, request_class): ...
Instead - this will not break collection. The TaskInstance is not initialized when the module is parsed,
@@ -658,8 +651,7 @@ parametrize specification is being parsed - even if test is marked as DB test.
),
],
)
- def test_rendered_task_detail_env_secret(patch_app, admin_client, request, env, expected):
- ...
+ def test_rendered_task_detail_env_secret(patch_app, admin_client, request, env, expected): ...
You can make the code conditional and mock out the Variable to avoid hitting the database.
@@ -704,8 +696,7 @@ You can make the code conditional and mock out the Variable to avoid hitting the
),
],
)
- def test_rendered_task_detail_env_secret(patch_app, admin_client, request, env, expected):
- ...
+ def test_rendered_task_detail_env_secret(patch_app, admin_client, request, env, expected): ...
You can also use fixture to create object that needs database just like this.
@@ -1056,8 +1047,7 @@ Example of the ``postgres`` only test:
.. code-block:: python
@pytest.mark.backend("postgres")
- def test_copy_expert(self):
- ...
+ def test_copy_expert(self): ...
Example of the ``postgres,mysql`` test (they are skipped with the ``sqlite`` backend):
@@ -1065,8 +1055,7 @@ Example of the ``postgres,mysql`` test (they are skipped with the ``sqlite`` bac
.. code-block:: python
@pytest.mark.backend("postgres", "mysql")
- def test_celery_executor(self):
- ...
+ def test_celery_executor(self): ...
You can use the custom ``--backend`` switch in pytest to only run tests specific for that backend.
diff --git a/docs/apache-airflow-providers-openlineage/guides/developer.rst b/docs/apache-airflow-providers-openlineage/guides/developer.rst
index d826ccde9a..6b291dc17a 100644
--- a/docs/apache-airflow-providers-openlineage/guides/developer.rst
+++ b/docs/apache-airflow-providers-openlineage/guides/developer.rst
@@ -63,16 +63,13 @@ OpenLineage defines a few methods for implementation in Operators. Those are ref
.. code-block:: python
- def get_openlineage_facets_on_start() -> OperatorLineage:
- ...
+ def get_openlineage_facets_on_start() -> OperatorLineage: ...
- def get_openlineage_facets_on_complete(ti: TaskInstance) -> OperatorLineage:
- ...
+ def get_openlineage_facets_on_complete(ti: TaskInstance) -> OperatorLineage: ...
- def get_openlineage_facets_on_failure(ti: TaskInstance) -> OperatorLineage:
- ...
+ def get_openlineage_facets_on_failure(ti: TaskInstance) -> OperatorLineage: ...
OpenLineage methods get called respectively when task instance changes state to:
diff --git a/docs/apache-airflow/core-concepts/dag-run.rst b/docs/apache-airflow/core-concepts/dag-run.rst
index 97fd4e28a7..df732d1260 100644
--- a/docs/apache-airflow/core-concepts/dag-run.rst
+++ b/docs/apache-airflow/core-concepts/dag-run.rst
@@ -100,6 +100,7 @@ in the configuration file. When turned off, the scheduler creates a DAG run only
Code that goes along with the Airflow tutorial located at:
https://github.com/apache/airflow/blob/main/airflow/example_dags/tutorial.py
"""
+
from airflow.models.dag import DAG
from airflow.providers.standard.operators.bash import BashOperator
diff --git a/docs/apache-airflow/core-concepts/dags.rst b/docs/apache-airflow/core-concepts/dags.rst
index 64726e0801..2b6fb0279b 100644
--- a/docs/apache-airflow/core-concepts/dags.rst
+++ b/docs/apache-airflow/core-concepts/dags.rst
@@ -663,6 +663,7 @@ This is especially useful if your tasks are built dynamically from configuration
"""
### My great DAG
"""
+
import pendulum
dag = DAG(