You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ep...@apache.org on 2024/01/15 11:14:28 UTC
(airflow) branch v2-8-test updated (24dd759927 -> a97462bc9f)
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a change to branch v2-8-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit 24dd759927 Update RELEASE_NOTES.rst
new a97462bc9f Update RELEASE_NOTES.rst
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (24dd759927)
\
N -- N -- N refs/heads/v2-8-test (a97462bc9f)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
RELEASE_NOTES.rst | 8 ++++----
airflow/reproducible_build.yaml | 4 ++--
2 files changed, 6 insertions(+), 6 deletions(-)
(airflow) 01/01: Update RELEASE_NOTES.rst
Posted by ep...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a commit to branch v2-8-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit a97462bc9f0867ad423b1e41088d1b3fb41aa398
Author: Ephraim Anierobi <sp...@gmail.com>
AuthorDate: Mon Jan 15 10:55:11 2024 +0100
Update RELEASE_NOTES.rst
---
RELEASE_NOTES.rst | 219 ++++++++++++++++++++++++------------
airflow/reproducible_build.yaml | 4 +-
newsfragments/36281.significant.rst | 4 -
newsfragments/36537.significant.rst | 41 -------
newsfragments/36647.significant.rst | 23 ----
newsfragments/config.toml | 2 +-
6 files changed, 151 insertions(+), 142 deletions(-)
diff --git a/RELEASE_NOTES.rst b/RELEASE_NOTES.rst
index cdd3a4e42d..fc3375e9b0 100644
--- a/RELEASE_NOTES.rst
+++ b/RELEASE_NOTES.rst
@@ -27,75 +27,152 @@ Airflow 2.8.1 (2024-01-19)
Significant Changes
^^^^^^^^^^^^^^^^^^^
-- Target version for core dependency ``pendulum`` package set to 3
-
- Support for pendulum 2.1.2 will be saved for a while, presumably until the next feature version of Airflow.
- It is advised to upgrade user code to use pendulum 3 as soon as possible. (#36281)
-- Airflow packaging specification follows modern Python packaging standards.
-
- We standardized Airflow dependency configuration to follow latest development in Python packaging by
- using pyproject.toml. Airflow is now compliant with those accepted PEPs:
-
- * `PEP-440 Version Identification and Dependency Specification <https://www.python.org/dev/peps/pep-0440/>`__
- * `PEP-517 A build-system independent format for source trees <https://www.python.org/dev/peps/pep-0517/>`__
- * `PEP-518 Specifying Minimum Build System Requirements for Python Projects <https://www.python.org/dev/peps/pep-0518/>`__
- * `PEP-561 Distributing and Packaging Type Information <https://www.python.org/dev/peps/pep-0561/>`__
- * `PEP-621 Storing project metadata in pyproject.toml <https://www.python.org/dev/peps/pep-0621/>`__
- * `PEP-660 Editable installs for pyproject.toml based builds (wheel based) <https://www.python.org/dev/peps/pep-0660/>`__
- * `PEP-685 Comparison of extra names for optional distribution dependencies <https://www.python.org/dev/peps/pep-0685/>`__
-
- Also we implement multiple license files support coming from Draft, not yet accepted (but supported by hatchling) PEP:
- * `PEP 639 Improving License Clarity with Better Package Metadata <https://peps.python.org/pep-0639/>`__
-
- This has almost no noticeable impact on users if they are using modern Python packaging and development tools, generally
- speaking Airflow should behave as it did before when installing it from PyPI and it should be much easier to install
- it for development purposes using ``pip install -e ".[devel]"``.
-
- The differences from the user side are:
-
- * Airflow extras now get extras normalized to ``-`` (following PEP-685) instead of ``_`` and ``.``
- (as it was before in some extras). When you install airflow with such extras (for example ``dbt.core`` or
- ``all_dbs``) you should use ``-`` instead of ``_`` and ``.``.
-
- In most modern tools this will work in backwards-compatible way, but in some old version of those tools you might need to
- replace ``_`` and ``.`` with ``-``. You can also get warnings that the extra you are installing does not exist - but usually
- this warning is harmless and the extra is installed anyway. It is, however, recommended to change to use ``-`` in extras in your dependency
- specifications for all Airflow extras.
+Target version for core dependency ``pendulum`` package set to 3 (#36281).
+""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+Support for pendulum 2.1.2 will be saved for a while, presumably until the next feature version of Airflow.
+It is advised to upgrade user code to use pendulum 3 as soon as possible.
- * Released airflow package does not contain ``devel``, ``devel-*``, ``doc`` and ``doc-gen`` extras.
- Those extras are only available when you install Airflow from sources in ``--editable`` mode. This is
- because those extras are only used for development and documentation building purposes and are not needed
- when you install Airflow for production use. Those dependencies had unspecified and varying behaviour for
- released packages anyway and you were not supposed to use them in released packages.
+Airflow packaging specification follows modern Python packaging standards (#36537).
+"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+We standardized Airflow dependency configuration to follow latest development in Python packaging by
+using ``pyproject.toml``. Airflow is now compliant with those accepted PEPs:
+
+* `PEP-440 Version Identification and Dependency Specification <https://www.python.org/dev/peps/pep-0440/>`__
+* `PEP-517 A build-system independent format for source trees <https://www.python.org/dev/peps/pep-0517/>`__
+* `PEP-518 Specifying Minimum Build System Requirements for Python Projects <https://www.python.org/dev/peps/pep-0518/>`__
+* `PEP-561 Distributing and Packaging Type Information <https://www.python.org/dev/peps/pep-0561/>`__
+* `PEP-621 Storing project metadata in pyproject.toml <https://www.python.org/dev/peps/pep-0621/>`__
+* `PEP-660 Editable installs for pyproject.toml based builds (wheel based) <https://www.python.org/dev/peps/pep-0660/>`__
+* `PEP-685 Comparison of extra names for optional distribution dependencies <https://www.python.org/dev/peps/pep-0685/>`__
+
+Also we implement multiple license files support coming from Draft, not yet accepted (but supported by hatchling) PEP:
+* `PEP 639 Improving License Clarity with Better Package Metadata <https://peps.python.org/pep-0639/>`__
+
+This has almost no noticeable impact on users if they are using modern Python packaging and development tools, generally
+speaking Airflow should behave as it did before when installing it from PyPI and it should be much easier to install
+it for development purposes using ``pip install -e ".[devel]"``.
+
+The differences from the user side are:
+
+* Airflow extras now get extras normalized to ``-`` (following PEP-685) instead of ``_`` and ``.``
+ (as it was before in some extras). When you install airflow with such extras (for example ``dbt.core`` or
+ ``all_dbs``) you should use ``-`` instead of ``_`` and ``.``.
+
+In most modern tools this will work in backwards-compatible way, but in some old version of those tools you might need to
+replace ``_`` and ``.`` with ``-``. You can also get warnings that the extra you are installing does not exist - but usually
+this warning is harmless and the extra is installed anyway. It is, however, recommended to change to use ``-`` in extras in your dependency
+specifications for all Airflow extras.
+
+* Released airflow package does not contain ``devel``, ``devel-*``, ``doc`` and ``doc-gen`` extras.
+ Those extras are only available when you install Airflow from sources in ``--editable`` mode. This is
+ because those extras are only used for development and documentation building purposes and are not needed
+ when you install Airflow for production use. Those dependencies had unspecified and varying behaviour for
+ released packages anyway and you were not supposed to use them in released packages.
+
+* The ``all`` and ``all-*`` extras were not always working correctly when installing Airflow using constraints
+ because they were also considered as development-only dependencies. With this change, those dependencies are
+ now properly handling constraints and they will install properly with constraints, pulling the right set
+ of providers and dependencies when constraints are used.
+
+Graphviz dependency is now an optional one, not required one (#36647).
+""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+The ``graphviz`` dependency has been problematic as Airflow required dependency - especially for
+ARM-based installations. Graphviz packages require binary graphviz libraries - which is already a
+limitation, but they also require to install graphviz Python bindings to be build and installed.
+This does not work for older Linux installation but - more importantly - when you try to install
+Graphviz libraries for Python 3.8, 3.9 for ARM M1 MacBooks, the packages fail to install because
+Python bindings compilation for M1 can only work for Python 3.10+.
- * The ``all`` and ``all-*`` extras were not always working correctly when installing Airflow using constraints
- because they were also considered as development-only dependencies. With this change, those dependencies are
- now properly handling constraints and they will install properly with constraints, pulling the right set
- of providers and dependencies when constraints are used. (#36537)
-- Graphviz dependency is now an optional one, not required one.
+This is not a breaking change technically - the CLIs to render the DAGs is still there and IF you
+already have graphviz installed, it will continue working as it did before. The only problem when it
+does not work is where you do not have graphviz installed it will raise an error and inform that you need it.
- The ``graphviz`` dependency has been problematic as Airflow required dependency - especially for
- ARM-based installations. Graphviz packages require binary graphviz libraries - which is already a
- limitation, but they also require to install graphviz Python bindings to be build and installed.
- This does not work for older Linux installation but - more importantly - when you try to install
- Graphviz libraries for Python 3.8, 3.9 for ARM M1 MacBooks, the packages fail to install because
- Python bindings compilation for M1 can only work for Python 3.10+.
+Graphviz will remain to be installed for most users:
- This is not a breaking change technically - the CLIs to render the DAGs is still there and IF you
- already have graphviz installed, it will continue working as it did before. The only problem when it
- does not work is where you do not have graphviz installed it will raise an error and inform that you need it.
+* the Airflow Image will still contain graphviz library, because
+ it is added there as extra
+* when previous version of Airflow has been installed already, then
+ graphviz library is already installed there and Airflow will
+ continue working as it did
- Graphviz will remain to be installed for most users:
+The only change will be a new installation of new version of Airflow from the scratch, where graphviz will
+need to be specified as extra or installed separately in order to enable DAG rendering option.
- * the Airflow Image will still contain graphviz library, because
- it is added there as extra
- * when previous version of Airflow has been installed already, then
- graphviz library is already installed there and Airflow will
- continue working as it did
+Bug Fixes
+"""""""""
+- Fix Callback exception when a removed task is the last one in the ``taskinstance`` list (#36693)
+- Allow anonymous user edit/show resource when set ``AUTH_ROLE_PUBLIC=admin`` (#36750)
+- Better error message when sqlite URL uses relative path (#36774)
+- Explicit string cast required to force integer-type run_ids to be passed as strings instead of integers (#36756)
+- Add log lookup exception for empty ``op`` subtypes (#35536)
+- Remove unused index on task instance (#36737)
+- Fix check on subclass for ``typing.Union`` in ``_infer_multiple_outputs`` for Python 3.10+ (#36728)
+- Make sure ``multiple_outputs`` is inferred correctly even when using ``TypedDict`` (#36652)
+- Add back FAB constant in legacy security manager (#36719)
+- Fix AttributeError when using ``Dagrun.update_state`` (#36712)
+- Do not let ``EventsTimetable`` schedule past events if ``catchup=False`` (#36134)
+- Support encryption for triggers parameters (#36492)
+- Rename concurrency label to max active tasks (#36691)
+- Fix the type hint for ``tis_query`` in ``_process_executor_events`` (#36655)
+- Redirect to index when user does not have permission to access a page (#36623)
+- Avoid using dict as default value in ``call_regular_interval`` (#36608)
+- Remove option to set a task instance to running state in UI (#36518)
+- Fix details tab not showing when using dynamic task mapping (#36522)
+- Raise error when ``DagRun`` fails while running ``dag test`` (#36517)
+- Refactor ``_manage_executor_state`` by refreshing TIs in batch (#36502)
+- Add flask config: ``MAX_CONTENT_LENGTH`` (#36401)
+- Fix get_leaves calculation for teardown in nested group (#36456)
+- Stop serializing timezone-naive datetime to timezone-aware datetime with UTC tz (#36379)
+- Make ``kubernetes`` decorator type annotation consistent with operator (#36405)
+- Fix Webserver returning 500 for POST requests to ``api/dag/*/dagrun`` from anonymous user (#36275)
+- Fix the required access for get_variable endpoint (#36396)
+- Fix datetime reference in ``DAG.is_fixed_time_schedule`` (#36370)
+- Fix AirflowSkipException message raised by BashOperator (#36354)
+- Allow PythonVirtualenvOperator.skip_on_exit_code to be zero (#36361)
+- Increase width of execution_date input in trigger.html (#36278)
+- Fix logging for pausing DAG (#36182)
+- Stop deserializing pickle when enable_xcom_pickling is False (#36255)
+- Check DAG read permission before accessing DAG code (#36257)
+- Enable mark task as failed/success always (#36254)
+- Create latest log dir symlink as relative link (#36019)
+- Fix Python-based decorators templating (#36103)
- The only change will be a new installation of new version of Airflow from the scratch, where graphviz will
- need to be specified as extra or installed separately in order to enable DAG rendering option. (#36647)
+Miscellaneous
+"""""""""""""
+- Restore function scoped ``httpx`` import in file_task_handler for performance (#36753)
+- Add support of Pendulum 3 (#36281)
+- Standardize airflow build process and switch to Hatchling build backend (#36537)
+- Get rid of ``pyarrow-hotfix`` for ``CVE-2023-47248`` (#36697)
+- Make ``graphviz`` dependency optional (#36647)
+- Announce MSSQL support end in Airflow 2.9.0, add migration script hints (#36509)
+- Set min ``pandas`` dependency to 1.2.5 for all providers and airflow (#36698)
+- Bump follow-redirects from 1.15.3 to 1.15.4 in ``/airflow/www`` (#36700)
+- Provide the logger_name param to base hook in order to override the logger name (#36674)
+- Fix run type icon alignment with run type text (#36616)
+- Follow BaseHook connection fields method signature in FSHook (#36444)
+- Remove redundant ``docker`` decorator type annotations (#36406)
+- Straighten typing in workday timetable (#36296)
+- Use ``batch_is_authorized_dag`` to check if user has permission to read DAGs (#36279)
+- Replace deprecated get_accessible_dag_ids and use get_readable_dags in get_dag_warnings (#36256)
+Doc Only Changes
+""""""""""""""""
+- Metrics tagging documentation (#36627)
+- In docs use logical_date instead of deprecated execution_date (#36654)
+- Add section about live-upgrading Airflow (#36637)
+- Replace ``numpy`` example with practical exercise demonstrating top-level code (#35097)
+- Improve and add more complete description in the architecture diagrams (#36513)
+- Improve the error message displayed when there is a webserver error (#36570)
+- Update ``dags.rst`` with information on DAG pausing (#36540)
+- Update installation prerequisites after upgrading to Debian Bookworm (#36521)
+- Add description on the ways how users should approach DB monitoring (#36483)
+- Add branching based on mapped task group example to dynamic-task-mapping.rst (#36480)
+- Add further details to replacement documentation (#36485)
+- Use cards when describing priority weighting methods (#36411)
+- Update ``metrics.rst`` for param ``dagrun.schedule_delay`` (#36404)
+- Update admonitions in Python operator doc to reflect sentiment (#36340)
+- Improve audit_logs.rst (#36213)
+- Remove Redshift mention from the list of managed Postgres backends (#36217)
Airflow 2.8.0 (2023-12-18)
--------------------------
@@ -103,16 +180,16 @@ Airflow 2.8.0 (2023-12-18)
Significant Changes
^^^^^^^^^^^^^^^^^^^
-- Raw HTML code in DAG docs and DAG params descriptions is disabled by default
-
- To ensure that no malicious javascript can be injected with DAG descriptions or trigger UI forms by DAG authors
- a new parameter ``webserver.allow_raw_html_descriptions`` was added with default value of ``False``.
- If you trust your DAG authors code and want to allow using raw HTML in DAG descriptions and params, you can restore the previous
- behavior by setting the configuration value to ``True``.
-
- To ensure Airflow is secure by default, the raw HTML support in trigger UI has been super-seeded by markdown support via
- the ``description_md`` attribute. If you have been using ``description_html`` please migrate to ``description_md``.
- The ``custom_html_form`` is now deprecated. (#35460)
+Raw HTML code in DAG docs and DAG params descriptions is disabled by default (#35460)
+"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+To ensure that no malicious javascript can be injected with DAG descriptions or trigger UI forms by DAG authors
+a new parameter ``webserver.allow_raw_html_descriptions`` was added with default value of ``False``.
+If you trust your DAG authors code and want to allow using raw HTML in DAG descriptions and params, you can restore the previous
+behavior by setting the configuration value to ``True``.
+
+To ensure Airflow is secure by default, the raw HTML support in trigger UI has been super-seeded by markdown support via
+the ``description_md`` attribute. If you have been using ``description_html`` please migrate to ``description_md``.
+The ``custom_html_form`` is now deprecated.
New Features
""""""""""""
diff --git a/airflow/reproducible_build.yaml b/airflow/reproducible_build.yaml
index af282b6a04..10999b8a56 100644
--- a/airflow/reproducible_build.yaml
+++ b/airflow/reproducible_build.yaml
@@ -1,2 +1,2 @@
-release-notes-hash: b0d8fdf0caf6f957b63ee7f4ee2147b9
-source-date-epoch: 1705311983
+release-notes-hash: 6ddec6e84b200832baabef9b499d4d3a
+source-date-epoch: 1705317232
diff --git a/newsfragments/36281.significant.rst b/newsfragments/36281.significant.rst
deleted file mode 100644
index 1207be0782..0000000000
--- a/newsfragments/36281.significant.rst
+++ /dev/null
@@ -1,4 +0,0 @@
-Target version for core dependency ``pendulum`` package set to 3
-
-Support for pendulum 2.1.2 will be saved for a while, presumably until the next feature version of Airflow.
-It is advised to upgrade user code to use pendulum 3 as soon as possible.
diff --git a/newsfragments/36537.significant.rst b/newsfragments/36537.significant.rst
deleted file mode 100644
index 22a288b076..0000000000
--- a/newsfragments/36537.significant.rst
+++ /dev/null
@@ -1,41 +0,0 @@
-Airflow packaging specification follows modern Python packaging standards.
-
-We standardized Airflow dependency configuration to follow latest development in Python packaging by
-using pyproject.toml. Airflow is now compliant with those accepted PEPs:
-
-* `PEP-440 Version Identification and Dependency Specification <https://www.python.org/dev/peps/pep-0440/>`__
-* `PEP-517 A build-system independent format for source trees <https://www.python.org/dev/peps/pep-0517/>`__
-* `PEP-518 Specifying Minimum Build System Requirements for Python Projects <https://www.python.org/dev/peps/pep-0518/>`__
-* `PEP-561 Distributing and Packaging Type Information <https://www.python.org/dev/peps/pep-0561/>`__
-* `PEP-621 Storing project metadata in pyproject.toml <https://www.python.org/dev/peps/pep-0621/>`__
-* `PEP-660 Editable installs for pyproject.toml based builds (wheel based) <https://www.python.org/dev/peps/pep-0660/>`__
-* `PEP-685 Comparison of extra names for optional distribution dependencies <https://www.python.org/dev/peps/pep-0685/>`__
-
-Also we implement multiple license files support coming from Draft, not yet accepted (but supported by hatchling) PEP:
-* `PEP 639 Improving License Clarity with Better Package Metadata <https://peps.python.org/pep-0639/>`__
-
-This has almost no noticeable impact on users if they are using modern Python packaging and development tools, generally
-speaking Airflow should behave as it did before when installing it from PyPI and it should be much easier to install
-it for development purposes using ``pip install -e ".[devel]"``.
-
-The differences from the user side are:
-
-* Airflow extras now get extras normalized to ``-`` (following PEP-685) instead of ``_`` and ``.``
- (as it was before in some extras). When you install airflow with such extras (for example ``dbt.core`` or
- ``all_dbs``) you should use ``-`` instead of ``_`` and ``.``.
-
-In most modern tools this will work in backwards-compatible way, but in some old version of those tools you might need to
-replace ``_`` and ``.`` with ``-``. You can also get warnings that the extra you are installing does not exist - but usually
-this warning is harmless and the extra is installed anyway. It is, however, recommended to change to use ``-`` in extras in your dependency
-specifications for all Airflow extras.
-
-* Released airflow package does not contain ``devel``, ``devel-*``, ``doc`` and ``doc-gen`` extras.
- Those extras are only available when you install Airflow from sources in ``--editable`` mode. This is
- because those extras are only used for development and documentation building purposes and are not needed
- when you install Airflow for production use. Those dependencies had unspecified and varying behaviour for
- released packages anyway and you were not supposed to use them in released packages.
-
-* The ``all`` and ``all-*`` extras were not always working correctly when installing Airflow using constraints
- because they were also considered as development-only dependencies. With this change, those dependencies are
- now properly handling constraints and they will install properly with constraints, pulling the right set
- of providers and dependencies when constraints are used.
diff --git a/newsfragments/36647.significant.rst b/newsfragments/36647.significant.rst
deleted file mode 100644
index dc3f0faad8..0000000000
--- a/newsfragments/36647.significant.rst
+++ /dev/null
@@ -1,23 +0,0 @@
-Graphviz dependency is now an optional one, not required one.
-
-The ``graphviz`` dependency has been problematic as Airflow required dependency - especially for
-ARM-based installations. Graphviz packages require binary graphviz libraries - which is already a
-limitation, but they also require to install graphviz Python bindings to be build and installed.
-This does not work for older Linux installation but - more importantly - when you try to install
-Graphviz libraries for Python 3.8, 3.9 for ARM M1 MacBooks, the packages fail to install because
-Python bindings compilation for M1 can only work for Python 3.10+.
-
-This is not a breaking change technically - the CLIs to render the DAGs is still there and IF you
-already have graphviz installed, it will continue working as it did before. The only problem when it
-does not work is where you do not have graphviz installed it will raise an error and inform that you need it.
-
-Graphviz will remain to be installed for most users:
-
-* the Airflow Image will still contain graphviz library, because
- it is added there as extra
-* when previous version of Airflow has been installed already, then
- graphviz library is already installed there and Airflow will
- continue working as it did
-
-The only change will be a new installation of new version of Airflow from the scratch, where graphviz will
-need to be specified as extra or installed separately in order to enable DAG rendering option.
diff --git a/newsfragments/config.toml b/newsfragments/config.toml
index 014284c133..4c0f7cc949 100644
--- a/newsfragments/config.toml
+++ b/newsfragments/config.toml
@@ -16,7 +16,7 @@
# under the License.
[tool.towncrier]
name = "Airflow"
-filename = "../RELEASE_NOTES.rst"
+filename = "RELEASE_NOTES.rst"
underlines = ["-", '^']
[[tool.towncrier.type]]