You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ka...@apache.org on 2021/09/11 11:40:41 UTC
[airflow] branch v2-2-test updated (db5ac64 -> e994d36)
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a change to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git.
from db5ac64 Reorder migrations to be compatible with 2.1.4 (#18153)
add a3c4784 Add C2FO to ``INTHEWILD.md`` (#18157)
add a773794 Make auto refresh interval configurable (#18107)
add 2776e08 Fix Airflow version for `[logging] worker_log_server_port` (#18158)
add 9140ad8 Use parameters instead of params (#18143)
add 723eeb4 Add script that validate version fields in config.yaml (#18130)
add f34893a Remove limits for dnspython (#18162)
new faff793 Bump version to `2.2.0b1`
new e994d36 Add 2.2.0b1 changelog
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGELOG.txt | 276 +++++++++++++++++++++
INTHEWILD.md | 1 +
airflow/config_templates/config.yml | 10 +-
airflow/config_templates/default_airflow.cfg | 4 +
airflow/configuration.py | 2 +-
.../example_dags/example_facebook_ads_to_gcs.py | 4 +-
.../google/cloud/transfers/facebook_ads_to_gcs.py | 27 +-
.../example_dags/example_display_video.py | 4 +-
.../marketing_platform/operators/display_video.py | 29 ++-
airflow/www/static/js/graph.js | 5 +-
airflow/www/static/js/tree.js | 4 +-
airflow/www/templates/airflow/graph.html | 1 +
airflow/www/templates/airflow/tree.html | 3 +-
airflow/www/views.py | 2 +
dev/README_RELEASE_AIRFLOW.md | 13 +
dev/validate_version_added_fields_in_config.py | 112 +++++++++
setup.py | 8 +-
.../cloud/transfers/test_facebook_ads_to_gcs.py | 8 +-
.../operators/test_display_video.py | 6 +-
19 files changed, 485 insertions(+), 34 deletions(-)
create mode 100755 dev/validate_version_added_fields_in_config.py
[airflow] 02/02: Add 2.2.0b1 changelog
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit e994d36ff4a22bff602a4ab1df81ae2759174775
Author: Jed Cunningham <66...@users.noreply.github.com>
AuthorDate: Fri Sep 10 17:04:16 2021 -0600
Add 2.2.0b1 changelog
fixup! Add 2.2.0b1 changelog
---
CHANGELOG.txt | 276 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 276 insertions(+)
diff --git a/CHANGELOG.txt b/CHANGELOG.txt
index 442942d..0a51759 100644
--- a/CHANGELOG.txt
+++ b/CHANGELOG.txt
@@ -1,3 +1,279 @@
+Airflow 2.2.0b1, TBD
+-------------------------
+
+New Features
+""""""""""""
+
+- AIP-39: Handle DAG scheduling with timetables (#15397)
+- AIP-39: ``DagRun.data_interval_start|end`` (#16352)
+- AIP-40: Add Deferrable "Async" Tasks (#15389)
+- Add Airflow Standalone command (#15826)
+- Ability to test connections from UI or API (#15795)
+- Add Next Run to UI (#17732)
+- Allow custom timetable as a DAG argument (#17414)
+- Add a calendar field to choose the execution date of the DAG when triggering it (#16141)
+- Allow setting specific ``cwd`` for BashOperator (#17751)
+- Show import errors in DAG views (#17818)
+- Add pre/post execution hooks [Experimental] (#17576)
+- Add triggerer to ``docker-compose.yaml`` file (#17745)
+- Added table to view providers in Airflow ui under admin tab (#15385)
+- Adds secrets backend/logging/auth information to provider yaml (#17625)
+- Add date format filters to Jinja environment (#17451)
+- Introduce ``RESTARTING`` state (#16681)
+- Webserver: Unpause DAG on manual trigger (#16569)
+- API endpoint to create new user (#16609)
+- Add ``insert_args`` for support transfer replace (#15825)
+- Add recursive flag to glob in filesystem sensor (#16894)
+- Add conn to jinja template context (#16686)
+- Add ``default_args`` for ``TaskGroup`` (#16557)
+- Allow adding duplicate connections from UI (#15574)
+- Allow specifying multiple URLs via the CORS config option (#17941)
+- Implement API endpoint for DAG deletion (#17980)
+- Add DAG run endpoint for marking a dagrun success or failed(#17839)
+- Add Triggerer warning banner in UI (#17565)
+- Add support for ``kinit`` options ``[-f|-F]`` and ``[-a|-A]`` (#17816)
+- Queue support for DaskExecutor using Dask Worker Resources (#16829)
+- Make auto refresh interval configurable (#18107)
+
+Improvements
+""""""""""""
+- Adding ``TaskGroup`` support in ``BaseOperator.chain()`` (#17456)
+- Fix deprecation error message rather than silencing it (#18126)
+- Make next_dagrun_info take a data interval (#18088)
+- Change XCom class methods to accept run_id argument (#18084)
+- Advises the kernel to not cache log files generated by Airflow (#18054)
+- Allow filtering DAGS by tags in the REST API (#18090)
+- Optimize imports of Providers Manager (#18052)
+- Adds capability of Warnings for incompatible community providers (#18020)
+- Serialize the ``template_ext`` attribute to show it in UI (#17985)
+- Add ``robots.txt`` and ``X-Robots-Tag`` header (#17946)
+- Refactor BranchDayOfWeekOperator, DayOfWeekSensor (#17940)
+- Update error message to guide the user into self-help mostly (#17929)
+- Update to Celery 5 (#17397)
+- Avoid redirect loop for users with no permissions (#17838)
+- Improve graph view load time for dags with open groups (#17821)
+- Add links to provider's documentation (#17736)
+- Separate infer_data_interval for data interval timetables (#17755)
+- Remove Marshmallow schema warnings (#17753)
+- Add ``logical_date`` to OpenAPI DAGRun schema (#17122)
+- Rename ``none_failed_or_skipped`` by ``none_failed_min_one_success`` trigger rule (#17683)
+- Remove ``[core] store_dag_code`` & use DB to get Dag Code (#16342)
+- Rename ``task_concurrency`` to ``max_active_tis_per_dag`` (#17708)
+- Import Hooks lazily individually in providers manager (#17682)
+- Adding support for multiple task-ids in the external task sensor (#17339)
+- Replace execution_date with run_id in airflow tasks run command (#16666)
+- Avoid endless redirect loop when user has no roles (#17613)
+- Make output from users cli command more consistent (#17642)
+- Open relative extra links in place (#17477)
+- Move worker_log_server_port option to the logging section (#17621)
+- Use gunicorn to serve logs generated by worker (#17591)
+- Improve validation of Group id (#17578)
+- Simplify 404 page (#17501)
+- Add XCom.clear so it's hookable in custom XCom backend (#17405)
+- add deprecation notice for SubDagOperator (#17488)
+- Make schema in DBApiHook private (#17423)
+- Support DAGS folder being in different location on scheduler and runners (#16860)
+- Remove /dagrun/create and disable edit form generated by F.A.B (#17376)
+- Enable specifying dictionary paths in `template_fields_renderers` (#17321)
+- error early if virtualenv is missing (#15788)
+- Handle connection parameters added to Extra and custom fields (#17269)
+- Fix `airflow celery stop` to accept the pid file. (#17278)
+- Remove DAG refresh buttons (#17263)
+- deprecate dummy trigger rule infavor of always (#17144)
+- Be verbose about failure to import airflow_local_settings (#17195)
+- Include exit code in AirflowException str when BashOperator fails. (#17151)
+- Adding EdgeModifier support for chain() (#17099)
+- Only allows supported field types to be used in custom connections (#17194)
+- Secrets backend failover (#16404)
+- Warn on Webserver when using ``SQLite`` or ``SequentialExecutor`` (#17133)
+- Extend ``init_containers`` defined in ``pod_override`` (#17537)
+- Client-side filter dag dependencies (#16253)
+- Improve executor validation in CLI (#17071)
+- Prevent running `airflow db init\upgrade` migrations and setup in parallel. (#17078)
+- Update chain() and cross_downstream() to support XComArgs (#16732)
+- Improve graph view refresh (#16696)
+- When a task instance fails with exception, log it (#16805)
+- Set process title for ``serve-logs`` and ``LocalExecutor`` (#16644)
+- Rename test_cycle to check_cycle (#16617)
+- Set Process title for Worker when using ``LocalExecutor`` (#16623)
+- Add schema as DbApiHook instance attribute (#16521)
+- Improve compatibility with MSSQL (#9973)
+- Add transparency for unsupported connection type (#16220)
+- Call resource based fab methods (#16190)
+- Format more dates with timezone (#16129)
+- Replace deprecated ``dag.sub_dag`` with ``dag.partial_subset`` (#16179)
+- Treat `AirflowSensorTimeout` as immediate failure without retrying (#12058)
+- Marking success/failed automatically clears failed downstream tasks (#13037)
+- UI: Updated table component (#15805)
+- Add close/open indicator for import dag errors (#16073)
+- Add collapsible import errors (#16072)
+- Always return a response in TI's ``action_clear`` view (#15980)
+- Add cli command to delete user by email (#15873)
+- Update `airflow tasks *` commands to lookup TaskInstances from DagRun Table (#16030)
+- Use resource and action names for FAB permissions (#16410)
+- Rename DAG concurrency (``[core] dag_concurrency``) settings for easier understanding (#16267)
+- Calendar UI improvements (#16226)
+- Refactor: ``SKIPPED`` should not be logged again as ``SUCCESS`` (#14822)
+- Remove limits for dnspython (#18162)
+
+Bug Fixes
+"""""""""
+- Change id collation for MySQL to case-sensitive (#18072)
+- Fix failed tests from run_as_user+error_file changes (#18013)
+- Logs task launch exception in StandardTaskRunner (#17967)
+- Require timetable class be registered via plugin (#17989)
+- Applied permissions to self._error_file (#15947)
+- Hide variable import form if user lacks permission (#18000)
+- Fix blank dag dependencies view (#17990)
+- Add missing menu access for dag dependencies and configurations pages (#17450)
+- Fix passing Jinja templates in ``DateTimeSensor`` (#17959)
+- Fixing bug which restricted the visibility of ImportErrors (#17924)
+- Fix grammar in `traceback.html` (#17942)
+- Fix ``DagRunState`` enum query for ``MySQLdb`` driver (#17886)
+- BugFix: ``TimeSensorAsync`` returns a naive datetime (#17875)
+- Fixed button size in "Actions" group. (#17902)
+- Fix log links on graph TI modal (#17862)
+- Only show import errors for DAGs a user can access (#17835)
+- Show all import_errors from zip files (#17759)
+- fix EXTRA_LOGGER_NAMES param and related docs (#17808)
+- Use one interpreter for Airflow and gunicorn (#17805)
+- Fix: Mysql 5.7 id utf8mb3 (#14535)
+- Fix dag_processing.last_duration metric random holes (#17769)
+- Automatically use utf8mb3_general_ci collation for mysql (#17729)
+- Fix ``TimeSensorAsync`` (#17748)
+- Ensure ``DateTimeTrigger`` receives a datetime object (#17747)
+- Fix wrong query on running tis (#17631)
+- Add root to tree refresh url (#17633)
+- Do not delete running DAG from the UI (#17630)
+- Fix ``triggerer`` query where limit is not supported in some MySQL version (#17601)
+- Rescue if a DagRun's DAG was removed from db (#17544)
+- fix: filter condition of TaskInstance does not work #17535 (#17548)
+- Fixed long delays in the Temporal trigger (#17564)
+- Regression on pid reset to allow task start after heartbeat (#17333)
+- Dont use TaskInstance in CeleryExecutor.trigger_tasks (#16248)
+- Remove locks for upgrades in mssql (#17213)
+- Create virtualenv via python call (#17156)
+- Ensure a DAG is acyclic when running DAG.cli() (#17105)
+- Translate non-ascii characters (#17057)
+- Change the logic of None comparison in model_list template (#16893)
+- Have UI and POST /task_instances_state API endpoint have same behaviour (#16539)
+- ensure task is skipped if missing sla (#16719)
+- Fix direct use of cached_property module. (#16710)
+- Fix TI success confirm page (#16650)
+- Cattrs 1.7.0 released by the end of May 2021 break lineage usage (#16173)
+- Fix Celery executor getting stuck randomly because of reset_signals in multiprocessing (#15989)
+- Modify return value check in python virtualenv jinja template (#16049)
+- Fix dag dependency search (#15924)
+- Make custom JSON encoder support ``Decimal`` (#16383)
+- Bugfix: Allow clearing tasks with just ``dag_id`` and empty ``subdir`` (#16513)
+- Convert port value to a number before calling test connection (#16497)
+- Handle missing/null serialized DAG dependencies (#16393)
+- Correctly set ``dag.fileloc`` when using the ``@dag`` decorator (#16384)
+- Fix TI success/failure links (#16233)
+
+Doc only changes
+""""""""""""""""
+- Update non-working example in documentation (#18067)
+- Improve the description of how to handle dynamic task generation (#17963)
+- Remove default_args pattern + added get_current_context() use for Core Airflow example DAGs (#16866)
+- Update max_tis_per_query to better render on the webpage (#17971)
+- Adds Github Oauth example with team based authorization (#17896)
+- Update docker.rst (#17882)
+- Example xcom update (#17749)
+- Add doc warning about connections added via envvars (#17915)
+- fix wrong documents around upgrade-check.rst (#17903)
+- Add Brent to Committers list (#17873)
+- Improves documentation about modules management (#17757)
+- Remove deprecated metrics from metrics.rst (#17772)
+- Make sure "podcution-readiness` of docker-compose is well explained (#17731)
+- Doc: Update Upgrade to 2 docs with Airflow 1.10.x EOL dates (#17710)
+- Doc: Replace deprecated param from docstrings (#17709)
+- docs(dagowner): describe dag owner more carefully (#17699)
+- docs(impersonation): update note so avoid misintrepretation (#17701)
+- Docs: Make ``DAG.is_active`` read-only in API (#17667)
+- Improve cross-links to operators and hooks references (#17622)
+- Update documentation regarding Python 3.9 support (#17611)
+- Fix MySQL database character set instruction (#17603)
+- Document overriding ``XCom.clear`` for data lifecycle management (#17589)
+- Path correction in docs for airflow core (#17567)
+- docs(celery): reworded, add actual multiple queues example (#17541)
+- Doc: Add FAQ to speed up parsing with tons of dag files (#17519)
+- Improve image building documentation for new users (#17409)
+- Doc: Strip unnecessary arguments from MariaDB JIRA URL (#17296)
+- Update warning about MariaDB and multiple schedulers (#17287)
+- Doc: Recommend using same configs on all Airflow components (#17146)
+- Move docs about masking to a new page (#17007)
+- Docs: Suggest use of Env vars instead of Airflow Vars in best practises doc (#16926)
+- Docs: Better description for `pod_template_file` (#16861)
+- Prevent Redis access directly from host (#16885)
+- Add Aneesh Joseph as Airflow Committer (#16835)
+- Docs: Added new pipeline example for the tutorial docs (#16548)
+- Remove upstart from docs (#16672)
+- Add new committers: ``Jed`` and ``TP`` (#16671)
+- Docs: Fix ``flask-ouathlib`` to ``flask-oauthlib`` in Upgrading docs (#16320)
+- Docs: Fix creating a connection docs (#16312)
+- Docker-compose: Don't run migration for adhoc command (#16255)
+- Docs: Fix url for ``Elasticsearch`` (#16275)
+- Small improvements for README.md files (#16244)
+- Fix docs for ``dag_concurrency`` (#16177)
+- Check synctatic correctness for code-snippets (#16005)
+- Improvements for Docker Image docs (#14843)
+- Add proper link for wheel packages in docs. (#15999)
+- Add Docs for ``default_pool`` slots (#15997)
+- Add memory usage warning in quick-start documentation (#15967)
+- Update example `KubernetesExecutor` `git-sync` pod template file (#15904)
+- Docs: Fix Taskflow API docs (#16574)
+- Added new pipeline example for the tutorial docs (#16084)
+- Updating the DAG docstring to include ``render_template_as_native_obj`` (#16534)
+- Update docs on setting up SMTP (#16523)
+- Docs: Fix API verb from ``POST`` to ``PATCH`` (#16511)
+
+Misc/Internal
+"""""""""""""
+- Test coverage on the autocomplete view (#15943)
+- Update version added fields in airflow/config_templates/config.yml (#18128)
+- Add "packaging" to core requirements (#18122)
+- Check for missing DagRun rows for "downstream" tables before migrating DB (#17030)
+- Rename FileToWasbOperator to LocalFilesystemToWasbOperator (#18109)
+- Adds LoggingMixins to BaseTrigger (#18106)
+- Limit colorlog version (6.x is incompatible) (#18099)
+- Change TaskInstance and TaskReschedule PK from execution_date to run_id (#17719)
+- Fix building documentation broken by upgrade of dnspython (#18046)
+- Fix building docs in `main` builds (#18035)
+- Adding missing init file in example_dags directory (#18019)
+- Bump `pip` version to `21.2.4` (#17746)
+- Do not let create_dagrun overwrite explicit run_id (#17728)
+- Remove upper-limit on ``tenacity`` (#17593)
+- Remove redundant ``numpy`` dependency (#17594)
+- Bump `mysql-connector-python` to latest version (#17596)
+- Make `pandas` an optional core dependency (#17575)
+- Add more typing to airflow.utils.helpers (#15582)
+- Chore: Some code cleanup in `airflow/utils/db.py` (#17090)
+- Refactor: Remove processor_factory from DAG processing (#16659)
+- Remove AbstractDagFileProcessorProcess from dag processing (#16816)
+- Update TaskGroup typing (#16811)
+- Update ``click`` to 8.x (#16779)
+- Remove remaining Pylint disables (#16760)
+- Remove duplicated try, there is already a try in create_session (#16701)
+- Removes pylint from our toolchain (#16682)
+- Refactor usage of unneeded function call (#16653)
+- Add type annotations to setup.py (#16658)
+- Remove SQLAlchemy <1.4 constraint (#16630)
+- Refactor `dag.clear` method (#16086)
+- Use ``DAG_ACTIONS`` constant (#16232)
+- Use updated ``_get_all_non_dag_permissions`` method (#16317)
+- Add updated-name wrappers for built-in FAB methods (#16077)
+- Pins ``docutils`` to <0.17 until breaking behaviour is fixed (#16133)
+- Remove ``TaskInstance.log_filepat``h attribute (#15217)
+- Bump ``pyupgrade`` `` v2.13.0`` to ``v2.18.1`` (#15991)
+- Removes unnecessary function call in ``airflow/www/app.py`` (#15956)
+- Correctly implement autocomplete early return in ``airflow/www/views.py`` (#15940)
+- Move ``plyvel`` to google provider extra (#15812)
+- Update ``Watchtower`` version to ``1.0.6`` (#16469)
+- Update permission migrations to use new naming scheme (#16400)
+- Use resource and action names for FAB (#16380)
+- Swap out calls to find_permission_view_menu for get_permission wrapper (#16377)
+
Airflow 2.1.3, 2021-08-23
-------------------------
[airflow] 01/02: Bump version to `2.2.0b1`
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit faff79381467b8f2b6abdc859a536b9dea6fb3bd
Author: Kaxil Naik <ka...@gmail.com>
AuthorDate: Sat Sep 11 12:36:30 2021 +0100
Bump version to `2.2.0b1`
---
setup.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/setup.py b/setup.py
index c52ee4f..af12aa5 100644
--- a/setup.py
+++ b/setup.py
@@ -41,7 +41,7 @@ PY39 = sys.version_info >= (3, 9)
logger = logging.getLogger(__name__)
-version = '2.2.0.dev0'
+version = '2.2.0b1'
my_dir = dirname(__file__)