You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ka...@apache.org on 2020/07/01 23:48:18 UTC
[airflow] branch v1-10-test updated (0e4c3a2 -> 9a212d1)
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a change to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git.
discard 0e4c3a2 Update version_added of configs added in 1.10.11
discard 011282e fixup! fixup! Add Changelog for 1.10.11
discard efe86e5 Update docs about the change to default auth for experimental API (#9617)
discard 3c8fb4a Replace old SubDag zoom screenshot with new (#9621)
discard ccf47ae Restrict changing XCom values from the Webserver (#9614)
discard a177e5d Replace old Variables View Screenshot with new (#9620)
discard cc3f09f Change 'initiate' to 'initialize' in installation.rst (#9619)
discard 670c7d4 Add docs to change Colors on the Webserver (#9607)
discard d56fdb4 fixup! Add Changelog for 1.10.11
discard 89329c4 Change default auth for experimental backend to deny_all (#9611)
omit fae2e63 Add Changelog for 1.10.11
new 92c5f46 Change default auth for experimental backend to deny_all (#9611)
new 127104b Add docs to change Colors on the Webserver (#9607)
new e5662b9 Change 'initiate' to 'initialize' in installation.rst (#9619)
new 0610df1 Replace old Variables View Screenshot with new (#9620)
new 30325f8 Restrict changing XCom values from the Webserver (#9614)
new 0663b4e Replace old SubDag zoom screenshot with new (#9621)
new 0d520f2 Update docs about the change to default auth for experimental API (#9617)
new 84fc6b4 Update version_added of configs added in 1.10.11
new c99194a Add __repr__ for DagTag so tags display properly in /dagmodel/show (#8719)
new 9a212d1 Add Changelog for 1.10.11
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (0e4c3a2)
\
N -- N -- N refs/heads/v1-10-test (9a212d1)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 10 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGELOG.txt | 1 +
airflow/models/dag.py | 3 +++
tests/models/test_dag.py | 12 +++++++++++-
3 files changed, 15 insertions(+), 1 deletion(-)
[airflow] 04/10: Replace old Variables View Screenshot with new
(#9620)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 0610df17ecb4ab6d740d09f5ed25d364798acffd
Author: Kaxil Naik <ka...@gmail.com>
AuthorDate: Wed Jul 1 23:51:18 2020 +0100
Replace old Variables View Screenshot with new (#9620)
(cherry picked from commit 05c88cb392e63e9f12f05915b94d216f22b652dd)
---
docs/img/variable_hidden.png | Bin 154299 -> 121301 bytes
1 file changed, 0 insertions(+), 0 deletions(-)
diff --git a/docs/img/variable_hidden.png b/docs/img/variable_hidden.png
index e081ca3..d982b92 100644
Binary files a/docs/img/variable_hidden.png and b/docs/img/variable_hidden.png differ
[airflow] 06/10: Replace old SubDag zoom screenshot with new (#9621)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 0663b4ecce46b5ea95febca345f5605cb232786a
Author: Kaxil Naik <ka...@gmail.com>
AuthorDate: Thu Jul 2 00:18:08 2020 +0100
Replace old SubDag zoom screenshot with new (#9621)
(cherry picked from commit 63a8c79aa9f2dfc3f0802e19b98b0c0b0f6b7858)
---
docs/img/subdag_zoom.png | Bin 150185 -> 255915 bytes
1 file changed, 0 insertions(+), 0 deletions(-)
diff --git a/docs/img/subdag_zoom.png b/docs/img/subdag_zoom.png
index 08fcf5c..fe5ce5a 100644
Binary files a/docs/img/subdag_zoom.png and b/docs/img/subdag_zoom.png differ
[airflow] 02/10: Add docs to change Colors on the Webserver (#9607)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 127104b7e4388be2d0e0b64e4316980d9b6928a9
Author: Kaxil Naik <ka...@gmail.com>
AuthorDate: Wed Jul 1 16:17:29 2020 +0100
Add docs to change Colors on the Webserver (#9607)
Feature was added in https://github.com/apache/airflow/pull/9520
(cherry picked from commit 65855e558f9b070004ba817e8cf94fd834d2a67a)
---
docs/howto/customize-state-colors-ui.rst | 70 +++++++++++++++++++++++++++
docs/howto/index.rst | 1 +
docs/img/change-ui-colors/dags-page-new.png | Bin 0 -> 483599 bytes
docs/img/change-ui-colors/dags-page-old.png | Bin 0 -> 493009 bytes
docs/img/change-ui-colors/graph-view-new.png | Bin 0 -> 56973 bytes
docs/img/change-ui-colors/graph-view-old.png | Bin 0 -> 54884 bytes
docs/img/change-ui-colors/tree-view-new.png | Bin 0 -> 36934 bytes
docs/img/change-ui-colors/tree-view-old.png | Bin 0 -> 21601 bytes
8 files changed, 71 insertions(+)
diff --git a/docs/howto/customize-state-colors-ui.rst b/docs/howto/customize-state-colors-ui.rst
new file mode 100644
index 0000000..c856950
--- /dev/null
+++ b/docs/howto/customize-state-colors-ui.rst
@@ -0,0 +1,70 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ .. http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Customizing state colours in UI
+===============================
+
+.. versionadded:: 1.10.11
+
+To change the colors for TaskInstance/DagRun State in the Airflow Webserver, perform the
+following steps:
+
+1. Create ``airflow_local_settings.py`` file and put in on ``$PYTHONPATH`` or
+ to ``$AIRFLOW_HOME/config`` folder. (Airflow adds ``$AIRFLOW_HOME/config`` on ``PYTHONPATH`` when
+ Airflow is initialized)
+
+2. Add the following contents to ``airflow_local_settings.py`` file. Change the colors to whatever you
+ would like.
+
+ .. code-block:: python
+
+ STATE_COLORS = {
+ "queued": 'darkgray',
+ "running": '#01FF70',
+ "success": '#2ECC40',
+ "failed": 'firebrick',
+ "up_for_retry": 'yellow',
+ "up_for_reschedule": 'turquoise',
+ "upstream_failed": 'orange',
+ "skipped": 'darkorchid',
+ "scheduled": 'tan',
+ }
+
+
+
+3. Restart Airflow Webserver.
+
+Screenshots
+-----------
+
+Before
+^^^^^^
+
+.. image:: ../img/change-ui-colors/dags-page-old.png
+
+.. image:: ../img/change-ui-colors/graph-view-old.png
+
+.. image:: ../img/change-ui-colors/tree-view-old.png
+
+After
+^^^^^^
+
+.. image:: ../img/change-ui-colors/dags-page-new.png
+
+.. image:: ../img/change-ui-colors/graph-view-new.png
+
+.. image:: ../img/change-ui-colors/tree-view-new.png
diff --git a/docs/howto/index.rst b/docs/howto/index.rst
index ae20c91..2b91c77 100644
--- a/docs/howto/index.rst
+++ b/docs/howto/index.rst
@@ -32,6 +32,7 @@ configuring an Airflow environment.
set-config
initialize-database
operator/index
+ customize-state-colors-ui
custom-operator
connection/index
secure-connections
diff --git a/docs/img/change-ui-colors/dags-page-new.png b/docs/img/change-ui-colors/dags-page-new.png
new file mode 100644
index 0000000..d2ffe1f
Binary files /dev/null and b/docs/img/change-ui-colors/dags-page-new.png differ
diff --git a/docs/img/change-ui-colors/dags-page-old.png b/docs/img/change-ui-colors/dags-page-old.png
new file mode 100644
index 0000000..5078d01
Binary files /dev/null and b/docs/img/change-ui-colors/dags-page-old.png differ
diff --git a/docs/img/change-ui-colors/graph-view-new.png b/docs/img/change-ui-colors/graph-view-new.png
new file mode 100644
index 0000000..b367461
Binary files /dev/null and b/docs/img/change-ui-colors/graph-view-new.png differ
diff --git a/docs/img/change-ui-colors/graph-view-old.png b/docs/img/change-ui-colors/graph-view-old.png
new file mode 100644
index 0000000..ceaf8d4
Binary files /dev/null and b/docs/img/change-ui-colors/graph-view-old.png differ
diff --git a/docs/img/change-ui-colors/tree-view-new.png b/docs/img/change-ui-colors/tree-view-new.png
new file mode 100644
index 0000000..6a5b2d7
Binary files /dev/null and b/docs/img/change-ui-colors/tree-view-new.png differ
diff --git a/docs/img/change-ui-colors/tree-view-old.png b/docs/img/change-ui-colors/tree-view-old.png
new file mode 100644
index 0000000..8dedde6
Binary files /dev/null and b/docs/img/change-ui-colors/tree-view-old.png differ
[airflow] 05/10: Restrict changing XCom values from the Webserver
(#9614)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 30325f8e87beb19148f29aa9d3aafc7439c9958a
Author: Kaxil Naik <ka...@gmail.com>
AuthorDate: Wed Jul 1 22:13:10 2020 +0100
Restrict changing XCom values from the Webserver (#9614)
(cherry-picked from 1655fa9253ba8f61ccda77780a9e94766c15f565)
---
UPDATING.md | 6 ++++++
airflow/www/views.py | 2 ++
airflow/www_rbac/views.py | 4 +---
3 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/UPDATING.md b/UPDATING.md
index ec193f9..61734bb 100644
--- a/UPDATING.md
+++ b/UPDATING.md
@@ -89,6 +89,12 @@ the previous behaviour on a new install by setting this in your airflow.cfg:
auth_backend = airflow.api.auth.backend.default
```
+### XCom Values can no longer be added or changed from the Webserver
+
+Since XCom values can contain pickled data, we would no longer allow adding or
+changing XCom values from the UI.
+
+
## Airflow 1.10.10
### Setting Empty string to a Airflow Variable will return an empty string
diff --git a/airflow/www/views.py b/airflow/www/views.py
index a3293c8..abd1b9e 100644
--- a/airflow/www/views.py
+++ b/airflow/www/views.py
@@ -2754,6 +2754,8 @@ class VariableView(wwwutils.DataProfilingMixin, AirflowModelView):
class XComView(wwwutils.SuperUserMixin, AirflowModelView):
+ can_create = False
+ can_edit = False
verbose_name = "XCom"
verbose_name_plural = "XComs"
diff --git a/airflow/www_rbac/views.py b/airflow/www_rbac/views.py
index 67a7493..96d4079 100644
--- a/airflow/www_rbac/views.py
+++ b/airflow/www_rbac/views.py
@@ -2233,12 +2233,10 @@ class XComModelView(AirflowModelView):
datamodel = AirflowModelView.CustomSQLAInterface(XCom)
- base_permissions = ['can_add', 'can_list', 'can_edit', 'can_delete']
+ base_permissions = ['can_list', 'can_delete']
search_columns = ['key', 'value', 'timestamp', 'execution_date', 'task_id', 'dag_id']
list_columns = ['key', 'value', 'timestamp', 'execution_date', 'task_id', 'dag_id']
- add_columns = ['key', 'value', 'execution_date', 'task_id', 'dag_id']
- edit_columns = ['key', 'value', 'execution_date', 'task_id', 'dag_id']
base_order = ('execution_date', 'desc')
base_filters = [['dag_id', DagFilter, lambda: []]]
[airflow] 08/10: Update version_added of configs added in 1.10.11
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 84fc6b482e4fe563f42f60914fe35cc744283715
Author: Kaxil Naik <ka...@gmail.com>
AuthorDate: Thu Jul 2 00:33:42 2020 +0100
Update version_added of configs added in 1.10.11
---
airflow/config_templates/config.yml | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/airflow/config_templates/config.yml b/airflow/config_templates/config.yml
index 0d52426..3dd0a58 100644
--- a/airflow/config_templates/config.yml
+++ b/airflow/config_templates/config.yml
@@ -705,7 +705,7 @@
description: |
If set to True, Airflow will track files in plugins_folder directory. When it detects changes,
then reload the gunicorn.
- version_added: ~
+ version_added: 1.10.11
type: boolean
example: ~
default: "False"
@@ -1749,7 +1749,7 @@
- name: pod_template_file
description: |
Path to the YAML pod file. If set, all other kubernetes-related fields are ignored.
- version_added: ~
+ version_added: 1.10.11
type: string
example: ~
default: ""
@@ -1776,7 +1776,7 @@
description: |
If False (and delete_worker_pods is True),
failed worker pods will not be deleted so users can investigate them.
- version_added: ~
+ version_added: 1.10.11
type: string
example: ~
default: "False"
@@ -1847,7 +1847,7 @@
- name: dags_volume_mount_point
description: |
For either git sync or volume mounted DAGs, the worker will mount the volume in this path
- version_added: ~
+ version_added: 1.10.11
type: string
example: ~
default: ""
[airflow] 01/10: Change default auth for experimental backend to
deny_all (#9611)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 92c5f46405497f4d9d9ce6d5cd3544b998be3714
Author: Ash Berlin-Taylor <as...@firemirror.com>
AuthorDate: Wed Jul 1 17:04:35 2020 +0100
Change default auth for experimental backend to deny_all (#9611)
In a move that should surprise no one, a number of users do not read,
and leave the API wide open by default. Safe is better than powned
(cherry picked from commit 9e305d6b810a2a21e2591a80a80ec41acb3afed0)
---
UPDATING.md | 16 ++++++++++++++++
airflow/config_templates/config.yml | 6 ++++--
airflow/config_templates/default_airflow.cfg | 6 ++++--
3 files changed, 24 insertions(+), 4 deletions(-)
diff --git a/UPDATING.md b/UPDATING.md
index 3dfda58..ec193f9 100644
--- a/UPDATING.md
+++ b/UPDATING.md
@@ -73,6 +73,22 @@ Before 1.10.11 it was possible to edit DagRun State in the `/admin/dagrun/` page
In Airflow 1.10.11+, the user can only choose the states from the list.
+### Experimental API will deny all request by default.
+
+The previous default setting was to allow all API requests without authentication, but this poses security
+risks to users who miss this fact. This changes the default for new installs to deny all requests by default.
+
+**Note**: This will not change the behavior for existing installs, please update check your airflow.cfg
+
+If you wish to have the experimental API work, and aware of the risks of enabling this without authentication
+(or if you have your own authentication layer in front of Airflow) you can get
+the previous behaviour on a new install by setting this in your airflow.cfg:
+
+```
+[api]
+auth_backend = airflow.api.auth.backend.default
+```
+
## Airflow 1.10.10
### Setting Empty string to a Airflow Variable will return an empty string
diff --git a/airflow/config_templates/config.yml b/airflow/config_templates/config.yml
index f632cd5..0d52426 100644
--- a/airflow/config_templates/config.yml
+++ b/airflow/config_templates/config.yml
@@ -524,11 +524,13 @@
options:
- name: auth_backend
description: |
- How to authenticate users of the API
+ How to authenticate users of the API. See
+ https://airflow.apache.org/docs/stable/security.html for possible values.
+ ("airflow.api.auth.backend.default" allows all requests for historic reasons)
version_added: ~
type: string
example: ~
- default: "airflow.api.auth.backend.default"
+ default: "airflow.api.auth.backend.deny_all"
- name: lineage
description: ~
options:
diff --git a/airflow/config_templates/default_airflow.cfg b/airflow/config_templates/default_airflow.cfg
index a061d46..63bd3cb 100644
--- a/airflow/config_templates/default_airflow.cfg
+++ b/airflow/config_templates/default_airflow.cfg
@@ -274,8 +274,10 @@ endpoint_url = http://localhost:8080
fail_fast = False
[api]
-# How to authenticate users of the API
-auth_backend = airflow.api.auth.backend.default
+# How to authenticate users of the API. See
+# https://airflow.apache.org/docs/stable/security.html for possible values.
+# ("airflow.api.auth.backend.default" allows all requests for historic reasons)
+auth_backend = airflow.api.auth.backend.deny_all
[lineage]
# what lineage backend to use
[airflow] 10/10: Add Changelog for 1.10.11
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 9a212d11eb4f96c38fac8d6bfc7c0ded0493fa42
Author: Kaxil Naik <ka...@gmail.com>
AuthorDate: Wed Jul 1 19:40:29 2020 +0100
Add Changelog for 1.10.11
---
.pre-commit-config.yaml | 3 +-
CHANGELOG.txt | 198 ++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 200 insertions(+), 1 deletion(-)
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 469c606..1b7d88a 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -245,7 +245,8 @@ repos:
(?x)
^airflow/contrib/hooks/cassandra_hook.py$|
^airflow/operators/hive_stats_operator.py$|
- ^tests/contrib/hooks/test_cassandra_hook.py
+ ^tests/contrib/hooks/test_cassandra_hook.py|
+ ^CHANGELOG.txt
- id: dont-use-safe-filter
language: pygrep
name: Don't use safe in templates
diff --git a/CHANGELOG.txt b/CHANGELOG.txt
index a8aa353..81be084 100644
--- a/CHANGELOG.txt
+++ b/CHANGELOG.txt
@@ -1,3 +1,201 @@
+Airflow 1.10.11, 2020-07-05
+-----------------------------
+
+New Features
+""""""""""""
+
+- Add task instance mutation hook (#8852)
+- Allow changing Task States Colors (#9520)
+- Add support for AWS Secrets Manager as Secrets Backend (#8186)
+- Add airflow info command to the CLI (#8704)
+- Add Local Filesystem Secret Backend (#8596)
+- Add Airflow config CLI command (#8694)
+- Add Support for Python 3.8 (#8836)(#8823)
+- Allow K8S worker pod to be configured from JSON/YAML file (#6230)
+- Add quarterly to crontab presets (#6873)
+- Add support for ephemeral storage on KubernetesPodOperator (#6337)
+- Add AirflowFailException to fail without any retry (#7133)
+- Add SQL Branch Operator (#8942)
+
+Bug Fixes
+"""""""""
+
+- Use NULL as dag.description default value (#7593)
+- BugFix: DAG trigger via UI error in RBAC UI (#8411)
+- Fix logging issue when running tasks (#9363)
+- Fix JSON encoding error in DockerOperator (#8287)
+- Fix alembic crash due to typing import (#6547)
+- Correctly restore upstream_task_ids when deserializing Operators (#8775)
+- Correctly store non-default Nones in serialized tasks/dags (#8772)
+- Correctly deserialize dagrun_timeout field on DAGs (#8735)
+- Fix tree view if config contains " (#9250)
+- Fix Dag Run UI execution date with timezone cannot be saved issue (#8902)
+- Fix Migration for MSSQL (#8385)
+- RBAC ui: Fix missing Y-axis labels with units in plots (#8252)
+- RBAC ui: Fix missing task runs being rendered as circles instead (#8253)
+- Fix: DagRuns page renders the state column with artifacts in old UI (#9612)
+- Fix task and dag stats on home page (#8865)
+- Fix the trigger_dag api in the case of nested subdags (#8081)
+- UX Fix: Prevent undesired text selection with DAG title selection in Chrome (#8912)
+- Fix connection add/edit for spark (#8685)
+- Fix retries causing constraint violation on MySQL with DAG Serialization (#9336)
+- [AIRFLOW-4472] Use json.dumps/loads for templating lineage data (#5253)
+- Restrict google-cloud-texttospeach to <v2 (#9137)
+- Fix pickling failure when spawning processes (#8671)
+- Pin Version of Azure Cosmos to <4 (#8956)
+- Azure storage 0.37.0 is not installable any more (#8833)
+- Fix modal_backdrop z-index in the UI (#7313)
+- Fix Extra Links in Gannt View (#8308)
+- Bug fix for EmrAddStepOperator init with cluster_name error (#9235)
+- Fix KubernetesPodOperator pod name length validation (#8829)
+- Fix non updating DAG code by checking against last modification time (#8266)
+- BugFix: Unpausing a DAG with catchup=False creates an extra DAG run (#8776)
+
+
+Improvements
+""""""""""""
+
+- Improve add_dag_code_table migration (#8176)
+- Persistent display/filtering of DAG status (#8106)
+- Set unique logger names (#7330)
+- Update the version of cattrs from 0.9 to 1.0 to support Python 3.8 (#7100)
+- Reduce response payload size of /dag_stats and /task_stats (#8655)
+- Add authenticator parameter to snowflake_hook (#8642)
+- Show "Task Reschedule" table in Airflow Webserver (#9521)
+- Change worker_refresh_interval fallback to default of 30 (#9588)
+- Use pfromat instead of str to render arguments in WebUI (#9587)
+- Simplify DagFileProcessorManager (#7521)
+- Reload gunicorn when plugins has beeen changed (#8997)
+- Fix the default value for store_dag_code (#9554)
+- Add support for fetching logs from running pods (#8626)
+- Persist start/end date and duration for DummyOperator Task Instance (#8663)
+- Ensure "started"/"ended" in tooltips are not shown if job not started (#8667)
+- Add context to execution_date_fn in ExternalTaskSensor (#8702)
+- Avoid color info in response of ``/dag_stats`` & ``/task_stats`` (#8742)
+- Make loading plugins from entrypoint fault-tolerant (#8732)
+- Refactor Kubernetes worker config (#7114)
+- Add default ``conf`` parameter to Spark JDBC Hook (#8787)
+- Allow passing backend_kwargs to AWS SSM client (#8802)
+- Filter dags by clicking on tag (#8897)
+- Support k8s auth method in Vault Secrets provider (#8640)
+- Monitor pods by labels instead of names (#6377)
+- Optimize count query on /home (#8729)
+- Fix json string escape in tree view (#8551)
+- Add TaskInstance state to TI Tooltip to be colour-blind friendlier (#8910)
+- Add a tip to trigger DAG screen (#9049)
+- Use Markup for htmlcontent for landing_times (#9242)
+- Pinning max pandas version to 2.0 (lesser than) to allow pandas 1.0 (#7954)
+- Update example webserver_config.py to show correct CSRF config (#8944)
+- Fix displaying Executor Class Name in "Base Job" table (#8679)
+- Use existing DagBag for 'dag_details' & 'trigger' Endpoints (#8501)
+- Flush pending Sentry exceptions before exiting (#7232)
+- Display DAG run conf in the list view (#6794)
+- Fix performance degradation when updating dagrun state (#8435)
+- Don't use the ``|safe`` filter in code, it's risky (#9180)
+- Validate only task commands are run by executors (#9178)
+- Show Deprecation warning on duplicate Task ids (#8728)
+- Move DAG._schedule_interval logic out of ``DAG.__init__`` (#8225)
+- Make retrieving Paused Dag ids a separate method (#7587)
+- Bulk fetch paused_dag_ids (#7476)
+- Add a configurable DAGs volume mount path for Kubernetes (#8147)
+- Add schedulername option for KubernetesPodOperator (#6088)
+- Support running git sync container as root (#6312)
+- Add extra options for Slack Webhook operator and Slack hook (#9409)
+- Monkey patch greenlet celery pools (#8559)
+- Decrypt secrets from SystemsManagerParameterStoreBackend (#9214)
+- Prevent clickable sorting on non sortable columns in TI view (#8681)
+- Make hive macros py3 compatible (#8598)
+- Fix SVG tooltip positioning with custom scripting (#8269)
+- Avoid unnecessary sleep to maintain local task job heart rate (#6553)
+- Include some missing RBAC roles on User and Viewer roles (#9133)
+- Show Dag's Markdown docs on Tree View (#9448)
+- Improved compatibility with Python 3.5+ - Convert signal.SIGTERM to int (#9207)
+- Add 'main' param to template_fields in DataprocSubmitPySparkJobOperator (#9154)
+- Make it possible to silence warnings from Airflow (#9208)
+- Remove redundant count query in BaseOperator.clear() (#9362)
+- Fix DB migration message (#8988)
+- Fix awkward log info in dbapi_hook (#8482)
+- Fix Celery default to no longer allow pickle (#7205)
+- Further validation that only task commands are run by executors (#9240)
+- Remove vendored nvd3 and slugify libraries (#9136)
+- Enable configurable git sync depth (#9094)
+- Reduce the required resources for the Kubernetes's sidecar (#6062)
+- Refactor K8S codebase with k8s API models (#5481)
+- Move k8sexecutor out of contrib to closer match master (#8904)
+- Allow filtering using "event" and "owner" in "Log" view (#4881)
+- Add Yandex.Cloud custom connection to 1.10 (#8791)
+- Add table-hover css class to DAGs table (#5033)
+- Show un/pause errors in dags view. (#7669)
+- Restructure database queries on /home (#4872)
+- Add Cross Site Scripting defence (#6913)
+- Make Gantt tooltip the same as Tree and Graph view (#8220)
+- Add config to only delete worker pod on task failure (#7507)(#8312)
+- Remove duplicate error message on chart connection failure (#8476)
+- Remove default value spark_binary (#8508)
+- Expose Airflow Webserver Port in Production Docker Image (#8228)
+- Commit after each alembic migration (#4797)
+- KubernetesPodOperator fixes and test (#6524)
+- Docker Image: Add ADDITIONAL_AIRFLOW_EXTRAS (#9032)
+- Docker Image: Add ADDITIONAL_PYTHON_DEPS (#9031)
+- Remove httplib2 from Google requirements (#9194)
+- Helm Chart: Remove importlib usage (#9613)
+- Merging multiple sql operators (#9124)
+- Adds hive as extra in pyhive dependency (#9075)
+- Change default auth for experimental backend to deny_all (#9611)
+- Restrict changing XCom values from the Webserver (#9614)
+- Add __repr__ for DagTag so tags display properly in /dagmodel/show (#8719)
+
+Doc only changes
+""""""""""""""""
+
+- Add docs on using DAGRun.conf (#9578)
+- Enforce code-block directives in doc (#9443)
+- Carefully parse warning messages when building documentation (#8693)
+- Make KubernetesPodOperator clear in docs (#8444)
+- Improve language in Pod Mutation Hook docs (#8445)
+- Fix formatting of Pool docs in concepts.rst (#8443)
+- Make doc clearer about Airflow Variables using Environment Variables (#8427)
+- Fix pools doc for LocalExecutor (#7643)
+- Add Local and Sequential Executors to Doc (#8084)
+- Add documentation for CLI command airflow dags test (#8251)
+- Fix typo in DAG Serialization documentation (#8317)
+- Add scheduler in production section (#7351)
+- Add a structural dag validation example (#6727)
+- Adding Task re-run documentation (#6295)
+- Fix outdated doc on settings.policy (#7532)
+- Add docs about reload_on_plugin_change opiton (#9575)
+- Add copy button to Code Blocks in Airflow Docs (#9450)
+- Update commands in docs for v1.10+ (#9585)
+- Add more info on dry-run CLI option (#9582)
+- Document default timeout value for SSHOperator (#8744)
+- Fix docs on creating CustomOperator (#8678)
+- Enhanced documentation around Cluster Policy (#8661)
+- Use sphinx syntax in concepts.rst (#7729)
+- Update README to remove Py 3.8 limitation for Master (#9451)
+- Add note about using dag_run.conf in BashOperator (#9143)
+- Improve tutorial - Include all imports statements (#8670)
+- Added more precise Python requirements to README.md (#8455)
+- Fix Airflow Stable version in README.md (#9360)
+- Update AWS connection example to show how to set from env var (#9191)
+- Fix list formatting of plugins doc. (#8873)
+- Add 'Version Added' on Secrets Backend docs (#8264)
+- Simplify language re roll-your-own secrets backend (#8257)
+- Add installation description for repeatable PyPi installation (#8513)
+- Add note extra links only render on when using RBAC webserver (#8788)
+- Remove unused airflow import from docs (#9274)
+- Add PR/issue note in Contribution Workflow Example (#9177)
+- Remove redundant airflowVersion from Helm Chart readme (#9592)
+- Fix typo of resultBackendConnection in chart README (#9537)
+- Remove non-existent chart value from readme (#9511)
+- Fix typo in helm chart upgrade command for 2.0 (#9484)
+- Don't use the term "whitelist" - language matters (#9174)
+- Add docs to change Colors on the Webserver (#9607)
+- Change 'initiate' to 'initialize' in installation.rst (#9619)
+- Replace old Variables View Screenshot with new (#9620)
+- Replace old SubDag zoom screenshot with new (#9621)
+- Update docs about the change to default auth for experimental API (#9617)
+
+
Airflow 1.10.10, 2020-04-09
---------------------------
[airflow] 09/10: Add __repr__ for DagTag so tags display properly
in /dagmodel/show (#8719)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit c99194a6d4bc4327061257aa249e9d50aebfaf1e
Author: Xiaodong DENG <xd...@hotmail.com>
AuthorDate: Tue May 5 20:32:18 2020 +0200
Add __repr__ for DagTag so tags display properly in /dagmodel/show (#8719)
(cherry picked from commit c717d12f47c604082afc106b7a4a1f71d91f73e2)
---
airflow/models/dag.py | 3 +++
tests/models/test_dag.py | 12 +++++++++++-
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/airflow/models/dag.py b/airflow/models/dag.py
index e24c164..94c6d2e 100644
--- a/airflow/models/dag.py
+++ b/airflow/models/dag.py
@@ -1711,6 +1711,9 @@ class DagTag(Base):
name = Column(String(100), primary_key=True)
dag_id = Column(String(ID_LEN), ForeignKey('dag.dag_id'), primary_key=True)
+ def __repr__(self):
+ return self.name
+
class DagModel(Base):
diff --git a/tests/models/test_dag.py b/tests/models/test_dag.py
index 5d9d05d..b711a8b 100644
--- a/tests/models/test_dag.py
+++ b/tests/models/test_dag.py
@@ -36,7 +36,7 @@ from mock import patch
from airflow import models, settings
from airflow.configuration import conf
from airflow.exceptions import AirflowException, AirflowDagCycleException
-from airflow.models import DAG, DagModel, TaskInstance as TI
+from airflow.models import DAG, DagModel, DagTag, TaskInstance as TI
from airflow.operators.bash_operator import BashOperator
from airflow.operators.dummy_operator import DummyOperator
from airflow.operators.subdag_operator import SubDagOperator
@@ -46,6 +46,7 @@ from airflow.utils.db import create_session
from airflow.utils.state import State
from airflow.utils.weight_rule import WeightRule
from tests.models import DEFAULT_DATE
+from tests.test_utils.db import clear_db_dags
class DagTest(unittest.TestCase):
@@ -657,6 +658,15 @@ class DagTest(unittest.TestCase):
self.assertEqual(prev_local.isoformat(), "2018-03-24T03:00:00+01:00")
self.assertEqual(prev.isoformat(), "2018-03-24T02:00:00+00:00")
+ def test_dagtag_repr(self):
+ clear_db_dags()
+ dag = DAG('dag-test-dagtag', start_date=DEFAULT_DATE, tags=['tag-1', 'tag-2'])
+ dag.sync_to_db()
+ with create_session() as session:
+ self.assertEqual({'tag-1', 'tag-2'},
+ {repr(t) for t in session.query(DagTag).filter(
+ DagTag.dag_id == 'dag-test-dagtag').all()})
+
@patch('airflow.models.dag.timezone.utcnow')
def test_sync_to_db(self, mock_now):
dag = DAG(
[airflow] 03/10: Change 'initiate' to 'initialize' in
installation.rst (#9619)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit e5662b96466408f5ed9472cee41e737a150471df
Author: Kaxil Naik <ka...@gmail.com>
AuthorDate: Wed Jul 1 23:34:25 2020 +0100
Change 'initiate' to 'initialize' in installation.rst (#9619)
`Initiating Airflow Database` -> `Initializing Airflow Database`
(cherry picked from commit bc3f48c96603dbd2a94a33f05dd816097ccab3f1)
---
docs/installation.rst | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/docs/installation.rst b/docs/installation.rst
index e7c8970..d5652cb 100644
--- a/docs/installation.rst
+++ b/docs/installation.rst
@@ -180,10 +180,10 @@ Here's the list of the subpackages and what they enable:
| vertica | ``pip install 'apache-airflow[vertica]'`` | Vertica hook support as an Airflow backend |
+---------------------+-----------------------------------------------------+----------------------------------------------------------------------+
-Initiating Airflow Database
-'''''''''''''''''''''''''''
+Initializing Airflow Database
+'''''''''''''''''''''''''''''
-Airflow requires a database to be initiated before you can run tasks. If
+Airflow requires a database to be initialized before you can run tasks. If
you're just experimenting and learning Airflow, you can stick with the
default SQLite option. If you don't want to use SQLite, then take a look at
:doc:`howto/initialize-database` to setup a different database.
[airflow] 07/10: Update docs about the change to default auth for
experimental API (#9617)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 0d520f28c6232d94d1aae6118641ae4a06fae44f
Author: Kaxil Naik <ka...@gmail.com>
AuthorDate: Wed Jul 1 22:59:13 2020 +0100
Update docs about the change to default auth for experimental API (#9617)
(cherry picked from commit 7ef7f5880dfefc6e33cb7bf331927aa08e1bb438)
---
docs/security.rst | 18 +++++++++++++++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/docs/security.rst b/docs/security.rst
index 863a454..3817c7f 100644
--- a/docs/security.rst
+++ b/docs/security.rst
@@ -159,15 +159,27 @@ only the dags which it is owner of, unless it is a superuser.
API Authentication
------------------
-Authentication for the API is handled separately to the Web Authentication. The default is to not
-require any authentication on the API i.e. wide open by default. This is not recommended if your
-Airflow webserver is publicly accessible, and you should probably use the ``deny all`` backend:
+Authentication for the API is handled separately to the Web Authentication. The default is to
+deny all requests:
.. code-block:: ini
[api]
auth_backend = airflow.api.auth.backend.deny_all
+.. versionchanged:: 1.10.11
+
+ In Airflow <1.10.11, the default setting was to allow all API requests without authentication, but this
+ posed security risks for if the Webserver is publicly accessible.
+
+If you wish to have the experimental API work, and aware of the risks of enabling this without authentication
+(or if you have your own authentication layer in front of Airflow) you can set the following in ``airflow.cfg``:
+
+.. code-block:: ini
+
+ [api]
+ auth_backend = airflow.api.auth.backend.default
+
Two "real" methods for authentication are currently supported for the API.
To enabled Password authentication, set the following in the configuration: