You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ka...@apache.org on 2020/06/15 17:20:21 UTC
[airflow] branch v1-10-test updated (09e5faa -> 8f552cc)
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a change to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git.
omit 09e5faa fixup! Don't use the `|safe` filter in code, it's risky (#9180)
omit b5e7914 [AIRFLOW-3973] Commit after each alembic migration (#4797)
omit 9c6ae4c Remove unused airflow import from docs (#9274)
omit 371345d fixup! Don't use the `|safe` filter in code, it's risky (#9180)
omit bf3c26b Don't use the `|safe` filter in code, it's risky (#9180)
new 60f631f Don't use the `|safe` filter in code, it's risky (#9180)
new 7b7c5b1 Remove unused airflow import from docs (#9274)
new 8f552cc [AIRFLOW-3973] Commit after each alembic migration (#4797)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (09e5faa)
\
N -- N -- N refs/heads/v1-10-test (8f552cc)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
[airflow] 02/03: Remove unused airflow import from docs (#9274)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 7b7c5b1d98224476342c8469abdcd798ef1eb16a
Author: Do Son Tung <do...@gmail.com>
AuthorDate: Sun Jun 14 07:21:37 2020 +0700
Remove unused airflow import from docs (#9274)
The bash code block which demonstrates the creating user import airflow but unused it. So that line is redundant
---
docs/security.rst | 1 -
1 file changed, 1 deletion(-)
diff --git a/docs/security.rst b/docs/security.rst
index c653e55..e6bced8 100644
--- a/docs/security.rst
+++ b/docs/security.rst
@@ -68,7 +68,6 @@ attack. Creating a new user has to be done via a Python REPL on the same machine
$ python
Python 2.7.9 (default, Feb 10 2015, 03:28:08)
Type "help", "copyright", "credits" or "license" for more information.
- >>> import airflow
>>> from airflow import models, settings
>>> from airflow.contrib.auth.backends.password_auth import PasswordUser
>>> user = PasswordUser(models.User())
[airflow] 01/03: Don't use the `|safe` filter in code,
it's risky (#9180)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 60f631f75dff1c0e4340cf43e520a88c5f2ab07e
Author: Ash Berlin-Taylor <as...@firemirror.com>
AuthorDate: Mon Jun 8 22:27:02 2020 +0100
Don't use the `|safe` filter in code, it's risky (#9180)
Most things already use the `Markup` class to correctly escape problem
areas, this commit just fixes the last instances so that we can assert
that `|safe` is never used.
(cherry-picked from 7fd3695766095a358803b79454bde26e5e28c1f0)
---
.pre-commit-config.yaml | 7 ++++
airflow/contrib/plugins/metastore_browser/main.py | 4 +--
.../templates/metastore_browser/dbs.html | 2 +-
airflow/www/static/main.css | 4 +++
airflow/www/templates/airflow/chart.html | 2 +-
airflow/www/templates/airflow/code.html | 2 +-
airflow/www/templates/airflow/config.html | 2 +-
airflow/www/templates/airflow/dag.html | 6 ++--
airflow/www/templates/airflow/dag_code.html | 2 +-
airflow/www/templates/airflow/duration_chart.html | 4 +--
airflow/www/templates/airflow/gantt.html | 4 +--
airflow/www/templates/airflow/graph.html | 14 ++++----
airflow/www/templates/airflow/list_dags.html | 4 +--
airflow/www/templates/airflow/query.html | 2 +-
airflow/www/templates/airflow/task.html | 8 +----
airflow/www/templates/airflow/task_instance.html | 2 +-
airflow/www/templates/airflow/ti_code.html | 2 +-
airflow/www/templates/airflow/tree.html | 2 +-
airflow/www/templates/airflow/xcom.html | 1 -
airflow/www/views.py | 38 ++++++++++++----------
airflow/www_rbac/static/css/main.css | 4 +++
airflow/www_rbac/templates/airflow/chart.html | 2 +-
airflow/www_rbac/templates/airflow/code.html | 2 +-
airflow/www_rbac/templates/airflow/config.html | 2 +-
airflow/www_rbac/templates/airflow/dag.html | 6 ++--
airflow/www_rbac/templates/airflow/dag_code.html | 2 +-
.../www_rbac/templates/airflow/duration_chart.html | 4 +--
airflow/www_rbac/templates/airflow/gantt.html | 6 ++--
airflow/www_rbac/templates/airflow/graph.html | 14 ++++----
airflow/www_rbac/templates/airflow/model_list.html | 2 +-
airflow/www_rbac/templates/airflow/task.html | 8 +----
.../www_rbac/templates/airflow/task_instance.html | 2 +-
airflow/www_rbac/templates/airflow/ti_code.html | 2 +-
airflow/www_rbac/templates/airflow/xcom.html | 1 -
airflow/www_rbac/utils.py | 19 +++++++----
airflow/www_rbac/views.py | 22 +++++--------
36 files changed, 108 insertions(+), 102 deletions(-)
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 7e49b77..88ab3cc 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -228,6 +228,13 @@ repos:
files: ^setup.py$|^INSTALL$|^CONTRIBUTING.rst$
pass_filenames: false
require_serial: true
+ - id: dont-use-safe-filter
+ language: pygrep
+ name: Don't use safe in templates
+ description: the Safe filter is error-prone, use Markup() in code instead
+ entry: "\\|\\s*safe"
+ files: \.html$
+ pass_filenames: true
- id: python2-fastcheck
language: pygrep
name: Find common Python 3 vs. 2.7 compatibility problems
diff --git a/airflow/contrib/plugins/metastore_browser/main.py b/airflow/contrib/plugins/metastore_browser/main.py
index b0643ac..6826030 100644
--- a/airflow/contrib/plugins/metastore_browser/main.py
+++ b/airflow/contrib/plugins/metastore_browser/main.py
@@ -20,7 +20,7 @@
from datetime import datetime
import json
-from flask import Blueprint, request
+from flask import Blueprint, Markup, request
from flask_admin import BaseView, expose
import pandas as pd
@@ -67,7 +67,7 @@ class MetastoreBrowserView(BaseView, wwwutils.DataProfilingMixin):
escape=False,
na_rep='',)
return self.render(
- "metastore_browser/dbs.html", table=table)
+ "metastore_browser/dbs.html", table=Markup(table))
@expose('/table/')
def table(self):
diff --git a/airflow/contrib/plugins/metastore_browser/templates/metastore_browser/dbs.html b/airflow/contrib/plugins/metastore_browser/templates/metastore_browser/dbs.html
index 9555a02..6a6e187 100644
--- a/airflow/contrib/plugins/metastore_browser/templates/metastore_browser/dbs.html
+++ b/airflow/contrib/plugins/metastore_browser/templates/metastore_browser/dbs.html
@@ -23,5 +23,5 @@
<h4>
<span>Hive Databases</span>
</h4>
- {{ table|safe }}
+ {{ table }}
{% endblock %}
diff --git a/airflow/www/static/main.css b/airflow/www/static/main.css
index 2740a0d..c5008b8 100644
--- a/airflow/www/static/main.css
+++ b/airflow/www/static/main.css
@@ -224,6 +224,10 @@ div.square {
box-shadow: inset 0 6px 6px rgba(0, 0, 0, 0.4);
}
+.dag-doc {
+ margin-bottom: 15px;
+}
+
.hll { background-color: #ffffcc }
.c { color: #408080; font-style: italic } /* Comment */
.err { border: 1px solid #FF0000 } /* Error */
diff --git a/airflow/www/templates/airflow/chart.html b/airflow/www/templates/airflow/chart.html
index 1fc3779..128364c 100644
--- a/airflow/www/templates/airflow/chart.html
+++ b/airflow/www/templates/airflow/chart.html
@@ -46,7 +46,7 @@
<input name="_csrf_token" type="hidden" value="{{ csrf_token() }}">
</form>
</div>
-<div style="clear: both;">{{ chart |safe }}</div>
+<div style="clear: both;">{{ chart }}</div>
<hr/>
{% endblock %}
diff --git a/airflow/www/templates/airflow/code.html b/airflow/www/templates/airflow/code.html
index 522c8f4..a91975f 100644
--- a/airflow/www/templates/airflow/code.html
+++ b/airflow/www/templates/airflow/code.html
@@ -40,6 +40,6 @@
{% endif %}
{% if code_html %}
- {{ code_html|safe }}
+ {{ code_html }}
{% endif %}
{% endblock %}
diff --git a/airflow/www/templates/airflow/config.html b/airflow/www/templates/airflow/config.html
index 0aedb17..f7786be 100644
--- a/airflow/www/templates/airflow/config.html
+++ b/airflow/www/templates/airflow/config.html
@@ -36,7 +36,7 @@
{% endif %}
{% if code_html %}
- {{ code_html|safe }}
+ {{ code_html }}
{% endif %}
<hr>
diff --git a/airflow/www/templates/airflow/dag.html b/airflow/www/templates/airflow/dag.html
index 89d3ce0..67e32a3 100644
--- a/airflow/www/templates/airflow/dag.html
+++ b/airflow/www/templates/airflow/dag.html
@@ -369,8 +369,8 @@ function updateQueryStringParameter(uri, key, value) {
});
subdag_id = sd;
execution_date = d;
- $('#task_id').html(t);
- $('#execution_date').html(d);
+ $('#task_id').text(t);
+ $('#execution_date').text(d);
$('#myModal').modal({});
$("#myModal").css("margin-top","0px")
if (subdag_id === undefined)
@@ -428,7 +428,7 @@ function updateQueryStringParameter(uri, key, value) {
function call_modal_dag(dag) {
id = dag && dag.id;
execution_date = dag && dag.execution_date;
- $('#dag_id').html(dag_id);
+ $('#dag_id').text(dag_id);
$('#dagModal').modal({});
$("#dagModal").css("margin-top","0px");
}
diff --git a/airflow/www/templates/airflow/dag_code.html b/airflow/www/templates/airflow/dag_code.html
index 05a5553..2649f72 100644
--- a/airflow/www/templates/airflow/dag_code.html
+++ b/airflow/www/templates/airflow/dag_code.html
@@ -26,7 +26,7 @@
<div class="active">
<a onclick="toggleWrap()">Toggle wrap</a>
</div>
- {{ html_code|safe }}
+ {{ html_code }}
{% endblock %}
{% block tail %}
diff --git a/airflow/www/templates/airflow/duration_chart.html b/airflow/www/templates/airflow/duration_chart.html
index 392dc17..ff5d8ab 100644
--- a/airflow/www/templates/airflow/duration_chart.html
+++ b/airflow/www/templates/airflow/duration_chart.html
@@ -50,8 +50,8 @@
<input name="_csrf_token" type="hidden" value="{{ csrf_token() }}">
</form>
</div>
-<div id="dur_chart" style="clear: both;">{{ chart |safe }}</div>
-<div id="cum_dur_chart" style="clear: both;">{{ cum_chart | safe}}</div>
+<div id="dur_chart" style="clear: both;">{{ chart }}</div>
+<div id="cum_dur_chart" style="clear: both;">{{ cum_chart }}</div>
<hr/>
{% endblock %}
diff --git a/airflow/www/templates/airflow/gantt.html b/airflow/www/templates/airflow/gantt.html
index a76b3851..f2bea89 100644
--- a/airflow/www/templates/airflow/gantt.html
+++ b/airflow/www/templates/airflow/gantt.html
@@ -33,7 +33,7 @@
Base date: {{ form.base_date(class_="form-control") }}
Number of runs: {{ form.num_runs(class_="form-control") }}
Run:<input type="hidden" value="{{ dag.dag_id }}" name="dag_id">
- {{ form.execution_date(class_="form-control") | safe }}
+ {{ form.execution_date(class_="form-control") }}
<input type="submit" value="Go" class="btn btn-default" action="" method="get">
<input type="hidden" name="root" value="{{ root if root else '' }}">
<input name="_csrf_token" type="hidden" value="{{ csrf_token() }}">
@@ -57,7 +57,7 @@
var dag_id = '{{ dag.dag_id }}';
var task_id = '';
var execution_date = '';
- data = {{ data |tojson|safe }};
+ data = {{ data |tojson }};
var gantt = d3.gantt()
.taskTypes(data.taskNames)
.taskStatus(data.taskStatus)
diff --git a/airflow/www/templates/airflow/graph.html b/airflow/www/templates/airflow/graph.html
index 0535bf8..dcbe4f9 100644
--- a/airflow/www/templates/airflow/graph.html
+++ b/airflow/www/templates/airflow/graph.html
@@ -32,7 +32,7 @@
{% block body %}
{{ super() }}
{% if doc_md %}
-<div class="rich_doc" style="margin-bottom: 15px;">{{ doc_md|safe }}</div>
+{{ doc_md }}
{% endif %}
<div class="form-inline">
<form method="get" style="float:left;">
@@ -40,9 +40,9 @@
Base date: {{ form.base_date(class_="form-control") }}
Number of runs: {{ form.num_runs(class_="form-control") }}
Run:
- {{ form.execution_date(class_="form-control") | safe }}
+ {{ form.execution_date(class_="form-control") }}
Layout:
- {{ form.arrange(class_="form-control") | safe }}
+ {{ form.arrange(class_="form-control") }}
<input type="hidden" name="root" value="{{ root }}">
<input type="hidden" value="{{ dag.dag_id }}" name="dag_id">
<input name="_csrf_token" type="hidden" value="{{ csrf_token() }}">
@@ -107,10 +107,10 @@
var upstream_color = "#2020A0";
var downstream_color = "#0000FF";
- var nodes = {{ nodes|tojson|safe }};
- var edges = {{ edges|tojson|safe }};
- var tasks = {{ tasks|tojson|safe }};
- var task_instances = {{ task_instances|tojson|safe }};
+ var nodes = {{ nodes|tojson }};
+ var edges = {{ edges|tojson }};
+ var tasks = {{ tasks|tojson }};
+ var task_instances = {{ task_instances|tojson }};
var execution_date = "{{ execution_date }}";
var arrange = "{{ arrange }}";
var g = dagreD3.json.decode(nodes, edges);
diff --git a/airflow/www/templates/airflow/list_dags.html b/airflow/www/templates/airflow/list_dags.html
index 3e9fe29..e9398eb 100644
--- a/airflow/www/templates/airflow/list_dags.html
+++ b/airflow/www/templates/airflow/list_dags.html
@@ -216,8 +216,8 @@
{% if filter_groups %}
var filter = new AdminFilters(
'#filter_form', '.field-filters',
- {{ filter_groups|tojson|safe }},
- {{ active_filters|tojson|safe }}
+ {{ filter_groups|tojson }},
+ {{ active_filters|tojson }}
);
{% endif %}
})(jQuery);
diff --git a/airflow/www/templates/airflow/query.html b/airflow/www/templates/airflow/query.html
index 6a41e7b..b5c4546 100644
--- a/airflow/www/templates/airflow/query.html
+++ b/airflow/www/templates/airflow/query.html
@@ -43,7 +43,7 @@
</div>
</div>
</form>
- {{ results|safe }}
+ {{ results }}
{% endblock %}
{% block tail %}
{{ super() }}
diff --git a/airflow/www/templates/airflow/task.html b/airflow/www/templates/airflow/task.html
index 2fe0b0e..fb53ca7 100644
--- a/airflow/www/templates/airflow/task.html
+++ b/airflow/www/templates/airflow/task.html
@@ -39,14 +39,11 @@
</tr>
{% endfor %}
</table>
- {% if html_code is defined %}
- {{ html_code|safe }}
- {% endif %}
</div>
<div>
{% for attr, value in special_attrs_rendered.items() %}
<h5>Attribute: {{ attr }}</h5>
- {{ value|safe }}
+ {{ value }}
{% endfor %}
<h5>Task Instance Attributes</h5>
<table class="table table-striped table-bordered">
@@ -74,8 +71,5 @@
</tr>
{% endfor %}
</table>
- {% if html_code is defined %}
- {{ html_code|safe }}
- {% endif %}
</div>
{% endblock %}
diff --git a/airflow/www/templates/airflow/task_instance.html b/airflow/www/templates/airflow/task_instance.html
index 2380a4b..e1e4819 100644
--- a/airflow/www/templates/airflow/task_instance.html
+++ b/airflow/www/templates/airflow/task_instance.html
@@ -35,7 +35,7 @@
{{ task_id }}
<input type="hidden" value="{{ dag.dag_id }}" name="dag_id">
</span>
- {{ form.execution_date(class_="form-control") | safe }}
+ {{ form.execution_date(class_="form-control") }}
</div>
</form>
diff --git a/airflow/www/templates/airflow/ti_code.html b/airflow/www/templates/airflow/ti_code.html
index 7531a01..5ca5f59 100644
--- a/airflow/www/templates/airflow/ti_code.html
+++ b/airflow/www/templates/airflow/ti_code.html
@@ -25,6 +25,6 @@
<h4>{{ title }}</h4>
{% for k, v in html_dict.items() %}
<h5>{{ k }}</h5>
- {{ v|safe }}
+ {{ v }}
{% endfor %}
{% endblock %}
diff --git a/airflow/www/templates/airflow/tree.html b/airflow/www/templates/airflow/tree.html
index 6bfb465..5804efc 100644
--- a/airflow/www/templates/airflow/tree.html
+++ b/airflow/www/templates/airflow/tree.html
@@ -84,7 +84,7 @@
$('span.status_square').tooltip({html: true});
var devicePixelRatio = window.devicePixelRatio || 1;
-var data = {{ data|tojson|safe }};
+var data = {{ data|tojson }};
var barHeight = 20;
var axisHeight = 40;
var square_x = parseInt(500 * devicePixelRatio);
diff --git a/airflow/www/templates/airflow/xcom.html b/airflow/www/templates/airflow/xcom.html
index b043e95..a79467b 100644
--- a/airflow/www/templates/airflow/xcom.html
+++ b/airflow/www/templates/airflow/xcom.html
@@ -36,6 +36,5 @@
</tr>
{% endfor %}
</table>
- {{ html_code|safe }}
</div>
{% endblock %}
diff --git a/airflow/www/views.py b/airflow/www/views.py
index a069f17..e98096d 100644
--- a/airflow/www/views.py
+++ b/airflow/www/views.py
@@ -236,20 +236,25 @@ def pygment_html_render(s, lexer=lexers.TextLexer):
def render(obj, lexer):
out = ""
if isinstance(obj, basestring):
- out += pygment_html_render(obj, lexer)
+ out += Markup(pygment_html_render(obj, lexer))
elif isinstance(obj, (tuple, list)):
for i, s in enumerate(obj):
- out += "<div>List item #{}</div>".format(i)
- out += "<div>" + pygment_html_render(s, lexer) + "</div>"
+ out += Markup("<div>List item #{}</div>".format(i))
+ out += Markup("<div>" + pygment_html_render(s, lexer) + "</div>")
elif isinstance(obj, dict):
for k, v in obj.items():
- out += '<div>Dict item "{}"</div>'.format(k)
- out += "<div>" + pygment_html_render(v, lexer) + "</div>"
+ out += Markup('<div>Dict item "{}"</div>'.format(k))
+ out += Markup("<div>" + pygment_html_render(v, lexer) + "</div>")
return out
-def wrapped_markdown(s):
- return '<div class="rich_doc">' + markdown.markdown(s) + "</div>"
+def wrapped_markdown(s, css_class=None):
+ if s is None:
+ return None
+
+ return Markup(
+ '<div class="rich_doc {css_class}" >' + markdown.markdown(s) + "</div>"
+ ).format(css_class=css_class)
attr_renderer = {
@@ -694,15 +699,15 @@ class Airflow(AirflowViewMixin, BaseView):
dag_id = request.args.get('dag_id')
dag_orm = models.DagModel.get_dagmodel(dag_id, session=session)
code = DagCode.get_code_by_fileloc(dag_orm.fileloc)
- html_code = highlight(
- code, lexers.PythonLexer(), HtmlFormatter(linenos=True))
+ html_code = Markup(highlight(
+ code, lexers.PythonLexer(), HtmlFormatter(linenos=True)))
except Exception as e:
all_errors += (
"Exception encountered during " +
"dag_id retrieval/dag retrieval fallback/code highlighting:\n\n{}\n".format(e)
)
- html_code = '<p>Failed to load file.</p><p>Details: {}</p>'.format(
+ html_code = Markup('<p>Failed to load file.</p><p>Details: {}</p>').format(
escape(all_errors))
return self.render(
@@ -819,8 +824,7 @@ class Airflow(AirflowViewMixin, BaseView):
if template_field in attr_renderer:
html_dict[template_field] = attr_renderer[template_field](content)
else:
- html_dict[template_field] = (
- "<pre><code>" + str(content) + "</pre></code>")
+ html_dict[template_field] = Markup("<pre><code>{}</pre></code>").format(str(content))
return self.render(
'airflow/ti_code.html',
@@ -1741,7 +1745,7 @@ class Airflow(AirflowViewMixin, BaseView):
if not tasks:
flash("No tasks found", "error")
session.commit()
- doc_md = markdown.markdown(dag.doc_md) if hasattr(dag, 'doc_md') and dag.doc_md else ''
+ doc_md = wrapped_markdown(getattr(dag, 'doc_md', None), css_class='dag-doc')
external_logs = conf.get('elasticsearch', 'frontend')
return self.render(
@@ -1873,8 +1877,8 @@ class Airflow(AirflowViewMixin, BaseView):
demo_mode=conf.getboolean('webserver', 'demo_mode'),
root=root,
form=form,
- chart=chart.htmlcontent,
- cum_chart=cum_chart.htmlcontent
+ chart=Markup(chart.htmlcontent),
+ cum_chart=Markup(cum_chart.htmlcontent)
)
@expose('/tries')
@@ -1940,7 +1944,7 @@ class Airflow(AirflowViewMixin, BaseView):
demo_mode=conf.getboolean('webserver', 'demo_mode'),
root=root,
form=form,
- chart=chart.htmlcontent
+ chart=Markup(chart.htmlcontent)
)
@expose('/landing_times')
@@ -2396,7 +2400,7 @@ class QueryView(wwwutils.DataProfilingMixin, AirflowViewMixin, BaseView):
return self.render(
'airflow/query.html', form=form,
title="Ad Hoc Query",
- results=results or '',
+ results=Markup(results or ''),
has_data=has_data)
diff --git a/airflow/www_rbac/static/css/main.css b/airflow/www_rbac/static/css/main.css
index f5636d9..8b2324b 100644
--- a/airflow/www_rbac/static/css/main.css
+++ b/airflow/www_rbac/static/css/main.css
@@ -291,6 +291,10 @@ label[for="timezone-other"],
display: block;
}
+.dag-doc {
+ margin-bottom: 15px;
+}
+
.hll { background-color: #ffc; }
.c { color: #408080; font-style: italic; } /* Comment */
.err { border: 1px solid #f00; } /* Error */
diff --git a/airflow/www_rbac/templates/airflow/chart.html b/airflow/www_rbac/templates/airflow/chart.html
index 07865c8..5e2a29f 100644
--- a/airflow/www_rbac/templates/airflow/chart.html
+++ b/airflow/www_rbac/templates/airflow/chart.html
@@ -42,7 +42,7 @@
<input name="_csrf_token" type="hidden" value="{{ csrf_token() }}">
</form>
</div>
-<div style="clear: both;">{{ chart |safe }}</div>
+<div style="clear: both;">{{ chart }}</div>
<hr/>
{% endblock %}
diff --git a/airflow/www_rbac/templates/airflow/code.html b/airflow/www_rbac/templates/airflow/code.html
index adbcfbf..725f642 100644
--- a/airflow/www_rbac/templates/airflow/code.html
+++ b/airflow/www_rbac/templates/airflow/code.html
@@ -39,6 +39,6 @@
{% endif %}
{% if code_html %}
- {{ code_html|safe }}
+ {{ code_html }}
{% endif %}
{% endblock %}
diff --git a/airflow/www_rbac/templates/airflow/config.html b/airflow/www_rbac/templates/airflow/config.html
index f617b68..3ea81c4 100644
--- a/airflow/www_rbac/templates/airflow/config.html
+++ b/airflow/www_rbac/templates/airflow/config.html
@@ -35,7 +35,7 @@
{% endif %}
{% if code_html %}
- {{ code_html|safe }}
+ {{ code_html }}
{% endif %}
<hr>
diff --git a/airflow/www_rbac/templates/airflow/dag.html b/airflow/www_rbac/templates/airflow/dag.html
index dda7cf9..b5f4d00 100644
--- a/airflow/www_rbac/templates/airflow/dag.html
+++ b/airflow/www_rbac/templates/airflow/dag.html
@@ -376,8 +376,8 @@ function updateQueryStringParameter(uri, key, value) {
});
subdag_id = sd;
execution_date = d;
- $('#task_id').html(t);
- $('#execution_date').html(d);
+ $('#task_id').text(t);
+ $('#execution_date').text(d);
$('#myModal').modal({});
$("#myModal").css("margin-top","0px");
$('#extra_links').prev('hr').hide();
@@ -474,7 +474,7 @@ function updateQueryStringParameter(uri, key, value) {
function call_modal_dag(dag) {
id = dag && dag.id;
execution_date = dag && dag.execution_date;
- $('#dag_id').html(dag_id);
+ $('#dag_id').text(dag_id);
$('#dagModal').modal({});
$("#dagModal").css("margin-top","0px");
}
diff --git a/airflow/www_rbac/templates/airflow/dag_code.html b/airflow/www_rbac/templates/airflow/dag_code.html
index 2650efc..92a74aa 100644
--- a/airflow/www_rbac/templates/airflow/dag_code.html
+++ b/airflow/www_rbac/templates/airflow/dag_code.html
@@ -24,7 +24,7 @@
<div class="active">
<a onclick="toggleWrap()">Toggle wrap</a>
</div>
- {{ html_code|safe }}
+ {{ html_code }}
{% endblock %}
{% block tail %}
diff --git a/airflow/www_rbac/templates/airflow/duration_chart.html b/airflow/www_rbac/templates/airflow/duration_chart.html
index 4955f11..b1b9fa0 100644
--- a/airflow/www_rbac/templates/airflow/duration_chart.html
+++ b/airflow/www_rbac/templates/airflow/duration_chart.html
@@ -46,8 +46,8 @@
<input name="_csrf_token" type="hidden" value="{{ csrf_token() }}">
</form>
</div>
-<div id="dur_chart" style="clear: both;">{{ chart |safe }}</div>
-<div id="cum_dur_chart" style="clear: both;">{{ cum_chart | safe}}</div>
+<div id="dur_chart" style="clear: both;">{{ chart }}</div>
+<div id="cum_dur_chart" style="clear: both;">{{ cum_chart }}</div>
<hr/>
{% endblock %}
diff --git a/airflow/www_rbac/templates/airflow/gantt.html b/airflow/www_rbac/templates/airflow/gantt.html
index 6e7fa9f..7a5e954 100644
--- a/airflow/www_rbac/templates/airflow/gantt.html
+++ b/airflow/www_rbac/templates/airflow/gantt.html
@@ -30,7 +30,7 @@
Base date: {{ form.base_date(class_="form-control") }}
Number of runs: {{ form.num_runs(class_="form-control") }}
Run:<input type="hidden" value="{{ dag.dag_id }}" name="dag_id">
- {{ form.execution_date(class_="form-control") | safe }}
+ {{ form.execution_date(class_="form-control") }}
<input type="submit" value="Go" class="btn btn-default" action="" method="get">
<input type="hidden" name="root" value="{{ root if root else '' }}">
<input name="_csrf_token" type="hidden" value="{{ csrf_token() }}">
@@ -52,8 +52,8 @@
$( document ).ready(function() {
var dag_id = '{{ dag.dag_id }}';
var task_id = '';
- var exection_date = '';
- data = {{ data |tojson|safe }};
+ var execution_date = '';
+ data = {{ data |tojson }};
var gantt = d3.gantt()
.taskTypes(data.taskNames)
.height(data.height)
diff --git a/airflow/www_rbac/templates/airflow/graph.html b/airflow/www_rbac/templates/airflow/graph.html
index 9585b67..e2ab0f1 100644
--- a/airflow/www_rbac/templates/airflow/graph.html
+++ b/airflow/www_rbac/templates/airflow/graph.html
@@ -27,7 +27,7 @@
{% block content %}
{{ super() }}
{% if doc_md %}
-<div class="rich_doc" style="margin-bottom: 15px;">{{ doc_md|safe }}</div>
+{{ doc_md }}
{% endif %}
<div class="form-inline">
<form method="get" style="float:left;">
@@ -35,9 +35,9 @@
Base date: {{ form.base_date(class_="form-control") }}
Number of runs: {{ form.num_runs(class_="form-control") }}
Run:
- {{ form.execution_date(class_="form-control") | safe }}
+ {{ form.execution_date(class_="form-control") }}
Layout:
- {{ form.arrange(class_="form-control") | safe }}
+ {{ form.arrange(class_="form-control") }}
<input type="hidden" name="root" value="{{ root }}">
<input type="hidden" value="{{ dag.dag_id }}" name="dag_id">
<input name="_csrf_token" type="hidden" value="{{ csrf_token() }}">
@@ -104,14 +104,14 @@
var initialStrokeWidth = '3px';
var highlightStrokeWidth = '5px';
- var nodes = {{ nodes|tojson|safe }};
- var edges = {{ edges|tojson|safe }};
+ var nodes = {{ nodes|tojson }};
+ var edges = {{ edges|tojson }};
var execution_date = "{{ execution_date }}";
var arrange = "{{ arrange }}";
// Below variables are being used in dag.js
- var tasks = {{ tasks|tojson|safe }};
- var task_instances = {{ task_instances|tojson|safe }};
+ var tasks = {{ tasks|tojson }};
+ var task_instances = {{ task_instances|tojson }};
var getTaskInstanceURL = "{{ url_for('Airflow.task_instances') }}" +
"?dag_id=" + encodeURIComponent(dag_id) + "&execution_date=" +
encodeURIComponent(execution_date);
diff --git a/airflow/www_rbac/templates/airflow/model_list.html b/airflow/www_rbac/templates/airflow/model_list.html
index aba6e52..dc5bec0 100644
--- a/airflow/www_rbac/templates/airflow/model_list.html
+++ b/airflow/www_rbac/templates/airflow/model_list.html
@@ -78,7 +78,7 @@
{% if formatter and formatter(item) %}
<td>{{ formatter(item) }}</td>
{% elif item[value] != None %}
- <td>{{ item[value]|safe }}</td>
+ <td>{{ item[value] }}</td>
{% else %}
<td></td>
{% endif %}
diff --git a/airflow/www_rbac/templates/airflow/task.html b/airflow/www_rbac/templates/airflow/task.html
index 02ad445..5278a74 100644
--- a/airflow/www_rbac/templates/airflow/task.html
+++ b/airflow/www_rbac/templates/airflow/task.html
@@ -37,14 +37,11 @@
</tr>
{% endfor %}
</table>
- {% if html_code is defined %}
- {{ html_code|safe }}
- {% endif %}
</div>
<div>
{% for attr, value in special_attrs_rendered.items() %}
<h5>Attribute: {{ attr }}</h5>
- {{ value|safe }}
+ {{ value }}
{% endfor %}
<h5>Task Instance Attributes</h5>
<table class="table table-striped table-bordered">
@@ -72,8 +69,5 @@
</tr>
{% endfor %}
</table>
- {% if html_code is defined %}
- {{ html_code|safe }}
- {% endif %}
</div>
{% endblock %}
diff --git a/airflow/www_rbac/templates/airflow/task_instance.html b/airflow/www_rbac/templates/airflow/task_instance.html
index b424c65..ec43800 100644
--- a/airflow/www_rbac/templates/airflow/task_instance.html
+++ b/airflow/www_rbac/templates/airflow/task_instance.html
@@ -32,7 +32,7 @@
{{ task_id }}
<input type="hidden" value="{{ task_id }}" name="task_id">
</span>
- {{ form.execution_date(class_="form-control") | safe }}
+ {{ form.execution_date(class_="form-control") }}
</div>
</form>
diff --git a/airflow/www_rbac/templates/airflow/ti_code.html b/airflow/www_rbac/templates/airflow/ti_code.html
index 32a0739..05f5472 100644
--- a/airflow/www_rbac/templates/airflow/ti_code.html
+++ b/airflow/www_rbac/templates/airflow/ti_code.html
@@ -23,6 +23,6 @@
<h4>{{ title }}</h4>
{% for k, v in html_dict.items() %}
<h5>{{ k }}</h5>
- {{ v|safe }}
+ {{ v }}
{% endfor %}
{% endblock %}
diff --git a/airflow/www_rbac/templates/airflow/xcom.html b/airflow/www_rbac/templates/airflow/xcom.html
index 16174b1..ac99c25 100644
--- a/airflow/www_rbac/templates/airflow/xcom.html
+++ b/airflow/www_rbac/templates/airflow/xcom.html
@@ -37,6 +37,5 @@
</tr>
{% endfor %}
</table>
- {{ html_code|safe }}
</div>
{% endblock %}
diff --git a/airflow/www_rbac/utils.py b/airflow/www_rbac/utils.py
index c2c551b..ffd79b1 100644
--- a/airflow/www_rbac/utils.py
+++ b/airflow/www_rbac/utils.py
@@ -300,20 +300,25 @@ def pygment_html_render(s, lexer=lexers.TextLexer):
def render(obj, lexer):
out = ""
if isinstance(obj, basestring):
- out += pygment_html_render(obj, lexer)
+ out += Markup(pygment_html_render(obj, lexer))
elif isinstance(obj, (tuple, list)):
for i, s in enumerate(obj):
- out += "<div>List item #{}</div>".format(i)
- out += "<div>" + pygment_html_render(s, lexer) + "</div>"
+ out += Markup("<div>List item #{}</div>").format(i)
+ out += Markup("<div>" + pygment_html_render(s, lexer) + "</div>")
elif isinstance(obj, dict):
for k, v in obj.items():
- out += '<div>Dict item "{}"</div>'.format(k)
- out += "<div>" + pygment_html_render(v, lexer) + "</div>"
+ out += Markup('<div>Dict item "{}"</div>').format(k)
+ out += Markup("<div>" + pygment_html_render(v, lexer) + "</div>")
return out
-def wrapped_markdown(s):
- return '<div class="rich_doc">' + markdown.markdown(s) + "</div>"
+def wrapped_markdown(s, css_class=None):
+ if s is None:
+ return None
+
+ return Markup(
+ '<div class="rich_doc {css_class}" >' + markdown.markdown(s) + "</div>"
+ ).format(css_class=css_class)
def get_attr_renderer():
diff --git a/airflow/www_rbac/views.py b/airflow/www_rbac/views.py
index ef2b751..f9f7604 100644
--- a/airflow/www_rbac/views.py
+++ b/airflow/www_rbac/views.py
@@ -33,7 +33,6 @@ from urllib.parse import unquote
import six
from six.moves.urllib.parse import quote
-import markdown
import pendulum
import sqlalchemy as sqla
from flask import (
@@ -540,15 +539,15 @@ class Airflow(AirflowBaseView):
dag_id = request.args.get('dag_id')
dag_orm = DagModel.get_dagmodel(dag_id, session=session)
code = DagCode.get_code_by_fileloc(dag_orm.fileloc)
- html_code = highlight(
- code, lexers.PythonLexer(), HtmlFormatter(linenos=True))
+ html_code = Markup(highlight(
+ code, lexers.PythonLexer(), HtmlFormatter(linenos=True)))
except Exception as e:
all_errors += (
"Exception encountered during " +
"dag_id retrieval/dag retrieval fallback/code highlighting:\n\n{}\n".format(e)
)
- html_code = '<p>Failed to load file.</p><p>Details: {}</p>'.format(
+ html_code = Markup('<p>Failed to load file.</p><p>Details: {}</p>').format(
escape(all_errors))
return self.render_template(
@@ -635,11 +634,9 @@ class Airflow(AirflowBaseView):
for template_field in task.template_fields:
content = getattr(task, template_field)
if template_field in wwwutils.get_attr_renderer():
- html_dict[template_field] = \
- wwwutils.get_attr_renderer()[template_field](content)
+ html_dict[template_field] = wwwutils.get_attr_renderer()[template_field](content)
else:
- html_dict[template_field] = (
- "<pre><code>" + str(content) + "</pre></code>")
+ html_dict[template_field] = Markup("<pre><code>{}</pre></code>").format(str(content))
return self.render_template(
'airflow/ti_code.html',
@@ -1618,8 +1615,7 @@ class Airflow(AirflowBaseView):
if not tasks:
flash("No tasks found", "error")
session.commit()
- doc_md = markdown.markdown(dag.doc_md) \
- if hasattr(dag, 'doc_md') and dag.doc_md else ''
+ doc_md = wwwutils.wrapped_markdown(getattr(dag, 'doc_md', None), css_class='dag-doc')
external_logs = conf.get('elasticsearch', 'frontend')
return self.render_template(
@@ -1751,8 +1747,8 @@ class Airflow(AirflowBaseView):
demo_mode=conf.getboolean('webserver', 'demo_mode'),
root=root,
form=form,
- chart=chart.htmlcontent,
- cum_chart=cum_chart.htmlcontent
+ chart=Markup(chart.htmlcontent),
+ cum_chart=Markup(cum_chart.htmlcontent),
)
@expose('/tries')
@@ -1819,7 +1815,7 @@ class Airflow(AirflowBaseView):
demo_mode=conf.getboolean('webserver', 'demo_mode'),
root=root,
form=form,
- chart=chart.htmlcontent
+ chart=Markup(chart.htmlcontent),
)
@expose('/landing_times')
[airflow] 03/03: [AIRFLOW-3973] Commit after each alembic migration
(#4797)
Posted by ka...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kaxilnaik pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 8f552cc09a54989f6212c735d2afca1a76431576
Author: Elliott Shugerman <es...@medianewsgroup.com>
AuthorDate: Tue Jun 9 14:12:47 2020 -0600
[AIRFLOW-3973] Commit after each alembic migration (#4797)
If `Variable`s are used in DAGs, and Postgres is used for the internal
database, a fresh `$ airflow initdb` (or `$ airflow resetdb`) spams the
logs with error messages (but does not fail).
This commit corrects this by running each migration in a separate
transaction.
Co-authored-by: Elliott Shugerman <ee...@medianewsgroup.com>
(cherry picked from commit ea95e9c7236969acc807c65de0f12633d04753a0)
---
airflow/migrations/env.py | 1 +
1 file changed, 1 insertion(+)
diff --git a/airflow/migrations/env.py b/airflow/migrations/env.py
index 2de0c2f..234c795 100644
--- a/airflow/migrations/env.py
+++ b/airflow/migrations/env.py
@@ -81,6 +81,7 @@ def run_migrations_online():
with connectable.connect() as connection:
context.configure(
connection=connection,
+ transaction_per_migration=True,
target_metadata=target_metadata,
compare_type=COMPARE_TYPE,
render_as_batch=True