You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/03/08 05:18:00 UTC
[GitHub] [airflow] retornam opened a new pull request #7653: [AIRFLOW-6914]
Add a default robots.txt
retornam opened a new pull request #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653
If an Airflow instance is public, Google, Bing,
DuckDuckGo and other search engines can index it.
If authentication has not been enabled, this can
create a serious security issue.
This fix adds a robots.txt to DENY all robot indexing,
adds the X-Robots-Tag header and also adds the
no-index meta tag
Signed-off-by: Raymond Etornam <re...@users.noreply.github.com>
---
Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg)
Make sure to mark the boxes below before creating PR: [x]
- [x] Description above provides context of the change
- [x] Commit message/PR title starts with `[AIRFLOW-NNNN]`. AIRFLOW-NNNN = JIRA ID<sup>*</sup>
- [x] Unit tests coverage for changes (not needed for documentation changes)
- [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [x] Relevant documentation is updated including usage instructions.
- [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
<sup>*</sup> For document-only changes commit message can start with `[AIRFLOW-XXXX]`.
---
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7653:
[AIRFLOW-6914] Add a default robots.txt
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#discussion_r404731596
##########
File path: airflow/www/webpack.config.js
##########
@@ -98,6 +98,10 @@ const config = {
{
test: /\.(ttf|eot|svg)(\?v=[0-9]\.[0-9]\.[0-9])?$/,
loader: 'file-loader',
+ },{
+ test: /\.txt$/,
+ include: STATIC_DIR,
+ loader: 'file-loader',
Review comment:
This isn't needed, you are including it from /static, not /static/dist.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #7653: [AIRFLOW-6914] Add a
default robots.txt
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#issuecomment-597072961
Tests are failing
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7653:
[AIRFLOW-6914] Add a default robots.txt
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#issuecomment-610136821
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=h1) Report
> Merging [#7653](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0dafdd0b9d635b4513b1413007337b19c3d96b17&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100.00%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7653/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7653 +/- ##
==========================================
- Coverage 88.31% 88.30% -0.02%
==========================================
Files 935 935
Lines 45170 45173 +3
==========================================
- Hits 39892 39888 -4
- Misses 5278 5285 +7
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/www/blueprints.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy93d3cvYmx1ZXByaW50cy5weQ==) | `100.00% <100.00%> (ø)` | |
| [airflow/utils/process\_utils.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9wcm9jZXNzX3V0aWxzLnB5) | `70.45% <0.00%> (-8.34%)` | :arrow_down: |
| [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `87.94% <0.00%> (-0.76%)` | :arrow_down: |
| [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `90.81% <0.00%> (+0.43%)` | :arrow_up: |
| [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.64% <0.00%> (+1.11%)` | :arrow_up: |
| [airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=) | `94.36% <0.00%> (+1.40%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=footer). Last update [0dafdd0...0bdd464](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7653:
[AIRFLOW-6914] Add a default robots.txt
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#discussion_r404730936
##########
File path: airflow/www/templates/airflow/dag.html
##########
@@ -21,6 +21,10 @@
{% block page_title %}{{ dag.dag_id }} - Airflow{% endblock %}
+{% block head_meta %}
+ <meta name="robots" content="noindex" />
Review comment:
!! This should call `{{ super() }}`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7653:
[AIRFLOW-6914] Add a default robots.txt
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#issuecomment-610136821
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=h1) Report
> Merging [#7653](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0dafdd0b9d635b4513b1413007337b19c3d96b17&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100.00%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7653/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7653 +/- ##
==========================================
- Coverage 88.31% 88.30% -0.02%
==========================================
Files 935 935
Lines 45170 45173 +3
==========================================
- Hits 39892 39888 -4
- Misses 5278 5285 +7
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/www/blueprints.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy93d3cvYmx1ZXByaW50cy5weQ==) | `100.00% <100.00%> (ø)` | |
| [airflow/utils/process\_utils.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9wcm9jZXNzX3V0aWxzLnB5) | `70.45% <0.00%> (-8.34%)` | :arrow_down: |
| [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `87.94% <0.00%> (-0.76%)` | :arrow_down: |
| [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `90.81% <0.00%> (+0.43%)` | :arrow_up: |
| [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.64% <0.00%> (+1.11%)` | :arrow_up: |
| [airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=) | `94.36% <0.00%> (+1.40%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=footer). Last update [0dafdd0...0bdd464](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7653:
[AIRFLOW-6914] Add a default robots.txt
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#discussion_r404731378
##########
File path: tests/www/test_views.py
##########
@@ -319,6 +319,11 @@ def test_not_found(self):
resp = self.client.get('/', follow_redirects=True)
self.assertEqual(resp.status_code, 404)
+ def test_robots_txt(self):
+ resp = self.client.get('/test/robots.txt', follow_redirects=True)
Review comment:
Does /test/robots.txt do anything? I thought this was only read if it lives at /robots.txt?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7653:
[AIRFLOW-6914] Add a default robots.txt
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#issuecomment-610136821
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=h1) Report
> Merging [#7653](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0dafdd0b9d635b4513b1413007337b19c3d96b17&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100.00%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7653/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7653 +/- ##
==========================================
- Coverage 88.31% 88.30% -0.02%
==========================================
Files 935 935
Lines 45170 45173 +3
==========================================
- Hits 39892 39888 -4
- Misses 5278 5285 +7
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/www/blueprints.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy93d3cvYmx1ZXByaW50cy5weQ==) | `100.00% <100.00%> (ø)` | |
| [airflow/utils/process\_utils.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9wcm9jZXNzX3V0aWxzLnB5) | `70.45% <0.00%> (-8.34%)` | :arrow_down: |
| [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `87.94% <0.00%> (-0.76%)` | :arrow_down: |
| [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `90.81% <0.00%> (+0.43%)` | :arrow_up: |
| [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.64% <0.00%> (+1.11%)` | :arrow_up: |
| [airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=) | `94.36% <0.00%> (+1.40%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=footer). Last update [0dafdd0...0bdd464](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] retornam commented on issue #7653: [AIRFLOW-6914] Add a
default robots.txt
Posted by GitBox <gi...@apache.org>.
retornam commented on issue #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#issuecomment-613670159
No I think I have enough based on the review, unless you have more to add. Thanks
> On Apr 14, 2020, at 13:06, Kaxil Naik <no...@github.com> wrote:
>
>
> Hey @retornam Do you need any more info to create a new PR to solve the same issue?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or unsubscribe.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7653:
[AIRFLOW-6914] Add a default robots.txt
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#discussion_r405104784
##########
File path: tests/www/test_views.py
##########
@@ -319,6 +319,11 @@ def test_not_found(self):
resp = self.client.get('/', follow_redirects=True)
self.assertEqual(resp.status_code, 404)
+ def test_robots_txt(self):
+ resp = self.client.get('/test/robots.txt', follow_redirects=True)
Review comment:
But serving /test/robots.txt won't do anything - nothing will ever request it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7653:
[AIRFLOW-6914] Add a default robots.txt
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#discussion_r405106036
##########
File path: airflow/www/templates/airflow/dag.html
##########
@@ -21,6 +21,10 @@
{% block page_title %}{{ dag.dag_id }} - Airflow{% endblock %}
+{% block head_meta %}
+ <meta name="robots" content="noindex" />
Review comment:
Look at the next block 4 lines down
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil merged pull request #7653: [AIRFLOW-6914] Add a
default robots.txt
Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7653:
[AIRFLOW-6914] Add a default robots.txt
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#issuecomment-610136821
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=h1) Report
> Merging [#7653](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0dafdd0b9d635b4513b1413007337b19c3d96b17&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100.00%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7653/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7653 +/- ##
==========================================
- Coverage 88.31% 88.30% -0.02%
==========================================
Files 935 935
Lines 45170 45173 +3
==========================================
- Hits 39892 39888 -4
- Misses 5278 5285 +7
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/www/blueprints.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy93d3cvYmx1ZXByaW50cy5weQ==) | `100.00% <100.00%> (ø)` | |
| [airflow/utils/process\_utils.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9wcm9jZXNzX3V0aWxzLnB5) | `70.45% <0.00%> (-8.34%)` | :arrow_down: |
| [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `87.94% <0.00%> (-0.76%)` | :arrow_down: |
| [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `90.81% <0.00%> (+0.43%)` | :arrow_up: |
| [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.64% <0.00%> (+1.11%)` | :arrow_up: |
| [airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=) | `94.36% <0.00%> (+1.40%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=footer). Last update [0dafdd0...0bdd464](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ad-m commented on issue #7653: [AIRFLOW-6914] Add a
default robots.txt
Posted by GitBox <gi...@apache.org>.
ad-m commented on issue #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#issuecomment-598986102
@retornam , could you make tests pass and ask GH to assign CVE?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] retornam commented on a change in pull request #7653:
[AIRFLOW-6914] Add a default robots.txt
Posted by GitBox <gi...@apache.org>.
retornam commented on a change in pull request #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#discussion_r405097960
##########
File path: tests/www/test_views.py
##########
@@ -319,6 +319,11 @@ def test_not_found(self):
resp = self.client.get('/', follow_redirects=True)
self.assertEqual(resp.status_code, 404)
+ def test_robots_txt(self):
+ resp = self.client.get('/test/robots.txt', follow_redirects=True)
Review comment:
@ashb we set the default url in the config for the test to include `/test/`
https://github.com/apache/airflow/blob/master/tests/www/test_views.py#L303
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] retornam commented on a change in pull request #7653:
[AIRFLOW-6914] Add a default robots.txt
Posted by GitBox <gi...@apache.org>.
retornam commented on a change in pull request #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#discussion_r405098671
##########
File path: airflow/www/templates/airflow/dag.html
##########
@@ -21,6 +21,10 @@
{% block page_title %}{{ dag.dag_id }} - Airflow{% endblock %}
+{% block head_meta %}
+ <meta name="robots" content="noindex" />
Review comment:
@ashb sorry can you be a bit clearer / detailed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on issue #7653: [AIRFLOW-6914] Add a
default robots.txt
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#issuecomment-610329723
@ad-m Lack of robots.txt is not CVE worthy. All it does is stop indexing -- if you are relying or expecting that to provide any protection you are in for a rude shock.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7653:
[AIRFLOW-6914] Add a default robots.txt
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#issuecomment-610136821
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=h1) Report
> Merging [#7653](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0dafdd0b9d635b4513b1413007337b19c3d96b17&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100.00%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7653/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7653 +/- ##
==========================================
- Coverage 88.31% 88.30% -0.02%
==========================================
Files 935 935
Lines 45170 45173 +3
==========================================
- Hits 39892 39888 -4
- Misses 5278 5285 +7
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/www/blueprints.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy93d3cvYmx1ZXByaW50cy5weQ==) | `100.00% <100.00%> (ø)` | |
| [airflow/utils/process\_utils.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9wcm9jZXNzX3V0aWxzLnB5) | `70.45% <0.00%> (-8.34%)` | :arrow_down: |
| [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `87.94% <0.00%> (-0.76%)` | :arrow_down: |
| [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `90.81% <0.00%> (+0.43%)` | :arrow_up: |
| [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.64% <0.00%> (+1.11%)` | :arrow_up: |
| [airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=) | `94.36% <0.00%> (+1.40%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=footer). Last update [0dafdd0...0bdd464](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #7653: [AIRFLOW-6914] Add a
default robots.txt
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#issuecomment-613654409
Hey @retornam Do you need any more info to create a new PR to solve the same issue?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #7653: [AIRFLOW-6914] Add a
default robots.txt
Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#issuecomment-610136821
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=h1) Report
> Merging [#7653](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0dafdd0b9d635b4513b1413007337b19c3d96b17&el=desc) will **decrease** coverage by `55.05%`.
> The diff coverage is `75.00%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7653/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7653 +/- ##
===========================================
- Coverage 88.31% 33.26% -55.06%
===========================================
Files 935 935
Lines 45170 45173 +3
===========================================
- Hits 39892 15025 -24867
- Misses 5278 30148 +24870
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/www/blueprints.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy93d3cvYmx1ZXByaW50cy5weQ==) | `71.42% <75.00%> (-28.58%)` | :arrow_down: |
| [airflow/hooks/S3\_hook.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9TM19ob29rLnB5) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [airflow/hooks/pig\_hook.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9waWdfaG9vay5weQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [airflow/hooks/hdfs\_hook.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9oZGZzX2hvb2sucHk=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [airflow/hooks/http\_hook.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9odHRwX2hvb2sucHk=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [airflow/hooks/jdbc\_hook.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9qZGJjX2hvb2sucHk=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [airflow/hooks/druid\_hook.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9kcnVpZF9ob29rLnB5) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [airflow/hooks/mssql\_hook.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9tc3NxbF9ob29rLnB5) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [airflow/hooks/mysql\_hook.py](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9teXNxbF9ob29rLnB5) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [806 more](https://codecov.io/gh/apache/airflow/pull/7653/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=footer). Last update [0dafdd0...e429583](https://codecov.io/gh/apache/airflow/pull/7653?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on issue #7653: [AIRFLOW-6914] Add a
default robots.txt
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #7653: [AIRFLOW-6914] Add a default robots.txt
URL: https://github.com/apache/airflow/pull/7653#issuecomment-610348247
(We've reverted this PR as it's not quite right yet -- we still want the feature though.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services