You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2023/02/20 01:06:21 UTC
[airflow] branch main updated: fix: add pre-commit to ban urlparse (#27841)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new a99254ffd3 fix: add pre-commit to ban urlparse (#27841)
a99254ffd3 is described below
commit a99254ffd36f9de06feda6fe45773495632e3255
Author: Wonhyeong Seo <wo...@kakao.com>
AuthorDate: Mon Feb 20 10:06:12 2023 +0900
fix: add pre-commit to ban urlparse (#27841)
review by @potiuk @uranusjr @eladkal
---
.pre-commit-config.yaml | 7 +++++++
STATIC_CODE_CHECKS.rst | 2 ++
dev/breeze/src/airflow_breeze/pre_commit_ids.py | 1 +
images/breeze/output-commands-hash.txt | 2 +-
images/breeze/output_static-checks.svg | 14 +++++++-------
5 files changed, 18 insertions(+), 8 deletions(-)
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 968024394f..05627f0db9 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -420,6 +420,13 @@ repos:
pass_filenames: true
files: \.py$
exclude: ^airflow/_vendor/|^docs/|^tests/
+ - id: check-urlparse-usage-in-code
+ language: pygrep
+ name: Don't use urlparse in code
+ description: urlparse is not recommended, use urlsplit() in code instead
+ entry: "^\\s*from urllib\\.parse import ((\\|, )(urlparse\\|urlunparse))+$"
+ pass_filenames: true
+ files: \.py$
- id: check-only-new-session-with-provide-session
name: Check NEW_SESSION is only used with @provide_session
language: python
diff --git a/STATIC_CODE_CHECKS.rst b/STATIC_CODE_CHECKS.rst
index ba6c3da1c6..814889782c 100644
--- a/STATIC_CODE_CHECKS.rst
+++ b/STATIC_CODE_CHECKS.rst
@@ -227,6 +227,8 @@ require Breeze Docker image to be build locally.
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
| check-system-tests-tocs | Check that system tests is properly added | |
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
+| check-urlparse-usage-in-code | Don't use urlparse in code | |
++-----------------------------------------------------------+------------------------------------------------------------------+---------+
| check-xml | Check XML files with xmllint | |
+-----------------------------------------------------------+------------------------------------------------------------------+---------+
| codespell | Run codespell to check for common misspellings in files | |
diff --git a/dev/breeze/src/airflow_breeze/pre_commit_ids.py b/dev/breeze/src/airflow_breeze/pre_commit_ids.py
index 6bf5931a13..cfa8ad27dc 100644
--- a/dev/breeze/src/airflow_breeze/pre_commit_ids.py
+++ b/dev/breeze/src/airflow_breeze/pre_commit_ids.py
@@ -67,6 +67,7 @@ PRE_COMMIT_LIST = [
"check-start-date-not-used-in-defaults",
"check-system-tests-present",
"check-system-tests-tocs",
+ "check-urlparse-usage-in-code",
"check-xml",
"codespell",
"compile-www-assets",
diff --git a/images/breeze/output-commands-hash.txt b/images/breeze/output-commands-hash.txt
index 598326d6eb..5757ec5977 100644
--- a/images/breeze/output-commands-hash.txt
+++ b/images/breeze/output-commands-hash.txt
@@ -55,7 +55,7 @@ setup:version:123b462a421884dc2320ffc5e54b2478
setup:56a2ef337c354362760d247df5d05365
shell:ab07ac2d57253e25367a7200ce686703
start-airflow:5e8460ac38f8e9ea2a0ac7e248fd7bc9
-static-checks:12e8fed2acbed0d823efc5121fd0eb58
+static-checks:af6504875660e76e870967ffb43f1724
stop:e5aa686b4e53707ced4039d8414d5cd6
testing:docker-compose-tests:b86c044b24138af0659a05ed6331576c
testing:helm-tests:94a442e7f3f63b34c4831a84d165690a
diff --git a/images/breeze/output_static-checks.svg b/images/breeze/output_static-checks.svg
index 41758e87ce..70577a3f7f 100644
--- a/images/breeze/output_static-checks.svg
+++ b/images/breeze/output_static-checks.svg
@@ -246,13 +246,13 @@
</text><text class="breeze-static-checks-r5" x="0" y="605.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-24)">│</text><text class="breeze-static-checks-r7" x="366" y="605.6" textLength="1073.6" clip-path="url(#breeze-static-checks-line-24)">check-providers-subpackages-init-file-exist | check-pydevd-left-in-code |               </text><text class="breeze-static-checks-r5" x="1451.8" y [...]
</text><text class="breeze-static-checks-r5" x="0" y="630" textLength="12.2" clip-path="url(#breeze-static-checks-line-25)">│</text><text class="breeze-static-checks-r7" x="366" y="630" textLength="1073.6" clip-path="url(#breeze-static-checks-line-25)">check-revision-heads-map | check-safe-filter-usage-in-html | check-setup-order |        </text><text class="breeze-static-checks-r5" x="1451.8" y="630" textLength="12.2" clip [...]
</text><text class="breeze-static-checks-r5" x="0" y="654.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-26)">│</text><text class="breeze-static-checks-r7" x="366" y="654.4" textLength="1073.6" clip-path="url(#breeze-static-checks-line-26)">check-start-date-not-used-in-defaults | check-system-tests-present |                    </text><text class="breeze-stati [...]
-</text><text class="breeze-static-checks-r5" x="0" y="678.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-27)">│</text><text class="breeze-static-checks-r7" x="366" y="678.8" textLength="1073.6" clip-path="url(#breeze-static-checks-line-27)">check-system-tests-tocs | check-xml | codespell | compile-www-assets |                  </text><text class="br [...]
-</text><text class="breeze-static-checks-r5" x="0" y="703.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-28)">│</text><text class="breeze-static-checks-r7" x="366" y="703.2" textLength="1073.6" clip-path="url(#breeze-static-checks-line-28)">compile-www-assets-dev | create-missing-init-py-files-tests | debug-statements |        </text><text class="breeze-static-checks-r5" x="1451.8" y="703.2" textLength="12.2 [...]
-</text><text class="breeze-static-checks-r5" x="0" y="727.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-29)">│</text><text class="breeze-static-checks-r7" x="366" y="727.6" textLength="1073.6" clip-path="url(#breeze-static-checks-line-29)">detect-private-key | doctoc | end-of-file-fixer | fix-encoding-pragma | flynt | identity</text><text class="breeze-static-checks-r5" x="1451.8" y="727.6" textLength="12.2" clip-path="ur [...]
-</text><text class="breeze-static-checks-r5" x="0" y="752" textLength="12.2" clip-path="url(#breeze-static-checks-line-30)">│</text><text class="breeze-static-checks-r7" x="366" y="752" textLength="1073.6" clip-path="url(#breeze-static-checks-line-30)">| insert-license | lint-chart-schema | lint-css | lint-dockerfile | lint-helm-chart |   </text><text class="breeze-static-checks-r5" x="1451.8" y="752" textLength="12.2" clip [...]
-</text><text class="breeze-static-checks-r5" x="0" y="776.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-31)">│</text><text class="breeze-static-checks-r7" x="366" y="776.4" textLength="1073.6" clip-path="url(#breeze-static-checks-line-31)">lint-json-schema | lint-markdown | lint-openapi | mixed-line-ending | pretty-format-json</text><text class="breeze-static-checks-r5" x="1451.8" y="776.4" textLength="12.2" clip-path="url(#breeze- [...]
-</text><text class="breeze-static-checks-r5" x="0" y="800.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-32)">│</text><text class="breeze-static-checks-r7" x="366" y="800.8" textLength="1073.6" clip-path="url(#breeze-static-checks-line-32)">| python-no-log-warn | replace-bad-characters | rst-backticks | ruff | run-mypy |       </text><text class="breeze-static-checks-r5" x="1451.8" y="800 [...]
-</text><text class="breeze-static-checks-r5" x="0" y="825.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-33)">│</text><text class="breeze-static-checks-r7" x="366" y="825.2" textLength="1073.6" clip-path="url(#breeze-static-checks-line-33)">run-shellcheck | trailing-whitespace | ts-compile-and-lint-javascript |                 </text><text class="breeze-static-che [...]
+</text><text class="breeze-static-checks-r5" x="0" y="678.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-27)">│</text><text class="breeze-static-checks-r7" x="366" y="678.8" textLength="1073.6" clip-path="url(#breeze-static-checks-line-27)">check-system-tests-tocs | check-urlparse-usage-in-code | check-xml | codespell |        </text><text class="breeze-static-checks-r5" x="1451.8" y="678.8" textLe [...]
+</text><text class="breeze-static-checks-r5" x="0" y="703.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-28)">│</text><text class="breeze-static-checks-r7" x="366" y="703.2" textLength="1073.6" clip-path="url(#breeze-static-checks-line-28)">compile-www-assets | compile-www-assets-dev | create-missing-init-py-files-tests |      </text><text class="breeze-static-checks-r5" x="1451.8" y="703.2" textLength="12.2" clip-pat [...]
+</text><text class="breeze-static-checks-r5" x="0" y="727.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-29)">│</text><text class="breeze-static-checks-r7" x="366" y="727.6" textLength="1073.6" clip-path="url(#breeze-static-checks-line-29)">debug-statements | detect-private-key | doctoc | end-of-file-fixer | fix-encoding-pragma</text><text class="breeze-static-checks-r5" x="1451.8" y="727.6" textLength="12.2" clip-path="url(#breeze- [...]
+</text><text class="breeze-static-checks-r5" x="0" y="752" textLength="12.2" clip-path="url(#breeze-static-checks-line-30)">│</text><text class="breeze-static-checks-r7" x="366" y="752" textLength="1073.6" clip-path="url(#breeze-static-checks-line-30)">| flynt | identity | insert-license | lint-chart-schema | lint-css | lint-dockerfile |  </text><text class="breeze-static-checks-r5" x="1451.8" y="752" textLength="12.2" [...]
+</text><text class="breeze-static-checks-r5" x="0" y="776.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-31)">│</text><text class="breeze-static-checks-r7" x="366" y="776.4" textLength="1073.6" clip-path="url(#breeze-static-checks-line-31)">lint-helm-chart | lint-json-schema | lint-markdown | lint-openapi | mixed-line-ending | </text><text class="breeze-static-checks-r5" x="1451.8" y="776.4" textLength="12.2" clip-path="ur [...]
+</text><text class="breeze-static-checks-r5" x="0" y="800.8" textLength="12.2" clip-path="url(#breeze-static-checks-line-32)">│</text><text class="breeze-static-checks-r7" x="366" y="800.8" textLength="1073.6" clip-path="url(#breeze-static-checks-line-32)">pretty-format-json | python-no-log-warn | replace-bad-characters | rst-backticks | ruff </text><text class="breeze-static-checks-r5" x="1451.8" y="800.8" textLength="12.2" clip-path="url(#br [...]
+</text><text class="breeze-static-checks-r5" x="0" y="825.2" textLength="12.2" clip-path="url(#breeze-static-checks-line-33)">│</text><text class="breeze-static-checks-r7" x="366" y="825.2" textLength="1073.6" clip-path="url(#breeze-static-checks-line-33)">| run-mypy | run-shellcheck | trailing-whitespace | ts-compile-and-lint-javascript |    </text><text class="breeze-static-checks-r5" x="1451.8" y="825.2" textLength="12.2" cli [...]
</text><text class="breeze-static-checks-r5" x="0" y="849.6" textLength="12.2" clip-path="url(#breeze-static-checks-line-34)">│</text><text class="breeze-static-checks-r7" x="366" y="849.6" textLength="1073.6" clip-path="url(#breeze-static-checks-line-34)">update-black-version | update-breeze-cmd-output | update-breeze-readme-config-hash |    </text><text class="breeze-static-checks-r5" x="1451.8" y="849.6" textLength="12.2" clip-path="url(#br [...]
</text><text class="breeze-static-checks-r5" x="0" y="874" textLength="12.2" clip-path="url(#breeze-static-checks-line-35)">│</text><text class="breeze-static-checks-r7" x="366" y="874" textLength="1073.6" clip-path="url(#breeze-static-checks-line-35)">update-common-sql-api-stubs | update-er-diagram | update-extras |                       </text><tex [...]
</text><text class="breeze-static-checks-r5" x="0" y="898.4" textLength="12.2" clip-path="url(#breeze-static-checks-line-36)">│</text><text class="breeze-static-checks-r7" x="366" y="898.4" textLength="1073.6" clip-path="url(#breeze-static-checks-line-36)">update-in-the-wild-to-be-sorted | update-inlined-dockerfile-scripts |                   </text><text class="breeze-static-che [...]