You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/08/20 22:36:43 UTC
[GitHub] [airflow] jedcunningham opened a new pull request #17759: Show all import_errors from zip files
jedcunningham opened a new pull request #17759:
URL: https://github.com/apache/airflow/pull/17759
Instead of showing a single import error from a zip file, show them all.
Before:
After:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a change in pull request #17759: Show all import_errors from zip files
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on a change in pull request #17759:
URL: https://github.com/apache/airflow/pull/17759#discussion_r694041319
##########
File path: airflow/dag_processing/processor.py
##########
@@ -513,7 +513,12 @@ def update_import_errors(session: Session, dagbag: DagBag) -> None:
"""
# Clear the errors of the processed files
for dagbag_file in dagbag.file_last_changed:
- session.query(errors.ImportError).filter(errors.ImportError.filename == dagbag_file).delete()
+ if dagbag_file.endswith(".zip"):
Review comment:
Ah, thanks! I think the easiest fix is to just use `startswith` for all of them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on pull request #17759: Show all import_errors from zip files
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on pull request #17759:
URL: https://github.com/apache/airflow/pull/17759#issuecomment-904829155
(The mssql failures are unrelated to this change)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham merged pull request #17759: Show all import_errors from zip files
Posted by GitBox <gi...@apache.org>.
jedcunningham merged pull request #17759:
URL: https://github.com/apache/airflow/pull/17759
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham closed pull request #17759: Show all import_errors from zip files
Posted by GitBox <gi...@apache.org>.
jedcunningham closed pull request #17759:
URL: https://github.com/apache/airflow/pull/17759
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #17759: Show all import_errors from zip files
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #17759:
URL: https://github.com/apache/airflow/pull/17759#discussion_r693405991
##########
File path: airflow/dag_processing/processor.py
##########
@@ -513,7 +513,12 @@ def update_import_errors(session: Session, dagbag: DagBag) -> None:
"""
# Clear the errors of the processed files
for dagbag_file in dagbag.file_last_changed:
- session.query(errors.ImportError).filter(errors.ImportError.filename == dagbag_file).delete()
+ if dagbag_file.endswith(".zip"):
Review comment:
IIRC the DAG parsing code (`correct_if_zipped`?) uses `zipfile.is_zipfile` and accepts zip files without the `.zip` suffix, so this probably should do the same.
Actually, I wonder if it’s worthwhile to separate the file’s display (which includes the part inside a zip) and the actual path to avoid this check altogether.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #17759: Show all import_errors from zip files
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #17759:
URL: https://github.com/apache/airflow/pull/17759#issuecomment-902996262
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org