You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/09/23 19:35:24 UTC
[GitHub] [airflow] potiuk opened a new pull request #18478: Improve guidance to users telling them what to do on import timeout
potiuk opened a new pull request #18478:
URL: https://github.com/apache/airflow/pull/18478
The message about import timeut does not explain an average user
what the root cause is. The updated one links to the docuementation
on how to reduce tpo-level Python code overhead and reduce
complexity of their DAGs.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #18478: Improve guidance to users telling them what to do on import timeout
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #18478:
URL: https://github.com/apache/airflow/pull/18478#issuecomment-927088306
All fixed @jedcunningham
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #18478: Improve guidance to users telling them what to do on import timeout
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #18478:
URL: https://github.com/apache/airflow/pull/18478#discussion_r715740117
##########
File path: airflow/models/dagbag.py
##########
@@ -316,7 +316,14 @@ def _load_modules_from_file(self, filepath, safe_mode):
if mod_name in sys.modules:
del sys.modules[mod_name]
- timeout_msg = f"DagBag import timeout for {filepath} after {self.DAGBAG_IMPORT_TIMEOUT}s"
+ timeout_msg = (
+ f"DagBag import timeout for {filepath} after {self.DAGBAG_IMPORT_TIMEOUT}s.\n"
Review comment:
Recovered :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #18478: Improve guidance to users telling them what to do on import timeout
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #18478:
URL: https://github.com/apache/airflow/pull/18478#discussion_r715186985
##########
File path: airflow/models/dagbag.py
##########
@@ -316,7 +316,14 @@ def _load_modules_from_file(self, filepath, safe_mode):
if mod_name in sys.modules:
del sys.modules[mod_name]
- timeout_msg = f"DagBag import timeout for {filepath} after {self.DAGBAG_IMPORT_TIMEOUT}s"
+ timeout_msg = (
+ f"DagBag import timeout for {filepath} after {self.DAGBAG_IMPORT_TIMEOUT}s.\n"
Review comment:
Good point. Let me fix it tomorrow, when I recover from ApacheCon Whiskey Birds of the Feather sesion ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #18478: Improve guidance to users telling them what to do on import timeout
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #18478:
URL: https://github.com/apache/airflow/pull/18478
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #18478: Improve guidance to users telling them what to do on import timeout
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #18478:
URL: https://github.com/apache/airflow/pull/18478#discussion_r715102283
##########
File path: airflow/models/dagbag.py
##########
@@ -316,7 +316,14 @@ def _load_modules_from_file(self, filepath, safe_mode):
if mod_name in sys.modules:
del sys.modules[mod_name]
- timeout_msg = f"DagBag import timeout for {filepath} after {self.DAGBAG_IMPORT_TIMEOUT}s"
+ timeout_msg = (
+ f"DagBag import timeout for {filepath} after {self.DAGBAG_IMPORT_TIMEOUT}s.\n"
Review comment:
let's use `get_docs_url` from `from airflow.utils.docs import get_docs_url`
e.g:
```
doc_link = get_docs_url("stable-rest-api-ref.html")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #18478: Improve guidance to users telling them what to do on import timeout
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #18478:
URL: https://github.com/apache/airflow/pull/18478#issuecomment-926098067
Should help with cases similar to https://github.com/apache/airflow/issues/18448
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a change in pull request #18478: Improve guidance to users telling them what to do on import timeout
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on a change in pull request #18478:
URL: https://github.com/apache/airflow/pull/18478#discussion_r715847803
##########
File path: airflow/models/dagbag.py
##########
@@ -316,7 +317,12 @@ def _load_modules_from_file(self, filepath, safe_mode):
if mod_name in sys.modules:
del sys.modules[mod_name]
- timeout_msg = f"DagBag import timeout for {filepath} after {self.DAGBAG_IMPORT_TIMEOUT}s"
+ timeout_msg = (
+ f"DagBag import timeout for {filepath} after {self.DAGBAG_IMPORT_TIMEOUT}s.\n"
+ "Please take a look at those docs to improve your DAG import time\n"
Review comment:
```suggestion
"Please take a look at these docs to improve your DAG import time:\n"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #18478: Improve guidance to users telling them what to do on import timeout
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #18478:
URL: https://github.com/apache/airflow/pull/18478#issuecomment-927089942
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #18478: Improve guidance to users telling them what to do on import timeout
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #18478:
URL: https://github.com/apache/airflow/pull/18478#issuecomment-926800987
Looks good - and seems that the increased setup timeout worked @ashb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org