You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/02/02 11:13:22 UTC
[GitHub] [airflow] kaxil opened a new pull request #7334: [AIRFLOW-6694]
Fixed import to print_stuff in kubernetes tests (#7306)
kaxil opened a new pull request #7334: [AIRFLOW-6694] Fixed import to print_stuff in kubernetes tests (#7306)
URL: https://github.com/apache/airflow/pull/7334
I get the following error when running master:
```
ERROR [airflow.models.dagbag.DagBag] Failed to import: /Users/kaxilnaik/airflow_master/dags/example_bigquery_1.py
Traceback (most recent call last):
File "/Users/kaxilnaik/Documents/GitHub/incubator-airflow/airflow/models/dagbag.py", line 248, in process_file
loader.exec_module(m)
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/Users/kaxilnaik/airflow_master/dags/example_bigquery_1.py", line 28, in <module>
from airflow.gcp.operators.bigquery import (
ModuleNotFoundError: No module named 'airflow.gcp'
ERROR [airflow.models.dagbag.DagBag] Failed to import: /Users/kaxilnaik/Documents/GitHub/incubator-airflow/airflow/example_dags/example_kubernetes_executor_config.py
Traceback (most recent call last):
File "/Users/kaxilnaik/Documents/GitHub/incubator-airflow/airflow/models/dagbag.py", line 248, in process_file
loader.exec_module(m)
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/Users/kaxilnaik/Documents/GitHub/incubator-airflow/airflow/example_dags/example_kubernetes_executor_config.py", line 24, in <module>
from libs.helper import print_stuff
ModuleNotFoundError: No module named 'libs'
```
---
Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg)
Make sure to mark the boxes below before creating PR: [x]
- [x] Description above provides context of the change
- [x] Commit message/PR title starts with `[AIRFLOW-NNNN]`. AIRFLOW-NNNN = JIRA ID<sup>*</sup>
- [x] Unit tests coverage for changes (not needed for documentation changes)
- [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [x] Relevant documentation is updated including usage instructions.
- [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
<sup>*</sup> For document-only changes commit message can start with `[AIRFLOW-XXXX]`.
---
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7334: [AIRFLOW-6694] Fixed
import to print_stuff in kubernetes tests (#7306)
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #7334: [AIRFLOW-6694] Fixed import to print_stuff in kubernetes tests (#7306)
URL: https://github.com/apache/airflow/pull/7334#issuecomment-581126320
I think the right change is to add "example_dags" to PYTHONPATH. I believe(?) this is the default behaviour for Airflow to have the dags folder in PYTHONPATH so that the relative imports work (or am I wrong about it ?)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #7334: [AIRFLOW-6694] Fixed
import to print_stuff in kubernetes tests (#7306)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #7334: [AIRFLOW-6694] Fixed import to print_stuff in kubernetes tests (#7306)
URL: https://github.com/apache/airflow/pull/7334#issuecomment-581126464
> I think the right change is to add "example_dags" to PYTHONPATH. I believe(?) this is the default behaviour for Airflow to have the dags folder in PYTHONPATH so that the relative imports work (or am I wrong about it ?)
This is already the case with all our 1.10.* versions:
https://github.com/apache/airflow/blob/c8597cbf143b970ad3c7b0d62e3b44d1dfdc8afe/airflow/contrib/example_dags/example_kubernetes_executor_config.py#L26-L30
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7334: [AIRFLOW-6694] Fixed
import to print_stuff in kubernetes tests (#7306)
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #7334: [AIRFLOW-6694] Fixed import to print_stuff in kubernetes tests (#7306)
URL: https://github.com/apache/airflow/pull/7334#issuecomment-581126152
This won't work in kubernetes. In kubernetes the example_dags folder is git-synced to "DAGS_FOLDER" so it is not any more in "airflow.example_dags" folder - instead "libs" folder is in PYTHONPATH.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #7334: [AIRFLOW-6694] Fixed
import to print_stuff in kubernetes tests (#7306)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #7334: [AIRFLOW-6694] Fixed import to print_stuff in kubernetes tests (#7306)
URL: https://github.com/apache/airflow/pull/7334#issuecomment-581126741
> This won't work in kubernetes. In kubernetes the example_dags folder is git-synced to "DAGS_FOLDER" so it is not any more in "airflow.example_dags" folder - instead "libs" folder is in PYTHONPATH.
It is bundled in the airflow package itself so don't need to be on PYTHONPATH
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil edited a comment on issue #7334: [AIRFLOW-6694]
Fixed import to print_stuff in kubernetes tests (#7306)
Posted by GitBox <gi...@apache.org>.
kaxil edited a comment on issue #7334: [AIRFLOW-6694] Fixed import to print_stuff in kubernetes tests (#7306)
URL: https://github.com/apache/airflow/pull/7334#issuecomment-581126464
> I think the right change is to add "example_dags" to PYTHONPATH. I believe(?) this is the default behaviour for Airflow to have the dags folder in PYTHONPATH so that the relative imports work (or am I wrong about it ?)
This is already the case with all our 1.10.* versions and works fine:
https://github.com/apache/airflow/blob/c8597cbf143b970ad3c7b0d62e3b44d1dfdc8afe/airflow/contrib/example_dags/example_kubernetes_executor_config.py#L26-L30
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil merged pull request #7334: [AIRFLOW-6694] Fixed
import to print_stuff in kubernetes tests (#7306)
Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #7334: [AIRFLOW-6694] Fixed import to print_stuff in kubernetes tests (#7306)
URL: https://github.com/apache/airflow/pull/7334
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #7334: [AIRFLOW-6694] Fixed
import to print_stuff in kubernetes tests (#7306)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #7334: [AIRFLOW-6694] Fixed import to print_stuff in kubernetes tests (#7306)
URL: https://github.com/apache/airflow/pull/7334#issuecomment-581126654
And example_dags folder is included from https://github.com/apache/airflow/blob/f67226a05ad07a1461efec3d877b7aeee5375737/airflow/utils/file.py#L126-L129
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7334: [AIRFLOW-6694] Fixed
import to print_stuff in kubernetes tests (#7306)
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #7334: [AIRFLOW-6694] Fixed import to print_stuff in kubernetes tests (#7306)
URL: https://github.com/apache/airflow/pull/7334#issuecomment-581127117
Ah. OK. I see. It's a bit strange though that we get the main dag from git and the import from airflow. It will work indeed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services