You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2023/02/12 18:45:54 UTC

[GitHub] [airflow] potiuk opened a new pull request, #29494: Fix circular imports when airflow starts from scratch

potiuk opened a new pull request, #29494:
URL: https://github.com/apache/airflow/pull/29494

   When airflow CLI is started without generated config file the CLI fails with circular imports. Some of the util classes imported during such start were importing "constants" (in fact variables) from the setttings and this happened before settings were fully initialized.
   
   This PR moves all relevant top-level imports to be local imports and moves initialization of the State class with settings-defined colors to initialization() method in order to avoid those circular imports.
   
   <!--
   Thank you for contributing! Please make sure that your code changes
   are covered with tests. And in case of new features or big changes
   remember to adjust the documentation.
   
   Feel free to ping committers for the review!
   
   In case of an existing issue, reference it using one of the following:
   
   closes: #ISSUE
   related: #ISSUE
   
   How to write a good git commit message:
   http://chris.beams.io/posts/git-commit/
   -->
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] uranusjr commented on a diff in pull request #29494: Fix circular imports when airflow starts

Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #29494:
URL: https://github.com/apache/airflow/pull/29494#discussion_r1104227705


##########
airflow/models/xcom.py:
##########
@@ -772,8 +772,8 @@ def _get_bound_query(self) -> Generator[Query, None, None]:
             yield self._query
             return
 
-        if Session is None:
-            raise
+        if getattr(setting, "Session", None) is None:

Review Comment:
   ```suggestion
           if getattr(settings, "Session", None) is None:
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk commented on a diff in pull request #29494: Fix circular imports when airflow starts

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #29494:
URL: https://github.com/apache/airflow/pull/29494#discussion_r1104906094


##########
airflow/settings.py:
##########
@@ -352,7 +353,8 @@ def dispose_orm():
     global Session
 
     if Session is not None:
-        Session.remove()
+
+Session.remove()

Review Comment:
   Bad merge. I knew I should not use GitHub UI for that. Lazy, Lazy Jarek.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] o-nikolas commented on pull request #29494: Fix circular imports when airflow starts

Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on PR #29494:
URL: https://github.com/apache/airflow/pull/29494#issuecomment-1428952682

   > Some time ago I tried to write specific tests for known and repeatable problem with circular imports.
   
   This would be so awesome to have. If you see the PR I linked (and the other one like it for `__init__.py`) the fix ultimately relies on `configuration.py` being imported early and intentionally. If we could find some way to test/assert at least that specifically that could help. 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk commented on pull request #29494: Fix circular imports when airflow starts from scratch

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #29494:
URL: https://github.com/apache/airflow/pull/29494#issuecomment-1427104975

   I have not found yet which change triggered it (for sure that was not the change by @o-nikolas #29257 as I reverted it and it did not help) but it seems our main cli `airflow` fails with circular imports. Maybe others can find where it is from but my changes fix it (@o-nikolas - this is another manifestation of the circular "dependencies" we have - between config and settings):
   
   Example stack trace I got before that one in various stages of the fix:
   
   ```
   Traceback (most recent call last):
     File "/Users/jarek/.pyenv/versions/airflow-3.9/bin/airflow", line 5, in <module>
       from airflow.__main__ import main
     File "/Users/jarek/IdeaProjects/airflow/airflow/__main__.py", line 27, in <module>
       from airflow.cli import cli_parser
     File "/Users/jarek/IdeaProjects/airflow/airflow/cli/cli_parser.py", line 33, in <module>
       from airflow import settings
     File "/Users/jarek/IdeaProjects/airflow/airflow/settings.py", line 39, in <module>
       from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf  # NOQA F401
     File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line 1794, in <module>
       conf.validate()
     File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line 343, in validate
       self._validate_config_dependencies()
     File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line 439, in _validate_config_dependencies
       executor, _ = ExecutorLoader.import_default_executor_cls()
     File "/Users/jarek/IdeaProjects/airflow/airflow/executors/executor_loader.py", line 151, in import_default_executor_cls
       return cls.import_executor_cls(executor_name)
     File "/Users/jarek/IdeaProjects/airflow/airflow/executors/executor_loader.py", line 126, in import_executor_cls
       return import_string(cls.executors[executor_name]), ConnectorSource.CORE
     File "/Users/jarek/IdeaProjects/airflow/airflow/utils/module_loading.py", line 36, in import_string
       module = import_module(module_path)
     File "/Users/jarek/.pyenv/versions/3.9.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
       return _bootstrap._gcd_import(name[level:], package, level)
     File "/Users/jarek/IdeaProjects/airflow/airflow/executors/sequential_executor.py", line 30, in <module>
       from airflow.executors.base_executor import BaseExecutor, CommandType
     File "/Users/jarek/IdeaProjects/airflow/airflow/executors/base_executor.py", line 34, in <module>
       from airflow.models.taskinstance import TaskInstance, TaskInstanceKey
     File "/Users/jarek/IdeaProjects/airflow/airflow/models/taskinstance.py", line 72, in <module>
       from airflow.datasets.manager import dataset_manager
     File "/Users/jarek/IdeaProjects/airflow/airflow/datasets/manager.py", line 27, in <module>
       from airflow.models.dataset import DatasetDagRunQueue, DatasetEvent, DatasetModel
     File "/Users/jarek/IdeaProjects/airflow/airflow/models/dataset.py", line 40, in <module>
       from airflow.utils import timezone
     File "/Users/jarek/IdeaProjects/airflow/airflow/utils/timezone.py", line 27, in <module>
       from airflow.settings import TIMEZONE
   ImportError: cannot import name 'TIMEZONE' from partially initialized module 'airflow.settings' (most likely due to a circular import) (/Users/jarek/IdeaProjects/airflow/airflow/settings.py)
   ``` 
   
   Then:
   
   ```
   Traceback (most recent call last):
     File "/Users/jarek/.pyenv/versions/airflow-3.9/bin/airflow", line 5, in <module>
       from airflow.__main__ import main
     File "/Users/jarek/IdeaProjects/airflow/airflow/__main__.py", line 27, in <module>
       from airflow.cli import cli_parser
     File "/Users/jarek/IdeaProjects/airflow/airflow/cli/cli_parser.py", line 33, in <module>
       from airflow import settings
     File "/Users/jarek/IdeaProjects/airflow/airflow/settings.py", line 39, in <module>
       from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf  # NOQA F401
     File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line 1794, in <module>
       conf.validate()
     File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line 343, in validate
       self._validate_config_dependencies()
     File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line 439, in _validate_config_dependencies
       executor, _ = ExecutorLoader.import_default_executor_cls()
     File "/Users/jarek/IdeaProjects/airflow/airflow/executors/executor_loader.py", line 151, in import_default_executor_cls
       return cls.import_executor_cls(executor_name)
     File "/Users/jarek/IdeaProjects/airflow/airflow/executors/executor_loader.py", line 126, in import_executor_cls
       return import_string(cls.executors[executor_name]), ConnectorSource.CORE
     File "/Users/jarek/IdeaProjects/airflow/airflow/utils/module_loading.py", line 36, in import_string
       module = import_module(module_path)
     File "/Users/jarek/.pyenv/versions/3.9.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
       return _bootstrap._gcd_import(name[level:], package, level)
     File "/Users/jarek/IdeaProjects/airflow/airflow/executors/sequential_executor.py", line 30, in <module>
       from airflow.executors.base_executor import BaseExecutor, CommandType
     File "/Users/jarek/IdeaProjects/airflow/airflow/executors/base_executor.py", line 34, in <module>
       from airflow.models.taskinstance import TaskInstance, TaskInstanceKey
     File "/Users/jarek/IdeaProjects/airflow/airflow/models/taskinstance.py", line 72, in <module>
       from airflow.datasets.manager import dataset_manager
     File "/Users/jarek/IdeaProjects/airflow/airflow/datasets/manager.py", line 29, in <module>
       from airflow.utils.log.logging_mixin import LoggingMixin
     File "/Users/jarek/IdeaProjects/airflow/airflow/utils/log/logging_mixin.py", line 29, in <module>
       from airflow.settings import IS_K8S_EXECUTOR_POD
   ImportError: cannot import name 'IS_K8S_EXECUTOR_POD' from partially initialized module 'airflow.settings' (most likely due to a circular import) (/Users/jarek/IdeaProjects/airflow/airflow/settings.py)
   ```
   
   Then:
   
   ```
   Traceback (most recent call last):
     File "/Users/jarek/.pyenv/versions/airflow-3.9/bin/airflow", line 5, in <module>
       from airflow.__main__ import main
     File "/Users/jarek/IdeaProjects/airflow/airflow/__main__.py", line 27, in <module>
       from airflow.cli import cli_parser
     File "/Users/jarek/IdeaProjects/airflow/airflow/cli/cli_parser.py", line 33, in <module>
       from airflow import settings
     File "/Users/jarek/IdeaProjects/airflow/airflow/settings.py", line 39, in <module>
       from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf  # NOQA F401
     File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line 1791, in <module>
       conf.validate()
     File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line 342, in validate
       self._validate_config_dependencies()
     File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line 438, in _validate_config_dependencies
       executor, _ = ExecutorLoader.import_default_executor_cls()
     File "/Users/jarek/IdeaProjects/airflow/airflow/executors/executor_loader.py", line 151, in import_default_executor_cls
       return cls.import_executor_cls(executor_name)
     File "/Users/jarek/IdeaProjects/airflow/airflow/executors/executor_loader.py", line 126, in import_executor_cls
       return import_string(cls.executors[executor_name]), ConnectorSource.CORE
     File "/Users/jarek/IdeaProjects/airflow/airflow/utils/module_loading.py", line 36, in import_string
       module = import_module(module_path)
     File "/Users/jarek/.pyenv/versions/3.9.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
       return _bootstrap._gcd_import(name[level:], package, level)
     File "/Users/jarek/IdeaProjects/airflow/airflow/executors/sequential_executor.py", line 30, in <module>
       from airflow.executors.base_executor import BaseExecutor, CommandType
     File "/Users/jarek/IdeaProjects/airflow/airflow/executors/base_executor.py", line 34, in <module>
       from airflow.models.taskinstance import TaskInstance, TaskInstanceKey
     File "/Users/jarek/IdeaProjects/airflow/airflow/models/taskinstance.py", line 91, in <module>
       from airflow.models.mappedoperator import MappedOperator
     File "/Users/jarek/IdeaProjects/airflow/airflow/models/mappedoperator.py", line 35, in <module>
       from airflow.models.abstractoperator import (
     File "/Users/jarek/IdeaProjects/airflow/airflow/models/abstractoperator.py", line 33, in <module>
       from airflow.utils.state import State, TaskInstanceState
     File "/Users/jarek/IdeaProjects/airflow/airflow/utils/state.py", line 73, in <module>
       class State:
     File "/Users/jarek/IdeaProjects/airflow/airflow/utils/state.py", line 123, in State
       from airflow.settings import STATE_COLORS
   ImportError: cannot import name 'STATE_COLORS' from partially initialized module 'airflow.settings' (most likely due to a circular import) (/Users/jarek/IdeaProjects/airflow/airflow/settings.py)
   ```
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] uranusjr commented on a diff in pull request #29494: Fix circular imports when airflow starts

Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #29494:
URL: https://github.com/apache/airflow/pull/29494#discussion_r1103956801


##########
airflow/models/xcom.py:
##########
@@ -772,6 +772,8 @@ def _get_bound_query(self) -> Generator[Query, None, None]:
             yield self._query
             return
 
+        if Session is None:
+            raise

Review Comment:
   ```suggestion
           if getattr(setting, "Session", None) is None:
               raise RuntimeError("Session must be set before!")
   ```
   
   This matches the pattern elsewhere; if this is called before `settings` is configured, the `Session` variable may actually not exist at all (instead of being `None`).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] o-nikolas commented on a diff in pull request #29494: Fix circular imports when airflow starts

Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on code in PR #29494:
URL: https://github.com/apache/airflow/pull/29494#discussion_r1104818974


##########
airflow/settings.py:
##########
@@ -352,7 +353,8 @@ def dispose_orm():
     global Session
 
     if Session is not None:
-        Session.remove()
+
+Session.remove()

Review Comment:
   This looks odd? Was this change intentional?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] o-nikolas commented on pull request #29494: Fix circular imports when airflow starts

Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on PR #29494:
URL: https://github.com/apache/airflow/pull/29494#issuecomment-1428950550

   I created a [new PR](https://github.com/apache/airflow/pull/29523) which fixes `__main__.py` in the same way that I fixed `__init__.py`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk commented on pull request #29494: Fix circular imports when airflow starts

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #29494:
URL: https://github.com/apache/airflow/pull/29494#issuecomment-1428905027

   Ah yeah i see you proposed a new one - now (sorry on mobile). 
   
   Yeah worth trying. I just know from the past that i had some nice ideas for fixing those problems and they often broke down when I tested them :) after some big changes.
   
   I would love to get those problems not reappear.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk commented on a diff in pull request #29494: Fix circular imports when airflow starts

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #29494:
URL: https://github.com/apache/airflow/pull/29494#discussion_r1104912531


##########
airflow/settings.py:
##########
@@ -352,7 +353,8 @@ def dispose_orm():
     global Session
 
     if Session is not None:
-        Session.remove()
+
+Session.remove()

Review Comment:
   Thanks @o-nikolas  for vigilance. Resolved.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] o-nikolas commented on pull request #29494: Fix circular imports when airflow starts

Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on PR #29494:
URL: https://github.com/apache/airflow/pull/29494#issuecomment-1428888690

   >The first traceback you 
   
   > I have not found yet which change triggered it (for sure that was not the change by @o-nikolas #29257 as I reverted it and it did not help) but it seems our main cli `airflow` fails with circular imports. Maybe others can find where it is from but my changes fix it (@o-nikolas - this is another manifestation of the circular "dependencies" we have - between config and settings):
   > 
   > Example stack trace I got before that one in various stages of the fix:
   > 
   > ```
   > Traceback (most recent call last):
   >   File "/Users/jarek/.pyenv/versions/airflow-3.9/bin/airflow", line 5, in <module>
   >     from airflow.__main__ import main
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/__main__.py", line 27, in <module>
   >     from airflow.cli import cli_parser
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/cli/cli_parser.py", line 33, in <module>
   >     from airflow import settings
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/settings.py", line 39, in <module>
   >     from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf  # NOQA F401
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line 1794, in <module>
   >     conf.validate()
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line 343, in validate
   >     self._validate_config_dependencies()
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/configuration.py", line 439, in _validate_config_dependencies
   >     executor, _ = ExecutorLoader.import_default_executor_cls()
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/executors/executor_loader.py", line 151, in import_default_executor_cls
   >     return cls.import_executor_cls(executor_name)
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/executors/executor_loader.py", line 126, in import_executor_cls
   >     return import_string(cls.executors[executor_name]), ConnectorSource.CORE
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/utils/module_loading.py", line 36, in import_string
   >     module = import_module(module_path)
   >   File "/Users/jarek/.pyenv/versions/3.9.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
   >     return _bootstrap._gcd_import(name[level:], package, level)
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/executors/sequential_executor.py", line 30, in <module>
   >     from airflow.executors.base_executor import BaseExecutor, CommandType
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/executors/base_executor.py", line 34, in <module>
   >     from airflow.models.taskinstance import TaskInstance, TaskInstanceKey
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/models/taskinstance.py", line 72, in <module>
   >     from airflow.datasets.manager import dataset_manager
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/datasets/manager.py", line 27, in <module>
   >     from airflow.models.dataset import DatasetDagRunQueue, DatasetEvent, DatasetModel
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/models/dataset.py", line 40, in <module>
   >     from airflow.utils import timezone
   >   File "/Users/jarek/IdeaProjects/airflow/airflow/utils/timezone.py", line 27, in <module>
   >     from airflow.settings import TIMEZONE
   > ImportError: cannot import name 'TIMEZONE' from partially initialized module 'airflow.settings' (most likely due to a circular import) (/Users/jarek/IdeaProjects/airflow/airflow/settings.py)
   > ```
   
   @potiuk I believe that a similar fix I applied in `airflow.__int__` (in #29257)  would also have fixed this case actually. Basically in `__main__.py` if `airflow.configuration.conf` is imported **before**  `airflow.cli.cli_parser` (which imports `airflow.settings`) then the cycle would be removed.
   We get cycles like this when `airflow.settings` **is the first module to import/load `airflow.configuration`** because the very first import of `airflow.configuration` triggers the conf object to be created and validated, but much of the conf validation code depends on `airflow.settings` downstream so we can't have it happen inside `airflow.settings`.
   
   So importing `airflow.configuration` early (thus initting/validating conf **outside** the context of `airflow.settings`) would have fixed the issue in `__main__.py` I think:
   ```diff
    from __future__ import annotations
    
    import os
    
    import argcomplete
    
   -from airflow.cli import cli_parser
    from airflow.configuration import conf
   +from airflow.cli import cli_parser
    
    
    def main():
        """Main executable function."""
        if conf.get("core", "security") == "kerberos":
            os.environ["KRB5CCNAME"] = conf.get("kerberos", "ccache")
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk merged pull request #29494: Fix circular imports when airflow starts

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #29494:
URL: https://github.com/apache/airflow/pull/29494


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] Taragolis commented on pull request #29494: Fix circular imports when airflow starts

Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis commented on PR #29494:
URL: https://github.com/apache/airflow/pull/29494#issuecomment-1428915301

   Some time ago I tried to write specific tests for known and repeatable problem with circular imports.
   
   There is couple issues (I guess all could be solved):
   1. Python 3.7 only show as regular ImportError without mentioning that is circular import
   2. Need to run all this kind of tests outside of regular `pytest` environment because everything already imported in the right way. I found a stupid solution - generate python file and execute it in subprocess
   3. If run in subprocess need to mock specific stuff very carefully.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk commented on pull request #29494: Fix circular imports when airflow starts

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #29494:
URL: https://github.com/apache/airflow/pull/29494#issuecomment-1428901618

   Well. It was failing after the fix from you. I checked it was there. I even reverted it to see if it changes anything. It did not. Maybe you  can try to revert my fix and see if you can reproduce it.
   
   But now when I am thinking if it - maybe this was somehow connected to installed entrypoint and 'pip install -e .' caused some weird interaction? 
   
   Because this is how I got it failing:
   
   * Activate venv
   * Run `pip install -e`
   * Remove all files generated in AIRFLOW_HOME
   * run ``airflow` 
   
   Maybe that was somehow my environmental problem ? Maybe you can double check it by reverting :) ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org