[GitHub] [airflow] leonsmith edited a comment on issue #15687: celery worker can't read dags from db

[GitBox <gi...@apache.org>]

leonsmith edited a comment on issue #15687:
URL: https://github.com/apache/airflow/issues/15687#issuecomment-929573349


   So after tweaking a couple of things from my work in progress merge request (to link the pickle to the dag) this is still throwing an error when the task is picked up from the worker.
   
   ```python
   ModuleNotFoundError: No module named 'unusual_prefix_d83562dd8d4afa96cb7d254b46b193a7106a58ef_test_logging'
     File "airflow/executors/celery_executor.py", line 121, in _execute_in_fork
       args.func(args)
     File "airflow/cli/cli_parser.py", line 48, in command
       return func(*args, **kwargs)
     File "airflow/utils/cli.py", line 92, in wrapper
       return f(*args, **kwargs)
     File "airflow/cli/commands/task_command.py", line 274, in task_run
       dag = get_dag_by_pickle(args.pickle)
     File "airflow/utils/session.py", line 70, in wrapper
       return func(*args, session=session, **kwargs)
     File "airflow/utils/cli.py", line 221, in get_dag_by_pickle
       dag_pickle = session.query(DagPickle).filter(DagPickle.id == pickle_id).first()
     File "sqlalchemy/orm/query.py", line 3429, in first
       ret = list(self[0:1])
     File "sqlalchemy/orm/query.py", line 3203, in __getitem__
       return list(res)
     File "sqlalchemy/orm/loading.py", line 100, in instances
       cursor.close()
     File "sqlalchemy/util/langhelpers.py", line 68, in __exit__
       compat.raise_(
     File "sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "sqlalchemy/orm/loading.py", line 80, in instances
       rows = [proc(row) for row in fetch]
     File "sqlalchemy/orm/loading.py", line 80, in <listcomp>
       rows = [proc(row) for row in fetch]
     File "sqlalchemy/orm/loading.py", line 579, in _instance
       _populate_full(
     File "sqlalchemy/orm/loading.py", line 725, in _populate_full
       dict_[key] = getter(row)
     File "sqlalchemy/sql/sqltypes.py", line 1723, in process
       return loads(value)
     File "dill/_dill.py", line 327, in loads
       return load(file, ignore, **kwds)
     File "dill/_dill.py", line 313, in load
       return Unpickler(file, ignore=ignore, **kwds).load()
     File "dill/_dill.py", line 525, in load
       obj = StockUnpickler.load(self)
     File "dill/_dill.py", line 515, in find_class
       return StockUnpickler.find_class(self, module, name)```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org