You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/24 05:18:00 UTC

[GitHub] [airflow] downeyh opened a new issue #13298: multiporcessing in task throwd daemonic processes are not allowed to have children

downeyh opened a new issue #13298:
URL: https://github.com/apache/airflow/issues/13298


   **Apache Airflow version**:
   2.0
   
   **Environment**:
   Python3.6
   
   
   **What happened**:
   I used mutiprocessing Process in PythonOperator callable function, but it reports an error.
   
     File "/home/xxx/airflow/dags/Gen/SecData.py", line 88, in gen
       w.start()
     File "/usr/lib64/python3.6/multiprocessing/process.py", line 103, in start
       'daemonic processes are not allowed to have children'
   AssertionError: daemonic processes are not allowed to have children
   
   **What you expected to happen**:
   In airflow1.10, I could use multiprocessing in task function, it works well,
   I'm not sure if is't a bug or there are other ways to use multiporcessing in PythonOperator task function.
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] downeyh commented on issue #13298: multiporcessing in task throwd daemonic processes are not allowed to have children

Posted by GitBox <gi...@apache.org>.
downeyh commented on issue #13298:
URL: https://github.com/apache/airflow/issues/13298#issuecomment-750763717


   export PYTHONOPTIMIZE=1
   solved the problem


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] downeyh closed issue #13298: multiporcessing in task throwd daemonic processes are not allowed to have children

Posted by GitBox <gi...@apache.org>.
downeyh closed issue #13298:
URL: https://github.com/apache/airflow/issues/13298


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] raphaelauv commented on issue #13298: multiporcessing in task throwd daemonic processes are not allowed to have children

Posted by GitBox <gi...@apache.org>.
raphaelauv commented on issue #13298:
URL: https://github.com/apache/airflow/issues/13298#issuecomment-803894927


   explanation there -> https://stackoverflow.com/a/57493983


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #13298: multiporcessing in task throwd daemonic processes are not allowed to have children

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #13298:
URL: https://github.com/apache/airflow/issues/13298#issuecomment-750745798


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org