You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/03/19 11:20:27 UTC

[GitHub] [airflow] JCoder01 opened a new issue #14892: Scheduler doesn't make command generic

JCoder01 opened a new issue #14892:
URL: https://github.com/apache/airflow/issues/14892


   
   
   **Apache Airflow version**: 2.0.1
   
   
   **Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
   
   **Environment**: EKS, On Prem
   
   - **Cloud provider or hardware configuration**:
   - **OS** (e.g. from /etc/os-release):
   - **Kernel** (e.g. `uname -a`):
   - **Install tools**:
   - **Others**:
   
   **What happened**:
   
   Scheduler includes full dag `fileloc` rather than generic `DAGS_FOLDER/path/to/dag.py`
   
   
   **What you expected to happen**:
   
   `--subdir` in task instance is genericized to allow running tasks on workers where `core.dags_folder` is different than scheduler.
   
   **How to reproduce it**:
   Run scheduler with Celery executor and observe log output.
   
   
   **Anything else we need to know**:
   
   In TaskInstance there is some logic to handle this,
   https://github.com/apache/airflow/blob/master/airflow/models/taskinstance.py#L405:411
    
   However, it seems in the latest scheduler, it just passes `dag.fileloc` from the metadata db. 
   https://github.com/apache/airflow/blob/master/airflow/jobs/scheduler_job.py#L1103
   
   Should the write to the db genericize the command like in TaskInstance?
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] mik-laj commented on issue #14892: Scheduler doesn't make command generic

Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #14892:
URL: https://github.com/apache/airflow/issues/14892#issuecomment-802782260


   Duplicate:  https://github.com/apache/airflow/issues/8061


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] mik-laj closed issue #14892: Scheduler doesn't make command generic

Posted by GitBox <gi...@apache.org>.
mik-laj closed issue #14892:
URL: https://github.com/apache/airflow/issues/14892


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org