[jira] [Commented] (AIRFLOW-154) Handle xcom parameters when running a single task

["Chris Riccomini (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15293467#comment-15293467 ] 

Chris Riccomini commented on AIRFLOW-154:
-----------------------------------------

{quote}Is it possible to specify any parameters from the commandline that would substitute the xcom pull from task 1?{quote}

I don't believe this is possible with {{airflow run}} right now:

{noformat}
usage: airflow run [-h] [-sd SUBDIR] [-m] [-f] [--pool POOL] [-l] [-i] [-I]
                   [--ship_dag] [-p PICKLE]
                   dag_id task_id execution_date

positional arguments:
  dag_id                The id of the dag
  task_id               The id of the task
  execution_date        The execution date of the DAG

optional arguments:
  -h, --help            show this help message and exit
  -sd SUBDIR, --subdir SUBDIR
                        File location or directory from which to look for the
                        dag
  -m, --mark_success    Mark jobs as succeeded without running them
  -f, --force           Force a run regardless or previous success
  --pool POOL           Resource pool to use
  -l, --local           Run the task using the LocalExecutor
  -i, --ignore_dependencies
                        Ignore upstream and depends_on_past dependencies
  -I, --ignore_depends_on_past
                        Ignore depends_on_past dependencies (but respect
                        upstream dependencies)
  --ship_dag            Pickles (serializes) the DAG and ships it to the
                        worker
  -p PICKLE, --pickle PICKLE
                        Serialized pickle object of the entire dag (used
                        internally)
{noformat}

{quote} What happens with the parameters pushed in task 2? Are they left behind somewhere?{quote}

I believe that they're pushed like any other task. The task will run with an execution date. That date will determine the ordering of the xcom variables relative to other xcom variables from the same task instance.

> Handle xcom parameters when running a single task
> -------------------------------------------------
>
>                 Key: AIRFLOW-154
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-154
>             Project: Apache Airflow
>          Issue Type: Wish
>          Components: xcom
>            Reporter: Timo
>            Priority: Minor
>
> What would be the correct way to handle xcom parameters when running a single task in a dag. Simplified example:
> task 1:
>   - Push parameter
> task 2:
>   - Pull parameter from task 1
>   - Push parameter
> task 3:
>   - Pull parameter from task 2
> I would like to only run task 2 manually.
>   1. Is it possible to specify any parameters from the commandline that would substitute the xcom pull from task 1?
>   2. What happens with the parameters pushed in task 2? Are they left behind somewhere?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)