[GitHub] [airflow] CYarros10 commented on issue #30007: BeamRunPythonPipelineOperator doesn't push xcom until Pipeline completes, leaving DataflowSensors worthless

["CYarros10 (via GitHub)" <gi...@apache.org>]

CYarros10 commented on issue #30007:
URL: https://github.com/apache/airflow/issues/30007#issuecomment-1549832192

   Have you tested this? the documentation is inconsistent and not reliable to solely go off of. for example, documentation you referenced states:
   
   `    wait_for_python_job_dataflow_runner_async_done = DataflowJobStatusSensor(
           task_id="wait-for-python-job-async-done",
           job_id="{{task_instance.xcom_pull('start_python_job_dataflow_runner_async')['dataflow_job_id']}}",
           expected_statuses={DataflowJobStatus.JOB_STATE_DONE},
           project_id=GCP_PROJECT_ID,
           location='us-central1',
       )`
       
       and dataflow_job_id is not actually in the xcom


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org