[I] DAG with BigQueryInsertJobOperator not generating job_id when triggered by asset event in Airflow 3 [airflow]

["txu-7777 (via GitHub)" <gi...@apache.org>]

txu-7777 opened a new issue, #55073:
URL: https://github.com/apache/airflow/issues/55073

   ### Apache Airflow Provider(s)
   
   google
   
   ### Versions of Apache Airflow Providers
   
   apache-airflow-providers-google==17.1.0
   
   
   ### Apache Airflow version
   
   3.0.5
   
   ### Operating System
   
   Mac
   
   ### Deployment
   
   Astronomer
   
   ### Deployment details
   
   Local
   
   ### What happened
   
   I have 2 DAGs. The 2nd DAG is scheduled based on an asset event from the first DAG. The 2nd DAG has a BigQueryInsertJobOperator - however due to some changes in Airflow 3 the `logical_date` is not passed through the context anymore with asset triggered dagruns. 
   
   Within the BigQueryInsertJobOperator there is a block of code in the `execute` method that auto generates the `job_id` based on the `logical_date`. Due to `logical_date` no longer being present, the operator is breaking.
   
   ```
   def execute(self, context: Any):
           hook = BigQueryHook(
               gcp_conn_id=self.gcp_conn_id,
               impersonation_chain=self.impersonation_chain,
           )
           self.hook = hook
           if self.project_id is None:
               self.project_id = hook.project_id
   
           self.job_id = hook.generate_job_id(
               job_id=self.job_id,
               dag_id=self.dag_id,
               task_id=self.task_id,
               logical_date=context["logical_date"],
               configuration=self.configuration,
               force_rerun=self.force_rerun,
           )
           ...
   ```
    
   
   ### What you think should happen instead
   
   BigQueryInsertJobOperator should be able to generate a job_id regardless of whether it is triggered in a scheduled / manually triggered DAG or through an asset event.
   
   ### How to reproduce
   
   Create 2 DAGs. Add a task in the DAG triggered by an asset that checks for logical_date=context["logical_date"]. When using logical_date=context.get("logical_date"), there is no error but the value returned is None. Interestingly, this is not the case in Airflow 2, where logical_date is returned as part of the context, and reflects the run_id of the dag. 
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] DAG with BigQueryInsertJobOperator not generating job_id when triggered by asset event in Airflow 3 [airflow]

["kaxil (via GitHub)" <gi...@apache.org>]

kaxil closed issue #55073: DAG with BigQueryInsertJobOperator not generating job_id when triggered by asset event in Airflow 3
URL: https://github.com/apache/airflow/issues/55073


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] DAG with BigQueryInsertJobOperator not generating job_id when triggered by asset event in Airflow 3 [airflow]

["boring-cyborg[bot] (via GitHub)" <gi...@apache.org>]

boring-cyborg[bot] commented on issue #55073:
URL: https://github.com/apache/airflow/issues/55073#issuecomment-3237469585

   Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org