You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/06/25 06:01:14 UTC

[GitHub] [airflow] erica-lee edited a comment on issue #10876: Serialized DAG with BigQueryOperator doesn't load for DAG specific UI views

erica-lee edited a comment on issue #10876:
URL: https://github.com/apache/airflow/issues/10876#issuecomment-865662451


   I have same error on GCP Composer.(Apache Airflow version: 1.10.15)
   
   It is caused by data type of serialized operator extra links.
   ```python
       # https://github.com/apache/airflow/blob/v1-10-stable/airflow/serialization/serialized_objects.py#L447
       @classmethod
       def _deserialize_operator_extra_links(
           cls,
           encoded_op_links
       ):
           # type: (list) -> Dict[str, BaseOperatorLink]
           ...
   ```
   
   If you use like this, it will work:
   ```python
   # example code: https://github.com/apache/airflow/blob/v2-0-stable/airflow/providers/google/cloud/example_dags/example_bigquery_queries.py
   import attr
   from airflow.contrib.operators.bigquery_operator import BigQueryConsoleLink, BigQueryConsoleIndexableLink
   from airflow.contrib.operators.bigquery_operator import BigQueryOperator
   
   
   @attr.s(auto_attribs=True)
   class BigQueryConsoleIndexableLink2(BigQueryConsoleIndexableLink):
       index: int = attr.ib()
       
   
   class BigQueryOperator2(BigQueryOperator):
       @property
       def operator_extra_links(self):
           """
           Return operator extra links
           """
           if isinstance(self.sql, str):
               return (
                   BigQueryConsoleLink(),
               )
           # * important * type: (list)
           return (
               [BigQueryConsoleIndexableLink2(i) for i, _ in enumerate(self.sql)]
           )
   
   with models.DAG('THIS_IS_DAG_ID', default_args={...}, user_defined_macros={"DATASET": DATASET_NAME, "TABLE": TABLE_1}, ...) as dag:
       bigquery_execute_multi_query = BigQueryOperator2(
           task_id="execute_multi_query_debug_dag",
           sql=[
               f"SELECT * FROM {DATASET_NAME}.{TABLE_2}",
               f"SELECT COUNT(*) FROM {DATASET_NAME}.{TABLE_2}",
           ],
           use_legacy_sql=False,
           location=location,
       )
       bigquery_execute_multi_query
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org