You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/26 15:22:30 UTC
[GitHub] [airflow] nathadfield opened a new issue #11856: BigQueryInsertJobOperator creates invalid job_ids
nathadfield opened a new issue #11856:
URL: https://github.com/apache/airflow/issues/11856
Apache Airflow version: 1.10.*
BackPort Packages version: 2020.10.29rc1
What happened:
BigQueryInsertJobOperator tries to start a BigQuery job by specifying a job_id which is a combination of dag_id, task_id, execution_date and an additional uniqueness_suffix.
https://github.com/PolideaInternal/airflow/blob/e5307990adb49a5a1c1c88a029940682efee0f9e/airflow/providers/google/cloud/operators/bigquery.py#L2073
However, it would appear that the regex pattern that is attempting to remove the `+` character is not correct as this still remain in `job_id` after `re.sub` is applied.
https://github.com/PolideaInternal/airflow/blob/e5307990adb49a5a1c1c88a029940682efee0f9e/airflow/providers/google/cloud/operators/bigquery.py#L2073
`Invalid job ID "airflow_my_test_dag_v1_test_2020_10_01T00_00_00+00_00_27f69de5cd17a012f3c89d414a70fe64". Job IDs must be alphanumeric (plus underscores and dashes) and must be at most 1024 characters long.
`
How to reproduce it:
- Create a DAG with the name my-test-dag-v1.
- Run any query against BigQuery using this operator
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek closed issue #11856: BigQueryInsertJobOperator creates invalid job_ids
Posted by GitBox <gi...@apache.org>.
turbaszek closed issue #11856:
URL: https://github.com/apache/airflow/issues/11856
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org