You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/26 16:01:24 UTC
[GitHub] [airflow] nathadfield opened a new pull request #11857: BigQuery job_id fix for `BigQueryInsertJobOperator`
nathadfield opened a new pull request #11857:
URL: https://github.com/apache/airflow/pull/11857
Ensures that the `:`, `-`, `+` and `.` characters are removed from the string that is built to create a BigQuery jobid.
Closes: #11856
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on pull request #11857: BigQuery job_id fix for `BigQueryInsertJobOperator`
Posted by GitBox <gi...@apache.org>.
turbaszek commented on pull request #11857:
URL: https://github.com/apache/airflow/pull/11857#issuecomment-717759618
@nathadfield please rebase due to #11892 and run `pre-commit run black --all-files` - this should fix the static check issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nathadfield commented on a change in pull request #11857: BigQuery job_id fix for `BigQueryInsertJobOperator`
Posted by GitBox <gi...@apache.org>.
nathadfield commented on a change in pull request #11857:
URL: https://github.com/apache/airflow/pull/11857#discussion_r512822744
##########
File path: airflow/providers/google/cloud/operators/bigquery.py
##########
@@ -2062,7 +2062,7 @@ def _job_id(self, context):
exec_date = context['execution_date'].isoformat()
job_id = f"airflow_{self.dag_id}_{self.task_id}_{exec_date}_{uniqueness_suffix}"
- return re.sub(r"\:|-|\+\.", "_", job_id)
+ return re.sub(r"[:\-+.]", "_", job_id)
Review comment:
@turbaszek Test added.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #11857: BigQuery job_id fix for `BigQueryInsertJobOperator`
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11857:
URL: https://github.com/apache/airflow/pull/11857#issuecomment-717493703
[The Workflow run](https://github.com/apache/airflow/actions/runs/332086602) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #11857: BigQuery job_id fix for `BigQueryInsertJobOperator`
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11857:
URL: https://github.com/apache/airflow/pull/11857#issuecomment-717494005
[The Workflow run](https://github.com/apache/airflow/actions/runs/332089055) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #11857: BigQuery job_id fix for `BigQueryInsertJobOperator`
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11857:
URL: https://github.com/apache/airflow/pull/11857#issuecomment-717494085
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek merged pull request #11857: BigQuery job_id fix for `BigQueryInsertJobOperator`
Posted by GitBox <gi...@apache.org>.
turbaszek merged pull request #11857:
URL: https://github.com/apache/airflow/pull/11857
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #11857: BigQuery job_id fix for `BigQueryInsertJobOperator`
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #11857:
URL: https://github.com/apache/airflow/pull/11857#discussion_r512084684
##########
File path: airflow/providers/google/cloud/operators/bigquery.py
##########
@@ -2062,7 +2062,7 @@ def _job_id(self, context):
exec_date = context['execution_date'].isoformat()
job_id = f"airflow_{self.dag_id}_{self.task_id}_{exec_date}_{uniqueness_suffix}"
- return re.sub(r"\:|-|\+\.", "_", job_id)
+ return re.sub(r"[:\-+.]", "_", job_id)
Review comment:
Can you add test for this? It will help us avoid regression. Probably using parametrize will be good idea
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nathadfield commented on pull request #11857: BigQuery job_id fix for `BigQueryInsertJobOperator`
Posted by GitBox <gi...@apache.org>.
nathadfield commented on pull request #11857:
URL: https://github.com/apache/airflow/pull/11857#issuecomment-717812294
@turbaszek Thanks! Tests are rerunning again now
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #11857: BigQuery job_id fix for `BigQueryInsertJobOperator`
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11857:
URL: https://github.com/apache/airflow/pull/11857#issuecomment-717494234
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nathadfield commented on pull request #11857: BigQuery job_id fix for `BigQueryInsertJobOperator`
Posted by GitBox <gi...@apache.org>.
nathadfield commented on pull request #11857:
URL: https://github.com/apache/airflow/pull/11857#issuecomment-717843617
Great success!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org