You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "tatiana (via GitHub)" <gi...@apache.org> on 2024/05/07 12:03:53 UTC
[PR] Fix `DataprocCreateBatchOperator` with `result_retry` raises `AttributeError` [airflow]
tatiana opened a new pull request, #39462:
URL: https://github.com/apache/airflow/pull/39462
Closes: #39394
When trying to run the `example_dataproc_batch.py` DAG locally, some of the tasks failed, including:
```
create_batch_2 = DataprocCreateBatchOperator(
task_id=create_batch_2,
project_id=PROJECT_ID,
region=REGION,
batch=BATCH_CONFIG,
batch_id=BATCH_ID_2,
result_retry=AsyncRetry(maximum=10.0, initial=10.0, multiplier=1.0),
)
```
With the error:
```
Traceback (most recent call last):
File /usr/local/lib/python3.11/site-packages/airflow/models/taskinstance.py, line 434, in _execute_task
result = execute_callable(context=context, **execute_callable_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File /usr/local/lib/python3.11/site-packages/airflow/providers/google/cloud/operators/dataproc.py, line 2537, in execute
result = hook.wait_for_operation(
^^^^^^^^^^^^^^^^^^^^^^^^
File /usr/local/lib/python3.11/site-packages/airflow/providers/google/cloud/hooks/dataproc.py, line 266, in wait_for_operation
error = operation.exception(timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File /usr/local/lib/python3.11/site-packages/google/api_core/future/polling.py, line 282, in exception
self._blocking_poll(timeout=timeout)
File /usr/local/lib/python3.11/site-packages/google/api_core/future/polling.py, line 137, in _blocking_poll
polling(self._done_or_raise)(retry=retry)
File /usr/local/lib/python3.11/site-packages/google/api_core/retry.py, line 372, in retry_wrapped_func
return retry_target(
^^^^^^^^^^^^^
File /usr/local/lib/python3.11/site-packages/google/api_core/retry.py, line 207, in retry_target
result = target()
^^^^^^^^
File /usr/local/lib/python3.11/site-packages/google/api_core/future/polling.py, line 119, in _done_or_raise
if not self.done(retry=retry):
^^^^^^^^^^^^^^^^^^^^^^
File /usr/local/lib/python3.11/site-packages/google/api_core/operation.py, line 174, in done
self._refresh_and_update(retry)
File /usr/local/lib/python3.11/site-packages/google/api_core/operation.py, line 161, in _refresh_and_update
if not self._operation.done:
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'coroutine' object has no attribute 'done'
```
This was due to an issue in the dependecy `google-api-core==2.18.0`. The DAG works if it is run with 2.17.0 or 2.19.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix `DataprocCreateBatchOperator` with `result_retry` raises `AttributeError` [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal merged PR #39462:
URL: https://github.com/apache/airflow/pull/39462
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org