You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "tatiana (via GitHub)" <gi...@apache.org> on 2024/05/07 12:03:53 UTC

[PR] Fix `DataprocCreateBatchOperator` with `result_retry` raises `AttributeError` [airflow]

tatiana opened a new pull request, #39462:
URL: https://github.com/apache/airflow/pull/39462

   Closes: #39394
       
   When trying to run the `example_dataproc_batch.py` DAG locally, some of the tasks failed, including:
       
   ```
           create_batch_2 = DataprocCreateBatchOperator(
               task_id=create_batch_2,
               project_id=PROJECT_ID,
               region=REGION,
               batch=BATCH_CONFIG,
               batch_id=BATCH_ID_2,
               result_retry=AsyncRetry(maximum=10.0, initial=10.0, multiplier=1.0),
           )
   ```
       
   With the error:
   ```
       Traceback (most recent call last):
         File /usr/local/lib/python3.11/site-packages/airflow/models/taskinstance.py, line 434, in _execute_task
           result = execute_callable(context=context, **execute_callable_kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         File /usr/local/lib/python3.11/site-packages/airflow/providers/google/cloud/operators/dataproc.py, line 2537, in execute
           result = hook.wait_for_operation(
                    ^^^^^^^^^^^^^^^^^^^^^^^^
         File /usr/local/lib/python3.11/site-packages/airflow/providers/google/cloud/hooks/dataproc.py, line 266, in wait_for_operation
           error = operation.exception(timeout=timeout)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         File /usr/local/lib/python3.11/site-packages/google/api_core/future/polling.py, line 282, in exception
           self._blocking_poll(timeout=timeout)
         File /usr/local/lib/python3.11/site-packages/google/api_core/future/polling.py, line 137, in _blocking_poll
           polling(self._done_or_raise)(retry=retry)
         File /usr/local/lib/python3.11/site-packages/google/api_core/retry.py, line 372, in retry_wrapped_func
           return retry_target(
                  ^^^^^^^^^^^^^
         File /usr/local/lib/python3.11/site-packages/google/api_core/retry.py, line 207, in retry_target
           result = target()
                    ^^^^^^^^
         File /usr/local/lib/python3.11/site-packages/google/api_core/future/polling.py, line 119, in _done_or_raise
           if not self.done(retry=retry):
                  ^^^^^^^^^^^^^^^^^^^^^^
         File /usr/local/lib/python3.11/site-packages/google/api_core/operation.py, line 174, in done
           self._refresh_and_update(retry)
         File /usr/local/lib/python3.11/site-packages/google/api_core/operation.py, line 161, in _refresh_and_update
           if not self._operation.done:
                  ^^^^^^^^^^^^^^^^^^^^
       AttributeError: 'coroutine' object has no attribute 'done'
   ```
       
   This was due to an issue in the dependecy `google-api-core==2.18.0`. The DAG works if it is run with 2.17.0 or 2.19.0.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix `DataprocCreateBatchOperator` with `result_retry` raises `AttributeError` [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal merged PR #39462:
URL: https://github.com/apache/airflow/pull/39462


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org