You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "anavrotski (via GitHub)" <gi...@apache.org> on 2026/02/10 09:26:13 UTC
[I] Error cloning repository [airflow]
anavrotski opened a new issue, #61716:
URL: https://github.com/apache/airflow/issues/61716
### Apache Airflow version
3.1.7
### If "Other Airflow 3 version" selected, which one?
3.1.6
### What happened?
We are observing a sporadic issue (**so I don't know how to reproduce**) with GitDagBundles when cloning DAG from _self-hosted GitLab_ (see log below), and this error obviously is not raising an alert by calling DAG callback, so it is "hidden" in some sense.
```
[2026-02-10 06:05:02] WARNING - Repository clone/open failed, cleaning up and retrying bundle_name=etl-dags-bundle version=09b6... bare_repo_path=PosixPath('/tmp/airflow/dag_bundles/.../bare') repo_path=PosixPath('/tmp/airflow/dag_bundles/.../versions/09b6...') versions_path=PosixPath('/tmp/airflow/dag_bundles/etl-dags-bundle/versions') git_conn_id=git_conn submodules= exc=InvalidGitRepositoryError('/tmp/airflow/dag_bundles/.../versions/09b6...') source=airflow.providers.git.bundles.git loc=git.py:180
[2026-02-10 06:05:02] INFO - Cloning repository bundle_name=... version=09b6... bare_repo_path=PosixPath('/tmp/airflow/dag_bundles/.../bare') repo_path=PosixPath('/tmp/airflow/dag_bundles/.../versions/09b6...') versions_path=PosixPath('/tmp/airflow/dag_bundles/.../versions') git_conn_id=git_conn submodules= source=airflow.providers.git.bundles.git loc=git.py:166
[2026-02-10 06:05:02] WARNING - Repository clone/open failed, cleaning up and retrying bundle_name=... version=09b6... bare_repo_path=PosixPath('/tmp/airflow/dag_bundles/.../bare') repo_path=PosixPath('/tmp/airflow/dag_bundles/.../versions/09b6...') versions_path=PosixPath('/tmp/airflow/dag_bundles/.../versions') git_conn_id=git_conn submodules= exc=GitCommandError(['git', 'clone', '-v', '--', '/tmp/airflow/dag_bundles/.../bare', '/tmp/airflow/dag_bundles/.../versions/09b6...'], 128, b"fatal: destination path '/tmp/airflow/dag_bundles/.../versions/09b6...' already exists and is not an empty directory.\n") source=airflow.providers.git.bundles.git loc=git.py:180
[2026-02-10 06:05:02] ERROR - Top level error source=task loc=task_runner.py:1562
RuntimeError: Error cloning repository
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/task_runner.py", line 1550 in main
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/task_runner.py", line 732 in startup
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/task_runner.py", line 618 in parse
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/git/bundles/git.py", line 155 in initialize
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/git/bundles/git.py", line 126 in _initialize
GitCommandError: Cmd('git') failed due to: exit code(128)
cmdline: git clone -v -- /tmp/airflow/dag_bundles/.../bare /tmp/airflow/dag_bundles/.../versions/09b6...
stderr: 'fatal: destination path '/tmp/airflow/dag_bundles/.../versions/09b6...' already exists and is not an empty directory.
'
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/git/bundles/git.py", line 124 in _initialize
File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 338 in wrapped_f
File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 477 in __call__
File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 378 in iter
File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 420 in exc_check
File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 187 in reraise
File "/usr/python/lib/python3.12/concurrent/futures/_base.py", line 449 in result
File "/usr/python/lib/python3.12/concurrent/futures/_base.py", line 401 in __get_result
File "/home/airflow/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 480 in __call__
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/git/bundles/git.py", line 169 in _clone_repo_if_required
File "/home/airflow/.local/lib/python3.12/site-packages/git/repo/base.py", line 1538 in clone_from
File "/home/airflow/.local/lib/python3.12/site-packages/git/repo/base.py", line 1409 in _clone
File "/home/airflow/.local/lib/python3.12/site-packages/git/util.py", line 512 in finalize_process
File "/home/airflow/.local/lib/python3.12/site-packages/git/cmd.py", line 419 in wait
[2026-02-10 06:05:02] WARNING - Process exited abnormally exit_code=1 source=task
```
### What you think should happen instead?
I think that in [this Git provider's file](https://github.com/apache/airflow/blob/main/providers/git/src/airflow/providers/git/bundles/git.py), the `@retry` decorator from `tenacity` should be supplemented with parameters such as `wait`, `wait_exponential`, or `wait_fixed` to introduce a small delay between retries. As you can see in the log above, the current retries occur less than 1 second apart, which causes an issue.
### How to reproduce
I do not know.
### Operating System
linux
### Versions of Apache Airflow Providers
git provider v 0.2.0
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
official helm chart 1.18
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Error cloning repository [airflow]
Posted by "Arunodoy18 (via GitHub)" <gi...@apache.org>.
Arunodoy18 commented on issue #61716:
URL: https://github.com/apache/airflow/issues/61716#issuecomment-3901596708
Thanks for the detailed investigation!
From the discussion and logs, this does look like a race between cleanup and clone operations across concurrent runs.
I’m interested in working on this. My initial thought is to explore adding version-level locking around clone/open operations to ensure only one process prepares a bundle version directory at a time, instead of relying on retries.
If that direction sounds reasonable, I’d be happy to prototype a fix and add concurrency tests.
Please let me know if you have any preferences on approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Error cloning repository [airflow]
Posted by "dominikhei (via GitHub)" <gi...@apache.org>.
dominikhei commented on issue #61716:
URL: https://github.com/apache/airflow/issues/61716#issuecomment-3876758210
This looks like some kind of race condition. Having a retry delay is an option, I think if we could implement some kind of lock this would be even better. I can have a look at it if deemed necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Error cloning repository [airflow]
Posted by "dominikhei (via GitHub)" <gi...@apache.org>.
dominikhei commented on issue #61716:
URL: https://github.com/apache/airflow/issues/61716#issuecomment-3876810881
> > This looks like some kind of race condition. Having a retry delay is an option, I think if we could implement some kind of lock this would be even better. I can have a look at it if deemed necessary.
>
> Not sure why, but I'm seeing this "WARNING - Repository clone/open failed, ..." in almost every DAG run.
`WARNING - Repository clone/open failed, cleaning up and retrying` The cleanup does not work as we have the root error: `stderr: 'fatal: destination path '/tmp/airflow/dag_bundles/.../versions/09b6...' already exists and is not an empty directory`. I am not sure how the git operator works, will have a look at it, but to me it looks like the cloning is tried before everything is cleaned up and thus either incomplete or not synchronized with the retry.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Error cloning repository [airflow]
Posted by "Arunodoy18 (via GitHub)" <gi...@apache.org>.
Arunodoy18 commented on issue #61716:
URL: https://github.com/apache/airflow/issues/61716#issuecomment-3901614423
Any updates
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Error cloning repository [airflow]
Posted by "ephraimbuddy (via GitHub)" <gi...@apache.org>.
ephraimbuddy commented on issue #61716:
URL: https://github.com/apache/airflow/issues/61716#issuecomment-3901810638
> > This looks like some kind of race condition. Having a retry delay is an option, I think if we could implement some kind of lock this would be even better. I can have a look at it if deemed necessary.
>
> Not sure why, but I'm seeing this "WARNING - Repository clone/open failed, ..." in almost every DAG run.
The particular case you mentioned is being addressed here: https://github.com/apache/airflow/pull/61847
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Error cloning repository [airflow]
Posted by "anavrotski (via GitHub)" <gi...@apache.org>.
anavrotski commented on issue #61716:
URL: https://github.com/apache/airflow/issues/61716#issuecomment-3876776376
> This looks like some kind of race condition. Having a retry delay is an option, I think if we could implement some kind of lock this would be even better. I can have a look at it if deemed necessary.
Not sure why, but I'm seeing this "WARNING - Repository clone/open failed, ..." in almost every DAG run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org