[arrow] branch master updated: ARROW-16684: [CI][Archery] Add retry mechanism to git fetch on GitError failures

[ks...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

kszucs pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
     new 01d8485d17 ARROW-16684: [CI][Archery] Add retry mechanism to git fetch on GitError failures
01d8485d17 is described below

commit 01d8485d17adacbe75dac2a9b97c34dc28ca31f5
Author: Raúl Cumplido <ra...@gmail.com>
AuthorDate: Mon May 30 16:31:55 2022 +0200

    ARROW-16684: [CI][Archery] Add retry mechanism to git fetch on GitError failures
    
    There has been some jobs failing on Archery when performing a git fetch with the following error:
    ```
      File "/home/runner/work/crossbow/crossbow/arrow/dev/archery/archery/crossbow/cli.py", line 238, in latest_prefix
        queue.fetch()
      File "/home/runner/work/crossbow/crossbow/arrow/dev/archery/archery/crossbow/core.py", line 271, in fetch
        self.origin.fetch([refspec])
      File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pygit2/remote.py", line 146, in fetch
        payload.check_error(err)
      File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pygit2/callbacks.py", line 93, in check_error
        check_error(error_code)
      File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pygit2/errors.py", line 65, in check_error
        raise GitError(message)
    _pygit2.GitError: SSL error: received early EOF
    Error: Process completed with exit code 1.
    ```
    Retrying the job usually works. The idea of this PR is to add a default retry mechanism on fetching the remote branches on archery.
    
    Closes #13258 from raulcd/ARROW-16684
    
    Authored-by: Raúl Cumplido <ra...@gmail.com>
    Signed-off-by: Krisztián Szűcs <sz...@gmail.com>
---
 dev/archery/archery/crossbow/core.py | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/dev/archery/archery/crossbow/core.py b/dev/archery/archery/crossbow/core.py
index 55cb87e77f..460e3f65a2 100644
--- a/dev/archery/archery/crossbow/core.py
+++ b/dev/archery/archery/crossbow/core.py
@@ -43,8 +43,10 @@ try:
     import pygit2
 except ImportError:
     PygitRemoteCallbacks = object
+    GitError = Exception
 else:
     PygitRemoteCallbacks = pygit2.RemoteCallbacks
+    GitError = pygit2.GitError
 
 from ..utils.source import ArrowSources
 
@@ -266,9 +268,18 @@ class Repo:
                                 "Crossbow: {}".format(remote.url))
         return remote
 
-    def fetch(self):
+    def fetch(self, retry=3):
         refspec = '+refs/heads/*:refs/remotes/origin/*'
-        self.origin.fetch([refspec])
+        attempt = 1
+        while True:
+            try:
+                self.origin.fetch([refspec])
+                break
+            except GitError as e:
+                if retry and attempt < retry:
+                    attempt += 1
+                else:
+                    raise e
 
     def push(self, refs=None, github_token=None):
         github_token = github_token or self.github_token