You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@devlake.apache.org by "gustavobini (via GitHub)" <gi...@apache.org> on 2024/05/14 11:44:14 UTC
[I] [Bug][GitHub] GitExtractor fails on empty repository due to shallow clone [incubator-devlake]
gustavobini opened a new issue, #7468:
URL: https://github.com/apache/incubator-devlake/issues/7468
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar issues.
### What happened
When syncing GitHub, a few repositories had errors as they were cloned with `--bare --shallow-since=2024-03-09T13:24:51Z` and the last commit on these repositories was over 3 months before the date.
### What do you expect to happen
I expected a warning to be shown and not an error, as it's expected for the extractor to not do anything if there's no repository.
### How to reproduce
Set the sync policy to only include the last day and sync a GitHub repository with changes before that date.
### Anything else
```
time="2024-05-14 00:04:15" level=info msg=" [pipeline service] [pipeline #773] [task #60301] executing subtask Clone Git Repo"
time="2024-05-14 00:04:15" level=debug msg=" [pipeline service] [pipeline #773] [task #60301] [Clone Git Repo] [gitcli] git [clone https://git:<redacted>@github.com/org/repo.git /tmp/gitextractor1800277984 --bare --progress --shallow-since=2024-03-09T13:24:51Z]"
time="2024-05-14 00:04:16" level=error msg=" [pipeline service] [pipeline #773] [task #60301] [Clone Git Repo] [gitcli] git exited with error\nstderr: Cloning into bare repository '/tmp/gitextractor1800277984'...\nstderr: fatal: error processing shallow info: 4\n\n\tcaused by: exit status 128"
time="2024-05-14 00:04:16" level=info msg=" [pipeline service] [pipeline #773] [task #60301] finished step: 1 / 4"
time="2024-05-14 00:04:16" level=info msg=" [pipeline service] [pipeline #773] [task #60301] executing subtask Collect Commits"
time="2024-05-14 00:04:16" level=error msg=" [pipeline service] [pipeline #773] [task #60301] run task failed with panic\n\tcaused by: run task failed with panic (github.com/apache/incubator-devlake/plugins/gitextractor/tasks.getGitRepo:87)\n\tWraps: (2) git repo is empty, please check subtask: clone repo\n\tError types: (1) *hintdetail.withDetail (2) *errors.errorString"
```
### Version
v1.0.0-beta7
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitHub] GitExtractor fails on empty repository due to shallow clone [incubator-devlake]
Posted by "gustavobini (via GitHub)" <gi...@apache.org>.
gustavobini commented on issue #7468:
URL: https://github.com/apache/incubator-devlake/issues/7468#issuecomment-2115080947
@klesh FYI the fix on #7475 causes this in some repos:
```
time="2024-05-16 11:57:30" level=debug msg=" [pipeline service] [pipeline #797] [task #61700] [Clone Git Repo] [gitcli] git [clone https://git:<token>@github.com/<org>/<repo>.git /tmp/gitextractor1830545665 --bare --progress --shallow-since=2024-05-16T00:02:53Z]"
time="2024-05-16 11:57:38" level=error msg=" [pipeline service] [pipeline #797] [task #61700] subtask Clone Git Repo ended unexpectedly\n\tWraps: (2) failed to deepen the cloned repo\n\tWraps: (3) exit status 128\n\tError types: (1) *hintdetail.withDetail (2) *hintdetail.withDetail (3) *exec.ExitError"
time="2024-05-16 11:57:38" level=error msg=" [pipeline service] [pipeline #797] [task #61700] error writing subtask Clone Git Repo status to DB\n\tcaused by: failed to encode args[0]: unable to encode 1 into binary format for bool (OID 16): cannot find encode plan (500)\n\tWraps: (2) failed to encode args[0]: unable to encode 1 into binary format for bool (OID 16): cannot find encode plan\n\tError types: (1) *hintdetail.withDetail (2) *errors.errorString"
```
Running with this image [release-v1.0_240516_0721_3ae23bf](https://hub.docker.com/layers/apache/devlake/release-v1.0_240516_0721_3ae23bf/images/sha256-9f2256954793f3551e0b4f8e005a352cceb81dbe660e7aa8935f72a481abd5fc?context=explore) (I was testing the Jenkins fix I did). I increased the time range but no luck.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitHub] GitExtractor fails on empty repository due to shallow clone [incubator-devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #7468:
URL: https://github.com/apache/incubator-devlake/issues/7468#issuecomment-2116560000
@gustavobini Can you share some of them if possible? I am having trouble to reproduce the problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitHub] GitExtractor fails on empty repository due to shallow clone [incubator-devlake]
Posted by "gustavobini (via GitHub)" <gi...@apache.org>.
gustavobini commented on issue #7468:
URL: https://github.com/apache/incubator-devlake/issues/7468#issuecomment-2116890360
@klesh I cloned the repo locally and executed the same commands Devlake does, and my issue is exactly the same as this one https://stackoverflow.com/questions/63878612/git-fatal-error-in-object-unshallow-sha-1 and also got resolved by `git repack -d`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitHub] GitExtractor fails on empty repository due to shallow clone [incubator-devlake]
Posted by "gustavobini (via GitHub)" <gi...@apache.org>.
gustavobini commented on issue #7468:
URL: https://github.com/apache/incubator-devlake/issues/7468#issuecomment-2116779161
> @gustavobini Can you share some of them if possible? I am having trouble to reproduce the problem.
@klesh they're private org repos. Let me see what I can do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitHub] GitExtractor fails on empty repository due to shallow clone [incubator-devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #7468:
URL: https://github.com/apache/incubator-devlake/issues/7468#issuecomment-2111465799
Thanks for reporting the problem, will look into it ASAP
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitHub] GitExtractor fails on empty repository due to shallow clone [incubator-devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #7468:
URL: https://github.com/apache/incubator-devlake/issues/7468#issuecomment-2116557508
@gustavobini Thanks for reporting the problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitHub] GitExtractor fails on empty repository due to shallow clone [incubator-devlake]
Posted by "abeizn (via GitHub)" <gi...@apache.org>.
abeizn closed issue #7468: [Bug][GitHub] GitExtractor fails on empty repository due to shallow clone
URL: https://github.com/apache/incubator-devlake/issues/7468
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitHub] GitExtractor fails on empty repository due to shallow clone [incubator-devlake]
Posted by "abeizn (via GitHub)" <gi...@apache.org>.
abeizn closed issue #7468: [Bug][GitHub] GitExtractor fails on empty repository due to shallow clone
URL: https://github.com/apache/incubator-devlake/issues/7468
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitHub] GitExtractor fails on empty repository due to shallow clone [incubator-devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #7468:
URL: https://github.com/apache/incubator-devlake/issues/7468#issuecomment-2119620661
@gustavobini Thanks for the information. Will submit another PR to fix the problem even though I do not fully understand why... 🤣
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org