You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@devlake.apache.org by "wouldd (via GitHub)" <gi...@apache.org> on 2024/04/30 14:19:33 UTC
[I] [Bug][gitextractor] Azure devops repos fail to clone with 400 errors [incubator-devlake]
wouldd opened a new issue, #7419:
URL: https://github.com/apache/incubator-devlake/issues/7419
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar issues.
### What happened
Soemwhere around the v1.0 release the underlying implementation for cloning git repos was changed, and since that point cloning azure deveops git repositories has not been working (on prem, though I would assume cloud is the same)
The error reported is a 400 response.
### What do you expect to happen
I expected git repos to continue to clone successfully, they were working fine before I updated the code base (to pick up an unrelated fix)
### How to reproduce
Configure the azure devops connector, provide a valid PAT and setup one or more repositories. run collect data.
I believe this would happen to azure cloud, if not then this could be specific to azure devops server which I know is technically unsupported. however this was working prior to picking up v1 changes and I can make it work with code changes now
### Anything else
So from what I can tell the azure devops git server requires an AUTHORIZATION Basic header to be set with the base64 encoded version of an empty user and the pat token, eg- :<token> encoded as base64. It seems to need this in addition to the values being passed on the commandline in the url where we are setting https://git:<pat>@url
In testing a fix locally I found that invoking the git cli directly as happens from this line: https://github.com/apache/incubator-devlake/blob/8a6bbe2b84d709e30d329cc7317ac85aa2398f10/backend/plugins/gitextractor/parser/clone_gitcli.go#L170C32-L171C2
did not work, something about hte way go was passing the parameters to the command were corrupting the header. However if I isntead call bash and pass the git command as an argument then repository cloning works no problem
Obviously my local fix is very specific to me, in taht I know I'm talking to azure devops so I can extract the PAT from the passed in url and encode it into a header etc. I'm not sure what hte general purpose fix for something like this would be.
### Version
master
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][gitextractor] Azure devops repos fail to clone with 400 errors [incubator-devlake]
Posted by "wouldd (via GitHub)" <gi...@apache.org>.
wouldd commented on issue #7419:
URL: https://github.com/apache/incubator-devlake/issues/7419#issuecomment-2100816668
So we are not using azure devops services, we're currently still using on-prem. so I guess if this is working for you then the issue relates to something odd about the on-prem setup. I'm happy to consider this a quirk of me being unsupported pointing to our server rather than services.
It's weird that it used to work prior to the code changes in the gitextractor logic. but I suppose for now I can continue to solve for our case with my local fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][gitextractor] Azure devops repos fail to clone with 400 errors [incubator-devlake]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #7419:
URL: https://github.com/apache/incubator-devlake/issues/7419#issuecomment-2227535846
This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][gitextractor] Azure devops repos fail to clone with 400 errors [incubator-devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #7419:
URL: https://github.com/apache/incubator-devlake/issues/7419#issuecomment-2095153262
Hi, can you share the error message of the 400 response? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][gitextractor] Azure devops repos fail to clone with 400 errors [incubator-devlake]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #7419:
URL: https://github.com/apache/incubator-devlake/issues/7419#issuecomment-2212645689
This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][gitextractor] Azure devops repos fail to clone with 400 errors [incubator-devlake]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #7419: [Bug][gitextractor] Azure devops repos fail to clone with 400 errors
URL: https://github.com/apache/incubator-devlake/issues/7419
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][gitextractor] Azure devops repos fail to clone with 400 errors [incubator-devlake]
Posted by "wouldd (via GitHub)" <gi...@apache.org>.
wouldd commented on issue #7419:
URL: https://github.com/apache/incubator-devlake/issues/7419#issuecomment-2097839338
@klesh sorry I didn't capture the exact wording, and my machine rebooted since. obviously I'm now running a version with my own local fix. Ostensibly it was just a 400 bad request response from AZD reported by the git cli. I guess I'm assuming that other people using AZD would be seeing the same issue with the latest code?
I can confirm that for me switching the base command to being bash and the args to be:
args := []string{"-c", "git -c http.extraHeader=\"Authorization: Basic "+sEnc+"\" clone "+ taskData.Options.Url+" "+localDir+" --bare --progress"}
fixed my issue. where I obviously pulled the PAT out of the Options.Url anad b64 encoded it into the sEnc value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][gitextractor] Azure devops repos fail to clone with 400 errors [incubator-devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #7419:
URL: https://github.com/apache/incubator-devlake/issues/7419#issuecomment-2100112493
@wouldd It is working fine on my local machine, I wonder if it is caused by the configuration on the Azure Portal. Is it possible for you to provide a Restricted token and Repo for me to investigate? You may find me on the Slack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org