You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by "adterskov (via GitHub)" <gi...@apache.org> on 2023/03/23 09:18:10 UTC
[GitHub] [incubator-devlake] adterskov opened a new issue, #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
adterskov opened a new issue, #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar issues.
### What happened
I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and didn't find similar issues.
### What do you expect to happen
When I try to configure connection with self-hosted GitLab an error occurred
**Test Failed. Please check your configuration.**
Logs in the Devlake container are following
`[GIN] 2023/03/23 - 11:47:58 | 500 | 878.823223ms | *.*.*.* | POST "/plugins/gitlab/test"
[GIN] 2023/03/23 - 11:48:06 | 200 | 10.393876ms | *.*.*.* | GET "/blueprints"
time="2023-03-23 11:48:08" level=error msg="HTTP 500 error
caused by: error requesting https://gitlab.example.com/api/v4/user
Wraps: (2) Get "https://gitlab.loc/api/v4/user": x509: certificate signed by unknown authority
Wraps: (3) Get "https://gitlab.loc/api/v4/user"
Wraps: (4) x509: certificate signed by unknown authority
Error types: (1) *hintdetail.withDetail (2) *hintdetail.withDetail (3) *url.Error (4) x509.UnknownAuthorityError"`
Environment variable is set
`IN_SECURE_SKIP_VERIFY=true`
I've already added CA certificate in docker image and run command
`/usr/sbin/update-ca-certificates`
Finaky I see CA certificate on the right path
`/etc/ssl/certs/ca-certificates.crt`
CA certificate was installed correctly and this check completed successfully
curl "https://gitlab.example.com/api/v4/user?access_token=gitlab-personal-access-token"
I also try to solve this issue with different Devlake versions
[v0.16.0-beta9](https://github.com/apache/incubator-devlake/releases/tag/v0.16.0-beta9)
[v0.15.1-beta5](https://github.com/apache/incubator-devlake/releases/tag/v0.15.1-beta5)
[v0.15.0](https://github.com/apache/incubator-devlake/releases/tag/v0.15.0)
### How to reproduce
Reproduce the steps from the official documentation
https://devlake.apache.org/docs/Configuration/GitLab
### Anything else
_No response_
### Version
v0.15.0
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] adterskov commented on issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "adterskov (via GitHub)" <gi...@apache.org>.
adterskov commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1485512377
> Can you verify if the CA is correct? For example:
>
> 1. access your gitlab from a computer without the root CA certificate installed, the browser would warn about the certificate
> 2. install the ca , make sure the warning is gone.
You can connect to our corporate GitLab only with an installed CA. I checked directly from the devlake container. After installing the certificate, the curl command is executed successfully
`curl "https://gitlab.example.com/api/v4/user?access_token=gitlab-personal-access-token "`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed) [incubator-devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1812451579
@khanh96le you may take a look at this doc:https://devlake.apache.org/docs/Troubleshooting/Configuration#failed-to-collect-data-from-the-server-with-a-self-signed-certificate
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] adterskov commented on issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "adterskov (via GitHub)" <gi...@apache.org>.
adterskov commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1494846858
I try to run GitExtractor localy with csv store but this option doesn't work properly
`go run main.go -url https://git:glpat-xxxxx@gitlab.example.com/project-1/extra/project.git -id gitlab:GitlabProject:2:2701 -output "/tmp/store"`
Despite the fact that the parameter **output** is set, it tries to connect to the database
> [error] failed to initialize database, got error dial tcp: lookup mysql: no such host"
> panic: dial tcp: lookup mysql: no such host
> Wraps: (2) dial tcp
> Wraps: (3) lookup mysql: no such host
> Error types: (1) *hintdetail.withDetail (2) *net.OpError (3) *net.DNSError
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] mindlesscloud commented on issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "mindlesscloud (via GitHub)" <gi...@apache.org>.
mindlesscloud commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1486820714
Hi @adterskov, Would you kindly confirm if the `IN_SECURE_SKIP_VERIFY` in the `.env` file was set to `true` please? The value of the variable was read from this file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] adterskov commented on issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "adterskov (via GitHub)" <gi...@apache.org>.
adterskov commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1496486987
@klesh yes, configuration in JSON is correct. I took it for local GitExtractor running and didn't have problems with it.
`go run main.go -url https://git:glpat-xxxxx@gitlab.example.com/project-1/extra/project.git -id gitlab:GitlabProject:2:2701 -output "/tmp/store"`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1488274476
Hi, @adterskov , I tried to reproduce the problem on `v0.16.0-beta9`, but it seemed to be working just fine.
Here is how I configured the `devlake`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1482200208
Can you verify if the CA is correct? For example:
1. access your gitlab from a computer without the root CA certificate installed, the browser would warn about the certificate
2. install the ca , make sure the warning is gone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] mappjzc commented on issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "mappjzc (via GitHub)" <gi...@apache.org>.
mappjzc commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1482139835
@adterskov Thanks for your issue.
Can you show more detail about your error?
Whether you can succeed in testing the connection?
We can not reproduce this issue.
Maybe you can tell me your WeChat or some others and take a look together?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed) [incubator-devlake]
Posted by "khanh96le (via GitHub)" <gi...@apache.org>.
khanh96le commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1813824726
Hi @klesh, thank you for your guidance.
Following the link you provided, I couldn't resolve the issue `GitExtractor task failed in a GitHub/GitLab/BitBucket blueprint`. See image below:
I am using image version `devlake.docker.scarf.sh/apache/devlake:v0.18.0`, which is > version `v0.15.x` suggested in the docs.
Download log files from "Historical Pipelines", the log does not show anything for debugging
```
time="2023-11-16 04:47:14" level=info msg=" [pipeline service] [pipeline #31] [task #263] start executing task: 263"
time="2023-11-16 04:47:14" level=info msg=" [pipeline service] [pipeline #31] [task #263] start plugin"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed) [incubator-devlake]
Posted by "khanh96le (via GitHub)" <gi...@apache.org>.
khanh96le commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1813856501
In Stage 2, there are 2 tasks: the first one runs success, the second one runs failed.
So I believe, the problem is related to the task that used GitExtractor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] archiephan78 commented on issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "archiephan78 (via GitHub)" <gi...@apache.org>.
archiephan78 commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1522771285
@adterskov How you resloved it, i have same issue with you, i added ca cert to container and add `IN_SECURE_SKIP_VERIFY ` to `.env` but gitextractor still cant work
```
time="2023-04-26 04:25:21" level=error msg=" [pipeline service] [pipeline #10] run tasks failed
caused by: Error running task 47.
Wraps: (2) error preparing task data for gitextractor
Wraps: (3) user rejected certificate for gitlab.example.com
Wraps: (4) user rejected certificate for gitlab.example.com
Error types: (1) *hintdetail.withDetail (2) *hintdetail.withDetail (3) *hintdetail.withDetail (4) *git.GitError
"
```
```
devlake:
image: apache/devlake:v0.16.0-beta17
ports:
- "127.0.0.1:8080:8080"
restart: always
volumes:
- ./.env:/app/.env
- ./logs:/app/logs
- ./gitlab.example.com.crt:/usr/local/share/ca-certificates/gitlab.example.com.crt
environment:
LOGGING_DIR: /app/logs
command: [ "sh", "-c", "update-ca-certificates; lake" ]
depends_on:
- mysql
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] adterskov commented on issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "adterskov (via GitHub)" <gi...@apache.org>.
adterskov commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1496488096
> I try to run GitExtractor localy with csv store but this option doesn't work properly `go run main.go -url https://git:glpat-xxxxx@gitlab.example.com/project-1/extra/project.git -id gitlab:GitlabProject:2:2701 -output "/tmp/store"`
>
> Despite the fact that the parameter **output** is set, it tries to connect to the database
>
> > [error] failed to initialize database, got error dial tcp: lookup mysql: no such host"
> > panic: dial tcp: lookup mysql: no such host
> > Wraps: (2) dial tcp
> > Wraps: (3) lookup mysql: no such host
> > Error types: (1) *hintdetail.withDetail (2) *net.OpError (3) *net.DNSError
Problem fixed. Dot env file with configuration was needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed) [incubator-devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1823926335
@khanh96le I'm not sure. How should we solve this then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed) [incubator-devlake]
Posted by "khanh96le (via GitHub)" <gi...@apache.org>.
khanh96le commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1815717473
This issue may be related to [this OpenSSL's issue ](https://github.com/openssl/openssl/issues/1418)
GitExtractor uses libgit2 which uses openssl for https verification.
My self-signed CA keyUsage is not set for keyCertSign, so openssl rejects it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1495645217
Hi, @adterskov I tried to reproduce your problem, and it doesn't appear. Weird.
Can you check to JSON output of the pipeline and see if the gitext configuration is correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1495224420
I see, will look into it ASAP
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] adterskov commented on issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "adterskov (via GitHub)" <gi...@apache.org>.
adterskov commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1487483460
Hello @mindlesscloud! Yes sure
`##########################
# Set if skip verify and connect with out trusted certificate when use https
##########################
IN_SECURE_SKIP_VERIFY="true"`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed) [incubator-devlake]
Posted by "khanh96le (via GitHub)" <gi...@apache.org>.
khanh96le commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1812028661
> Problem fixed. Dot env file with configuration was needed.
@adterskov Could you elaborate more on this? Which .env file, which values in it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed) [incubator-devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1813830423
@khanh96le So... what did you do extactly? did you install the rootCA and what steps did you take to verify the CA is installed correctly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed) [incubator-devlake]
Posted by "khanh96le (via GitHub)" <gi...@apache.org>.
khanh96le commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1813848979
@klesh
Yes I did install a self-signed rootCA by mounting the certificate file to the container, run `update-ca-certificates`, etc.
I verified it worked by running: `curl "https://my-private-gitlab.com/api/v4/user?access_token=gitlab-personal-access-token"`
And as I said in previous message, some information from Gitlab actually fetched successfully like the data about the pull requests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] adterskov commented on issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "adterskov (via GitHub)" <gi...@apache.org>.
adterskov commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1494727624
I deleted the current installation and reinstalled version v0.16.0-beta11. GitLab connection was added successfully.
But now when I run Blueprint job, an error from GitExtractor plugin is accured.
> error preparing task data for gitextractor
> Wraps: (2) user rejected certificate for gitlab.example.com
But CA certificate is installed, and I can perform git clone command from container (I get this URL from developers tool so GitExtractor use the same one)
`git clone https://git:glpat-xxxxx@gitlab.example.com/project-1/extra/project.git`
So GitLab access token is correct and User have necessary permission.
Maybe GitExtractor plugin need additional configuring?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] likyh closed issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "likyh (via GitHub)" <gi...@apache.org>.
likyh closed issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
URL: https://github.com/apache/incubator-devlake/issues/4756
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] adterskov commented on issue #4756: [Bug][GitLab] Connection to self-hosted GitLab failed (CA certificate has already installed)
Posted by "adterskov (via GitHub)" <gi...@apache.org>.
adterskov commented on issue #4756:
URL: https://github.com/apache/incubator-devlake/issues/4756#issuecomment-1485521038
> @adterskov Thanks for your issue. Can you show more detail about your error? Whether you can succeed in testing the connection? We can not reproduce this issue. Maybe you can tell me your WeChat or some others and take a look together?
As I wrote in the description, the check fails with an error
> Test Failed. Please check your configuration.
There is an error in the datalake container logs
> caused by: error requesting https://gitlab.example.com/api/v4/user
Wraps: (2) Get "https://gitlab.example.com/api/v4/user": x509: certificate signed by unknown authority
Wraps: (3) Get "https://gitlab.example.com/api/v4/user"
Wraps: (4) x509: certificate signed by unknown authority
Please tell me what information is still needed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org