You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by GitBox <gi...@apache.org> on 2022/06/08 16:05:59 UTC
[GitHub] [incubator-devlake] likyh opened a new pull request, #2128: save github connection in db and support create multi connection
likyh opened a new pull request, #2128:
URL: https://github.com/apache/incubator-devlake/pull/2128
# Summary
<!--
Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes.
Please fill out as many sections below as possible.
-->
Save github connection in db and support create multi connection.
It's the first PR for #1980. Only can collect data by the connection named `GitHub` or fail.
### Does this close any open issues?
#1980
### Screenshots
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] likyh commented on a diff in pull request #2128: save github connection in db and support create multi connection
Posted by GitBox <gi...@apache.org>.
likyh commented on code in PR #2128:
URL: https://github.com/apache/incubator-devlake/pull/2128#discussion_r894168776
##########
plugins/github/models/connection.go:
##########
@@ -17,31 +17,33 @@ limitations under the License.
package models
-type GithubConnection struct {
- Endpoint string `mapstructure:"endpoint" validate:"required" env:"GITHUB_ENDPOINT" json:"endpoint"`
- Auth string `mapstructure:"auth" validate:"required" env:"GITHUB_AUTH" json:"auth"`
- Proxy string `mapstructure:"proxy" env:"GITHUB_PROXY" json:"proxy"`
+import "github.com/apache/incubator-devlake/plugins/helper"
- Config `mapstructure:",squash"`
+type TestConnectionRequest struct {
+ Endpoint string `json:"endpoint" validate:"required,url"`
+ Auth string `json:"auth" validate:"required"`
+ Proxy string `json:"proxy"`
}
-type Config struct {
- PrType string `mapstructure:"prType" env:"GITHUB_PR_TYPE" json:"prType"`
- PrComponent string `mapstructure:"prComponent" env:"GITHUB_PR_COMPONENT" json:"prComponent"`
- IssueSeverity string `mapstructure:"issueSeverity" env:"GITHUB_ISSUE_SEVERITY" json:"issueSeverity"`
- IssuePriority string `mapstructure:"issuePriority" env:"GITHUB_ISSUE_PRIORITY" json:"issuePriority"`
- IssueComponent string `mapstructure:"issueComponent" env:"GITHUB_ISSUE_COMPONENT" json:"issueComponent"`
- IssueTypeBug string `mapstructure:"issueTypeBug" env:"GITHUB_ISSUE_TYPE_BUG" json:"issueTypeBug"`
- IssueTypeIncident string `mapstructure:"issueTypeIncident" env:"GITHUB_ISSUE_TYPE_INCIDENT" json:"issueTypeIncident"`
- IssueTypeRequirement string `mapstructure:"issueTypeRequirement" env:"GITHUB_ISSUE_TYPE_REQUIREMENT" json:"issueTypeRequirement"`
+type GithubConnection struct {
+ helper.RestConnection `mapstructure:",squash"`
+ Auth string `mapstructure:"auth" validate:"required" json:"auth"`
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] e2corporation commented on pull request #2128: save github connection in db and support create multi connection
Posted by GitBox <gi...@apache.org>.
e2corporation commented on PR #2128:
URL: https://github.com/apache/incubator-devlake/pull/2128#issuecomment-1154047460
> @e2corporation Would you take a look at the front end part ?
Why was multi-connection on UI done separately under 2 PRs, this will cause a conflict with the other open one for Gitlab. Ideally all multi-connection changes for UI should have been made on the same branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on a diff in pull request #2128: save github connection in db and support create multi connection
Posted by GitBox <gi...@apache.org>.
klesh commented on code in PR #2128:
URL: https://github.com/apache/incubator-devlake/pull/2128#discussion_r893617864
##########
plugins/github/github.go:
##########
@@ -95,7 +96,16 @@ func (plugin Github) PrepareTaskData(taskCtx core.TaskContext, options map[strin
if op.Repo == "" {
return nil, fmt.Errorf("repo is required for GitHub execution")
}
- apiClient, err := tasks.CreateApiClient(taskCtx)
+
+ // find the only github now
+ // FIXME to query by connection id when multi connection support
+ connection := &models.GithubConnection{}
+ err = taskCtx.GetDb().First(connection, "name = ?", `GitHub`).Error
Review Comment:
Use `GetDal` pls
##########
plugins/github/models/connection.go:
##########
@@ -17,31 +17,33 @@ limitations under the License.
package models
-type GithubConnection struct {
- Endpoint string `mapstructure:"endpoint" validate:"required" env:"GITHUB_ENDPOINT" json:"endpoint"`
- Auth string `mapstructure:"auth" validate:"required" env:"GITHUB_AUTH" json:"auth"`
- Proxy string `mapstructure:"proxy" env:"GITHUB_PROXY" json:"proxy"`
+import "github.com/apache/incubator-devlake/plugins/helper"
- Config `mapstructure:",squash"`
+type TestConnectionRequest struct {
+ Endpoint string `json:"endpoint" validate:"required,url"`
+ Auth string `json:"auth" validate:"required"`
+ Proxy string `json:"proxy"`
}
-type Config struct {
- PrType string `mapstructure:"prType" env:"GITHUB_PR_TYPE" json:"prType"`
- PrComponent string `mapstructure:"prComponent" env:"GITHUB_PR_COMPONENT" json:"prComponent"`
- IssueSeverity string `mapstructure:"issueSeverity" env:"GITHUB_ISSUE_SEVERITY" json:"issueSeverity"`
- IssuePriority string `mapstructure:"issuePriority" env:"GITHUB_ISSUE_PRIORITY" json:"issuePriority"`
- IssueComponent string `mapstructure:"issueComponent" env:"GITHUB_ISSUE_COMPONENT" json:"issueComponent"`
- IssueTypeBug string `mapstructure:"issueTypeBug" env:"GITHUB_ISSUE_TYPE_BUG" json:"issueTypeBug"`
- IssueTypeIncident string `mapstructure:"issueTypeIncident" env:"GITHUB_ISSUE_TYPE_INCIDENT" json:"issueTypeIncident"`
- IssueTypeRequirement string `mapstructure:"issueTypeRequirement" env:"GITHUB_ISSUE_TYPE_REQUIREMENT" json:"issueTypeRequirement"`
+type GithubConnection struct {
+ helper.RestConnection `mapstructure:",squash"`
+ Auth string `mapstructure:"auth" validate:"required" json:"auth"`
Review Comment:
I think this one should be `helper.AccessToken` instead
##########
plugins/github/models/connection.go:
##########
@@ -17,31 +17,33 @@ limitations under the License.
package models
-type GithubConnection struct {
- Endpoint string `mapstructure:"endpoint" validate:"required" env:"GITHUB_ENDPOINT" json:"endpoint"`
- Auth string `mapstructure:"auth" validate:"required" env:"GITHUB_AUTH" json:"auth"`
- Proxy string `mapstructure:"proxy" env:"GITHUB_PROXY" json:"proxy"`
+import "github.com/apache/incubator-devlake/plugins/helper"
- Config `mapstructure:",squash"`
+type TestConnectionRequest struct {
+ Endpoint string `json:"endpoint" validate:"required,url"`
+ Auth string `json:"auth" validate:"required"`
+ Proxy string `json:"proxy"`
}
-type Config struct {
- PrType string `mapstructure:"prType" env:"GITHUB_PR_TYPE" json:"prType"`
- PrComponent string `mapstructure:"prComponent" env:"GITHUB_PR_COMPONENT" json:"prComponent"`
- IssueSeverity string `mapstructure:"issueSeverity" env:"GITHUB_ISSUE_SEVERITY" json:"issueSeverity"`
- IssuePriority string `mapstructure:"issuePriority" env:"GITHUB_ISSUE_PRIORITY" json:"issuePriority"`
- IssueComponent string `mapstructure:"issueComponent" env:"GITHUB_ISSUE_COMPONENT" json:"issueComponent"`
- IssueTypeBug string `mapstructure:"issueTypeBug" env:"GITHUB_ISSUE_TYPE_BUG" json:"issueTypeBug"`
- IssueTypeIncident string `mapstructure:"issueTypeIncident" env:"GITHUB_ISSUE_TYPE_INCIDENT" json:"issueTypeIncident"`
- IssueTypeRequirement string `mapstructure:"issueTypeRequirement" env:"GITHUB_ISSUE_TYPE_REQUIREMENT" json:"issueTypeRequirement"`
+type GithubConnection struct {
+ helper.RestConnection `mapstructure:",squash"`
+ Auth string `mapstructure:"auth" validate:"required" json:"auth"`
+ Config `mapstructure:",squash"`
}
-// This object conforms to what the frontend currently expects.
-type GithubResponse struct {
- Name string `json:"name"`
- ID int `json:"id"`
+type Config struct {
Review Comment:
These are `TransformationRules` in the new user flow, we don't need to save them with connection anymore.
They will be passed in by `task.options` in the new user flow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on pull request #2128: save github connection in db and support create multi connection
Posted by GitBox <gi...@apache.org>.
klesh commented on PR #2128:
URL: https://github.com/apache/incubator-devlake/pull/2128#issuecomment-1154051981
> > @e2corporation Would you take a look at the front end part ?
>
> Why was multi-connection on UI done separately under 2 PRs, this will cause a conflict with the other open one for Gitlab. Ideally all multi-connection changes for UI should have been made on the same branch.
Sorry, I did it too fast, @likyh is working on new PR to fix further issues.
@likyh Please take a look at @e2corporation 's comment. and fix it on next PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on a diff in pull request #2128: save github connection in db and support create multi connection
Posted by GitBox <gi...@apache.org>.
klesh commented on code in PR #2128:
URL: https://github.com/apache/incubator-devlake/pull/2128#discussion_r895826550
##########
plugins/github/github.go:
##########
@@ -95,7 +96,16 @@ func (plugin Github) PrepareTaskData(taskCtx core.TaskContext, options map[strin
if op.Repo == "" {
return nil, fmt.Errorf("repo is required for GitHub execution")
}
- apiClient, err := tasks.CreateApiClient(taskCtx)
+
+ // find the only github now
+ // FIXME to query by connection id when multi connection support
+ connection := &models.GithubConnection{}
+ err = taskCtx.GetDb().First(connection, "name = ?", `GitHub`).Error
Review Comment:
Should be using connection helper to load the connection for you, otherwise, the Accesstoken wont be decrypted properly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] likyh commented on a diff in pull request #2128: save github connection in db and support create multi connection
Posted by GitBox <gi...@apache.org>.
likyh commented on code in PR #2128:
URL: https://github.com/apache/incubator-devlake/pull/2128#discussion_r894180992
##########
plugins/github/models/connection.go:
##########
@@ -17,31 +17,33 @@ limitations under the License.
package models
-type GithubConnection struct {
- Endpoint string `mapstructure:"endpoint" validate:"required" env:"GITHUB_ENDPOINT" json:"endpoint"`
- Auth string `mapstructure:"auth" validate:"required" env:"GITHUB_AUTH" json:"auth"`
- Proxy string `mapstructure:"proxy" env:"GITHUB_PROXY" json:"proxy"`
+import "github.com/apache/incubator-devlake/plugins/helper"
- Config `mapstructure:",squash"`
+type TestConnectionRequest struct {
+ Endpoint string `json:"endpoint" validate:"required,url"`
+ Auth string `json:"auth" validate:"required"`
+ Proxy string `json:"proxy"`
}
-type Config struct {
- PrType string `mapstructure:"prType" env:"GITHUB_PR_TYPE" json:"prType"`
- PrComponent string `mapstructure:"prComponent" env:"GITHUB_PR_COMPONENT" json:"prComponent"`
- IssueSeverity string `mapstructure:"issueSeverity" env:"GITHUB_ISSUE_SEVERITY" json:"issueSeverity"`
- IssuePriority string `mapstructure:"issuePriority" env:"GITHUB_ISSUE_PRIORITY" json:"issuePriority"`
- IssueComponent string `mapstructure:"issueComponent" env:"GITHUB_ISSUE_COMPONENT" json:"issueComponent"`
- IssueTypeBug string `mapstructure:"issueTypeBug" env:"GITHUB_ISSUE_TYPE_BUG" json:"issueTypeBug"`
- IssueTypeIncident string `mapstructure:"issueTypeIncident" env:"GITHUB_ISSUE_TYPE_INCIDENT" json:"issueTypeIncident"`
- IssueTypeRequirement string `mapstructure:"issueTypeRequirement" env:"GITHUB_ISSUE_TYPE_REQUIREMENT" json:"issueTypeRequirement"`
+type GithubConnection struct {
+ helper.RestConnection `mapstructure:",squash"`
+ Auth string `mapstructure:"auth" validate:"required" json:"auth"`
+ Config `mapstructure:",squash"`
}
-// This object conforms to what the frontend currently expects.
-type GithubResponse struct {
- Name string `json:"name"`
- ID int `json:"id"`
+type Config struct {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on a diff in pull request #2128: save github connection in db and support create multi connection
Posted by GitBox <gi...@apache.org>.
klesh commented on code in PR #2128:
URL: https://github.com/apache/incubator-devlake/pull/2128#discussion_r895832161
##########
plugins/github/github.go:
##########
@@ -95,7 +96,16 @@ func (plugin Github) PrepareTaskData(taskCtx core.TaskContext, options map[strin
if op.Repo == "" {
return nil, fmt.Errorf("repo is required for GitHub execution")
}
- apiClient, err := tasks.CreateApiClient(taskCtx)
+
+ // find the only github now
+ // FIXME to query by connection id when multi connection support
+ connection := &models.GithubConnection{}
+ err = taskCtx.GetDb().First(connection, "name = ?", `GitHub`).Error
Review Comment:
Should be using ConnectionHelper to load connection, otherwise, the AccessToken wont be decrypted properly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on pull request #2128: save github connection in db and support create multi connection
Posted by GitBox <gi...@apache.org>.
klesh commented on PR #2128:
URL: https://github.com/apache/incubator-devlake/pull/2128#issuecomment-1154042666
@e2corporation Would you take a look at the front end part ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] likyh commented on a diff in pull request #2128: save github connection in db and support create multi connection
Posted by GitBox <gi...@apache.org>.
likyh commented on code in PR #2128:
URL: https://github.com/apache/incubator-devlake/pull/2128#discussion_r894110785
##########
plugins/github/github.go:
##########
@@ -95,7 +96,16 @@ func (plugin Github) PrepareTaskData(taskCtx core.TaskContext, options map[strin
if op.Repo == "" {
return nil, fmt.Errorf("repo is required for GitHub execution")
}
- apiClient, err := tasks.CreateApiClient(taskCtx)
+
+ // find the only github now
+ // FIXME to query by connection id when multi connection support
+ connection := &models.GithubConnection{}
+ err = taskCtx.GetDb().First(connection, "name = ?", `GitHub`).Error
Review Comment:
this will change next PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on a diff in pull request #2128: save github connection in db and support create multi connection
Posted by GitBox <gi...@apache.org>.
klesh commented on code in PR #2128:
URL: https://github.com/apache/incubator-devlake/pull/2128#discussion_r895839024
##########
plugins/github/github.go:
##########
@@ -95,7 +96,16 @@ func (plugin Github) PrepareTaskData(taskCtx core.TaskContext, options map[strin
if op.Repo == "" {
return nil, fmt.Errorf("repo is required for GitHub execution")
}
- apiClient, err := tasks.CreateApiClient(taskCtx)
+
+ // find the only github now
+ // FIXME to query by connection id when multi connection support
+ connection := &models.GithubConnection{}
+ err = taskCtx.GetDb().First(connection, "name = ?", `GitHub`).Error
Review Comment:
Will be addressed by next PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh merged pull request #2128: save github connection in db and support create multi connection
Posted by GitBox <gi...@apache.org>.
klesh merged PR #2128:
URL: https://github.com/apache/incubator-devlake/pull/2128
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org