You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by "sstojak1 (via GitHub)" <gi...@apache.org> on 2024/06/02 08:39:22 UTC

[PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

sstojak1 opened a new pull request, #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559

   <!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at
   
       http://www.apache.org/licenses/LICENSE-2.0
   
   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
   -->
   ### ⚠️ Pre Checklist
   
   > Please complete _ALL_ items in this checklist, and remove before submitting
   
   - [ ] I have read through the [Contributing Documentation](https://devlake.apache.org/community/).
   - [ ] I have added relevant tests.
   - [ ] I have added relevant documentation.
   - [ ] I will add labels to the PR, such as `pr-type/bug-fix`, `pr-type/feature-development`, etc.
   
   <!--
   Thanks for submitting a pull request!
   
   We appreciate you spending the time to work on these changes.
   Please fill out as many sections below as possible.
   -->
   
   ### Summary
   What does this PR do?
   
   ### Does this close any open issues?
   Closes xx
   
   ### Screenshots
   Include any relevant screenshots here.
   
   ### Other Information
   Any other information that is important to this PR.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#issuecomment-2151767667

   @sstojak1 Hi, sorry for the late reply.
   We support using PostgreSQL as a storage option for advanced users who prefer writing their own dashboards without any preset dashboards.
   In other words: the `devlake` implementation written in Golang supports both PG and Mysql while the embedded dashboards for grafana only available for Mysql
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "d4x1 (via GitHub)" <gi...@apache.org>.
d4x1 commented on code in PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#discussion_r1623726769


##########
backend/core/models/migrationscripts/20240603_add_pull_request_id_index_for_pr_commits_table.go:
##########
@@ -0,0 +1,29 @@
+package migrationscripts
+
+import (
+	"github.com/apache/incubator-devlake/core/context"
+	"github.com/apache/incubator-devlake/core/errors"
+	"github.com/apache/incubator-devlake/helpers/migrationhelper"
+)
+
+type addPullRequestIdIndexToPullRequestCommits struct{}
+
+type pullRequestCommits20240602 struct {
+	PullRequestId string `gorm:"index"`

Review Comment:
   https://github.com/apache/incubator-devlake/blob/main/backend/core/models/domainlayer/code/pull_request_commit.go#L28,  `PullRequestId` and `CommitSha` are the primary keys, and it works as expected in db tables.
   
   So I think there is no need to add a new `index` in this field.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#issuecomment-2154397167

   Hi, @sstojak1 we are so close, there is one conflict left before I can merge the PR. Would you like to resolve it? Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#issuecomment-2151777606

   @sstojak1 And yes, we have some scripts designated MySQL, sometimes because they are MySQL specific nuances, which were unnecessary for PG from what  I know.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "sstojak1 (via GitHub)" <gi...@apache.org>.
sstojak1 commented on code in PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#discussion_r1624265161


##########
backend/core/models/migrationscripts/20240603_add_pull_request_id_index_for_pr_commits_table.go:
##########
@@ -0,0 +1,29 @@
+package migrationscripts
+
+import (
+	"github.com/apache/incubator-devlake/core/context"
+	"github.com/apache/incubator-devlake/core/errors"
+	"github.com/apache/incubator-devlake/helpers/migrationhelper"
+)
+
+type addPullRequestIdIndexToPullRequestCommits struct{}
+
+type pullRequestCommits20240602 struct {
+	PullRequestId string `gorm:"index"`

Review Comment:
   I agree. I changed the pull_request_commits script. Now it's changing the primary key columns' order.
   <img width="1036" alt="image" src="https://github.com/apache/incubator-devlake/assets/18380216/667954c3-e261-4272-8720-00747cbbf271">
   <img width="1311" alt="image" src="https://github.com/apache/incubator-devlake/assets/18380216/5e712be7-6131-4c3d-a076-30916f6c0586">
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "sstojak1 (via GitHub)" <gi...@apache.org>.
sstojak1 commented on PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#issuecomment-2152291236

   @klesh makes sense. I have added support for PostgreSQL. Please take a look.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on code in PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#discussion_r1624208311


##########
backend/core/models/migrationscripts/20240603_add_pull_request_id_index_for_pr_commits_table.go:
##########
@@ -0,0 +1,29 @@
+package migrationscripts
+
+import (
+	"github.com/apache/incubator-devlake/core/context"
+	"github.com/apache/incubator-devlake/core/errors"
+	"github.com/apache/incubator-devlake/helpers/migrationhelper"
+)
+
+type addPullRequestIdIndexToPullRequestCommits struct{}
+
+type pullRequestCommits20240602 struct {
+	PullRequestId string `gorm:"index"`

Review Comment:
   @sstojak1 True, we should consider which one is more likely to happen. It is easier to predict someone might need to group records by `pull_request_id` while it is harder to think of how `commit_sha` being prioritized could be beneficial for any case, maybe "Counting how many PRs contain the same commit?" which doesn't make too much sense.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "sstojak1 (via GitHub)" <gi...@apache.org>.
sstojak1 commented on code in PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#discussion_r1623916520


##########
backend/core/models/migrationscripts/20240603_add_pull_request_id_index_for_pr_commits_table.go:
##########
@@ -0,0 +1,29 @@
+package migrationscripts
+
+import (
+	"github.com/apache/incubator-devlake/core/context"
+	"github.com/apache/incubator-devlake/core/errors"
+	"github.com/apache/incubator-devlake/helpers/migrationhelper"
+)
+
+type addPullRequestIdIndexToPullRequestCommits struct{}
+
+type pullRequestCommits20240602 struct {
+	PullRequestId string `gorm:"index"`

Review Comment:
   with `PullRequestId`  index:
   ![image](https://github.com/apache/incubator-devlake/assets/18380216/f53bc438-4495-4657-867f-7a4523976d14)
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "sstojak1 (via GitHub)" <gi...@apache.org>.
sstojak1 commented on PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#issuecomment-2151465871

   Hi @klesh any updates here?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "sstojak1 (via GitHub)" <gi...@apache.org>.
sstojak1 commented on PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#issuecomment-2147574324

   > Why are two tables handled differently? Besides, can we support postgresql as well? Apache DevLake supports pg as a storage option.
   
   Those two tables don't have the same primary key definition. That's the reason why migration scripts are different. 
   
   Is the PostgreSQL supported? There are also a few scripts where this kind of schema check is added...
   Also, check this out:
   https://devlake-io.slack.com/archives/C03APJ20VM4/p1713164942109869
   https://devlake-io.slack.com/archives/C03APJ20VM4/p1716386100745249


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "abeizn (via GitHub)" <gi...@apache.org>.
abeizn commented on PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#issuecomment-2230463621

   @sstojak1 Already released in [v1.0.1-beta3](https://github.com/apache/incubator-devlake/releases/tag/v1.0.1-beta3)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on code in PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#discussion_r1624164846


##########
backend/core/models/migrationscripts/20240603_add_pull_request_id_index_for_pr_commits_table.go:
##########
@@ -0,0 +1,29 @@
+package migrationscripts
+
+import (
+	"github.com/apache/incubator-devlake/core/context"
+	"github.com/apache/incubator-devlake/core/errors"
+	"github.com/apache/incubator-devlake/helpers/migrationhelper"
+)
+
+type addPullRequestIdIndexToPullRequestCommits struct{}
+
+type pullRequestCommits20240602 struct {
+	PullRequestId string `gorm:"index"`

Review Comment:
   Ahh.. The order of the primary key columns seems off to me, the `PullRequestId` should be placed before the `CommitSha` column.
   However, adding a secondary index is acceptable IMO if adjusting the order is too hard to do so.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "sstojak1 (via GitHub)" <gi...@apache.org>.
sstojak1 commented on code in PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#discussion_r1624265161


##########
backend/core/models/migrationscripts/20240603_add_pull_request_id_index_for_pr_commits_table.go:
##########
@@ -0,0 +1,29 @@
+package migrationscripts
+
+import (
+	"github.com/apache/incubator-devlake/core/context"
+	"github.com/apache/incubator-devlake/core/errors"
+	"github.com/apache/incubator-devlake/helpers/migrationhelper"
+)
+
+type addPullRequestIdIndexToPullRequestCommits struct{}
+
+type pullRequestCommits20240602 struct {
+	PullRequestId string `gorm:"index"`

Review Comment:
   I agree. I changed the pull_request_commits script. Now it's changing the primary key columns order.
   <img width="1036" alt="image" src="https://github.com/apache/incubator-devlake/assets/18380216/667954c3-e261-4272-8720-00747cbbf271">
   <img width="1311" alt="image" src="https://github.com/apache/incubator-devlake/assets/18380216/5e712be7-6131-4c3d-a076-30916f6c0586">
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh merged PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "sstojak1 (via GitHub)" <gi...@apache.org>.
sstojak1 commented on code in PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#discussion_r1624183653


##########
backend/core/models/migrationscripts/20240603_add_pull_request_id_index_for_pr_commits_table.go:
##########
@@ -0,0 +1,29 @@
+package migrationscripts
+
+import (
+	"github.com/apache/incubator-devlake/core/context"
+	"github.com/apache/incubator-devlake/core/errors"
+	"github.com/apache/incubator-devlake/helpers/migrationhelper"
+)
+
+type addPullRequestIdIndexToPullRequestCommits struct{}
+
+type pullRequestCommits20240602 struct {
+	PullRequestId string `gorm:"index"`

Review Comment:
   This could work I think:
   `ALTER TABLE pull_request_commits DROP PRIMARY KEY;
   ALTER TABLE pull_request_commits ADD PRIMARY KEY (pull_request_id, commit_sha);`
   However, this change might impact the rest of the application, especially if there are custom queries that end users have that expect `commit_sha` to be first in the index sequence.
   
   What do you think?
   
   



##########
backend/core/models/migrationscripts/20240603_add_pull_request_id_index_for_pr_commits_table.go:
##########
@@ -0,0 +1,29 @@
+package migrationscripts
+
+import (
+	"github.com/apache/incubator-devlake/core/context"
+	"github.com/apache/incubator-devlake/core/errors"
+	"github.com/apache/incubator-devlake/helpers/migrationhelper"
+)
+
+type addPullRequestIdIndexToPullRequestCommits struct{}
+
+type pullRequestCommits20240602 struct {
+	PullRequestId string `gorm:"index"`

Review Comment:
   This could work I think:
   `ALTER TABLE pull_request_commits DROP PRIMARY KEY;`
   `ALTER TABLE pull_request_commits ADD PRIMARY KEY (pull_request_id, commit_sha);`
   However, this change might impact the rest of the application, especially if there are custom queries that end users have that expect `commit_sha` to be first in the index sequence.
   
   What do you think?
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "sstojak1 (via GitHub)" <gi...@apache.org>.
sstojak1 commented on code in PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#discussion_r1623915550


##########
backend/core/models/migrationscripts/20240603_add_pull_request_id_index_for_pr_commits_table.go:
##########
@@ -0,0 +1,29 @@
+package migrationscripts
+
+import (
+	"github.com/apache/incubator-devlake/core/context"
+	"github.com/apache/incubator-devlake/core/errors"
+	"github.com/apache/incubator-devlake/helpers/migrationhelper"
+)
+
+type addPullRequestIdIndexToPullRequestCommits struct{}
+
+type pullRequestCommits20240602 struct {
+	PullRequestId string `gorm:"index"`

Review Comment:
   true in case you are filtering by `CommitSha` or `CommitSha` + `PullRequestId`.
   If you are filtering only by `PullRequestId` the composite index won't be used - [docs](https://dev.mysql.com/doc/refman/8.4/en/multiple-column-indexes.html)
   
   ![image](https://github.com/apache/incubator-devlake/assets/18380216/5c96baaf-877d-4ba0-88d6-61cf2585abac)
   ![image](https://github.com/apache/incubator-devlake/assets/18380216/480f1a6b-aadd-402e-98a1-57a0eb8226e8)
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "sstojak1 (via GitHub)" <gi...@apache.org>.
sstojak1 commented on PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#issuecomment-2158350723

   @klesh can we merge it? :) 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] adding pull_request_id index to pull_request_commits/comments tables [incubator-devlake]

Posted by "sstojak1 (via GitHub)" <gi...@apache.org>.
sstojak1 commented on PR #7559:
URL: https://github.com/apache/incubator-devlake/pull/7559#issuecomment-2154488814

   @klesh done!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org