[GitHub] [incubator-devlake] warren830 opened a new pull request, #3706: Fix update xx commit incremental

[GitBox <gi...@apache.org>]

warren830 opened a new pull request, #3706:
URL: https://github.com/apache/incubator-devlake/pull/3706

   # Summary
   
   As tapd hasn't been used very frequently, so there are some issues when collecting by incremental
   1. we misused time.Format for the field `modified` which is used in url,
   - already fixed by using `since.Local().Format("2006-01-02")`
   2. we used a wrong way to collect xxCommits, the old way will miss data which created after last modification of story/bug/task. For example, we already have story_commit which created on 2022-11-02, and that story was modified on 2022-11-01, then we collect both data on 2022-11-03, and we connected another commit to any story on 2022-11-04, then we tried to collect data again on 2022-11-05, we will not collect commits for the story which modified before 2022-11-02, because connect a commit to a story, the story's filed `modified` will not be changed.
   - fixed by the way similar to  #2274, by this way,  every time we will collect story_commits as below:
   -  will collect story_commit for two kinds of stories described as below:
   	 1. stories have no story commits (story got modified after creation)
   	 2. stories have story commits and stories got modified after last time that we collected story commit (story got modified after creation)
   3. another minor issue(never influence our previous data report):
   - the entity `task` used a wrong tag `name` for the field `title`
   
   
   ### Does this close any open issues?
   Relate to #3632 
   
   ### Screenshots
   Include any relevant screenshots here.
   
   ### Other Information
   this new way still has side effect, we cannot collect story commits for the story which already collected commits and never got modified after last collection


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-devlake] warren830 closed pull request #3706: Fix update xx commit incremental

[GitBox <gi...@apache.org>]

warren830 closed pull request #3706: Fix update xx commit incremental
URL: https://github.com/apache/incubator-devlake/pull/3706


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-devlake] warren830 commented on pull request #3706: Fix update xx commit incremental

[GitBox <gi...@apache.org>]

warren830 commented on PR #3706:
URL: https://github.com/apache/incubator-devlake/pull/3706#issuecomment-1310268407

   As new logic still will miss data, will tackle this later


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org