Re: [I] [Feature][GitExtractor] Support incremental sync when collecting Git data [incubator-devlake]

["d4x1 (via GitHub)" <gi...@apache.org>]

d4x1 commented on issue #6853:
URL: https://github.com/apache/incubator-devlake/issues/6853#issuecomment-1929550742

   GitExtractor now collects repo's all commits every time when it's executed.  I'll update it and make GitExtractor only collect new commits after last run.
   But GitExtractor will ignore project's sync policy such `FullSync` or `TimeAfter`. There are some reasons:
   1. When `components` config is updated, all data in `commit_file_components` should be recalculated.  But there is no entry point updating it in Config UI. So supporting `FullSync` is unecessary so far.
   2. In a repository, commit id may change when `rebase` happens.  After rebase operation, old commits will be dangling comits, and if these commits still exist in database, they have no side effect. And new commit ids will be collected in the next  run.
   In a summary, GitExtractor will not support `FullSync` so far.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org