You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by "klesh (via GitHub)" <gi...@apache.org> on 2023/02/27 01:57:42 UTC
[GitHub] [incubator-devlake] klesh opened a new issue, #4529: [Feature][infra] Refresh data in non-incremental manner to mitigate records being deleted in DataSource
klesh opened a new issue, #4529:
URL: https://github.com/apache/incubator-devlake/issues/4529
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar feature requirement.
### Use case
Records were deleted from the data source side and users want them gone in DevLake while keeping the time filter unchanged
### Description
Currently, we have `diffSync` activated by default without any user intervention, it works fine in most scenarios, however, records might be deleted from the Data Source but still exist in the DevLake database no matter how many times they try to recollect data due to the limitation of the `diffSync`.
To force a data refresh to remove those records, one would have to push the time filter to an earlier time to trigger a Data Purge Procedure to make things right, this is unintuitive and ugly.
We should design a Flow to allow users to Refresh Data explicitly.
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on issue #4529: [Feature][infra] Refresh data in non-incremental manner to mitigate records being deleted in DataSource
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #4529:
URL: https://github.com/apache/incubator-devlake/issues/4529#issuecomment-1451438230
API: add option to SyncPolicy
config-ui: adopts the option
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on issue #4529: [Feature][infra] Refresh data in non-incremental manner to mitigate records being deleted in DataSource
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #4529:
URL: https://github.com/apache/incubator-devlake/issues/4529#issuecomment-1449193506
@Startrekzky @yumengwang03 need prototype for user flow design.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] github-actions[bot] closed issue #4529: [Feature][infra] Refresh data in non-incremental manner to mitigate records being deleted in DataSource
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #4529: [Feature][infra] Refresh data in non-incremental manner to mitigate records being deleted in DataSource
URL: https://github.com/apache/incubator-devlake/issues/4529
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 closed issue #4529: [Feature][infra] Refresh data in non-incremental manner to mitigate records being deleted in DataSource
Posted by "keon94 (via GitHub)" <gi...@apache.org>.
keon94 closed issue #4529: [Feature][infra] Refresh data in non-incremental manner to mitigate records being deleted in DataSource
URL: https://github.com/apache/incubator-devlake/issues/4529
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] github-actions[bot] commented on issue #4529: [Feature][infra] Refresh data in non-incremental manner to mitigate records being deleted in DataSource
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4529:
URL: https://github.com/apache/incubator-devlake/issues/4529#issuecomment-1553836629
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] github-actions[bot] commented on issue #4529: [Feature][infra] Refresh data in non-incremental manner to mitigate records being deleted in DataSource
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4529:
URL: https://github.com/apache/incubator-devlake/issues/4529#issuecomment-1544922266
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #4529: [Feature][infra] Refresh data in non-incremental manner to mitigate records being deleted in DataSource
Posted by "keon94 (via GitHub)" <gi...@apache.org>.
keon94 commented on issue #4529:
URL: https://github.com/apache/incubator-devlake/issues/4529#issuecomment-1571348144
Refreshing Data can now be accomplished by deleting a scope and re-adding it. So closing this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org