You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by "CamilleTeruel (via GitHub)" <gi...@apache.org> on 2023/07/31 03:50:20 UTC

[GitHub] [incubator-devlake] CamilleTeruel opened a new issue, #4880: [Feature][pydevlake] Better support for incremental collection

CamilleTeruel opened a new issue, #4880:
URL: https://github.com/apache/incubator-devlake/issues/4880

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar feature requirement.
   
   
   ### Use case
   
   As a python plugin developer, I want to have an easier way of declaring which attribute are used as checkpoint for incremental collection.
   
   ### Description
   
   Currently, `collect` methods get access to a state dict parameter that they can fill with arbitrary data that they will need in subsequent execution to implement incremental data collection.
   This is too low-level and complexifies the implementation of those methods.
   We should introduce a higher-level "bookmark" or "checkpoint" concept that is used to extract and compare specific JSON properties in order to limit collection to new data only. 
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] github-actions[bot] commented on issue #4880: [Feature][pydevlake] Better support for incremental collection

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4880:
URL: https://github.com/apache/incubator-devlake/issues/4880#issuecomment-1646699819

   This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature][pydevlake] Better support for incremental collection [incubator-devlake]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #4880: [Feature][pydevlake] Better support for incremental collection
URL: https://github.com/apache/incubator-devlake/issues/4880


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] github-actions[bot] commented on issue #4880: [Feature][pydevlake] Better support for incremental collection

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4880:
URL: https://github.com/apache/incubator-devlake/issues/4880#issuecomment-1668733945

   This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature][pydevlake] Better support for incremental collection [incubator-devlake]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4880:
URL: https://github.com/apache/incubator-devlake/issues/4880#issuecomment-1806953618

   This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] github-actions[bot] closed issue #4880: [Feature][pydevlake] Better support for incremental collection

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #4880: [Feature][pydevlake] Better support for incremental collection
URL: https://github.com/apache/incubator-devlake/issues/4880


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] github-actions[bot] commented on issue #4880: [Feature][pydevlake] Better support for incremental collection

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4880:
URL: https://github.com/apache/incubator-devlake/issues/4880#issuecomment-1537579761

   This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] github-actions[bot] commented on issue #4880: [Feature][pydevlake] Better support for incremental collection

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4880:
URL: https://github.com/apache/incubator-devlake/issues/4880#issuecomment-1635083511

   This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature][pydevlake] Better support for incremental collection [incubator-devlake]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4880:
URL: https://github.com/apache/incubator-devlake/issues/4880#issuecomment-1793589173

   This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-devlake] github-actions[bot] closed issue #4880: [Feature][pydevlake] Better support for incremental collection

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #4880: [Feature][pydevlake] Better support for incremental collection
URL: https://github.com/apache/incubator-devlake/issues/4880


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org