You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by "SteveYurongSu (via GitHub)" <gi...@apache.org> on 2025/06/18 11:03:00 UTC
[PR] Pipe/IoTV2: Persist progress index locally before shutdown to accurate recovery after restart [iotdb]
SteveYurongSu opened a new pull request, #15779:
URL: https://github.com/apache/iotdb/pull/15779
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe/IoTV2: Persist progress index locally before shutdown to accurate recovery after restart [iotdb]
Posted by "SteveYurongSu (via GitHub)" <gi...@apache.org>.
SteveYurongSu merged PR #15779:
URL: https://github.com/apache/iotdb/pull/15779
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe/IoTV2: Persist progress index locally before shutdown to accurate recovery after restart [iotdb]
Posted by "Pengzna (via GitHub)" <gi...@apache.org>.
Pengzna commented on code in PR #15779:
URL: https://github.com/apache/iotdb/pull/15779#discussion_r2154548680
##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/agent/task/meta/PipeRuntimeMeta.java:
##########
@@ -140,6 +140,15 @@ public void setIsStoppedByRuntimeException(boolean isStoppedByRuntimeException)
this.isStoppedByRuntimeException.set(isStoppedByRuntimeException);
}
+ public void persistProgressIndex() {
+ // Iterate through all the task metas and persist their progress index
+ for (final PipeTaskMeta taskMeta : consensusGroupId2TaskMetaMap.values()) {
+ if (taskMeta.getProgressIndex() != null) {
Review Comment:
Objects.nonNull(progressIndexPersistFile) && PipeConfig.getInstance().isPipeProgressIndexPersistEnabled() && taskMeta.getProgressIndex() != null?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe/IoTV2: Persist progress index locally before shutdown to accurate recovery after restart [iotdb]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #15779:
URL: https://github.com/apache/iotdb/pull/15779#issuecomment-2983914387
## [Codecov](https://app.codecov.io/gh/apache/iotdb/pull/15779?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: Patch coverage is `0%` with `17 lines` in your changes missing coverage. Please review.
> Project coverage is 39.11%. Comparing base [(`bbe031a`)](https://app.codecov.io/gh/apache/iotdb/commit/bbe031aaebe24a5587937139bd5837a8c572f358?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`766c8c1`)](https://app.codecov.io/gh/apache/iotdb/commit/766c8c1afa1b780ba9d3dfced8f17075fb506a0b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 2 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/iotdb/pull/15779?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...otdb/db/pipe/agent/task/PipeDataNodeTaskAgent.java](https://app.codecov.io/gh/apache/iotdb/pull/15779?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fagent%2Ftask%2FPipeDataNodeTaskAgent.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9hZ2VudC90YXNrL1BpcGVEYXRhTm9kZVRhc2tBZ2VudC5qYXZh) | 0.00% | [11 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/15779?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../commons/pipe/agent/task/meta/PipeRuntimeMeta.java](https://app.codecov.io/gh/apache/iotdb/pull/15779?src=pr&el=tree&filepath=iotdb-core%2Fnode-commons%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fcommons%2Fpipe%2Fagent%2Ftask%2Fmeta%2FPipeRuntimeMeta.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9ub2RlLWNvbW1vbnMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2lvdGRiL2NvbW1vbnMvcGlwZS9hZ2VudC90YXNrL21ldGEvUGlwZVJ1bnRpbWVNZXRhLmphdmE=) | 0.00% | [5 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/15779?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../apache/iotdb/db/service/DataNodeShutdownHook.java](https://app.codecov.io/gh/apache/iotdb/pull/15779?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fservice%2FDataNodeShutdownHook.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvc2VydmljZS9EYXRhTm9kZVNodXRkb3duSG9vay5qYXZh) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/15779?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #15779 +/- ##
============================================
- Coverage 39.12% 39.11% -0.01%
Complexity 198 198
============================================
Files 4837 4837
Lines 314486 314506 +20
Branches 39449 39452 +3
============================================
- Hits 123033 123021 -12
- Misses 191453 191485 +32
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/iotdb/pull/15779?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe/IoTV2: Persist progress index locally before shutdown to accurate recovery after restart [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #15779:
URL: https://github.com/apache/iotdb/pull/15779#issuecomment-2983757895
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15779) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=15779&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=15779&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=15779&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=15779&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=15779&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15779)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe/IoTV2: Persist progress index locally before shutdown to accurate recovery after restart [iotdb]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #15779:
URL: https://github.com/apache/iotdb/pull/15779#discussion_r2154318502
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/task/PipeDataNodeTaskAgent.java:
##########
@@ -840,6 +840,25 @@ private boolean isSnapshotMode(final PipeParameters parameters) {
return isSnapshotMode;
}
+ ///////////////////////// Shutdown Logic /////////////////////////
+
+ public void persistAllProgressIndexLocally() {
+ if (!tryReadLockWithTimeOut(10)) {
Review Comment:
[nitpick] Avoid using a magic number for the timeout; extract '10' into a named constant or configuration parameter for clarity and flexibility.
```suggestion
if (!tryReadLockWithTimeOut(DEFAULT_READ_LOCK_TIMEOUT_SECONDS)) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org