You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by "shuwenwei (via GitHub)" <gi...@apache.org> on 2026/01/08 03:31:52 UTC
[PR] Detect inconsistent chunk type when querying or compacting tsfile [iotdb]
shuwenwei opened a new pull request, #16999:
URL: https://github.com/apache/iotdb/pull/16999
## Description
1. Detect inconsistent chunk type when querying or compacting tsfile
2. Add statistics for count of chunk with inconsistent chunk type
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Detect inconsistent chunk type when querying or compacting tsfile [iotdb]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #16999:
URL: https://github.com/apache/iotdb/pull/16999#issuecomment-3721843493
## [Codecov](https://app.codecov.io/gh/apache/iotdb/pull/16999?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `64.51613%` with `11 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 39.20%. Comparing base ([`412fec5`](https://app.codecov.io/gh/apache/iotdb/commit/412fec5a7c1c6858c365784f3ddc02f824932c77?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`e18a936`](https://app.codecov.io/gh/apache/iotdb/commit/e18a93641483f3f6d1e904a2a3e041fdff8ec0a4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/iotdb/pull/16999?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...r/fast/FastNonAlignedSeriesCompactionExecutor.java](https://app.codecov.io/gh/apache/iotdb/pull/16999?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fstorageengine%2Fdataregion%2Fcompaction%2Fexecute%2Futils%2Fexecutor%2Ffast%2FFastNonAlignedSeriesCompactionExecutor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvc3RvcmFnZWVuZ2luZS9kYXRhcmVnaW9uL2NvbXBhY3Rpb24vZXhlY3V0ZS91dGlscy9leGVjdXRvci9mYXN0L0Zhc3ROb25BbGlnbmVkU2VyaWVzQ29tcGFjdGlvbkV4ZWN1dG9yLmphdmE=) | 33.33% | [4 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/16999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...b/db/exception/ChunkTypeInconsistentException.java](https://app.codecov.io/gh/apache/iotdb/pull/16999?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fexception%2FChunkTypeInconsistentException.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvZXhjZXB0aW9uL0NodW5rVHlwZUluY29uc2lzdGVudEV4Y2VwdGlvbi5qYXZh) | 33.33% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/16999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ueryengine/execution/fragment/QueryStatistics.java](https://app.codecov.io/gh/apache/iotdb/pull/16999?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fexecution%2Ffragment%2FQueryStatistics.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvZXhlY3V0aW9uL2ZyYWdtZW50L1F1ZXJ5U3RhdGlzdGljcy5qYXZh) | 50.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/16999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...utor/readchunk/SingleSeriesCompactionExecutor.java](https://app.codecov.io/gh/apache/iotdb/pull/16999?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fstorageengine%2Fdataregion%2Fcompaction%2Fexecute%2Futils%2Fexecutor%2Freadchunk%2FSingleSeriesCompactionExecutor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvc3RvcmFnZWVuZ2luZS9kYXRhcmVnaW9uL2NvbXBhY3Rpb24vZXhlY3V0ZS91dGlscy9leGVjdXRvci9yZWFkY2h1bmsvU2luZ2xlU2VyaWVzQ29tcGFjdGlvbkV4ZWN1dG9yLmphdmE=) | 50.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/16999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...e/statistics/FragmentInstanceStatisticsDrawer.java](https://app.codecov.io/gh/apache/iotdb/pull/16999?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fstatistics%2FFragmentInstanceStatisticsDrawer.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvc3RhdGlzdGljcy9GcmFnbWVudEluc3RhbmNlU3RhdGlzdGljc0RyYXdlci5qYXZh) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/16999?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #16999 +/- ##
============================================
- Coverage 39.27% 39.20% -0.07%
Complexity 212 212
============================================
Files 5073 5080 +7
Lines 339848 340599 +751
Branches 43401 43481 +80
============================================
+ Hits 133484 133548 +64
- Misses 206364 207051 +687
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/iotdb/pull/16999?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Detect inconsistent chunk type when querying or compacting tsfile & fix TsTable(TsTable origin) constructor [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #16999:
URL: https://github.com/apache/iotdb/pull/16999#issuecomment-3722345887
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=16999) **Quality Gate passed**
Issues
 [9 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=16999&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=16999&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=16999&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=16999&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=16999&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=16999)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Detect inconsistent chunk type when querying or compacting tsfile & fix TsTable(TsTable origin) constructor [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #16999:
URL: https://github.com/apache/iotdb/pull/16999#issuecomment-3722454535
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=16999) **Quality Gate passed**
Issues
 [9 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=16999&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=16999&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=16999&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=16999&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=16999&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=16999)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Detect inconsistent chunk type when querying or compacting tsfile [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #16999:
URL: https://github.com/apache/iotdb/pull/16999#issuecomment-3721742949
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=16999) **Quality Gate passed**
Issues
 [1 New issue](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=16999&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=16999&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=16999&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=16999&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=16999&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=16999)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Detect inconsistent chunk type when querying or compacting tsfile [iotdb]
Posted by "JackieTien97 (via GitHub)" <gi...@apache.org>.
JackieTien97 commented on code in PR #16999:
URL: https://github.com/apache/iotdb/pull/16999#discussion_r2671030759
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/read/reader/chunk/DiskChunkLoader.java:
##########
@@ -86,6 +92,21 @@ public IChunkReader getChunkReader(IChunkMetadata chunkMetaData, Filter globalTi
chunkMetaData.getDeleteIntervalList(),
chunkMetaData.getStatistics(),
context);
+ byte chunkType = chunk.getHeader().getChunkType();
+ if (chunkType != MetaMarker.CHUNK_HEADER
+ && chunkType != MetaMarker.ONLY_ONE_PAGE_CHUNK_HEADER) {
+ if (context.getQueryStatistics().getChunkWithMetadataErrorsCount().getAndAdd(1) == 0) {
+ LOGGER.warn(
+ "Unexpected chunk type detected when reading non-aligned chunk reader. "
+ + "The chunk metadata indicates a non-aligned chunk, "
+ + "but the actual chunk read from tsfile is a value chunk of aligned series. "
+ + "tsFile={}, chunkOffset={}, chunkType={}",
+ resource.getTsFilePath(),
+ chunkMetaData.getOffsetOfChunkHeader(),
+ chunkType);
Review Comment:
don't print it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Detect inconsistent chunk type when querying or compacting tsfile & fix TsTable(TsTable origin) constructor [iotdb]
Posted by "JackieTien97 (via GitHub)" <gi...@apache.org>.
JackieTien97 merged PR #16999:
URL: https://github.com/apache/iotdb/pull/16999
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org