You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by "HTHou (via GitHub)" <gi...@apache.org> on 2023/11/01 09:03:48 UTC
[PR] fix empty resource recovered from wal [iotdb]
HTHou opened a new pull request, #11452:
URL: https://github.com/apache/iotdb/pull/11452
## Description
#11404 fixed when inserting empty Tablet and flush, some empty TsFileResource generated. However, after inserting empty Tablet, killing and restarting DataNode, the empty TsFileResource will still be generated.
To fix it, we need to add the deletion logic of #11404 to the position of resource recovered from WAL.
## How to reproduce
1. start 1C1D iotdb and execute the following code.
```java
public static void main(String[] args)
throws IoTDBConnectionException, StatementExecutionException {
session =
new Session.Builder()
.host(LOCAL_HOST)
.port(6667)
.username("root")
.password("root")
.version(Version.V_1_0)
.build();
session.open(false);
insertTablet();
session.close();
}
private static void insertTablet() throws IoTDBConnectionException, StatementExecutionException {
// The schema of measurements of one device
// only measurementId and data type in MeasurementSchema take effects in Tablet
List<MeasurementSchema> schemaList = new ArrayList<>();
schemaList.add(new MeasurementSchema("s1", TSDataType.INT64));
schemaList.add(new MeasurementSchema("s2", TSDataType.INT64));
schemaList.add(new MeasurementSchema("s3", TSDataType.INT64));
Tablet tablet = new Tablet(ROOT_SG1_D1, schemaList, 100);
// Method 1 to add tablet data
long timestamp = System.currentTimeMillis();
for (long row = 0; row < 100; row++) {
int rowIndex = tablet.rowSize++;
tablet.addTimestamp(rowIndex, timestamp);
for (int s = 0; s < 3; s++) {
tablet.addValue(schemaList.get(s).getMeasurementId(), rowIndex, null);
}
if (tablet.rowSize == tablet.getMaxRowNumber()) {
session.insertTablet(tablet, true);
tablet.reset();
}
timestamp++;
}
if (tablet.rowSize != 0) {
session.insertTablet(tablet);
tablet.reset();
}
}
```
2. kill datanode by `kill -9 <pid>`
3. restart datanode, then print the TsFileResource. You will see the TsFileResource doesn't contains any devices.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix empty resource recovered from wal [iotdb]
Posted by "xingtanzjr (via GitHub)" <gi...@apache.org>.
xingtanzjr merged PR #11452:
URL: https://github.com/apache/iotdb/pull/11452
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix empty resource recovered from wal [iotdb]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #11452:
URL: https://github.com/apache/iotdb/pull/11452#issuecomment-1788642867
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=11452)
[](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=11452&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=11452&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=11452&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=11452&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=11452&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=11452&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=11452&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=11452&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=11452&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=11452&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=11452&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=11452&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=11452&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=11452&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=11452&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=11452&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix empty resource recovered from wal [iotdb]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #11452:
URL: https://github.com/apache/iotdb/pull/11452#issuecomment-1788715477
## [Codecov](https://app.codecov.io/gh/apache/iotdb/pull/11452?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
> Merging [#11452](https://app.codecov.io/gh/apache/iotdb/pull/11452?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (9db0ad9) into [master](https://app.codecov.io/gh/apache/iotdb/commit/e7592c09d912ef315cfd060b18383c4eea23b669?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (e7592c0) will **decrease** coverage by `0.03%`.
> Report is 1 commits behind head on master.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #11452 +/- ##
============================================
- Coverage 49.14% 49.12% -0.03%
+ Complexity 24556 24546 -10
============================================
Files 2768 2768
Lines 172197 172200 +3
Branches 20603 20604 +1
============================================
- Hits 84619 84585 -34
- Misses 87578 87615 +37
```
| [Files](https://app.codecov.io/gh/apache/iotdb/pull/11452?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../iotdb/db/storageengine/dataregion/DataRegion.java](https://app.codecov.io/gh/apache/iotdb/pull/11452?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvc3RvcmFnZWVuZ2luZS9kYXRhcmVnaW9uL0RhdGFSZWdpb24uamF2YQ==) | `50.61% <0.00%> (-0.11%)` | :arrow_down: |
... and [20 files with indirect coverage changes](https://app.codecov.io/gh/apache/iotdb/pull/11452/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org