You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by "shuwenwei (via GitHub)" <gi...@apache.org> on 2023/11/01 10:17:01 UTC
[PR] Check empty target file after sequence InnerSpaceCompactionTask [iotdb]
shuwenwei opened a new pull request, #11454:
URL: https://github.com/apache/iotdb/pull/11454
## Description
1. Mark target file as DELETED status if the target file is empty in ReadChunkCompactionPerformer
2. Skip the target file which is marked as DELETED status when validate compaction result
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Check empty target file after sequence InnerSpaceCompactionTask [iotdb]
Posted by "caozj1011 (via GitHub)" <gi...@apache.org>.
caozj1011 commented on code in PR #11454:
URL: https://github.com/apache/iotdb/pull/11454#discussion_r1381208623
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/task/AbstractCompactionTask.java:
##########
@@ -372,21 +374,56 @@ public boolean isDiskSpaceCheckPassed() {
return CompactionUtils.isDiskHasSpace();
}
- protected void validateTsFileResource(
- List<TsFileResource> targetTsFileList, boolean needValidateOverlap) {
+ protected void validateCompactionResult(
+ List<TsFileResource> sourceSeqFiles,
+ List<TsFileResource> sourceUnseqFiles,
+ List<TsFileResource> targetFiles)
+ throws CompactionValidationFailedException {
+ // skip TsFileResource which is marked as DELETED status
+ List<TsFileResource> validTargetFiles =
+ targetFiles.stream().filter(resource -> !resource.isDeleted()).collect(Collectors.toList());
+ CompactionTaskType taskType = getCompactionTaskType();
+ boolean needToValidateTsFileCorrectness = taskType != CompactionTaskType.INSERTION;
Review Comment:
These **CompactionTaskType** are supposed to be specified by task new or another step, but I can't find a place to assign a value to the task, make sure the task corresponds to its task type
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Check empty target file after sequence InnerSpaceCompactionTask [iotdb]
Posted by "caozj1011 (via GitHub)" <gi...@apache.org>.
caozj1011 commented on code in PR #11454:
URL: https://github.com/apache/iotdb/pull/11454#discussion_r1381200837
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/performer/impl/ReadChunkCompactionPerformer.java:
##########
@@ -92,6 +92,9 @@ public void perform()
targetResource.updatePlanIndexes(tsFileResource);
}
writer.endFile();
+ if (writer.isEmptyTargetFile()) {
Review Comment:
Do you need to judge other performers as well？
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/performer/impl/ReadChunkCompactionPerformer.java:
##########
@@ -92,6 +92,9 @@ public void perform()
targetResource.updatePlanIndexes(tsFileResource);
}
writer.endFile();
+ if (writer.isEmptyTargetFile()) {
Review Comment:
Do you need to judge other performers as well？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Check empty target file after sequence InnerSpaceCompactionTask [iotdb]
Posted by "xingtanzjr (via GitHub)" <gi...@apache.org>.
xingtanzjr merged PR #11454:
URL: https://github.com/apache/iotdb/pull/11454
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org