You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by "Caideyipi (via GitHub)" <gi...@apache.org> on 2026/06/29 04:34:46 UTC
[PR] Improve pipe runtime memory allocation [iotdb]
Caideyipi opened a new pull request, #18051:
URL: https://github.com/apache/iotdb/pull/18051
## Summary
- allocate pipe TsFile parser, sink batch, and TsFile read buffers from PipeMemoryManager only while active
- resize tablet and IoTConsensusV2 batch memory by actual buffered data, and release it on success, discard, or close
- bound TsFile read buffer allocation by the current file size across thrift, airgap, legacy, and IoTConsensusV2 sinks
## Tests
- mvn spotless:apply -pl iotdb-core/datanode
- git diff --check
- mvn -pl iotdb-core/datanode -DskipTests compile (fails on existing non-pipe compile issues: missing commons-collections4 ComparatorChain, missing SystemConstant authentication keys, missing ColumnHeaderConstant showCreate headers, and related generated aggregation symbols)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #18051:
URL: https://github.com/apache/iotdb/pull/18051#issuecomment-4849797142
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051) **Quality Gate failed**
Failed conditions
 [11.3% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=18051&metric=new_duplicated_lines_density&view=list) (required ≤ 5%)
<!-- sqra:qg_revision=8aa6dfefc9288cb7ef2b1306d65c363c000f9ad9ad2494199c54cc214ddd3e1e -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "Caideyipi (via GitHub)" <gi...@apache.org>.
Caideyipi commented on code in PR #18051:
URL: https://github.com/apache/iotdb/pull/18051#discussion_r3517854500
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/parser/scan/TsFileInsertionEventScanParser.java:
##########
@@ -235,6 +231,7 @@ public TabletInsertionEvent next() {
final Tablet tablet = getNextTablet();
// Record tablet metrics
recordTabletMetrics(tablet);
+ releaseTabletMemoryBlock();
Review Comment:
Applied in c940475af1d: the previous parser-owned tablet memory is released before fetching the next tablet, so the ownership/order is clearer.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/payload/evolvable/batch/PipeTabletEventBatch.java:
##########
@@ -126,7 +129,28 @@ public synchronized boolean onEvent(final TabletInsertionEvent event)
protected abstract boolean constructBatch(final TabletInsertionEvent event)
throws WALPipeException, IOException;
+ protected void increaseTotalBufferSizeAndUpdateMemoryBlock(final long bufferSize) {
+ if (bufferSize <= 0) {
+ return;
+ }
+
+ final long newTotalBufferSize = totalBufferSize + bufferSize;
+ PipeDataNodeResourceManager.memory()
+ .forceResize(allocatedMemoryBlock, Math.min(newTotalBufferSize, maxBatchSizeInBytes));
+ totalBufferSize = newTotalBufferSize;
Review Comment:
Applied in c940475af1d: totalBufferSize is clamped with Math.min(totalBufferSize + bufferSize, maxBatchSizeInBytes) before resizing.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/protocol/iotconsensusv2/payload/builder/IoTConsensusV2TransferBatchReqBuilder.java:
##########
@@ -137,27 +113,80 @@ public synchronized boolean onEvent(TabletInsertionEvent event)
return false;
}
- final long requestCommitId = ((EnrichedEvent) event).getReplicateIndexForIoTV2();
+ final EnrichedEvent enrichedEvent = (EnrichedEvent) event;
+ final long requestCommitId = enrichedEvent.getReplicateIndexForIoTV2();
// The deduplication logic here is to avoid the accumulation of the same event in a batch when
// retrying.
if ((events.isEmpty() || !events.get(events.size() - 1).equals(event))) {
- events.add((EnrichedEvent) event);
- requestCommitIds.add(requestCommitId);
- final int bufferSize = buildTabletInsertionBuffer(event);
-
- ((EnrichedEvent) event)
- .increaseReferenceCount(IoTConsensusV2TransferBatchReqBuilder.class.getName());
+ if (!enrichedEvent.increaseReferenceCount(
+ IoTConsensusV2TransferBatchReqBuilder.class.getName())) {
+ LOGGER.warn(DataNodePipeMessages.CANNOT_INCREASE_REFERENCE_COUNT_FOR_EVENT_IGNORE, event);
+ return shouldEmit();
+ }
- if (firstEventProcessingTime == Long.MIN_VALUE) {
- firstEventProcessingTime = System.currentTimeMillis();
+ final int previousEventsSize = events.size();
+ final int previousRequestCommitIdsSize = requestCommitIds.size();
+ final int previousBatchReqsSize = batchReqs.size();
+ try {
+ events.add(enrichedEvent);
+ requestCommitIds.add(requestCommitId);
+ final int bufferSize = buildTabletInsertionBuffer(event);
+ increaseTotalBufferSizeAndUpdateMemoryBlock(bufferSize);
+
+ if (firstEventProcessingTime == Long.MIN_VALUE) {
+ firstEventProcessingTime = System.currentTimeMillis();
+ }
+ } catch (final Exception e) {
+ rollbackTo(previousEventsSize, previousRequestCommitIdsSize, previousBatchReqsSize);
+ if (events.isEmpty()) {
+ resetMemoryUsage();
+ }
+ enrichedEvent.decreaseReferenceCount(
+ IoTConsensusV2TransferBatchReqBuilder.class.getName(), false);
+ throw e;
}
+ }
+
+ return shouldEmit();
+ }
- totalBufferSize += bufferSize;
+ private boolean shouldEmit() {
+ return !events.isEmpty()
+ && (totalBufferSize >= getMaxBatchSizeInBytes()
+ || System.currentTimeMillis() - firstEventProcessingTime >= maxDelayInMs);
+ }
+
+ private void increaseTotalBufferSizeAndUpdateMemoryBlock(final long bufferSize) {
+ if (bufferSize <= 0) {
+ return;
}
- return totalBufferSize >= getMaxBatchSizeInBytes()
- || System.currentTimeMillis() - firstEventProcessingTime >= maxDelayInMs;
+ final long newTotalBufferSize = totalBufferSize + bufferSize;
+ PipeDataNodeResourceManager.memory()
+ .forceResize(allocatedMemoryBlock, Math.min(newTotalBufferSize, getMaxBatchSizeInBytes()));
+ totalBufferSize = newTotalBufferSize;
+ }
+
+ private void rollbackTo(
+ final int previousEventsSize,
+ final int previousRequestCommitIdsSize,
+ final int previousBatchReqsSize) {
+ while (events.size() > previousEventsSize) {
+ events.remove(events.size() - 1);
+ }
+ while (requestCommitIds.size() > previousRequestCommitIdsSize) {
+ requestCommitIds.remove(requestCommitIds.size() - 1);
+ }
+ while (batchReqs.size() > previousBatchReqsSize) {
+ batchReqs.remove(batchReqs.size() - 1);
+ }
+ }
Review Comment:
Applied in c940475af1d: rollback now uses subList(...).clear() for events, commit ids, and batch requests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "jt2594838 (via GitHub)" <gi...@apache.org>.
jt2594838 commented on code in PR #18051:
URL: https://github.com/apache/iotdb/pull/18051#discussion_r3526075948
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/task/PipeDataNodeTaskAgent.java:
##########
Review Comment:
i18n
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/protocol/thrift/sync/IoTDBDataRegionSyncSink.java:
##########
@@ -612,73 +612,114 @@ protected void transferFilePieces(
final RandomAccessFile reader = new RandomAccessFile(file, "r")) {
final byte[] readBuffer = new byte[readFileBufferSize];
long position = 0;
- while (true) {
- mayLimitRateAndRecordIO(readFileBufferSize);
- final int readLength = reader.read(readBuffer);
- if (readLength == -1) {
- break;
- }
+ int readLength;
+ while ((readLength = readNextFilePiece(reader, readBuffer, readFileBufferSize)) != -1) {
+ position =
+ transferFilePiece(
+ pipe2WeightMap,
+ file,
+ clientAndStatus,
+ isMultiFile,
+ readBuffer,
+ position,
+ readLength,
+ reader);
+ }
+ }
+ }
- final byte[] payLoad =
- readLength == readFileBufferSize
- ? readBuffer
- : Arrays.copyOfRange(readBuffer, 0, readLength);
- final PipeTransferFilePieceResp resp;
- try {
- final TPipeTransferReq req =
- compressIfNeeded(
- isMultiFile
- ? getTransferMultiFilePieceReq(file.getName(), position, payLoad)
- : getTransferSingleFilePieceReq(file.getName(), position, payLoad));
- pipe2WeightMap.forEach(
- (namePair, weight) ->
- rateLimitIfNeeded(
- namePair.getLeft(),
- namePair.getRight(),
- clientAndStatus.getLeft().getEndPoint(),
- (long) (req.getBody().length * weight)));
- resp =
- PipeTransferFilePieceResp.fromTPipeTransferResp(
- clientAndStatus.getLeft().pipeTransfer(req));
- } catch (final Exception e) {
- clientAndStatus.setRight(false);
- throw new PipeConnectionException(
- String.format(
- "Network error when transfer file %s, because %s.", file, e.getMessage()),
- e);
- }
+ private int readNextFilePiece(
+ final RandomAccessFile reader, final byte[] readBuffer, final int readFileBufferSize)
+ throws IOException {
+ mayLimitRateAndRecordIO(readFileBufferSize);
+ return reader.read(readBuffer);
+ }
- position += readLength;
+ private long transferFilePiece(
+ final Map<Pair<String, Long>, Double> pipe2WeightMap,
+ final File file,
+ final Pair<IoTDBSyncClient, Boolean> clientAndStatus,
+ final boolean isMultiFile,
+ final byte[] readBuffer,
+ final long position,
+ final int readLength,
+ final RandomAccessFile reader)
+ throws PipeException, IOException {
+ final byte[] payLoad = buildFilePiecePayload(readBuffer, readLength);
+ final PipeTransferFilePieceResp resp =
+ doTransferFilePiece(pipe2WeightMap, file, clientAndStatus, isMultiFile, payLoad, position);
+ return handleTransferFilePieceResp(file, clientAndStatus, reader, position + readLength, resp);
+ }
- final TSStatus status = resp.getStatus();
- if (status.getCode() == TSStatusCode.PIPE_TRANSFER_FILE_OFFSET_RESET.getStatusCode()) {
- position = resp.getEndWritingOffset();
- reader.seek(position);
- LOGGER.info(DataNodePipeMessages.REDIRECT_FILE_POSITION_TO, position);
- continue;
- }
+ private byte[] buildFilePiecePayload(final byte[] readBuffer, final int readLength) {
+ return readLength == readBuffer.length
+ ? readBuffer
+ : Arrays.copyOfRange(readBuffer, 0, readLength);
+ }
- if (status.getCode()
- == TSStatusCode.PIPE_CONFIG_RECEIVER_HANDSHAKE_NEEDED.getStatusCode()) {
- getClientManager().sendHandshakeReq(clientAndStatus);
- }
+ private PipeTransferFilePieceResp doTransferFilePiece(
+ final Map<Pair<String, Long>, Double> pipe2WeightMap,
+ final File file,
+ final Pair<IoTDBSyncClient, Boolean> clientAndStatus,
+ final boolean isMultiFile,
+ final byte[] payLoad,
+ final long position)
+ throws PipeException, IOException {
+ try {
+ final TPipeTransferReq req =
+ compressIfNeeded(
+ isMultiFile
+ ? getTransferMultiFilePieceReq(file.getName(), position, payLoad)
+ : getTransferSingleFilePieceReq(file.getName(), position, payLoad));
+ pipe2WeightMap.forEach(
+ (namePair, weight) ->
+ rateLimitIfNeeded(
+ namePair.getLeft(),
+ namePair.getRight(),
+ clientAndStatus.getLeft().getEndPoint(),
+ (long) (req.getBody().length * weight)));
+ return PipeTransferFilePieceResp.fromTPipeTransferResp(
+ clientAndStatus.getLeft().pipeTransfer(req));
+ } catch (final Exception e) {
+ clientAndStatus.setRight(false);
+ throw new PipeConnectionException(
+ String.format("Network error when transfer file %s, because %s.", file, e.getMessage()),
+ e);
+ }
Review Comment:
i18n
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "Caideyipi (via GitHub)" <gi...@apache.org>.
Caideyipi commented on code in PR #18051:
URL: https://github.com/apache/iotdb/pull/18051#discussion_r3526260240
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/protocol/thrift/sync/IoTDBDataRegionSyncSink.java:
##########
@@ -612,73 +612,114 @@ protected void transferFilePieces(
final RandomAccessFile reader = new RandomAccessFile(file, "r")) {
final byte[] readBuffer = new byte[readFileBufferSize];
long position = 0;
- while (true) {
- mayLimitRateAndRecordIO(readFileBufferSize);
- final int readLength = reader.read(readBuffer);
- if (readLength == -1) {
- break;
- }
+ int readLength;
+ while ((readLength = readNextFilePiece(reader, readBuffer, readFileBufferSize)) != -1) {
+ position =
+ transferFilePiece(
+ pipe2WeightMap,
+ file,
+ clientAndStatus,
+ isMultiFile,
+ readBuffer,
+ position,
+ readLength,
+ reader);
+ }
+ }
+ }
- final byte[] payLoad =
- readLength == readFileBufferSize
- ? readBuffer
- : Arrays.copyOfRange(readBuffer, 0, readLength);
- final PipeTransferFilePieceResp resp;
- try {
- final TPipeTransferReq req =
- compressIfNeeded(
- isMultiFile
- ? getTransferMultiFilePieceReq(file.getName(), position, payLoad)
- : getTransferSingleFilePieceReq(file.getName(), position, payLoad));
- pipe2WeightMap.forEach(
- (namePair, weight) ->
- rateLimitIfNeeded(
- namePair.getLeft(),
- namePair.getRight(),
- clientAndStatus.getLeft().getEndPoint(),
- (long) (req.getBody().length * weight)));
- resp =
- PipeTransferFilePieceResp.fromTPipeTransferResp(
- clientAndStatus.getLeft().pipeTransfer(req));
- } catch (final Exception e) {
- clientAndStatus.setRight(false);
- throw new PipeConnectionException(
- String.format(
- "Network error when transfer file %s, because %s.", file, e.getMessage()),
- e);
- }
+ private int readNextFilePiece(
+ final RandomAccessFile reader, final byte[] readBuffer, final int readFileBufferSize)
+ throws IOException {
+ mayLimitRateAndRecordIO(readFileBufferSize);
+ return reader.read(readBuffer);
+ }
- position += readLength;
+ private long transferFilePiece(
+ final Map<Pair<String, Long>, Double> pipe2WeightMap,
+ final File file,
+ final Pair<IoTDBSyncClient, Boolean> clientAndStatus,
+ final boolean isMultiFile,
+ final byte[] readBuffer,
+ final long position,
+ final int readLength,
+ final RandomAccessFile reader)
+ throws PipeException, IOException {
+ final byte[] payLoad = buildFilePiecePayload(readBuffer, readLength);
+ final PipeTransferFilePieceResp resp =
+ doTransferFilePiece(pipe2WeightMap, file, clientAndStatus, isMultiFile, payLoad, position);
+ return handleTransferFilePieceResp(file, clientAndStatus, reader, position + readLength, resp);
+ }
- final TSStatus status = resp.getStatus();
- if (status.getCode() == TSStatusCode.PIPE_TRANSFER_FILE_OFFSET_RESET.getStatusCode()) {
- position = resp.getEndWritingOffset();
- reader.seek(position);
- LOGGER.info(DataNodePipeMessages.REDIRECT_FILE_POSITION_TO, position);
- continue;
- }
+ private byte[] buildFilePiecePayload(final byte[] readBuffer, final int readLength) {
+ return readLength == readBuffer.length
+ ? readBuffer
+ : Arrays.copyOfRange(readBuffer, 0, readLength);
+ }
- if (status.getCode()
- == TSStatusCode.PIPE_CONFIG_RECEIVER_HANDSHAKE_NEEDED.getStatusCode()) {
- getClientManager().sendHandshakeReq(clientAndStatus);
- }
+ private PipeTransferFilePieceResp doTransferFilePiece(
+ final Map<Pair<String, Long>, Double> pipe2WeightMap,
+ final File file,
+ final Pair<IoTDBSyncClient, Boolean> clientAndStatus,
+ final boolean isMultiFile,
+ final byte[] payLoad,
+ final long position)
+ throws PipeException, IOException {
+ try {
+ final TPipeTransferReq req =
+ compressIfNeeded(
+ isMultiFile
+ ? getTransferMultiFilePieceReq(file.getName(), position, payLoad)
+ : getTransferSingleFilePieceReq(file.getName(), position, payLoad));
+ pipe2WeightMap.forEach(
+ (namePair, weight) ->
+ rateLimitIfNeeded(
+ namePair.getLeft(),
+ namePair.getRight(),
+ clientAndStatus.getLeft().getEndPoint(),
+ (long) (req.getBody().length * weight)));
+ return PipeTransferFilePieceResp.fromTPipeTransferResp(
+ clientAndStatus.getLeft().pipeTransfer(req));
+ } catch (final Exception e) {
+ clientAndStatus.setRight(false);
+ throw new PipeConnectionException(
+ String.format("Network error when transfer file %s, because %s.", file, e.getMessage()),
+ e);
+ }
Review Comment:
Applied in 1370353caf5: reused the existing transfer-file network error i18n constant and added en/zh i18n for the transfer file status message.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/agent/task/PipeDataNodeTaskAgent.java:
##########
Review Comment:
Applied in 1370353caf5: moved the pipe meta report logs to DataNodePipeMessages with en/zh i18n constants.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #18051:
URL: https://github.com/apache/iotdb/pull/18051#issuecomment-4829250709
## [Codecov](https://app.codecov.io/gh/apache/iotdb/pull/18051?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `17.99163%` with `196 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 41.40%. Comparing base ([`fcd52d7`](https://app.codecov.io/gh/apache/iotdb/commit/fcd52d7ff064e7644fd9429d9fa174c37103d0bf?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`b8a28f5`](https://app.codecov.io/gh/apache/iotdb/commit/b8a28f54586309e3d5fa09ac77672be0f0a4815a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/iotdb/pull/18051?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../protocol/thrift/sync/IoTDBDataRegionSyncSink.java](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fsink%2Fprotocol%2Fthrift%2Fsync%2FIoTDBDataRegionSyncSink.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9zaW5rL3Byb3RvY29sL3RocmlmdC9zeW5jL0lvVERCRGF0YVJlZ2lvblN5bmNTaW5rLmphdmE=) | 0.00% | [55 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...builder/IoTConsensusV2TransferBatchReqBuilder.java](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fsink%2Fprotocol%2Fiotconsensusv2%2Fpayload%2Fbuilder%2FIoTConsensusV2TransferBatchReqBuilder.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9zaW5rL3Byb3RvY29sL2lvdGNvbnNlbnN1c3YyL3BheWxvYWQvYnVpbGRlci9Jb1RDb25zZW5zdXNWMlRyYW5zZmVyQmF0Y2hSZXFCdWlsZGVyLmphdmE=) | 0.00% | [50 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...oad/evolvable/batch/PipeTabletEventPlainBatch.java](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fsink%2Fpayload%2Fevolvable%2Fbatch%2FPipeTabletEventPlainBatch.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9zaW5rL3BheWxvYWQvZXZvbHZhYmxlL2JhdGNoL1BpcGVUYWJsZXRFdmVudFBsYWluQmF0Y2guamF2YQ==) | 14.28% | [18 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ler/IoTConsensusV2TsFileInsertionEventHandler.java](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fsink%2Fprotocol%2Fiotconsensusv2%2Fhandler%2FIoTConsensusV2TsFileInsertionEventHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9zaW5rL3Byb3RvY29sL2lvdGNvbnNlbnN1c3YyL2hhbmRsZXIvSW9UQ29uc2Vuc3VzVjJUc0ZpbGVJbnNlcnRpb25FdmVudEhhbmRsZXIuamF2YQ==) | 0.00% | [18 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...hrift/async/handler/PipeTransferTsFileHandler.java](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fsink%2Fprotocol%2Fthrift%2Fasync%2Fhandler%2FPipeTransferTsFileHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9zaW5rL3Byb3RvY29sL3RocmlmdC9hc3luYy9oYW5kbGVyL1BpcGVUcmFuc2ZlclRzRmlsZUhhbmRsZXIuamF2YQ==) | 0.00% | [18 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../payload/evolvable/batch/PipeTabletEventBatch.java](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fsink%2Fpayload%2Fevolvable%2Fbatch%2FPipeTabletEventBatch.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9zaW5rL3BheWxvYWQvZXZvbHZhYmxlL2JhdGNoL1BpcGVUYWJsZXRFdmVudEJhdGNoLmphdmE=) | 61.76% | [13 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...rotocol/iotconsensusv2/IoTConsensusV2SyncSink.java](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fsink%2Fprotocol%2Fiotconsensusv2%2FIoTConsensusV2SyncSink.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9zaW5rL3Byb3RvY29sL2lvdGNvbnNlbnN1c3YyL0lvVENvbnNlbnN1c1YyU3luY1NpbmsuamF2YQ==) | 0.00% | [10 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...pipe/sink/protocol/legacy/IoTDBLegacyPipeSink.java](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fsink%2Fprotocol%2Flegacy%2FIoTDBLegacyPipeSink.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9zaW5rL3Byb3RvY29sL2xlZ2FjeS9Jb1REQkxlZ2FjeVBpcGVTaW5rLmphdmE=) | 0.00% | [9 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ad/evolvable/batch/PipeTabletEventTsFileBatch.java](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fpipe%2Fsink%2Fpayload%2Fevolvable%2Fbatch%2FPipeTabletEventTsFileBatch.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcGlwZS9zaW5rL3BheWxvYWQvZXZvbHZhYmxlL2JhdGNoL1BpcGVUYWJsZXRFdmVudFRzRmlsZUJhdGNoLmphdmE=) | 44.44% | [5 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18051?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #18051 +/- ##
==========================================
Coverage 41.40% 41.40%
Complexity 318 318
==========================================
Files 5286 5286
Lines 369547 369685 +138
Branches 47815 47828 +13
==========================================
+ Hits 153008 153086 +78
- Misses 216539 216599 +60
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/iotdb/pull/18051?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #18051:
URL: https://github.com/apache/iotdb/pull/18051#issuecomment-4888615706
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051) **Quality Gate passed**
Issues
 [20 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=18051&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=18051&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=18051&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=18051&metric=new_coverage&view=list)
 [0.8% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=18051&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #18051:
URL: https://github.com/apache/iotdb/pull/18051#issuecomment-4875581127
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051) **Quality Gate passed**
Issues
 [18 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=18051&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=18051&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=18051&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=18051&metric=new_coverage&view=list)
 [0.8% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=18051&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #18051:
URL: https://github.com/apache/iotdb/pull/18051#issuecomment-4849849103
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051) **Quality Gate failed**
Failed conditions
 [11.3% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=18051&metric=new_duplicated_lines_density&view=list) (required ≤ 5%)
<!-- sqra:qg_revision=8aa6dfefc9288cb7ef2b1306d65c363c000f9ad9ad2494199c54cc214ddd3e1e -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #18051:
URL: https://github.com/apache/iotdb/pull/18051#issuecomment-4851312723
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051) **Quality Gate failed**
Failed conditions
 [11.3% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=18051&metric=new_duplicated_lines_density&view=list) (required ≤ 5%)
<!-- sqra:qg_revision=8aa6dfefc9288cb7ef2b1306d65c363c000f9ad9ad2494199c54cc214ddd3e1e -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #18051:
URL: https://github.com/apache/iotdb/pull/18051#issuecomment-4874194680
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051) **Quality Gate failed**
Failed conditions
 [11.2% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=18051&metric=new_duplicated_lines_density&view=list) (required ≤ 5%)
<!-- sqra:qg_revision=bba28e099c2081ba3471e980e0293ea9b29f8db493b62181781225ef689c39d4 -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "jt2594838 (via GitHub)" <gi...@apache.org>.
jt2594838 merged PR #18051:
URL: https://github.com/apache/iotdb/pull/18051
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #18051:
URL: https://github.com/apache/iotdb/pull/18051#issuecomment-4888839798
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051) **Quality Gate passed**
Issues
 [19 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=18051&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=18051&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=18051&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=18051&metric=new_coverage&view=list)
 [0.8% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=18051&metric=new_duplicated_lines_density&view=list)
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #18051:
URL: https://github.com/apache/iotdb/pull/18051#issuecomment-4829064334
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051) **Quality Gate failed**
Failed conditions
 [11.7% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=18051&metric=new_duplicated_lines_density&view=list) (required ≤ 5%)
<!-- sqra:qg_revision=53af214e32797387677891d6283ab6cd39278323db52c4e9b1bc4b3f8295f0ec -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #18051:
URL: https://github.com/apache/iotdb/pull/18051#issuecomment-4873428815
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051) **Quality Gate failed**
Failed conditions
 [11.4% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=18051&metric=new_duplicated_lines_density&view=list) (required ≤ 5%)
<!-- sqra:qg_revision=e8bc148948b616acaef601d98c27b201ae1cf697b0d179fbfdd958931cf2d2ef -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve pipe runtime memory allocation [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #18051:
URL: https://github.com/apache/iotdb/pull/18051#issuecomment-4875257779
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051) **Quality Gate failed**
Failed conditions
 [9.8% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=18051&metric=new_duplicated_lines_density&view=list) (required ≤ 5%)
<!-- sqra:qg_revision=e2a2be6928e253735bf5d6dd143af2498e8f71ed7c65eb4c93d4cd52c4c20288 -->
<!-- sqra-placement-anchor -->
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=18051)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org