You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by "Caideyipi (via GitHub)" <gi...@apache.org> on 2026/07/21 04:27:35 UTC
[PR] [Pipe] Back off TsFile parser admission retries [iotdb]
Caideyipi opened a new pull request, #18262:
URL: https://github.com/apache/iotdb/pull/18262
## Description
TsFile parser admission currently retries with a fixed sleep when Pipe memory is exhausted. Concurrent parser requests can therefore wake in lockstep, and a request may keep sleeping even after another parser releases its reservation.
This PR changes parser admission retries to:
- use bounded exponential backoff starting from `pipe_check_memory_enough_interval_ms`;
- cap the interval at the initial interval multiplied by `pipe_memory_allocate_max_retries`;
- add 50%-100% jitter to avoid synchronized retries;
- wait on `PipeMemoryManager` so parser and Pipe memory releases wake admission attempts immediately;
- preserve the existing overall parser admission timeout.
This is independent of receiver/sink backoff in #18236, which does not cover parser admission. Fair scheduling across Pipes remains isolated in #18260.
### Tests
- Verify exponential growth, overflow-safe capping, and jitter bounds.
- Exhaust parser memory with a real TsFile parser request, configure a 10-second retry interval, release one reservation, and verify the parser is notified and admitted within 3 seconds.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] concurrent read and write
- [x] added unit tests for parser admission backoff and release notification.
<hr>
##### Key changed/added classes
- `PipeTsFileInsertionEvent`
- `PipeMemoryManager`
- `PipeTsFileInsertionEventAdmissionTest`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Pipe] Back off TsFile parser admission retries [iotdb]
Posted by "Caideyipi (via GitHub)" <gi...@apache.org>.
Caideyipi commented on code in PR #18262:
URL: https://github.com/apache/iotdb/pull/18262#discussion_r3627886213
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/PipeTsFileInsertionEvent.java:
##########
@@ -900,22 +918,35 @@ private void waitForResourceEnough4Parsing(final long timeoutMs) throws Interrup
waitTimeSeconds);
}
- if (waitTimeSeconds * 1000 > timeoutMs) {
- // should contain 'TimeoutException' in exception message
- throw new PipeRuntimeOutOfMemoryCriticalException(
- String.format(
- DataNodePipeMessages
- .PIPE_EXCEPTION_TIMEOUTEXCEPTION_WAITED_S_SECONDS_FOR_MEMORY_TO_PARSE_TSFILE_0E4EF8FD,
- waitTimeSeconds));
+ if (tryReserveTsFileParserMemory(memoryManager)) {
+ LOGGER.info(
+ DataNodePipeMessages.WAIT_FOR_MEMORY_ENOUGH_FOR_PARSING_FOR,
+ resource != null ? resource.getTsFilePath() : "tsfile",
+ waitTimeSeconds);
+ return;
}
+
+ memoryCheckIntervalMs =
+ getNextMemoryCheckIntervalMs(memoryCheckIntervalMs, maxMemoryCheckIntervalMs);
}
+ }
+
+ static long getMaxMemoryCheckIntervalMs(final long initialIntervalMs, final int maxRetries) {
+ final long multiplier = Math.max(1, maxRetries);
+ return initialIntervalMs > Long.MAX_VALUE / multiplier
+ ? Long.MAX_VALUE
+ : initialIntervalMs * multiplier;
+ }
Review Comment:
Thanks, addressed in 606c24f3270. I added a description next to pipeMemoryAllocateMaxRetries clarifying that, besides limiting allocation retries, it also caps the TsFile parser memory admission backoff at pipeCheckMemoryEnoughIntervalMs * max(1, pipeMemoryAllocateMaxRetries).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Pipe] Back off TsFile parser admission retries [iotdb]
Posted by "jt2594838 (via GitHub)" <gi...@apache.org>.
jt2594838 merged PR #18262:
URL: https://github.com/apache/iotdb/pull/18262
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Pipe] Back off TsFile parser admission retries [iotdb]
Posted by "jt2594838 (via GitHub)" <gi...@apache.org>.
jt2594838 commented on code in PR #18262:
URL: https://github.com/apache/iotdb/pull/18262#discussion_r3626792434
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/PipeTsFileInsertionEvent.java:
##########
@@ -900,22 +918,35 @@ private void waitForResourceEnough4Parsing(final long timeoutMs) throws Interrup
waitTimeSeconds);
}
- if (waitTimeSeconds * 1000 > timeoutMs) {
- // should contain 'TimeoutException' in exception message
- throw new PipeRuntimeOutOfMemoryCriticalException(
- String.format(
- DataNodePipeMessages
- .PIPE_EXCEPTION_TIMEOUTEXCEPTION_WAITED_S_SECONDS_FOR_MEMORY_TO_PARSE_TSFILE_0E4EF8FD,
- waitTimeSeconds));
+ if (tryReserveTsFileParserMemory(memoryManager)) {
+ LOGGER.info(
+ DataNodePipeMessages.WAIT_FOR_MEMORY_ENOUGH_FOR_PARSING_FOR,
+ resource != null ? resource.getTsFilePath() : "tsfile",
+ waitTimeSeconds);
+ return;
}
+
+ memoryCheckIntervalMs =
+ getNextMemoryCheckIntervalMs(memoryCheckIntervalMs, maxMemoryCheckIntervalMs);
}
+ }
+
+ static long getMaxMemoryCheckIntervalMs(final long initialIntervalMs, final int maxRetries) {
+ final long multiplier = Math.max(1, maxRetries);
+ return initialIntervalMs > Long.MAX_VALUE / multiplier
+ ? Long.MAX_VALUE
+ : initialIntervalMs * multiplier;
+ }
Review Comment:
If the semantic of the maxRetries has changed, you should supplement some description in the config.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org