You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/11/02 07:52:02 UTC
[GitHub] [druid] jon-wei commented on a diff in pull request #13296: Make RecordSupplierInputSource respect sampler timeout when stream is empty
jon-wei commented on code in PR #13296:
URL: https://github.com/apache/druid/pull/13296#discussion_r1011327598
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/RecordSupplierInputSource.java:
##########
@@ -123,13 +129,23 @@ CloseableIterator<InputEntity> createEntityIterator()
private Iterator<OrderedPartitionableRecord<PartitionIdType, SequenceOffsetType, RecordType>> recordIterator;
private Iterator<? extends ByteEntity> bytesIterator;
private volatile boolean closed;
+ private long createTime = System.currentTimeMillis();
+ private Long terminationTime = samplerConfigTimeoutMs != null ? createTime + samplerConfigTimeoutMs : null;
Review Comment:
I had made it accept a null which was this was for, but changed this to check for 0 to be consistent with the way it's used elsewhere (to indicate no timeout specified)
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/RecordSupplierInputSource.java:
##########
@@ -123,13 +129,23 @@ CloseableIterator<InputEntity> createEntityIterator()
private Iterator<OrderedPartitionableRecord<PartitionIdType, SequenceOffsetType, RecordType>> recordIterator;
private Iterator<? extends ByteEntity> bytesIterator;
private volatile boolean closed;
+ private long createTime = System.currentTimeMillis();
+ private Long terminationTime = samplerConfigTimeoutMs != null ? createTime + samplerConfigTimeoutMs : null;
private void waitNextIteratorIfNecessary()
{
while (!closed && (bytesIterator == null || !bytesIterator.hasNext())) {
while (!closed && (recordIterator == null || !recordIterator.hasNext())) {
+ if (terminationTime != null) {
+ if (System.currentTimeMillis() > terminationTime) {
+ LOG.info("Configured sampler timeout has been exceeded, returning without a bytesIterator.");
Review Comment:
Added the timeout ms to the log
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org