You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/09/04 08:41:46 UTC
[GitHub] [druid] dylwylie opened a new pull request #10356: Avoid large limits causing int overflow in buffer size checks
dylwylie opened a new pull request #10356:
URL: https://github.com/apache/druid/pull/10356
### Description
The size check in LimitedBufferHashGrouper can overflow and hence pass inadvertently. This results in the limited grouper being used when the buffer is too small and so the buffer.limit call within init fails with an IllegalArgumentException.
This is likely to only occur at large limits which aren't common but we've seen it occur on our production cluster. The following test added to LimitedBufferHashGrouperTest fails prior to this commit and passes after, I'm not sure it's useful enough to include in the commit itself but included here for anyone interested
` @Test
public void testBufferTooSmallOverflowingLimit()
{
expectedException.expect(IAE.class);
expectedException.expectMessage("LimitedBufferHashGrouper initialized with insufficient buffer capacity");
final TestColumnSelectorFactory columnSelectorFactory = GrouperTestUtil.newColumnSelectorFactory();
// Bucket size for the grouper is 28 bytes
int limitToOverflow = Integer.MAX_VALUE / Integer.BYTES / 28;
makeGrouper(columnSelectorFactory, 10000, 50000, limitToOverflow);
}
`
This PR has:
- [x] been self-reviewed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] suneet-s merged pull request #10356: Avoid large limits causing int overflow in buffer size checks
Posted by GitBox <gi...@apache.org>.
suneet-s merged pull request #10356:
URL: https://github.com/apache/druid/pull/10356
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] lgtm-com[bot] commented on pull request #10356: Avoid large limits causing int overflow in buffer size checks
Posted by GitBox <gi...@apache.org>.
lgtm-com[bot] commented on pull request #10356:
URL: https://github.com/apache/druid/pull/10356#issuecomment-687047936
This pull request **introduces 1 alert** when merging 06cabfd7da436cfcbd69ddb788877c0f16a73502 into d7fcff3aba47dbd840c1cdca24f4802168ce1d9c - [view on LGTM.com](https://lgtm.com/projects/g/apache/druid/rev/pr-9bea0cfb157e3fca38e81ad4b42efac4adee7877)
**new alerts:**
* 1 for Result of multiplication cast to wider type
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org