You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Robert Stupp (JIRA)" <ji...@apache.org> on 2019/04/09 12:20:00 UTC

[jira] [Commented] (CASSANDRA-14773) Overflow of 32-bit integer during compaction.

    [ https://issues.apache.org/jira/browse/CASSANDRA-14773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813323#comment-16813323 ] 

Robert Stupp commented on CASSANDRA-14773:
------------------------------------------

TBH, I doubt that the changes in the proposed PR are enough to fix the existing issues in {{StreamingTombstoneHistogramBuilder}}. Some comments:
 * Nit: the method {{roundKey}} should be called {{ceilKey}}
 * The overall benefit of {{DistanceHolder}} is probably outweighed by the overhead to manage this structure. Maybe it's better to revert to the previous way (iterate to find the shortest distance). Removing {{DistanceHolder}} would also simplify the code a lot.
 * The calculation of {{sum}} in {{merge()}} is prone to an int overflow. Better make {{sum}} a {{long}} and guard the calculation of {{newPoint}} with {{Math.toExactInt()}}.
 * {{wrap}} should take {{value}} as a {{long}} and prevent getting a negative result (int overflow).
 * The code sequence {{boxed().map()}} found in the class can be simplified to {{mapToObj()}}
 * The calculations in {{roundKey}} should use {{long}}s and use a saturating cast (limit return values to 0..Cell.MAX_DELETION_TIME)
 * Deserialization in TombstoneHistogram is also prone to int overflows and should use a saturating cast (see CASSANDRA-14092)
 * {{tryCell}} should also not just blindly do a {{+=}} but a saturating cast of the old value + delta
 * Test cases for all the possible int overflows are missing

 

> Overflow of 32-bit integer during compaction.
> ---------------------------------------------
>
>                 Key: CASSANDRA-14773
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-14773
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Local/Compaction
>            Reporter: Vladimir Bukhtoyarov
>            Assignee: Vladimir Bukhtoyarov
>            Priority: Urgent
>             Fix For: 4.x
>
>
> In scope of CASSANDRA-13444 the compaction was significantly improved from CPU and memory perspective. Hovewer this improvement introduces the bug in rounding. When rounding the expriration time which is close to  *Cell.MAX_DELETION_TIME*(it is just *Integer.MAX_VALUE*) the math overflow happens(because in scope of -CASSANDRA-13444-) data type for point was changed from Long to Integer in order to reduce memory footprint), as result point became negative and acts as silent poison for internal structures of StreamingTombstoneHistogramBuilder like *DistanceHolder* and *DataHolder*. Then depending of point intervals:
>  * The TombstoneHistogram produces wrong values when interval of points is less then binSize, it is not critical.
>  * Compaction crashes with ArrayIndexOutOfBoundsException if amount of point intervals is great then  binSize, this case is very critical.
>  
> This is pull request [https://github.com/apache/cassandra/pull/273] that reproduces the issue and provides the fix. 
>  
> The stacktrace when running(on codebase without fix) *testMathOverflowDuringRoundingOfLargeTimestamp* without -ea JVM flag
> {noformat}
> java.lang.ArrayIndexOutOfBoundsException
> at java.lang.System.arraycopy(Native Method)
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$DistanceHolder.add(StreamingTombstoneHistogramBuilder.java:208)
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.flushValue(StreamingTombstoneHistogramBuilder.java:140)
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$$Lambda$1/1967205423.consume(Unknown Source)
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$Spool.forEach(StreamingTombstoneHistogramBuilder.java:574)
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.flushHistogram(StreamingTombstoneHistogramBuilder.java:124)
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.build(StreamingTombstoneHistogramBuilder.java:184)
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilderTest.testMathOverflowDuringRoundingOfLargeTimestamp(StreamingTombstoneHistogramBuilderTest.java:183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
> at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:159)
> at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> {noformat}
>  
> The stacktrace when running(on codebase without fix)  *testMathOverflowDuringRoundingOfLargeTimestamp* with enabled -ea JVM flag
> {noformat}
> ava.lang.AssertionError: point2 should follow point1
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$DataHolder.merge(StreamingTombstoneHistogramBuilder.java:324)
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.mergeBin(StreamingTombstoneHistogramBuilder.java:158)
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.flushValue(StreamingTombstoneHistogramBuilder.java:145)
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$$Lambda$1/1967205423.consume(Unknown Source)
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$Spool.forEach(StreamingTombstoneHistogramBuilder.java:574)
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.flushHistogram(StreamingTombstoneHistogramBuilder.java:124)
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.build(StreamingTombstoneHistogramBuilder.java:184)
> at org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilderTest.testMathOverflowDuringRoundingOfLargeTimestamp(StreamingTombstoneHistogramBuilderTest.java:183)
> {noformat}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org