You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean R. Owen (Jira)" <ji...@apache.org> on 2021/10/03 21:11:00 UTC
[jira] [Commented] (SPARK-36900) "SPARK-36464: size returns correct
positive number even with over 2GB data" will oom with JDK17
[ https://issues.apache.org/jira/browse/SPARK-36900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423727#comment-17423727 ]
Sean R. Owen commented on SPARK-36900:
--------------------------------------
We're just going to up the mem limit in tests for this right? I think it may be 'normal' for JDK 17
> "SPARK-36464: size returns correct positive number even with over 2GB data" will oom with JDK17
> ------------------------------------------------------------------------------------------------
>
> Key: SPARK-36900
> URL: https://issues.apache.org/jira/browse/SPARK-36900
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Affects Versions: 3.3.0
> Reporter: Yang Jie
> Priority: Major
>
> Execute
>
> {code:java}
> build/mvn clean install -pl core -am -Dtest=none -DwildcardSuites=org.apache.spark.util.io.ChunkedByteBufferOutputStreamSuite
> {code}
> with JDK 17,
> {code:java}
> ChunkedByteBufferOutputStreamSuite:
> - empty output
> - write a single byte
> - write a single near boundary
> - write a single at boundary
> - single chunk output
> - single chunk output at boundary size
> - multiple chunk output
> - multiple chunk output at boundary size
> *** RUN ABORTED ***
> java.lang.OutOfMemoryError: Java heap space
> at java.base/java.lang.Integer.valueOf(Integer.java:1081)
> at scala.runtime.BoxesRunTime.boxToInteger(BoxesRunTime.java:67)
> at org.apache.spark.util.io.ChunkedByteBufferOutputStream.allocateNewChunkIfNeeded(ChunkedByteBufferOutputStream.scala:87)
> at org.apache.spark.util.io.ChunkedByteBufferOutputStream.write(ChunkedByteBufferOutputStream.scala:75)
> at java.base/java.io.OutputStream.write(OutputStream.java:127)
> at org.apache.spark.util.io.ChunkedByteBufferOutputStreamSuite.$anonfun$new$22(ChunkedByteBufferOutputStreamSuite.scala:127)
> at org.apache.spark.util.io.ChunkedByteBufferOutputStreamSuite$$Lambda$179/0x00000008011a75d8.apply(Unknown Source)
> at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
> at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
> at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org