You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2025/03/03 18:05:00 UTC
[jira] [Comment Edited] (SPARK-51365) OOM occurred during macOS daily tests
[ https://issues.apache.org/jira/browse/SPARK-51365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932038#comment-17932038 ]
Dongjoon Hyun edited comment on SPARK-51365 at 3/3/25 6:04 PM:
---------------------------------------------------------------
Thank you for pinging me, [~LuciferYang]. However, the root cause of OOM is a well-known known issue of GitHub Action runners. FYI, macOS runner has only 7GB while linux runner has 16GB.
- [Standard GitHub-hosted runners for public repositories](https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories)
{code}
Virtual Machine Processor (CPU) Memory (RAM) Storage (SSD) Architecture
Linux 4 16 GB 14 GB x64
macOS 3 (M1) 7 GB 14 GB arm64
{code}
was (Author: dongjoon):
Thank you for pinging me, [~LuciferYang]. However, the root cause of OOM is a well-known known issue of GitHub Action runners. FYI, macOS runner has only 7GB while linux runner has 16GB.
- https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners
{code}
Virtual Machine Processor (CPU) Memory (RAM) Storage (SSD) Architecture
Linux 4 16 GB 14 GB x64
macOS 3 (M1) 7 GB 14 GB arm64
{code}
> OOM occurred during macOS daily tests
> -------------------------------------
>
> Key: SPARK-51365
> URL: https://issues.apache.org/jira/browse/SPARK-51365
> Project: Spark
> Issue Type: Bug
> Components: Tests
> Affects Versions: 4.1.0
> Reporter: Yang Jie
> Priority: Major
>
> * [https://github.com/apache/spark/actions/runs/13316147273/job/37299839380]
> {code:java}
> Warning: [343.044s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 4096k, guardsize: 16k, detached.
> 11372Warning: [343.044s][warning][os,thread] Failed to start the native thread for java.lang.Thread "shuffle-exchange-1529"
> 11373*** RUN ABORTED ***
> 11374An exception or error caused a run to abort: unable to create native thread: possibly out of memory or process/resource limits reached
> 11375 java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
> 11376 at java.base/java.lang.Thread.start0(Native Method)
> 11377 at java.base/java.lang.Thread.start(Thread.java:1553)
> 11378 at java.base/java.lang.System$2.start(System.java:2577)
> 11379 at java.base/jdk.internal.vm.SharedThreadContainer.start(SharedThreadContainer.java:152)
> 11380 at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:953)
> 11381 at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
> 11382 at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
> 11383 at java.base/java.util.concurrent.CompletableFuture.asyncSupplyStage(CompletableFuture.java:1782)
> 11384 at java.base/java.util.concurrent.CompletableFuture.supplyAsync(CompletableFuture.java:2005)
> 11385 at org.apache.spark.sql.execution.SQLExecution$.withThreadLocalCaptured(SQLExecution.scala:329)
> 11386 ...
> 11387Warning: The requested profile "volcano" could not be activated because it does not exist.
> 11388Warning: The requested profile "hive" could not be activated because it does not exist.
> 11389Error: Failed to execute goal org.scalatest:scalatest-maven-plugin:2.2.0:test (test) on project spark-sql_2.13: There are test failures -> [Help 1]
> 11390Error:
> 11391Error: To see the full stack trace of the errors, re-run Maven with the -e switch.
> 11392Error: Re-run Maven using the -X switch to enable full debug logging.
> 11393Error:
> 11394Error: For more information about the errors and possible solutions, please read the following articles:
> 11395Error: [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> 11396Error: Process completed with exit code 1. {code}
>
> * [https://github.com/apache/spark/actions/runs/13316147273/job/37299839259]
>
> {code:java}
> - group-by-ordinal.sql
> 10493- group-by-ordinal.sql_analyzer_test
> 10494Warning: [495.950s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 4096k, guardsize: 16k, detached.
> 10495Warning: [495.950s][warning][os,thread] Failed to start the native thread for java.lang.Thread "shuffle-exchange-1799"
> 1049616:17:17.464 ERROR org.apache.spark.sql.SQLQueryTestSuite: Error using configs: spark.sql.codegen.wholeStage=false,spark.sql.codegen.factoryMode=CODEGEN_ONLY
> 10497
> 10498*** RUN ABORTED ***
> 10499An exception or error caused a run to abort: unable to create native thread: possibly out of memory or process/resource limits reached
> 10500 java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
> 10501 at java.base/java.lang.Thread.start0(Native Method)
> 10502 at java.base/java.lang.Thread.start(Thread.java:1553)
> 10503 at java.base/java.lang.System$2.start(System.java:2577)
> 10504 at java.base/jdk.internal.vm.SharedThreadContainer.start(SharedThreadContainer.java:152)
> 10505 at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:953)
> 10506 at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
> 10507 at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
> 10508 at java.base/java.util.concurrent.CompletableFuture.asyncSupplyStage(CompletableFuture.java:1782)
> 10509 at java.base/java.util.concurrent.CompletableFuture.supplyAsync(CompletableFuture.java:2005)
> 10510 at org.apache.spark.sql.execution.SQLExecution$.withThreadLocalCaptured(SQLExecution.scala:329)
> 10511 ... {code}
>
>
> The root cause is unknown for now, and we need to investigate it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org