You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ma...@apache.org on 2026/10/21 06:33:49 UTC
(spark) branch branch-4.x updated: [SPARK-60363][SQL][TEST] Raise PROCESSES of the Oracle test container to avoid ORA-12516
This is an automated email from the ASF dual-hosted git repository.
MaxGekk pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 6e650eba3da0 [SPARK-60363][SQL][TEST] Raise PROCESSES of the Oracle test container to avoid ORA-12516
6e650eba3da0 is described below
commit 6e650eba3da087d5932eb5897734c96ea8dac46f
Author: Max Gekk <ma...@gmail.com>
AuthorDate: Wed Oct 21 08:33:29 2026 +0200
[SPARK-60363][SQL][TEST] Raise PROCESSES of the Oracle test container to avoid ORA-12516
### What changes were proposed in this pull request?
This PR raises the `PROCESSES` parameter of the Oracle Free container used by the JDBC docker integration tests from the image default of 200 to 1000.
`PROCESSES` is a static parameter, so it has to be in the spfile before the instance starts. `OracleDatabaseOnDocker` now starts the container through a small wrapper entrypoint, `oracle-docker-entrypoint.sh`. The script starts the instance in NOMOUNT, runs `ALTER SYSTEM SET PROCESSES = 1000 SCOPE = SPFILE`, aborts the instance and then `exec`s the image's own `container-entrypoint.sh`, which starts the listener and the database as before. The script is copied to a temporary directory [...]
The change happens before the listener exists, so the harness can never connect to the database and then lose that connection to a restart. A script in `/container-entrypoint-startdb.d` would have that race, because those scripts run after the database is already open and the passwords are already set.
### Why are the changes needed?
`org.apache.spark.sql.jdbc.v2.OracleIntegrationSuite` fails intermittently, about a minute into the suite, in the `scan with aggregate push-down: ... without DISTINCT` tests. The error is `java.sql.SQLException: ORA-12516: Cannot connect to database. Listener ... does not have a protocol handler for TCP ready or registered for service freepdb1`. The failures come in one or two bursts of 1 to 8 tests. Each burst lasts under a second, and when there are two they are about 3 seconds apar [...]
The instance never runs out of processes. During a full run of the suite, `v$resource_limit` shows `processes` at about 88 current and 92 peak out of 200, so no connections are leaking. The limit that is hit is the listener's own estimate of the load. The listener refuses a dedicated connection once the process count the instance last reported to it, plus every connection it has handed off since, reaches `PROCESSES`. The instance (LREG) sends that report only every 3 seconds or more, [...]
The v2 suite opens about 770 short-lived connections in about 40 seconds. Each query with `numPartitions = 4` opens a connection per partition, on top of the driver-side metadata connections. In the listener log of a local run, the busiest 3-second window had 99 connections, against the 112 that are available. A slightly denser burst on a CI runner, or a report that arrives a little late, crosses the line. Raising the instance's startup timeout (SPARK-57603) does not help, because the [...]
With `PROCESSES = 1000`, the same 8-, 16- and 32-wide bursts that were refused before are all accepted, and the listener never leaves `state:ready`. The fixed suite's busiest listener window is about 220 connections, against roughly 910 available.
### Does this PR introduce _any_ user-facing change?
No. Only the test harness changes.
### How was this patch tested?
Locally, with Docker and `ENABLE_DOCKER_INTEGRATION_TESTS=1`:
- `build/sbt -Pdocker-integration-tests "docker-integration-tests/testOnly org.apache.spark.sql.jdbc.v2.OracleIntegrationSuite"` passed in 3 of 3 runs: 63 tests succeeded, 22 ignored. During each run I polled `v$resource_limit` and `lsnrctl services` inside the container. The limit was 1000, `processes` peaked at 90 to 91, the listener refused 0 connections, and it was never blocked.
- `docker-integration-tests/testOnly org.apache.spark.sql.jdbc.OracleIntegrationSuite org.apache.spark.sql.jdbc.v2.OracleNamespaceSuite org.apache.spark.sql.jdbc.v2.join.OracleJoinPushdownIntegrationSuite` passed: 55 tests succeeded, 5 ignored.
- The flake did not reproduce locally without the fix, including with the container capped to 1 CPU. The evidence for the cause is the listener measurements above.
- After restricting the wrapper to `oracle-free` images, the v2 `OracleIntegrationSuite` passed again (63 succeeded, 22 ignored).
- `docker-integration-tests/scalastyle` and `docker-integration-tests/Test/scalastyle` pass.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5.5 and Claude Fable 5.1)
Closes #59611 from MaxGekk/oracle-12516.
Authored-by: Max Gekk <ma...@gmail.com>
Signed-off-by: Max Gekk <ma...@gmail.com>
(cherry picked from commit 8d5b30a06f687d9daa62f62d38a744ffc41321a0)
Signed-off-by: Max Gekk <ma...@gmail.com>
---
.../src/test/resources/oracle-docker-entrypoint.sh | 29 ++++++++++++++++
.../spark/sql/jdbc/OracleDatabaseOnDocker.scala | 39 ++++++++++++++++++++++
2 files changed, 68 insertions(+)
diff --git a/connector/docker-integration-tests/src/test/resources/oracle-docker-entrypoint.sh b/connector/docker-integration-tests/src/test/resources/oracle-docker-entrypoint.sh
new file mode 100755
index 000000000000..4ecd7b6403c9
--- /dev/null
+++ b/connector/docker-integration-tests/src/test/resources/oracle-docker-entrypoint.sh
@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Raises PROCESSES in the spfile before the image's own entrypoint starts the database, so that
+# the listener does not refuse connections with ORA-12516 while the tests open many short-lived
+# connections. See OracleDatabaseOnDocker for details.
+set -e
+sqlplus -s / as sysdba <<SQL
+WHENEVER SQLERROR EXIT SQL.SQLCODE
+STARTUP NOMOUNT;
+ALTER SYSTEM SET PROCESSES = 1000 SCOPE = SPFILE;
+SHUTDOWN ABORT;
+SQL
+exec container-entrypoint.sh "$@"
diff --git a/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala b/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala
index 28f1a8d8f907..fe118e3c29e3 100644
--- a/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala
+++ b/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala
@@ -17,7 +17,13 @@
package org.apache.spark.sql.jdbc
+import java.io.File
+import java.nio.file.Files
+
+import com.github.dockerjava.api.model.{AccessMode, Bind, ContainerConfig, HostConfig, Volume}
+
import org.apache.spark.internal.Logging
+import org.apache.spark.util.Utils
class OracleDatabaseOnDocker extends DatabaseOnDocker with Logging {
// sarutak/oracle-free is a custom fork of gvenzl/oracle-free which allows to set timeout for
@@ -39,6 +45,39 @@ class OracleDatabaseOnDocker extends DatabaseOnDocker with Logging {
// test is not flaky on slow CI runners.
override def connectionTimeout: Option[String] = Some("15min")
+ // The listener refuses a dedicated connection with ORA-12516 once the processes the instance
+ // last reported to it, plus the connections it has handed off since, reach PROCESSES. The
+ // instance reports only every few seconds, the image's default PROCESSES of 200 leaves room
+ // for about 110 connections on top of its background processes, and the JDBC v2 tests open
+ // close to 100 short-lived connections within one such window. PROCESSES is a static
+ // parameter, so a wrapper entrypoint raises it in the spfile before the image's own entrypoint
+ // (container-entrypoint.sh of gvenzl/oracle-free and its forks) starts the database. Other
+ // images, such as those built from oracle/docker-images, have no such entrypoint and start as
+ // they are.
+ private val entryPointDir = "/docker-entrypoint-oracle"
+ private val entryPointScript = "oracle-docker-entrypoint.sh"
+ private def wrapsEntryPoint: Boolean = imageName.contains("oracle-free")
+
+ override def getEntryPoint: Option[String] =
+ if (wrapsEntryPoint) Some(s"$entryPointDir/$entryPointScript") else None
+
+ override def beforeContainerStart(
+ hostConfigBuilder: HostConfig,
+ containerConfigBuilder: ContainerConfig): Unit = if (wrapsEntryPoint) {
+ val dir = Utils.createTempDir()
+ val script = new File(dir, entryPointScript)
+ Utils.tryWithResource(getClass.getClassLoader.getResourceAsStream(entryPointScript)) { in =>
+ Files.copy(in, script.toPath)
+ }
+ // The database runs as the `oracle` user of the container, not as the owner of these files.
+ Seq(dir, script).foreach { f =>
+ f.setReadable(true, false)
+ f.setExecutable(true, false)
+ }
+ val bind = new Bind(dir.getAbsolutePath, new Volume(entryPointDir), AccessMode.ro)
+ hostConfigBuilder.withBinds(hostConfigBuilder.getBinds :+ bind: _*)
+ }
+
override def getJdbcUrl(ip: String, port: Int): String = {
s"jdbc:oracle:thin:system/$oracle_password@//$ip:$port/freepdb1"
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org