You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@livy.apache.org by "Nilesh Rathi (Jira)" <ji...@apache.org> on 2026/09/17 15:29:00 UTC
[jira] [Assigned] (LIVY-1086) Fix macOS unit test failures by binding Spark driver host in test fixtures
[ https://issues.apache.org/jira/browse/LIVY-1086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nilesh Rathi reassigned LIVY-1086:
----------------------------------
Assignee: Nilesh Rathi
> Fix macOS unit test failures by binding Spark driver host in test fixtures
> --------------------------------------------------------------------------
>
> Key: LIVY-1086
> URL: https://issues.apache.org/jira/browse/LIVY-1086
> Project: Livy
> Issue Type: Bug
> Reporter: Nilesh Rathi
> Assignee: Nilesh Rathi
> Priority: Major
>
> h2. Problem
> Follow-up to [LIVY-1065|https://issues.apache.org/jira/browse/LIVY-1065] (merged in [PR #538|https://github.com/apache/livy/pull/538]).
> PR #538 pinned Livy, RSC, YARN mini-cluster, and several unit-test services to loopback ({{127.0.0.1}}) via {{TestUtils.TEST_BIND_HOST}} so tests pass on macOS. However, several unit tests still start Spark directly without setting {{spark.driver.host}} and {{spark.driver.bindAddress}}.
> On macOS, Spark's driver can advertise the machine's LAN IP (from {{InetAddress.getLocalHost()}}), which peer processes on the same host cannot reach. This causes failures such as:
> * {{RSCClient instance stopped}} and connect timeouts to the LAN IP (e.g. {{Failed to connect to /192.168.x.x:...}})
> * Spark stage failures / {{RemoteClassLoaderError}} when executors cannot reach the driver
> * Timeouts in REPL, RSC, server interactive, and thriftserver session tests
> Linux CI behavior is unchanged ({{127.0.0.1}} is equally valid there).
> h2. Proposed fix
> Extend the LIVY-1065 approach by setting {{spark.driver.host}} and {{spark.driver.bindAddress}} to {{TestUtils.TEST_BIND_HOST}} in test fixtures that create Spark sessions/drivers directly.
> Add {{TestUtils.SPARK_DRIVER_HOST}} and {{TestUtils.SPARK_DRIVER_BIND_ADDRESS}} constants in {{client-common/TestUtils.java}} to avoid duplicating raw config key strings.
> Fixtures/modules affected include (non-exhaustive):
> * {{repl}}: {{BaseSessionSpec}}, {{SessionSpec}}, {{ReplDriverSuite}}, interpreter specs
> * {{rsc}}: {{TestSparkClient}}
> * {{scala-api}}: {{ScalaClientTest}}
> * {{server}}: {{BaseInteractiveServletSpec}}, {{InteractiveSessionSpec}}
> * {{thriftserver}}: {{ThriftServerBaseTest}}, {{ThriftSessionTest}}, {{ColumnBufferTest}}
> * {{integration-test}}: {{MiniCluster}} (use {{TestUtils.SPARK_DRIVER_HOST}} constant)
> h2. Scope
> Test-harness only — no production code paths change.
> h2. Related work
> * [LIVY-1065|https://issues.apache.org/jira/browse/LIVY-1065] — initial macOS loopback binding for Livy/RSC/YARN services
> * [PR #538|https://github.com/apache/livy/pull/538] — merged LIVY-1065 fix
> * Follow-up PR — pins Spark driver to loopback in remaining test fixtures (requested by reviewer as a dedicated ticket)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)