[jira] [Resolved] (LIVY-1065) Ensure all tests pass on macOS

["Gyorgy Gal (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LIVY-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gyorgy Gal resolved LIVY-1065.
------------------------------
    Resolution: Fixed

> Ensure all tests pass on macOS
> ------------------------------
>
>                 Key: LIVY-1065
>                 URL: https://issues.apache.org/jira/browse/LIVY-1065
>             Project: Livy
>          Issue Type: Bug
>    Affects Versions: 1.0.0
>            Reporter: Gabor Roczei
>            Assignee: Gabor Roczei
>            Priority: Major
>             Fix For: 1.0.0
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> h2. Problem
> Running the Livy unit-test and integration-test suites on macOS is currently broken. Several Livy and Spark services used by the harness bind to the LAN IP returned by {{{}InetAddress.getLocalHost{}}}, but on macOS the canonical hostname routinely resolves to a loopback-only address (or the LAN IP is otherwise unreachable from a peer process on the same box), so the mini-cluster YARN containers and JDBC clients running on the same host cannot reach the server side. Concrete symptoms observed on macOS:
>  * {{JdbcIT}} SASL handshake timeout when the client tries {{{}jdbc:hive2://<LAN-IP>:10090{}}}.
>  * {{RSCClient instance stopped}} caused by {{ConnectTimeoutException}} on {{/LAN-IP:10000}} — the Spark driver inside the YARN AM cannot reach the Livy RSC server.
>  * {{Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered}} — the driver AM advertises {{spark://YarnAM@<LAN-IP>:<port>}} and the executor container never registers.
>  * {{java.io.IOException: Cannot run program "Rscript"/"python": error=2, No such file or directory}} in SparkRIT and PySparkIT — YARN's default sanitised PATH omits {{/opt/homebrew/bin}} on macOS, where Homebrew installs the R and Python interpreters that macOS developers use.
> Linux CI hosts already resolve the primary hostname to a routable address and do not hit this, so the same suites are green on CI today.
> h2. Fix
> Force every affected bind point to {{127.0.0.1}} in the test harness, and propagate the host shell {{PATH}} into the YARN AM and executor containers. Because 127.0.0.1 is equally reachable on Linux CI, this is a no-op on CI and a fix on macOS.
> Bind points touched:
>  * {{{}livy.server.host{}}}, {{{}livy.server.thrift.bind.host{}}}, {{livy.rsc.rpc.server.address}} in {{{}MiniLivyMain{}}}.
>  * {{{}spark.driver.host{}}}, {{{}spark.yarn.appMasterEnv.SPARK_LOCAL_IP{}}}, {{spark.executorEnv.SPARK_LOCAL_IP}} in {{{}MiniCluster.deploy(){}}}'s generated spark-defaults.conf. {{SPARK_LOCAL_IP}} is what the YARN AM driver actually reads on macOS; {{spark.driver.host}} on the submit side alone is not honoured in cluster deploy mode.
>  * {{RSCConf.Entry.RPC_SERVER_ADDRESS = 127.0.0.1}} in every unit-test fixture that spins up an in-process RSC server: {{{}BaseSessionSpec{}}}, {{{}ReplDriverSuite{}}}, {{{}TestSparkClient{}}}, {{{}ScalaClientTest{}}}, {{{}BaseInteractiveServletSpec{}}}, {{{}InteractiveSessionSpec{}}}, {{{}ThriftServerBaseTest{}}}, {{{}ThriftSessionTest{}}}.
>  * {{WebServer}} bind host from {{0.0.0.0}} / {{InetAddress.getLocalHost}} to {{127.0.0.1}} in {{HttpClientSpec}} and {{{}LivyConnectionSpec{}}}, and switch the URIs those tests build to use {{server.host}} so the client and server agree on the address.
> Also propagate the host shell {{PATH}} via {{spark.yarn.appMasterEnv.PATH}} / {{spark.executorEnv.PATH}} so {{RRunner}} and {{PythonRunner}} can resolve {{Rscript}} / {{python}} from {{/opt/homebrew/bin}} on macOS.
> h2. Scope
> Test-harness / CI only — no production code paths change. 
> Split out from LIVY-1041 (Add support for Spark 4 via a new Maven profile) because these fixes are logically independent and were bundled there only because they surfaced most visibly while getting the Spark 4 matrix green on macOS.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)