Re: Spark 3.5.0 and issue SPARK-45593 (SPARK-45201)

[Steven B Jones <sb...@us.ibm.com>]

A follow-up to my note yesterday.

Issue SPARK-45201 has similar externals to SPARK-45593 and is written to cover target release 3.5.0.  Remarkably, the issue only affects self-created distributions, and not the one(s) provided by Spark development itself.

I'll let you read SPARK-45201<https://issues.apache.org/jira/browse/SPARK-45201> and see how it should be handled.  Personally, it seems like it might want to make Spark 3.5.1 for all of us Spark distributors, but I leave it in your hands.

I will watch SPARK-45201.  For now, I consider this email thread to be closed.  Thank you.

Steve


SBJ

Steven B Jones

Senior Software Engineer



Phone:  845-435-5220

Mobile: 845-264-2288

Fax:      845-435-1752

Email: sbj@us.ibm.com



2455 South Rd, MS P380

Poughkeepsie, NY



www.IBM.com<http://www.ibm.com/>

 [cid:24b62431-97f4-4c96-bd24-a6b414a6d67c]







________________________________
From: Steven B Jones
Sent: Wednesday, December 13, 2023 9:52 PM
To: dev@spark.apache.org <de...@spark.apache.org>
Subject: Spark 3.5.0 and issue SPARK-45593

Hello,
I maintain a version of Apache Spark that runs on z/OS. I'm porting Spark 3.5.0 to our platform, and having the problem described by https://issues.apache.org/jira/projects/SPARK/issues/SPARK-45593<https://issues.apache.org/jira/projects/SPARK/issues/SPARK-45593?filter=allissues>  in spark-shell. (Building a runnable distribution from master code running spark-sql raise error "java.lang.ClassNotFoundException: org.sparkproject.guava.util.concurrent.internal.InternalFutureFailureAccess") That issue says the affects Spark 4.0, but as I said, I'm seeing it on Spark 3.5.0 with spark-shell.

Reading the issue, it opened "hot" with a quick PR that was supposed to fix it, but I haven't seen any activity since October 26.

Is no one affected by this on Spark 3.5.0?  Does spark regression testing cover the "build a runnable distribution" case listed, prior to 4.0.0?

Thanks and regards,
Steve


SBJ

Steven B Jones

Senior Software Engineer



Phone:  845-435-5220

Mobile: 845-264-2288

Fax:      845-435-1752

Email: sbj@us.ibm.com



2455 South Rd, MS P380

Poughkeepsie, NY



www.IBM.com<http://www.ibm.com/>

 [cid:a333ee6b-e1d3-42fd-abf3-aa3df204db69]