You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2026/09/27 22:44:06 UTC

(spark) branch master updated: [SPARK-59553][CONNECT] Make SparkSession.addArtifact honor spark.jars.ivySettings

This is an automated email from the ASF dual-hosted git repository.

HyukjinKwon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 29da84aa64eb [SPARK-59553][CONNECT] Make SparkSession.addArtifact honor spark.jars.ivySettings
29da84aa64eb is described below

commit 29da84aa64eb807f6127b1bcd4edad90ad9ea0e9
Author: qianlan717 <qi...@databricks.com>
AuthorDate: Mon Sep 28 07:43:51 2026 +0900

    [SPARK-59553][CONNECT] Make SparkSession.addArtifact honor spark.jars.ivySettings
    
    ### What changes were proposed in this pull request?
    
    This PR makes `SparkSession.addArtifact` and `SparkSession.addArtifacts` honor the existing `spark.jars.ivySettings` Spark configuration when resolving `ivy://` dependencies.
    
    Spark resolves Ivy dependencies through a runtime dependency resolver configured from the driver `SparkConf`. Spark Connect adds a typed Maven dependency to the ordered artifact batch and resolves it on the server, where the Ivy settings file, repository network, and credentials are available. Support is negotiated through the `serverSideMavenArtifacts.v1` capability. A new client falls back to the existing client-side resolution only when the server does not advertise that capability.
    
    The server preserves the order of mixed local and Ivy artifacts and resolves the complete batch before registering it. Ivy URIs with explicit repositories supplied through `?repos=` remain client-resolved, so the server never accesses arbitrary client-requested repositories. Other Ivy URIs use an isolated server-side cache per Spark session, bounded concurrency, an overall deadline, cancellation, quotas, and content-collision checks.
    
    Runtime Ivy repository requests use two new general, static Spark configurations:
    
    - `spark.jars.ivyConnectTimeout`, with a default of `30s`;
    - `spark.jars.ivyReadTimeout`, with a default of `5m`.
    
    The configurations must be set before the `SparkContext` starts. They are not Connect-specific, so other Maven resolution paths can adopt the same centralized controls in follow-up changes. The Connect server caps the configured values by the remaining overall resolution deadline.
    
    The implementation also uses Scala 2.13-compatible array conversion in the runtime resolver and an explicit lambda when passing the overloaded artifact-resolution method to `flatMap`. The public API and configuration documentation describe the Ivy settings, repository, timeout, and Connect client-resolution behavior.
    
    ### Why are the changes needed?
    
    `SparkSession.addArtifact` currently ignores `spark.jars.ivySettings`. In Spark Connect, Maven dependencies are also resolved by the client, which often cannot read a server-local Ivy settings file or access the server's Maven network and credentials. As a result, an `ivy://` dependency can fail through `addArtifact` even though the same server configuration works for other Maven dependency surfaces.
    
    The timeout configurations bound runtime repository operations while allowing deployments with different network characteristics to override the defaults. Their general names leave room for centralized Ivy timeout control across additional Spark Maven resolution paths in follow-up changes.
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes. `ivy://` dependencies passed to classic or Connect `SparkSession.addArtifact(s)` now use `spark.jars.ivySettings` and `spark.jars.repositories` when configured. Runtime Ivy requests use the new `spark.jars.ivyConnectTimeout` and `spark.jars.ivyReadTimeout` settings, defaulting to `30s` and `5m`, respectively.
    
    Spark Connect resolves Ivy dependencies on capable servers, except URIs with `?repos=`, which retain client-side resolution. New clients connected to older servers also retain the previous client-side resolution behavior. Client-resolved URIs do not use the server's Ivy settings.
    
    ### How was this patch tested?
    
    Added tests covering:
    
    - custom Ivy settings, repository policy, resolver timeouts, and cancellation;
    - both classic `SparkSession` artifact APIs, per-session Ivy cache isolation, timeout defaults, and configured timeout overrides;
    - ordered mixed Connect artifacts, capability negotiation, old-server fallback, client-side
      resolution for `?repos=`, and propagation of capability RPC failures;
    - server-side Maven resolution before artifact registration while preserving order.
    
    The Python protobuf bindings were regenerated with `dev/connect-gen-protos.sh`. `git diff --check`, Scala parser and type checks, protobuf Java generation, and generated-binding smoke checks passed.
    
    The following targeted suites pass with Java 17:
    
    - `MavenUtilsSuite`: 17 tests;
    - `ArtifactManagerIvySettingsSuite`: 2 tests;
    - `ArtifactSuite`: 15 tests;
    - `SparkConnectClientSuite`: 71 tests;
    - `AddArtifactsHandlerSuite`: 8 tests.
    
    The two suites affected by the latest timeout configuration change were rerun together: all 10 tests passed.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: OpenAI Codex (GPT-5)
    
    Closes #58874 from qianlan717/SPARK-59553-addArtifact.
    
    Authored-by: qianlan717 <qi...@databricks.com>
    Signed-off-by: Hyukjin Kwon <hy...@databricks.com>
---
 .../scala/org/apache/spark/util/MavenUtils.scala   |  83 +++++-
 .../spark/util/RuntimeDependencyResolver.scala     | 125 +++++++++
 .../org/apache/spark/util/MavenUtilsSuite.scala    | 162 +++++++++++
 .../org/apache/spark/internal/config/package.scala |  35 ++-
 docs/configuration.md                              |  36 ++-
 python/pyspark/sql/connect/proto/base_pb2.py       | 298 +++++++++++----------
 python/pyspark/sql/connect/proto/base_pb2.pyi      |  79 +++++-
 .../scala/org/apache/spark/sql/SparkSession.scala  |  21 +-
 .../spark/sql/connect/client/ArtifactSuite.scala   | 110 +++++++-
 .../connect/client/SparkConnectClientSuite.scala   |  19 +-
 .../src/main/protobuf/spark/connect/base.proto     |  20 ++
 .../spark/sql/connect/client/ArtifactManager.scala | 140 +++++++---
 .../sql/connect/client/SparkConnectClient.scala    |  43 ++-
 .../service/SparkConnectAddArtifactsHandler.scala  | 167 ++++++++++--
 .../service/SparkConnectAnalyzeHandler.scala       |   1 +
 .../connect/planner/SparkConnectServiceSuite.scala |  13 +
 .../connect/service/AddArtifactsHandlerSuite.scala | 216 ++++++++++++++-
 .../spark/sql/artifact/ArtifactManager.scala       |  35 ++-
 .../apache/spark/sql/classic/SparkSession.scala    |   4 +-
 .../artifact/ArtifactManagerIvySettingsSuite.scala |  87 ++++++
 20 files changed, 1455 insertions(+), 239 deletions(-)

diff --git a/common/utils/src/main/scala/org/apache/spark/util/MavenUtils.scala b/common/utils/src/main/scala/org/apache/spark/util/MavenUtils.scala
index 5e923ad35a3a..8ca1f9efe649 100644
--- a/common/utils/src/main/scala/org/apache/spark/util/MavenUtils.scala
+++ b/common/utils/src/main/scala/org/apache/spark/util/MavenUtils.scala
@@ -21,6 +21,10 @@ import java.io.{File, IOException, PrintStream}
 import java.net.URI
 import java.text.ParseException
 import java.util.UUID
+import java.util.concurrent.{CancellationException, TimeUnit}
+import java.util.concurrent.locks.ReentrantLock
+
+import scala.jdk.CollectionConverters._
 
 import org.apache.ivy.Ivy
 import org.apache.ivy.core.LogOptions
@@ -29,10 +33,10 @@ import org.apache.ivy.core.module.id.{ArtifactId, ModuleId, ModuleRevisionId}
 import org.apache.ivy.core.report.{DownloadStatus, ResolveReport}
 import org.apache.ivy.core.resolve.ResolveOptions
 import org.apache.ivy.core.retrieve.RetrieveOptions
-import org.apache.ivy.core.settings.IvySettings
+import org.apache.ivy.core.settings.{IvySettings, NamedTimeoutConstraint}
 import org.apache.ivy.plugins.matcher.GlobPatternMatcher
 import org.apache.ivy.plugins.repository.file.FileRepository
-import org.apache.ivy.plugins.resolver.{ChainResolver, FileSystemResolver, IBiblioResolver}
+import org.apache.ivy.plugins.resolver.{AbstractResolver, ChainResolver, FileSystemResolver, IBiblioResolver}
 
 import org.apache.spark.SparkException
 import org.apache.spark.internal.{Logging, LogKeys}
@@ -42,6 +46,35 @@ import org.apache.spark.util.ArrayImplicits._
 private[spark] object MavenUtils extends Logging {
   val JAR_IVY_SETTING_PATH_KEY: String = "spark.jars.ivySettings"
 
+  private val ivyLock = new ReentrantLock()
+  private val ivyLockPollIntervalMs = 100L
+
+  private def acquireIvyLock(isCancelled: () => Boolean): Unit = {
+    def checkCancelled(): Unit = {
+      if (isCancelled() || Thread.currentThread().isInterrupted) {
+        throw new CancellationException("Maven dependency resolution was cancelled")
+      }
+    }
+
+    checkCancelled()
+    try {
+      while (!ivyLock.tryLock(ivyLockPollIntervalMs, TimeUnit.MILLISECONDS)) {
+        checkCancelled()
+      }
+    } catch {
+      case _: InterruptedException =>
+        Thread.currentThread().interrupt()
+        throw new CancellationException("Maven dependency resolution was cancelled")
+    }
+    try {
+      checkCancelled()
+    } catch {
+      case e: Throwable =>
+        ivyLock.unlock()
+        throw e
+    }
+  }
+
   // Exposed for testing
   // var printStream = SparkSubmit.printStream
 
@@ -340,6 +373,28 @@ private[spark] object MavenUtils extends Logging {
     ivySettings
   }
 
+  /** Apply bounded network timeouts to every resolver in an Ivy settings graph. */
+  private[spark] def setResolverTimeouts(
+      ivySettings: IvySettings,
+      connectTimeoutMs: Int,
+      readTimeoutMs: Int): Unit = {
+    require(connectTimeoutMs > 0, "The Ivy connection timeout must be positive")
+    require(readTimeoutMs > 0, "The Ivy read timeout must be positive")
+
+    val name = s"spark-runtime-${UUID.randomUUID()}"
+    val timeout = new NamedTimeoutConstraint(name)
+    timeout.setConnectionTimeout(connectTimeoutMs)
+    timeout.setReadTimeout(readTimeoutMs)
+    ivySettings.addConfigured(timeout)
+
+    ivySettings.getResolvers.asScala.foreach {
+      case resolver: AbstractResolver =>
+        resolver.setTimeoutConstraint(name)
+        resolver.validate()
+      case _ =>
+    }
+  }
+
   /* Set ivy settings for location of cache, if option is supplied */
   private[util] def processIvyPathArg(ivySettings: IvySettings, ivyPath: Option[String]): Unit = {
     val alternateIvyDir = ivyPath.filterNot(_.trim.isEmpty).getOrElse {
@@ -454,10 +509,31 @@ private[spark] object MavenUtils extends Logging {
       noCacheIvySettings: Option[IvySettings] = None,
       transitive: Boolean,
       exclusions: Seq[String] = Nil,
-      isTest: Boolean = false)(implicit printStream: PrintStream): Seq[String] = {
+      isTest: Boolean = false)(
+      implicit printStream: PrintStream): Seq[String] = {
+    resolveMavenCoordinatesWithCancellation(
+      coordinates,
+      ivySettings,
+      noCacheIvySettings,
+      transitive,
+      exclusions,
+      isTest,
+      () => false)
+  }
+
+  private[spark] def resolveMavenCoordinatesWithCancellation(
+      coordinates: String,
+      ivySettings: IvySettings,
+      noCacheIvySettings: Option[IvySettings],
+      transitive: Boolean,
+      exclusions: Seq[String],
+      isTest: Boolean,
+      isCancelled: () => Boolean)(
+      implicit printStream: PrintStream): Seq[String] = {
     if (coordinates == null || coordinates.trim.isEmpty) {
       Nil
     } else {
+      acquireIvyLock(isCancelled)
       val sysOut = System.out
       // Default configuration name for ivy
       val ivyConfName = "default"
@@ -550,6 +626,7 @@ private[spark] object MavenUtils extends Logging {
         if (md != null) {
           clearIvyResolutionFiles(md.getModuleRevisionId, ivySettings.getDefaultCache, ivyConfName)
         }
+        ivyLock.unlock()
       }
     }
   }
diff --git a/common/utils/src/main/scala/org/apache/spark/util/RuntimeDependencyResolver.scala b/common/utils/src/main/scala/org/apache/spark/util/RuntimeDependencyResolver.scala
new file mode 100644
index 000000000000..ae18eb3f18bc
--- /dev/null
+++ b/common/utils/src/main/scala/org/apache/spark/util/RuntimeDependencyResolver.scala
@@ -0,0 +1,125 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.util
+
+import java.io.PrintStream
+import java.net.URI
+import java.nio.file.{Path, Paths}
+import java.util.concurrent.CancellationException
+
+import org.apache.spark.util.ArrayImplicits._
+
+/** Resolves runtime Ivy dependencies using one immutable Spark configuration snapshot. */
+private[spark] final class RuntimeDependencyResolver(
+    ivySettingsPath: Option[String],
+    ivyPath: Option[String],
+    localIvyPath: Option[String] = None) {
+  import RuntimeDependencyResolver._
+
+  /** Resolve an ivy URI. Calls are serialized because Ivy mutates process-wide state. */
+  def resolve(
+      uri: URI,
+      connectTimeoutMs: Int,
+      readTimeoutMs: Int,
+      repositoryPolicy: RepositoryPolicy = AllowRequestedRepositories,
+      isCancelled: () => Boolean = () => false): Seq[Path] = {
+    checkCancelled(isCancelled)
+    require(uri.getScheme == "ivy", s"Expected an ivy URI, found: $uri")
+    val authority = Option(uri.getAuthority).getOrElse {
+      throw new IllegalArgumentException(
+        s"Invalid Ivy URI authority in uri $uri: Expected 'org:module:version', found null.")
+    }
+    if (authority.split(":").length != 3) {
+      throw new IllegalArgumentException(
+        s"Invalid Ivy URI authority in uri $uri: " +
+          s"Expected 'org:module:version', found $authority.")
+    }
+
+    checkCancelled(isCancelled)
+    val (transitive, exclusions, requestedRepositories) = MavenUtils.parseQueryParams(uri)
+    val requested = requestedRepositories
+      .split(",")
+      .map(_.trim)
+      .filter(_.nonEmpty)
+      .toImmutableArraySeq
+    val repositories = repositoryPolicy
+      .validate(requested)
+      .iterator
+      .map(_.trim)
+      .filter(_.nonEmpty)
+      .toSeq
+      .distinct
+
+    implicit val printStream: PrintStream = System.err
+    val ivySettings = ivySettingsPath.filter(_.trim.nonEmpty) match {
+      case Some(path) =>
+        MavenUtils.loadIvySettings(path, repositoriesOption(repositories), localIvyPath)
+      case None => MavenUtils.buildIvySettings(repositoriesOption(repositories), localIvyPath)
+    }
+    // Resolver roots retain the configured/default local Ivy directory, while cache metadata and
+    // retrieved jars are isolated in the runtime directory.
+    MavenUtils.processIvyPathArg(ivySettings, ivyPath)
+    MavenUtils.setResolverTimeouts(ivySettings, connectTimeoutMs, readTimeoutMs)
+
+    checkCancelled(isCancelled)
+    val exclusionsList = exclusions
+      .split(",")
+      .map(_.trim)
+      .filter(_.nonEmpty)
+      .toImmutableArraySeq
+    val result = MavenUtils.resolveMavenCoordinatesWithCancellation(
+      authority,
+      ivySettings,
+      noCacheIvySettings = None,
+      transitive = transitive,
+      exclusions = exclusionsList,
+      isTest = false,
+      isCancelled = isCancelled)
+    checkCancelled(isCancelled)
+    result.map(Paths.get(_))
+  }
+}
+
+private[spark] object RuntimeDependencyResolver {
+  trait RepositoryPolicy {
+    def validate(requestedRepositories: Seq[String]): Seq[String]
+  }
+
+  object AllowRequestedRepositories extends RepositoryPolicy {
+    override def validate(requestedRepositories: Seq[String]): Seq[String] = requestedRepositories
+  }
+
+  object RejectRequestedRepositories extends RepositoryPolicy {
+    override def validate(requestedRepositories: Seq[String]): Seq[String] = {
+      if (requestedRepositories.nonEmpty) {
+        throw new IllegalArgumentException(
+          "Server-side Maven dependencies do not allow repositories from the ivy URI")
+      }
+      Nil
+    }
+  }
+
+  private def repositoriesOption(repositories: Seq[String]): Option[String] =
+    Option(repositories.mkString(",")).filter(_.nonEmpty)
+
+  private def checkCancelled(isCancelled: () => Boolean): Unit = {
+    if (isCancelled() || Thread.currentThread().isInterrupted) {
+      throw new CancellationException("Runtime Maven dependency resolution was cancelled")
+    }
+  }
+}
diff --git a/common/utils/src/test/scala/org/apache/spark/util/MavenUtilsSuite.scala b/common/utils/src/test/scala/org/apache/spark/util/MavenUtilsSuite.scala
index d30422ca8dd5..e473a90110cd 100644
--- a/common/utils/src/test/scala/org/apache/spark/util/MavenUtilsSuite.scala
+++ b/common/utils/src/test/scala/org/apache/spark/util/MavenUtilsSuite.scala
@@ -21,8 +21,12 @@ import java.io.{File, OutputStream, PrintStream}
 import java.net.URI
 import java.nio.charset.StandardCharsets
 import java.nio.file.{Files, Paths}
+import java.util.concurrent.{CancellationException, CountDownLatch, Executors, TimeUnit}
+import java.util.concurrent.atomic.{AtomicBoolean, AtomicInteger}
 
 import scala.collection.mutable.ArrayBuffer
+import scala.concurrent.{ExecutionContext, Future}
+import scala.concurrent.duration._
 import scala.jdk.CollectionConverters._
 
 import org.apache.ivy.core.module.descriptor.MDArtifact
@@ -37,6 +41,7 @@ class MavenUtilsSuite
     extends AnyFunSuite // scalastyle:ignore funsuite
     with BeforeAndAfterEach {
 
+  private val resolverTimeoutMs = 30 * 1000
   private var tempIvyPath: String = _
 
   private val noOpOutputStream = new OutputStream {
@@ -100,6 +105,21 @@ class MavenUtilsSuite
     }
   }
 
+  test("runtime resolver configures Ivy network timeouts") {
+    val settings = MavenUtils.buildIvySettings(None, Some(tempIvyPath))
+
+    MavenUtils.setResolverTimeouts(settings, connectTimeoutMs = 1234, readTimeoutMs = 5678)
+
+    val resolvers = settings.getDefaultResolver
+      .asInstanceOf[ChainResolver]
+      .getResolvers
+      .asScala
+      .collect { case resolver: AbstractResolver => resolver }
+    assert(resolvers.nonEmpty)
+    assert(resolvers.forall(_.getTimeoutConstraint.getConnectionTimeout == 1234))
+    assert(resolvers.forall(_.getTimeoutConstraint.getReadTimeout == 5678))
+  }
+
   test("add dependencies works correctly") {
     val md = MavenUtils.getModuleDescriptor
     val artifacts = MavenUtils.extractMavenCoordinates("com.databricks:spark-csv_2.12:0.1," +
@@ -145,6 +165,148 @@ class MavenUtilsSuite
     }
   }
 
+  test("runtime dependency resolver loads custom Ivy settings") {
+    val main = MavenCoordinate("my.runtime.lib", "mylib", "0.1")
+    IvyTestUtils.withRepository(main, None, None) { repo =>
+      val settings = Paths.get(tempIvyPath, "ivysettings.xml")
+      Files.writeString(
+        settings,
+        s"""<ivysettings>
+           |  <settings defaultResolver="runtime"/>
+           |  <resolvers>
+           |    <ibiblio name="runtime" m2compatible="true" root="$repo"/>
+           |  </resolvers>
+           |</ivysettings>""".stripMargin)
+      val resolver = new RuntimeDependencyResolver(
+        ivySettingsPath = Some(settings.toString),
+        ivyPath = Some(tempIvyPath))
+
+      val resolved = resolver.resolve(
+        URI.create(s"ivy://${main.toString}"),
+        resolverTimeoutMs,
+        resolverTimeoutMs)
+
+      assert(resolved.exists(_.getFileName.toString.contains("my.runtime.lib_mylib-0.1")))
+      assert(resolved.forall(_.startsWith(Paths.get(tempIvyPath))))
+    }
+  }
+
+  test("runtime dependency resolver applies the repository policy") {
+    val resolver = new RuntimeDependencyResolver(None, Some(tempIvyPath))
+    val error = intercept[IllegalArgumentException] {
+      resolver.resolve(
+        URI.create("ivy://my.runtime.lib:mylib:0.1?repos=https://example.com/repository"),
+        resolverTimeoutMs,
+        resolverTimeoutMs,
+        RuntimeDependencyResolver.RejectRequestedRepositories)
+    }
+
+    assert(error.getMessage.contains("do not allow repositories"))
+  }
+
+  test("runtime dependency resolver honors cancellation before resolution") {
+    val resolver = new RuntimeDependencyResolver(None, Some(tempIvyPath))
+
+    intercept[CancellationException] {
+      resolver.resolve(
+        URI.create("ivy://my.runtime.lib:mylib:0.1"),
+        resolverTimeoutMs,
+        resolverTimeoutMs,
+        isCancelled = () => true)
+    }
+  }
+
+  test("runtime dependency resolver honors cancellation while waiting for Ivy") {
+    val executor = Executors.newFixedThreadPool(2)
+    implicit val executionContext: ExecutionContext =
+      ExecutionContext.fromExecutorService(executor)
+    try {
+      val main = MavenCoordinate("my.runtime.waiting", "mylib", "0.1")
+      IvyTestUtils.withRepository(main, None, None) { repo =>
+        val firstEnteredIvy = new CountDownLatch(1)
+        val releaseFirst = new CountDownLatch(1)
+        val blockingPrintStream = new PrintStream(noOpOutputStream) {
+          private val blocked = new AtomicBoolean(false)
+          // scalastyle:off println
+          override def println(line: String): Unit = {
+            if (blocked.compareAndSet(false, true)) {
+              firstEnteredIvy.countDown()
+              releaseFirst.await()
+            }
+          }
+          // scalastyle:on println
+        }
+        val firstSettings = MavenUtils.buildIvySettings(
+          Some(repo),
+          Some(Paths.get(tempIvyPath, "first").toString))
+        val first = Future {
+          MavenUtils.resolveMavenCoordinates(
+            main.toString,
+            firstSettings,
+            transitive = true,
+            isTest = true)(blockingPrintStream)
+        }
+
+        assert(firstEnteredIvy.await(5, TimeUnit.SECONDS))
+        val cancelled = new AtomicBoolean(false)
+        val cancellationChecks = new AtomicInteger(0)
+        val waitingForIvy = new CountDownLatch(1)
+        val resolver = new RuntimeDependencyResolver(
+          ivySettingsPath = None,
+          ivyPath = Some(Paths.get(tempIvyPath, "second").toString))
+        val second = Future {
+          resolver.resolve(
+            URI.create(s"ivy://${main.toString}"),
+            resolverTimeoutMs,
+            resolverTimeoutMs,
+            isCancelled = () => {
+              if (cancellationChecks.incrementAndGet() >= 5) {
+                waitingForIvy.countDown()
+              }
+              cancelled.get()
+            })
+        }
+
+        try {
+          assert(waitingForIvy.await(5, TimeUnit.SECONDS))
+          cancelled.set(true)
+          intercept[CancellationException] {
+            SparkThreadUtils.awaitResultNoSparkExceptionConversion(second, 5.seconds)
+          }
+        } finally {
+          releaseFirst.countDown()
+        }
+        SparkThreadUtils.awaitResultNoSparkExceptionConversion(first, 5.seconds)
+      }
+    } finally {
+      executor.shutdownNow()
+    }
+  }
+
+  test("runtime dependency resolver preserves the local Ivy repository") {
+    val main = MavenCoordinate("my.runtime.local", "mylib", "0.1")
+    val localIvyHome = Paths.get(tempIvyPath, "local-ivy")
+    val runtimeIvyPath = Paths.get(tempIvyPath, "runtime-ivy")
+    IvyTestUtils.withRepository(
+      main,
+      dependencies = None,
+      rootDir = Some(localIvyHome.resolve("local").toFile),
+      useIvyLayout = true) { _ =>
+      val resolver = new RuntimeDependencyResolver(
+        ivySettingsPath = None,
+        ivyPath = Some(runtimeIvyPath.toString),
+        localIvyPath = Some(localIvyHome.toString))
+
+      val resolved = resolver.resolve(
+        URI.create(s"ivy://${main.toString}"),
+        resolverTimeoutMs,
+        resolverTimeoutMs)
+
+      assert(resolved.exists(_.getFileName.toString.contains("my.runtime.local_mylib-0.1")))
+      assert(resolved.forall(_.startsWith(runtimeIvyPath)))
+    }
+  }
+
   test("search for artifact at local repositories") {
     val main = new MavenCoordinate("my.great.lib", "mylib", "0.1")
     val dep = "my.great.dep:mydep:0.5"
diff --git a/core/src/main/scala/org/apache/spark/internal/config/package.scala b/core/src/main/scala/org/apache/spark/internal/config/package.scala
index b08bb116e984..1a0c583d9b53 100644
--- a/core/src/main/scala/org/apache/spark/internal/config/package.scala
+++ b/core/src/main/scala/org/apache/spark/internal/config/package.scala
@@ -3059,9 +3059,12 @@ package object config {
   private[spark] val JAR_IVY_SETTING_PATH =
     ConfigBuilder(MavenUtils.JAR_IVY_SETTING_PATH_KEY)
       .doc("Path to an Ivy settings file to customize resolution of jars specified " +
-        "using spark.jars.packages instead of the built-in defaults, such as maven central. " +
-        "Additional repositories given by the command-line option --repositories " +
-        "or spark.jars.repositories will also be included. " +
+        "using spark.jars.packages or ivy:// URIs passed to SparkSession.addArtifact instead " +
+        "of the built-in defaults, such as maven central. " +
+        "For spark.jars.packages, additional repositories from spark.jars.repositories will " +
+        "also be included. " +
+        "Client-resolved Spark Connect Ivy URIs do not use this setting. " +
+        "The spark-submit --repositories option applies to submission-time resolution. " +
         "Useful for allowing Spark to resolve artifacts from behind a firewall " +
         "e.g. via an in-house artifact server like Artifactory. " +
         "Details on the settings file format can be found at Settings Files")
@@ -3069,6 +3072,32 @@ package object config {
       .stringConf
       .createOptional
 
+  private[spark] val JAR_IVY_CONNECT_TIMEOUT =
+    ConfigBuilder("spark.jars.ivyConnectTimeout")
+      .doc("Connection timeout for Ivy repository requests made by " +
+        "SparkSession.addArtifact. Client-resolved Spark Connect Ivy URIs do not use this " +
+        "setting. This must be set before the SparkContext starts.")
+      .version("4.4.0")
+      .withBindingPolicy(ConfigBindingPolicy.NOT_APPLICABLE)
+      .timeConf(TimeUnit.MILLISECONDS)
+      .checkValue(
+        timeout => timeout > 0 && timeout <= Int.MaxValue,
+        s"Timeout must be positive and no greater than ${Int.MaxValue} milliseconds.")
+      .createWithDefaultString("30s")
+
+  private[spark] val JAR_IVY_READ_TIMEOUT =
+    ConfigBuilder("spark.jars.ivyReadTimeout")
+      .doc("Read timeout for Ivy repository requests made by SparkSession.addArtifact. " +
+        "Client-resolved Spark Connect Ivy URIs do not use this setting. " +
+        "This must be set before the SparkContext starts.")
+      .version("4.4.0")
+      .withBindingPolicy(ConfigBindingPolicy.NOT_APPLICABLE)
+      .timeConf(TimeUnit.MILLISECONDS)
+      .checkValue(
+        timeout => timeout > 0 && timeout <= Int.MaxValue,
+        s"Timeout must be positive and no greater than ${Int.MaxValue} milliseconds.")
+      .createWithDefaultString("5m")
+
   private[spark] val JAR_PACKAGES =
     ConfigBuilder("spark.jars.packages")
       .doc("Comma-separated list of Maven coordinates of jars to include " +
diff --git a/docs/configuration.md b/docs/configuration.md
index c96cbc74820b..fef5d4b3ba2a 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -1071,9 +1071,16 @@ Apart from these, the following properties are also available, and may be useful
   <td><code>spark.jars.ivySettings</code></td>
   <td></td>
   <td>
-    Path to an Ivy settings file to customize resolution of jars specified using <code>spark.jars.packages</code>
-    instead of the built-in defaults, such as maven central. Additional repositories given by the command-line
-    option <code>--repositories</code> or <code>spark.jars.repositories</code> will also be included.
+    Path to an Ivy settings file to customize resolution of jars specified using
+    <code>spark.jars.packages</code> or <code>ivy://</code> URIs passed to
+    <code>SparkSession.addArtifact</code> instead of the built-in defaults, such as maven central.
+    For <code>spark.jars.packages</code>, additional repositories from
+    <code>spark.jars.repositories</code> will also be included.
+    The <code>spark-submit --repositories</code> option applies to submission-time resolution.
+    In Spark Connect, Ivy URIs with a <code>repos</code> query parameter or sent to a server without
+    server-side Maven resolution are resolved by the client and do not use this setting. Other Ivy
+    URIs are resolved by the server, so Maven and Ivy repositories local to the client, such as
+    <code>~/.m2/repository</code> and <code>~/.ivy2.5.2/local</code>, are not searched.
     Useful for allowing Spark to resolve artifacts from behind a firewall e.g. via an in-house
     artifact server like Artifactory. Details on the settings file format can be
     found at <a href="http://ant.apache.org/ivy/history/latest-milestone/settings.html">Settings Files</a>.
@@ -1085,7 +1092,28 @@ Apart from these, the following properties are also available, and may be useful
   </td>
   <td>2.2.0</td>
 </tr>
- <tr>
+<tr>
+  <td><code>spark.jars.ivyConnectTimeout</code></td>
+  <td>30s</td>
+  <td>
+    Connection timeout for Ivy repository requests made by
+    <code>SparkSession.addArtifact</code>.
+    Client-resolved Spark Connect Ivy URIs do not use this setting.
+    This must be set before the SparkContext starts.
+  </td>
+  <td>4.4.0</td>
+</tr>
+<tr>
+  <td><code>spark.jars.ivyReadTimeout</code></td>
+  <td>5m</td>
+  <td>
+    Read timeout for Ivy repository requests made by <code>SparkSession.addArtifact</code>.
+    Client-resolved Spark Connect Ivy URIs do not use this setting.
+    This must be set before the SparkContext starts.
+  </td>
+  <td>4.4.0</td>
+</tr>
+<tr>
   <td><code>spark.jars.repositories</code></td>
   <td></td>
   <td>
diff --git a/python/pyspark/sql/connect/proto/base_pb2.py b/python/pyspark/sql/connect/proto/base_pb2.py
index ee3ddb48ba5c..432f50f8d421 100644
--- a/python/pyspark/sql/connect/proto/base_pb2.py
+++ b/python/pyspark/sql/connect/proto/base_pb2.py
@@ -46,7 +46,7 @@ from pyspark.sql.connect.proto import pipelines_pb2 as spark_dot_connect_dot_pip
 
 
 DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(
-    b'\n\x18spark/connect/base.proto\x12\rspark.connect\x1a\x19google/protobuf/any.proto\x1a\x1cspark/connect/commands.proto\x1a\x1aspark/connect/common.proto\x1a\x1fspark/connect/expressions.proto\x1a\x1dspark/connect/relations.proto\x1a\x19spark/connect/types.proto\x1a\x16spark/connect/ml.proto\x1a\x1dspark/connect/pipelines.proto"\xe3\x03\n\x04Plan\x12-\n\x04root\x18\x01 \x01(\x0b\x32\x17.spark.connect.RelationH\x00R\x04root\x12\x32\n\x07\x63ommand\x18\x02 \x01(\x0b\x32\x16.spark.conn [...]
+    b'\n\x18spark/connect/base.proto\x12\rspark.connect\x1a\x19google/protobuf/any.proto\x1a\x1cspark/connect/commands.proto\x1a\x1aspark/connect/common.proto\x1a\x1fspark/connect/expressions.proto\x1a\x1dspark/connect/relations.proto\x1a\x19spark/connect/types.proto\x1a\x16spark/connect/ml.proto\x1a\x1dspark/connect/pipelines.proto"\xe3\x03\n\x04Plan\x12-\n\x04root\x18\x01 \x01(\x0b\x32\x17.spark.connect.RelationH\x00R\x04root\x12\x32\n\x07\x63ommand\x18\x02 \x01(\x0b\x32\x16.spark.conn [...]
 )
 
 _globals = globals()
@@ -71,8 +71,8 @@ if not _descriptor._USE_C_DESCRIPTORS:
     _globals[
         "_FETCHERRORDETAILSRESPONSE_SPARKTHROWABLE_MESSAGEPARAMETERSENTRY"
     ]._serialized_options = b"8\001"
-    _globals["_COMPRESSIONCODEC"]._serialized_start = 20046
-    _globals["_COMPRESSIONCODEC"]._serialized_end = 20127
+    _globals["_COMPRESSIONCODEC"]._serialized_start = 20406
+    _globals["_COMPRESSIONCODEC"]._serialized_end = 20487
     _globals["_PLAN"]._serialized_start = 275
     _globals["_PLAN"]._serialized_end = 758
     _globals["_PLAN_COMPRESSEDOPERATION"]._serialized_start = 477
@@ -114,7 +114,7 @@ if not _descriptor._USE_C_DESCRIPTORS:
     _globals["_ANALYZEPLANREQUEST_JSONTODDL"]._serialized_start = 3448
     _globals["_ANALYZEPLANREQUEST_JSONTODDL"]._serialized_end = 3492
     _globals["_ANALYZEPLANRESPONSE"]._serialized_start = 3565
-    _globals["_ANALYZEPLANRESPONSE"]._serialized_end = 5486
+    _globals["_ANALYZEPLANRESPONSE"]._serialized_end = 5522
     _globals["_ANALYZEPLANRESPONSE_SCHEMA"]._serialized_start = 4861
     _globals["_ANALYZEPLANRESPONSE_SCHEMA"]._serialized_end = 4918
     _globals["_ANALYZEPLANRESPONSE_EXPLAIN"]._serialized_start = 4920
@@ -128,159 +128,163 @@ if not _descriptor._USE_C_DESCRIPTORS:
     _globals["_ANALYZEPLANRESPONSE_INPUTFILES"]._serialized_start = 5105
     _globals["_ANALYZEPLANRESPONSE_INPUTFILES"]._serialized_end = 5139
     _globals["_ANALYZEPLANRESPONSE_SPARKVERSION"]._serialized_start = 5141
-    _globals["_ANALYZEPLANRESPONSE_SPARKVERSION"]._serialized_end = 5181
-    _globals["_ANALYZEPLANRESPONSE_DDLPARSE"]._serialized_start = 5183
-    _globals["_ANALYZEPLANRESPONSE_DDLPARSE"]._serialized_end = 5242
-    _globals["_ANALYZEPLANRESPONSE_SAMESEMANTICS"]._serialized_start = 5244
-    _globals["_ANALYZEPLANRESPONSE_SAMESEMANTICS"]._serialized_end = 5283
-    _globals["_ANALYZEPLANRESPONSE_SEMANTICHASH"]._serialized_start = 5285
-    _globals["_ANALYZEPLANRESPONSE_SEMANTICHASH"]._serialized_end = 5323
+    _globals["_ANALYZEPLANRESPONSE_SPARKVERSION"]._serialized_end = 5217
+    _globals["_ANALYZEPLANRESPONSE_DDLPARSE"]._serialized_start = 5219
+    _globals["_ANALYZEPLANRESPONSE_DDLPARSE"]._serialized_end = 5278
+    _globals["_ANALYZEPLANRESPONSE_SAMESEMANTICS"]._serialized_start = 5280
+    _globals["_ANALYZEPLANRESPONSE_SAMESEMANTICS"]._serialized_end = 5319
+    _globals["_ANALYZEPLANRESPONSE_SEMANTICHASH"]._serialized_start = 5321
+    _globals["_ANALYZEPLANRESPONSE_SEMANTICHASH"]._serialized_end = 5359
     _globals["_ANALYZEPLANRESPONSE_PERSIST"]._serialized_start = 3111
     _globals["_ANALYZEPLANRESPONSE_PERSIST"]._serialized_end = 3120
     _globals["_ANALYZEPLANRESPONSE_UNPERSIST"]._serialized_start = 3264
     _globals["_ANALYZEPLANRESPONSE_UNPERSIST"]._serialized_end = 3275
-    _globals["_ANALYZEPLANRESPONSE_GETSTORAGELEVEL"]._serialized_start = 5349
-    _globals["_ANALYZEPLANRESPONSE_GETSTORAGELEVEL"]._serialized_end = 5432
-    _globals["_ANALYZEPLANRESPONSE_JSONTODDL"]._serialized_start = 5434
-    _globals["_ANALYZEPLANRESPONSE_JSONTODDL"]._serialized_end = 5476
-    _globals["_EXECUTEPLANREQUEST"]._serialized_start = 5489
-    _globals["_EXECUTEPLANREQUEST"]._serialized_end = 6260
-    _globals["_EXECUTEPLANREQUEST_REQUESTOPTION"]._serialized_start = 5923
-    _globals["_EXECUTEPLANREQUEST_REQUESTOPTION"]._serialized_end = 6184
-    _globals["_EXECUTEPLANRESPONSE"]._serialized_start = 6263
-    _globals["_EXECUTEPLANRESPONSE"]._serialized_end = 9854
-    _globals["_EXECUTEPLANRESPONSE_SQLCOMMANDRESULT"]._serialized_start = 8363
-    _globals["_EXECUTEPLANRESPONSE_SQLCOMMANDRESULT"]._serialized_end = 8434
-    _globals["_EXECUTEPLANRESPONSE_ARROWBATCH"]._serialized_start = 8437
-    _globals["_EXECUTEPLANRESPONSE_ARROWBATCH"]._serialized_end = 8685
-    _globals["_EXECUTEPLANRESPONSE_METRICS"]._serialized_start = 8688
-    _globals["_EXECUTEPLANRESPONSE_METRICS"]._serialized_end = 9205
-    _globals["_EXECUTEPLANRESPONSE_METRICS_METRICOBJECT"]._serialized_start = 8783
-    _globals["_EXECUTEPLANRESPONSE_METRICS_METRICOBJECT"]._serialized_end = 9115
+    _globals["_ANALYZEPLANRESPONSE_GETSTORAGELEVEL"]._serialized_start = 5385
+    _globals["_ANALYZEPLANRESPONSE_GETSTORAGELEVEL"]._serialized_end = 5468
+    _globals["_ANALYZEPLANRESPONSE_JSONTODDL"]._serialized_start = 5470
+    _globals["_ANALYZEPLANRESPONSE_JSONTODDL"]._serialized_end = 5512
+    _globals["_EXECUTEPLANREQUEST"]._serialized_start = 5525
+    _globals["_EXECUTEPLANREQUEST"]._serialized_end = 6296
+    _globals["_EXECUTEPLANREQUEST_REQUESTOPTION"]._serialized_start = 5959
+    _globals["_EXECUTEPLANREQUEST_REQUESTOPTION"]._serialized_end = 6220
+    _globals["_EXECUTEPLANRESPONSE"]._serialized_start = 6299
+    _globals["_EXECUTEPLANRESPONSE"]._serialized_end = 9890
+    _globals["_EXECUTEPLANRESPONSE_SQLCOMMANDRESULT"]._serialized_start = 8399
+    _globals["_EXECUTEPLANRESPONSE_SQLCOMMANDRESULT"]._serialized_end = 8470
+    _globals["_EXECUTEPLANRESPONSE_ARROWBATCH"]._serialized_start = 8473
+    _globals["_EXECUTEPLANRESPONSE_ARROWBATCH"]._serialized_end = 8721
+    _globals["_EXECUTEPLANRESPONSE_METRICS"]._serialized_start = 8724
+    _globals["_EXECUTEPLANRESPONSE_METRICS"]._serialized_end = 9241
+    _globals["_EXECUTEPLANRESPONSE_METRICS_METRICOBJECT"]._serialized_start = 8819
+    _globals["_EXECUTEPLANRESPONSE_METRICS_METRICOBJECT"]._serialized_end = 9151
     _globals[
         "_EXECUTEPLANRESPONSE_METRICS_METRICOBJECT_EXECUTIONMETRICSENTRY"
-    ]._serialized_start = 8992
+    ]._serialized_start = 9028
     _globals[
         "_EXECUTEPLANRESPONSE_METRICS_METRICOBJECT_EXECUTIONMETRICSENTRY"
-    ]._serialized_end = 9115
-    _globals["_EXECUTEPLANRESPONSE_METRICS_METRICVALUE"]._serialized_start = 9117
-    _globals["_EXECUTEPLANRESPONSE_METRICS_METRICVALUE"]._serialized_end = 9205
-    _globals["_EXECUTEPLANRESPONSE_OBSERVEDMETRICS"]._serialized_start = 9208
-    _globals["_EXECUTEPLANRESPONSE_OBSERVEDMETRICS"]._serialized_end = 9483
-    _globals["_EXECUTEPLANRESPONSE_RESULTCOMPLETE"]._serialized_start = 9485
-    _globals["_EXECUTEPLANRESPONSE_RESULTCOMPLETE"]._serialized_end = 9501
-    _globals["_EXECUTEPLANRESPONSE_EXECUTIONPROGRESS"]._serialized_start = 9504
-    _globals["_EXECUTEPLANRESPONSE_EXECUTIONPROGRESS"]._serialized_end = 9837
-    _globals["_EXECUTEPLANRESPONSE_EXECUTIONPROGRESS_STAGEINFO"]._serialized_start = 9660
-    _globals["_EXECUTEPLANRESPONSE_EXECUTIONPROGRESS_STAGEINFO"]._serialized_end = 9837
-    _globals["_KEYVALUE"]._serialized_start = 9856
-    _globals["_KEYVALUE"]._serialized_end = 9921
-    _globals["_CONFIGREQUEST"]._serialized_start = 9924
-    _globals["_CONFIGREQUEST"]._serialized_end = 11123
-    _globals["_CONFIGREQUEST_OPERATION"]._serialized_start = 10232
-    _globals["_CONFIGREQUEST_OPERATION"]._serialized_end = 10730
-    _globals["_CONFIGREQUEST_SET"]._serialized_start = 10732
-    _globals["_CONFIGREQUEST_SET"]._serialized_end = 10824
-    _globals["_CONFIGREQUEST_GET"]._serialized_start = 10826
-    _globals["_CONFIGREQUEST_GET"]._serialized_end = 10851
-    _globals["_CONFIGREQUEST_GETWITHDEFAULT"]._serialized_start = 10853
-    _globals["_CONFIGREQUEST_GETWITHDEFAULT"]._serialized_end = 10916
-    _globals["_CONFIGREQUEST_GETOPTION"]._serialized_start = 10918
-    _globals["_CONFIGREQUEST_GETOPTION"]._serialized_end = 10949
-    _globals["_CONFIGREQUEST_GETALL"]._serialized_start = 10951
-    _globals["_CONFIGREQUEST_GETALL"]._serialized_end = 10999
-    _globals["_CONFIGREQUEST_UNSET"]._serialized_start = 11001
-    _globals["_CONFIGREQUEST_UNSET"]._serialized_end = 11028
-    _globals["_CONFIGREQUEST_ISMODIFIABLE"]._serialized_start = 11030
-    _globals["_CONFIGREQUEST_ISMODIFIABLE"]._serialized_end = 11064
-    _globals["_CONFIGRESPONSE"]._serialized_start = 11126
-    _globals["_CONFIGRESPONSE"]._serialized_end = 11301
-    _globals["_ADDARTIFACTSREQUEST"]._serialized_start = 11304
-    _globals["_ADDARTIFACTSREQUEST"]._serialized_end = 12306
-    _globals["_ADDARTIFACTSREQUEST_ARTIFACTCHUNK"]._serialized_start = 11779
-    _globals["_ADDARTIFACTSREQUEST_ARTIFACTCHUNK"]._serialized_end = 11832
-    _globals["_ADDARTIFACTSREQUEST_SINGLECHUNKARTIFACT"]._serialized_start = 11834
-    _globals["_ADDARTIFACTSREQUEST_SINGLECHUNKARTIFACT"]._serialized_end = 11945
-    _globals["_ADDARTIFACTSREQUEST_BATCH"]._serialized_start = 11947
-    _globals["_ADDARTIFACTSREQUEST_BATCH"]._serialized_end = 12040
-    _globals["_ADDARTIFACTSREQUEST_BEGINCHUNKEDARTIFACT"]._serialized_start = 12043
-    _globals["_ADDARTIFACTSREQUEST_BEGINCHUNKEDARTIFACT"]._serialized_end = 12236
-    _globals["_ADDARTIFACTSRESPONSE"]._serialized_start = 12309
-    _globals["_ADDARTIFACTSRESPONSE"]._serialized_end = 12581
-    _globals["_ADDARTIFACTSRESPONSE_ARTIFACTSUMMARY"]._serialized_start = 12500
-    _globals["_ADDARTIFACTSRESPONSE_ARTIFACTSUMMARY"]._serialized_end = 12581
-    _globals["_ARTIFACTSTATUSESREQUEST"]._serialized_start = 12584
-    _globals["_ARTIFACTSTATUSESREQUEST"]._serialized_end = 12910
-    _globals["_ARTIFACTSTATUSESRESPONSE"]._serialized_start = 12913
-    _globals["_ARTIFACTSTATUSESRESPONSE"]._serialized_end = 13265
-    _globals["_ARTIFACTSTATUSESRESPONSE_STATUSESENTRY"]._serialized_start = 13108
-    _globals["_ARTIFACTSTATUSESRESPONSE_STATUSESENTRY"]._serialized_end = 13223
-    _globals["_ARTIFACTSTATUSESRESPONSE_ARTIFACTSTATUS"]._serialized_start = 13225
-    _globals["_ARTIFACTSTATUSESRESPONSE_ARTIFACTSTATUS"]._serialized_end = 13265
-    _globals["_INTERRUPTREQUEST"]._serialized_start = 13268
-    _globals["_INTERRUPTREQUEST"]._serialized_end = 13871
-    _globals["_INTERRUPTREQUEST_INTERRUPTTYPE"]._serialized_start = 13671
-    _globals["_INTERRUPTREQUEST_INTERRUPTTYPE"]._serialized_end = 13799
-    _globals["_INTERRUPTRESPONSE"]._serialized_start = 13874
-    _globals["_INTERRUPTRESPONSE"]._serialized_end = 14018
-    _globals["_REATTACHOPTIONS"]._serialized_start = 14020
-    _globals["_REATTACHOPTIONS"]._serialized_end = 14073
-    _globals["_RESULTCHUNKINGOPTIONS"]._serialized_start = 14076
-    _globals["_RESULTCHUNKINGOPTIONS"]._serialized_end = 14257
-    _globals["_REATTACHEXECUTEREQUEST"]._serialized_start = 14260
-    _globals["_REATTACHEXECUTEREQUEST"]._serialized_end = 14666
-    _globals["_RELEASEEXECUTEREQUEST"]._serialized_start = 14669
-    _globals["_RELEASEEXECUTEREQUEST"]._serialized_end = 15254
-    _globals["_RELEASEEXECUTEREQUEST_RELEASEALL"]._serialized_start = 15123
-    _globals["_RELEASEEXECUTEREQUEST_RELEASEALL"]._serialized_end = 15135
-    _globals["_RELEASEEXECUTEREQUEST_RELEASEUNTIL"]._serialized_start = 15137
-    _globals["_RELEASEEXECUTEREQUEST_RELEASEUNTIL"]._serialized_end = 15184
-    _globals["_RELEASEEXECUTERESPONSE"]._serialized_start = 15257
-    _globals["_RELEASEEXECUTERESPONSE"]._serialized_end = 15422
-    _globals["_RELEASESESSIONREQUEST"]._serialized_start = 15425
-    _globals["_RELEASESESSIONREQUEST"]._serialized_end = 15637
-    _globals["_RELEASESESSIONRESPONSE"]._serialized_start = 15639
-    _globals["_RELEASESESSIONRESPONSE"]._serialized_end = 15747
-    _globals["_FETCHERRORDETAILSREQUEST"]._serialized_start = 15750
-    _globals["_FETCHERRORDETAILSREQUEST"]._serialized_end = 16082
-    _globals["_FETCHERRORDETAILSRESPONSE"]._serialized_start = 16085
-    _globals["_FETCHERRORDETAILSRESPONSE"]._serialized_end = 18094
-    _globals["_FETCHERRORDETAILSRESPONSE_STACKTRACEELEMENT"]._serialized_start = 16314
-    _globals["_FETCHERRORDETAILSRESPONSE_STACKTRACEELEMENT"]._serialized_end = 16488
-    _globals["_FETCHERRORDETAILSRESPONSE_QUERYCONTEXT"]._serialized_start = 16491
-    _globals["_FETCHERRORDETAILSRESPONSE_QUERYCONTEXT"]._serialized_end = 16859
-    _globals["_FETCHERRORDETAILSRESPONSE_QUERYCONTEXT_CONTEXTTYPE"]._serialized_start = 16822
-    _globals["_FETCHERRORDETAILSRESPONSE_QUERYCONTEXT_CONTEXTTYPE"]._serialized_end = 16859
-    _globals["_FETCHERRORDETAILSRESPONSE_SPARKTHROWABLE"]._serialized_start = 16862
-    _globals["_FETCHERRORDETAILSRESPONSE_SPARKTHROWABLE"]._serialized_end = 17412
+    ]._serialized_end = 9151
+    _globals["_EXECUTEPLANRESPONSE_METRICS_METRICVALUE"]._serialized_start = 9153
+    _globals["_EXECUTEPLANRESPONSE_METRICS_METRICVALUE"]._serialized_end = 9241
+    _globals["_EXECUTEPLANRESPONSE_OBSERVEDMETRICS"]._serialized_start = 9244
+    _globals["_EXECUTEPLANRESPONSE_OBSERVEDMETRICS"]._serialized_end = 9519
+    _globals["_EXECUTEPLANRESPONSE_RESULTCOMPLETE"]._serialized_start = 9521
+    _globals["_EXECUTEPLANRESPONSE_RESULTCOMPLETE"]._serialized_end = 9537
+    _globals["_EXECUTEPLANRESPONSE_EXECUTIONPROGRESS"]._serialized_start = 9540
+    _globals["_EXECUTEPLANRESPONSE_EXECUTIONPROGRESS"]._serialized_end = 9873
+    _globals["_EXECUTEPLANRESPONSE_EXECUTIONPROGRESS_STAGEINFO"]._serialized_start = 9696
+    _globals["_EXECUTEPLANRESPONSE_EXECUTIONPROGRESS_STAGEINFO"]._serialized_end = 9873
+    _globals["_KEYVALUE"]._serialized_start = 9892
+    _globals["_KEYVALUE"]._serialized_end = 9957
+    _globals["_CONFIGREQUEST"]._serialized_start = 9960
+    _globals["_CONFIGREQUEST"]._serialized_end = 11159
+    _globals["_CONFIGREQUEST_OPERATION"]._serialized_start = 10268
+    _globals["_CONFIGREQUEST_OPERATION"]._serialized_end = 10766
+    _globals["_CONFIGREQUEST_SET"]._serialized_start = 10768
+    _globals["_CONFIGREQUEST_SET"]._serialized_end = 10860
+    _globals["_CONFIGREQUEST_GET"]._serialized_start = 10862
+    _globals["_CONFIGREQUEST_GET"]._serialized_end = 10887
+    _globals["_CONFIGREQUEST_GETWITHDEFAULT"]._serialized_start = 10889
+    _globals["_CONFIGREQUEST_GETWITHDEFAULT"]._serialized_end = 10952
+    _globals["_CONFIGREQUEST_GETOPTION"]._serialized_start = 10954
+    _globals["_CONFIGREQUEST_GETOPTION"]._serialized_end = 10985
+    _globals["_CONFIGREQUEST_GETALL"]._serialized_start = 10987
+    _globals["_CONFIGREQUEST_GETALL"]._serialized_end = 11035
+    _globals["_CONFIGREQUEST_UNSET"]._serialized_start = 11037
+    _globals["_CONFIGREQUEST_UNSET"]._serialized_end = 11064
+    _globals["_CONFIGREQUEST_ISMODIFIABLE"]._serialized_start = 11066
+    _globals["_CONFIGREQUEST_ISMODIFIABLE"]._serialized_end = 11100
+    _globals["_CONFIGRESPONSE"]._serialized_start = 11162
+    _globals["_CONFIGRESPONSE"]._serialized_end = 11337
+    _globals["_ADDARTIFACTSREQUEST"]._serialized_start = 11340
+    _globals["_ADDARTIFACTSREQUEST"]._serialized_end = 12666
+    _globals["_ADDARTIFACTSREQUEST_ARTIFACTCHUNK"]._serialized_start = 11815
+    _globals["_ADDARTIFACTSREQUEST_ARTIFACTCHUNK"]._serialized_end = 11868
+    _globals["_ADDARTIFACTSREQUEST_SINGLECHUNKARTIFACT"]._serialized_start = 11870
+    _globals["_ADDARTIFACTSREQUEST_SINGLECHUNKARTIFACT"]._serialized_end = 11981
+    _globals["_ADDARTIFACTSREQUEST_MAVENDEPENDENCY"]._serialized_start = 11983
+    _globals["_ADDARTIFACTSREQUEST_MAVENDEPENDENCY"]._serialized_end = 12018
+    _globals["_ADDARTIFACTSREQUEST_ARTIFACTENTRY"]._serialized_start = 12021
+    _globals["_ADDARTIFACTSREQUEST_ARTIFACTENTRY"]._serialized_end = 12228
+    _globals["_ADDARTIFACTSREQUEST_BATCH"]._serialized_start = 12231
+    _globals["_ADDARTIFACTSREQUEST_BATCH"]._serialized_end = 12400
+    _globals["_ADDARTIFACTSREQUEST_BEGINCHUNKEDARTIFACT"]._serialized_start = 12403
+    _globals["_ADDARTIFACTSREQUEST_BEGINCHUNKEDARTIFACT"]._serialized_end = 12596
+    _globals["_ADDARTIFACTSRESPONSE"]._serialized_start = 12669
+    _globals["_ADDARTIFACTSRESPONSE"]._serialized_end = 12941
+    _globals["_ADDARTIFACTSRESPONSE_ARTIFACTSUMMARY"]._serialized_start = 12860
+    _globals["_ADDARTIFACTSRESPONSE_ARTIFACTSUMMARY"]._serialized_end = 12941
+    _globals["_ARTIFACTSTATUSESREQUEST"]._serialized_start = 12944
+    _globals["_ARTIFACTSTATUSESREQUEST"]._serialized_end = 13270
+    _globals["_ARTIFACTSTATUSESRESPONSE"]._serialized_start = 13273
+    _globals["_ARTIFACTSTATUSESRESPONSE"]._serialized_end = 13625
+    _globals["_ARTIFACTSTATUSESRESPONSE_STATUSESENTRY"]._serialized_start = 13468
+    _globals["_ARTIFACTSTATUSESRESPONSE_STATUSESENTRY"]._serialized_end = 13583
+    _globals["_ARTIFACTSTATUSESRESPONSE_ARTIFACTSTATUS"]._serialized_start = 13585
+    _globals["_ARTIFACTSTATUSESRESPONSE_ARTIFACTSTATUS"]._serialized_end = 13625
+    _globals["_INTERRUPTREQUEST"]._serialized_start = 13628
+    _globals["_INTERRUPTREQUEST"]._serialized_end = 14231
+    _globals["_INTERRUPTREQUEST_INTERRUPTTYPE"]._serialized_start = 14031
+    _globals["_INTERRUPTREQUEST_INTERRUPTTYPE"]._serialized_end = 14159
+    _globals["_INTERRUPTRESPONSE"]._serialized_start = 14234
+    _globals["_INTERRUPTRESPONSE"]._serialized_end = 14378
+    _globals["_REATTACHOPTIONS"]._serialized_start = 14380
+    _globals["_REATTACHOPTIONS"]._serialized_end = 14433
+    _globals["_RESULTCHUNKINGOPTIONS"]._serialized_start = 14436
+    _globals["_RESULTCHUNKINGOPTIONS"]._serialized_end = 14617
+    _globals["_REATTACHEXECUTEREQUEST"]._serialized_start = 14620
+    _globals["_REATTACHEXECUTEREQUEST"]._serialized_end = 15026
+    _globals["_RELEASEEXECUTEREQUEST"]._serialized_start = 15029
+    _globals["_RELEASEEXECUTEREQUEST"]._serialized_end = 15614
+    _globals["_RELEASEEXECUTEREQUEST_RELEASEALL"]._serialized_start = 15483
+    _globals["_RELEASEEXECUTEREQUEST_RELEASEALL"]._serialized_end = 15495
+    _globals["_RELEASEEXECUTEREQUEST_RELEASEUNTIL"]._serialized_start = 15497
+    _globals["_RELEASEEXECUTEREQUEST_RELEASEUNTIL"]._serialized_end = 15544
+    _globals["_RELEASEEXECUTERESPONSE"]._serialized_start = 15617
+    _globals["_RELEASEEXECUTERESPONSE"]._serialized_end = 15782
+    _globals["_RELEASESESSIONREQUEST"]._serialized_start = 15785
+    _globals["_RELEASESESSIONREQUEST"]._serialized_end = 15997
+    _globals["_RELEASESESSIONRESPONSE"]._serialized_start = 15999
+    _globals["_RELEASESESSIONRESPONSE"]._serialized_end = 16107
+    _globals["_FETCHERRORDETAILSREQUEST"]._serialized_start = 16110
+    _globals["_FETCHERRORDETAILSREQUEST"]._serialized_end = 16442
+    _globals["_FETCHERRORDETAILSRESPONSE"]._serialized_start = 16445
+    _globals["_FETCHERRORDETAILSRESPONSE"]._serialized_end = 18454
+    _globals["_FETCHERRORDETAILSRESPONSE_STACKTRACEELEMENT"]._serialized_start = 16674
+    _globals["_FETCHERRORDETAILSRESPONSE_STACKTRACEELEMENT"]._serialized_end = 16848
+    _globals["_FETCHERRORDETAILSRESPONSE_QUERYCONTEXT"]._serialized_start = 16851
+    _globals["_FETCHERRORDETAILSRESPONSE_QUERYCONTEXT"]._serialized_end = 17219
+    _globals["_FETCHERRORDETAILSRESPONSE_QUERYCONTEXT_CONTEXTTYPE"]._serialized_start = 17182
+    _globals["_FETCHERRORDETAILSRESPONSE_QUERYCONTEXT_CONTEXTTYPE"]._serialized_end = 17219
+    _globals["_FETCHERRORDETAILSRESPONSE_SPARKTHROWABLE"]._serialized_start = 17222
+    _globals["_FETCHERRORDETAILSRESPONSE_SPARKTHROWABLE"]._serialized_end = 17772
     _globals[
         "_FETCHERRORDETAILSRESPONSE_SPARKTHROWABLE_MESSAGEPARAMETERSENTRY"
-    ]._serialized_start = 17289
+    ]._serialized_start = 17649
     _globals[
         "_FETCHERRORDETAILSRESPONSE_SPARKTHROWABLE_MESSAGEPARAMETERSENTRY"
-    ]._serialized_end = 17357
-    _globals["_FETCHERRORDETAILSRESPONSE_BREAKINGCHANGEINFO"]._serialized_start = 17415
-    _globals["_FETCHERRORDETAILSRESPONSE_BREAKINGCHANGEINFO"]._serialized_end = 17665
-    _globals["_FETCHERRORDETAILSRESPONSE_MITIGATIONCONFIG"]._serialized_start = 17667
-    _globals["_FETCHERRORDETAILSRESPONSE_MITIGATIONCONFIG"]._serialized_end = 17725
-    _globals["_FETCHERRORDETAILSRESPONSE_ERROR"]._serialized_start = 17728
-    _globals["_FETCHERRORDETAILSRESPONSE_ERROR"]._serialized_end = 18075
-    _globals["_CHECKPOINTCOMMANDRESULT"]._serialized_start = 18096
-    _globals["_CHECKPOINTCOMMANDRESULT"]._serialized_end = 18186
-    _globals["_CLONESESSIONREQUEST"]._serialized_start = 18189
-    _globals["_CLONESESSIONREQUEST"]._serialized_end = 18551
-    _globals["_CLONESESSIONRESPONSE"]._serialized_start = 18554
-    _globals["_CLONESESSIONRESPONSE"]._serialized_end = 18758
-    _globals["_GETSTATUSREQUEST"]._serialized_start = 18761
-    _globals["_GETSTATUSREQUEST"]._serialized_end = 19356
-    _globals["_GETSTATUSREQUEST_OPERATIONSTATUSREQUEST"]._serialized_start = 19160
-    _globals["_GETSTATUSREQUEST_OPERATIONSTATUSREQUEST"]._serialized_end = 19276
-    _globals["_GETSTATUSRESPONSE"]._serialized_start = 19359
-    _globals["_GETSTATUSRESPONSE"]._serialized_end = 20044
-    _globals["_GETSTATUSRESPONSE_OPERATIONSTATUS"]._serialized_start = 19617
-    _globals["_GETSTATUSRESPONSE_OPERATIONSTATUS"]._serialized_end = 20044
-    _globals["_GETSTATUSRESPONSE_OPERATIONSTATUS_OPERATIONSTATE"]._serialized_start = 19814
-    _globals["_GETSTATUSRESPONSE_OPERATIONSTATUS_OPERATIONSTATE"]._serialized_end = 20044
-    _globals["_SPARKCONNECTSERVICE"]._serialized_start = 20130
-    _globals["_SPARKCONNECTSERVICE"]._serialized_end = 21249
+    ]._serialized_end = 17717
+    _globals["_FETCHERRORDETAILSRESPONSE_BREAKINGCHANGEINFO"]._serialized_start = 17775
+    _globals["_FETCHERRORDETAILSRESPONSE_BREAKINGCHANGEINFO"]._serialized_end = 18025
+    _globals["_FETCHERRORDETAILSRESPONSE_MITIGATIONCONFIG"]._serialized_start = 18027
+    _globals["_FETCHERRORDETAILSRESPONSE_MITIGATIONCONFIG"]._serialized_end = 18085
+    _globals["_FETCHERRORDETAILSRESPONSE_ERROR"]._serialized_start = 18088
+    _globals["_FETCHERRORDETAILSRESPONSE_ERROR"]._serialized_end = 18435
+    _globals["_CHECKPOINTCOMMANDRESULT"]._serialized_start = 18456
+    _globals["_CHECKPOINTCOMMANDRESULT"]._serialized_end = 18546
+    _globals["_CLONESESSIONREQUEST"]._serialized_start = 18549
+    _globals["_CLONESESSIONREQUEST"]._serialized_end = 18911
+    _globals["_CLONESESSIONRESPONSE"]._serialized_start = 18914
+    _globals["_CLONESESSIONRESPONSE"]._serialized_end = 19118
+    _globals["_GETSTATUSREQUEST"]._serialized_start = 19121
+    _globals["_GETSTATUSREQUEST"]._serialized_end = 19716
+    _globals["_GETSTATUSREQUEST_OPERATIONSTATUSREQUEST"]._serialized_start = 19520
+    _globals["_GETSTATUSREQUEST_OPERATIONSTATUSREQUEST"]._serialized_end = 19636
+    _globals["_GETSTATUSRESPONSE"]._serialized_start = 19719
+    _globals["_GETSTATUSRESPONSE"]._serialized_end = 20404
+    _globals["_GETSTATUSRESPONSE_OPERATIONSTATUS"]._serialized_start = 19977
+    _globals["_GETSTATUSRESPONSE_OPERATIONSTATUS"]._serialized_end = 20404
+    _globals["_GETSTATUSRESPONSE_OPERATIONSTATUS_OPERATIONSTATE"]._serialized_start = 20174
+    _globals["_GETSTATUSRESPONSE_OPERATIONSTATUS_OPERATIONSTATE"]._serialized_end = 20404
+    _globals["_SPARKCONNECTSERVICE"]._serialized_start = 20490
+    _globals["_SPARKCONNECTSERVICE"]._serialized_end = 21609
 # @@protoc_insertion_point(module_scope)
diff --git a/python/pyspark/sql/connect/proto/base_pb2.pyi b/python/pyspark/sql/connect/proto/base_pb2.pyi
index 7750dc672654..9e746c564acc 100644
--- a/python/pyspark/sql/connect/proto/base_pb2.pyi
+++ b/python/pyspark/sql/connect/proto/base_pb2.pyi
@@ -896,14 +896,26 @@ class AnalyzePlanResponse(google.protobuf.message.Message):
         DESCRIPTOR: google.protobuf.descriptor.Descriptor
 
         VERSION_FIELD_NUMBER: builtins.int
+        CAPABILITIES_FIELD_NUMBER: builtins.int
         version: builtins.str
+        @property
+        def capabilities(
+            self,
+        ) -> google.protobuf.internal.containers.RepeatedScalarFieldContainer[builtins.str]:
+            """Capabilities supported by the server. Capability names are versioned so clients can
+            negotiate behavior without inferring it from the Spark version or a configuration value.
+            """
         def __init__(
             self,
             *,
             version: builtins.str = ...,
+            capabilities: collections.abc.Iterable[builtins.str] | None = ...,
         ) -> None: ...
         def ClearField(
-            self, field_name: typing_extensions.Literal["version", b"version"]
+            self,
+            field_name: typing_extensions.Literal[
+                "capabilities", b"capabilities", "version", b"version"
+            ],
         ) -> None: ...
 
     class DDLParse(google.protobuf.message.Message):
@@ -2488,26 +2500,87 @@ class AddArtifactsRequest(google.protobuf.message.Message):
             self, field_name: typing_extensions.Literal["data", b"data", "name", b"name"]
         ) -> None: ...
 
+    class MavenDependency(google.protobuf.message.Message):
+        """A Maven dependency that must be resolved by the server. The URI uses Spark's existing
+        ivy://group:module:version syntax.
+        """
+
+        DESCRIPTOR: google.protobuf.descriptor.Descriptor
+
+        URI_FIELD_NUMBER: builtins.int
+        uri: builtins.str
+        def __init__(
+            self,
+            *,
+            uri: builtins.str = ...,
+        ) -> None: ...
+        def ClearField(self, field_name: typing_extensions.Literal["uri", b"uri"]) -> None: ...
+
+    class ArtifactEntry(google.protobuf.message.Message):
+        """An ordered artifact batch entry used by clients that support server-side Maven resolution."""
+
+        DESCRIPTOR: google.protobuf.descriptor.Descriptor
+
+        ARTIFACT_FIELD_NUMBER: builtins.int
+        MAVEN_DEPENDENCY_FIELD_NUMBER: builtins.int
+        @property
+        def artifact(self) -> global___AddArtifactsRequest.SingleChunkArtifact: ...
+        @property
+        def maven_dependency(self) -> global___AddArtifactsRequest.MavenDependency: ...
+        def __init__(
+            self,
+            *,
+            artifact: global___AddArtifactsRequest.SingleChunkArtifact | None = ...,
+            maven_dependency: global___AddArtifactsRequest.MavenDependency | None = ...,
+        ) -> None: ...
+        def HasField(
+            self,
+            field_name: typing_extensions.Literal[
+                "artifact", b"artifact", "maven_dependency", b"maven_dependency", "value", b"value"
+            ],
+        ) -> builtins.bool: ...
+        def ClearField(
+            self,
+            field_name: typing_extensions.Literal[
+                "artifact", b"artifact", "maven_dependency", b"maven_dependency", "value", b"value"
+            ],
+        ) -> None: ...
+        def WhichOneof(
+            self, oneof_group: typing_extensions.Literal["value", b"value"]
+        ) -> typing_extensions.Literal["artifact", "maven_dependency"] | None: ...
+
     class Batch(google.protobuf.message.Message):
         """A number of `SingleChunkArtifact` batched into a single RPC."""
 
         DESCRIPTOR: google.protobuf.descriptor.Descriptor
 
         ARTIFACTS_FIELD_NUMBER: builtins.int
+        ENTRIES_FIELD_NUMBER: builtins.int
         @property
         def artifacts(
             self,
         ) -> google.protobuf.internal.containers.RepeatedCompositeFieldContainer[
             global___AddArtifactsRequest.SingleChunkArtifact
-        ]: ...
+        ]:
+            """Uploaded artifacts in batches that do not use ordered entries."""
+        @property
+        def entries(
+            self,
+        ) -> google.protobuf.internal.containers.RepeatedCompositeFieldContainer[
+            global___AddArtifactsRequest.ArtifactEntry
+        ]:
+            """Ordered mixture of uploaded artifacts and server-resolved Maven dependencies."""
         def __init__(
             self,
             *,
             artifacts: collections.abc.Iterable[global___AddArtifactsRequest.SingleChunkArtifact]
             | None = ...,
+            entries: collections.abc.Iterable[global___AddArtifactsRequest.ArtifactEntry]
+            | None = ...,
         ) -> None: ...
         def ClearField(
-            self, field_name: typing_extensions.Literal["artifacts", b"artifacts"]
+            self,
+            field_name: typing_extensions.Literal["artifacts", b"artifacts", "entries", b"entries"],
         ) -> None: ...
 
     class BeginChunkedArtifact(google.protobuf.message.Message):
diff --git a/sql/api/src/main/scala/org/apache/spark/sql/SparkSession.scala b/sql/api/src/main/scala/org/apache/spark/sql/SparkSession.scala
index d6444feffc0c..8aa554a879a1 100644
--- a/sql/api/src/main/scala/org/apache/spark/sql/SparkSession.scala
+++ b/sql/api/src/main/scala/org/apache/spark/sql/SparkSession.scala
@@ -561,7 +561,12 @@ abstract class SparkSession extends Serializable with Closeable {
   /**
    * Add a single artifact to the current session.
    *
-   * Currently only local files with extensions .jar and .class are supported.
+   * Currently it supports local files with extensions .jar and .class and Apache Ivy URIs. Ivy
+   * URIs are resolved using `spark.jars.ivySettings` when it is configured. When using Spark
+   * Connect, Ivy URIs with repositories in the `repos` query parameter, or sent to servers that
+   * do not support server-side Maven resolution, are resolved by the client without the server's
+   * Ivy settings. Otherwise, Ivy URIs are resolved by the server, which does not search Maven or
+   * Ivy repositories local to the client.
    *
    * @since 4.0.0
    */
@@ -571,7 +576,12 @@ abstract class SparkSession extends Serializable with Closeable {
   /**
    * Add a single artifact to the current session.
    *
-   * Currently it supports local files with extensions .jar and .class and Apache Ivy URIs.
+   * Currently it supports local files with extensions .jar and .class and Apache Ivy URIs. Ivy
+   * URIs are resolved using `spark.jars.ivySettings` when it is configured. When using Spark
+   * Connect, Ivy URIs with repositories in the `repos` query parameter, or sent to servers that
+   * do not support server-side Maven resolution, are resolved by the client without the server's
+   * Ivy settings. Otherwise, Ivy URIs are resolved by the server, which does not search Maven or
+   * Ivy repositories local to the client.
    *
    * @since 4.0.0
    */
@@ -622,7 +632,12 @@ abstract class SparkSession extends Serializable with Closeable {
   /**
    * Add one or more artifacts to the session.
    *
-   * Currently it supports local files with extensions .jar and .class and Apache Ivy URIs
+   * Currently it supports local files with extensions .jar and .class and Apache Ivy URIs. Ivy
+   * URIs are resolved using `spark.jars.ivySettings` when it is configured. When using Spark
+   * Connect, Ivy URIs with repositories in the `repos` query parameter, or sent to servers that
+   * do not support server-side Maven resolution, are resolved by the client without the server's
+   * Ivy settings. Otherwise, Ivy URIs are resolved by the server, which does not search Maven or
+   * Ivy repositories local to the client.
    *
    * @since 4.0.0
    */
diff --git a/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/ArtifactSuite.scala b/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/ArtifactSuite.scala
index bbe3fd16a969..9deb57a1d8c5 100644
--- a/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/ArtifactSuite.scala
+++ b/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/ArtifactSuite.scala
@@ -23,8 +23,10 @@ import java.util.concurrent.TimeUnit
 import java.util.jar.{JarEntry, JarOutputStream}
 import java.util.zip.CRC32
 
+import scala.jdk.CollectionConverters._
+
 import com.google.protobuf.ByteString
-import io.grpc.{ManagedChannel, Server}
+import io.grpc.{ManagedChannel, Server, Status}
 import io.grpc.inprocess.{InProcessChannelBuilder, InProcessServerBuilder}
 
 import org.apache.spark.connect.proto.AddArtifactsRequest
@@ -255,6 +257,112 @@ class ArtifactSuite extends ConnectFunSuite {
     assertFileDataEquality(artifacts.get(1).getData, Paths.get(file2))
   }
 
+  test("server-side Maven dependencies preserve artifact order") {
+    val classFile = artifactFilePath.resolve("smallClassFile.class").toUri
+    val ivyUri = URI.create("ivy://my.artifactsuite.lib:mylib:0.1")
+    val jarFile = artifactFilePath.resolve("smallJar.jar").toUri
+
+    artifactManager.addArtifacts(Seq(classFile, ivyUri, jarFile), serverSideMavenArtifacts = true)
+
+    val requests = service.getAndClearLatestAddArtifactRequests()
+    assert(requests.size == 1)
+    val batch = requests.head.getBatch
+    assert(batch.getArtifactsCount == 0)
+    assert(batch.getEntriesCount == 3)
+    assert(batch.getEntries(0).getArtifact.getName == "classes/smallClassFile.class")
+    assert(batch.getEntries(1).getMavenDependency.getUri == ivyUri.toString)
+    assert(batch.getEntries(2).getArtifact.getName == "jars/smallJar.jar")
+  }
+
+  test("server-side Maven mode resolves requested repositories on the client") {
+    val main = new MavenCoordinate("my.artifactsuite.client", "mylib", "0.1")
+    IvyTestUtils.withRepository(main, None, None) { repo =>
+      val classFile = artifactFilePath.resolve("smallClassFile.class").toUri
+      val clientIvyUri = URI.create(s"ivy://${main.toString}?repos=$repo")
+      val serverIvyUri = URI.create("ivy://my.artifactsuite.server:mylib:0.1")
+
+      artifactManager.addArtifacts(
+        Seq(classFile, clientIvyUri, serverIvyUri),
+        serverSideMavenArtifacts = true)
+
+      val requests = service.getAndClearLatestAddArtifactRequests()
+      assert(requests.size == 1)
+      val batch = requests.head.getBatch
+      assert(batch.getEntriesCount == 3)
+      assert(batch.getEntries(0).getArtifact.getName == "classes/smallClassFile.class")
+      assert(
+        batch.getEntries(1).getArtifact.getName.contains("my.artifactsuite.client_mylib-0.1"))
+      assert(batch.getEntries(2).getMavenDependency.getUri == serverIvyUri.toString)
+    }
+  }
+
+  test("Spark Connect uses server-side Maven resolution when advertised") {
+    service.serverCapabilities = Seq(SparkConnectClient.SERVER_SIDE_MAVEN_ARTIFACTS_CAPABILITY)
+    client = new SparkConnectClient(Configuration(), channel)
+    val ivyUri = URI.create("ivy://my.artifactsuite.lib:mylib:0.1")
+
+    client.addArtifact(ivyUri)
+
+    val requests = service.getAndClearLatestAddArtifactRequests()
+    assert(requests.size == 1)
+    assert(requests.head.getBatch.getEntriesCount == 1)
+    assert(requests.head.getBatch.getEntries(0).getMavenDependency.getUri == ivyUri.toString)
+  }
+
+  test("Spark Connect resolves requested Maven repositories on the client") {
+    val main = new MavenCoordinate("my.artifactsuite.clientcapability", "mylib", "0.1")
+    IvyTestUtils.withRepository(main, None, None) { repo =>
+      service.errorToThrowOnAnalyze = Some(Status.UNAVAILABLE.asRuntimeException())
+      client = new SparkConnectClient(Configuration(), channel)
+
+      client.addArtifact(URI.create(s"ivy://${main.toString}?repos=$repo"))
+
+      assert(service.errorToThrowOnAnalyze.nonEmpty)
+      val requests = service.getAndClearLatestAddArtifactRequests()
+      assert(requests.size == 1)
+      val batch = requests.head.getBatch
+      assert(batch.getArtifactsCount == 1)
+      assert(
+        batch.getArtifacts(0).getName.contains("my.artifactsuite.clientcapability_mylib-0.1"))
+    }
+  }
+
+  test("large Maven entries do not produce empty batches") {
+    val largeUri = URI.create(s"ivy://org.example:${"a" * CHUNK_SIZE}:1.0")
+    val otherUri = URI.create("ivy://org.example:other:1.0")
+
+    artifactManager.addArtifacts(Seq(largeUri, otherUri), serverSideMavenArtifacts = true)
+
+    val requests = service.getAndClearLatestAddArtifactRequests()
+    assert(requests.size == 2)
+    assert(requests.forall(_.getBatch.getEntriesCount == 1))
+    assert(requests.head.getBatch.getEntries(0).getMavenDependency.getUri == largeUri.toString)
+    assert(requests(1).getBatch.getEntries(0).getMavenDependency.getUri == otherUri.toString)
+  }
+
+  test("Spark Connect falls back only when the Maven capability is absent") {
+    val main = new MavenCoordinate("my.artifactsuite.fallback", "mylib", "0.1")
+    IvyTestUtils.withRepository(main, None, None) { repo =>
+      client = new SparkConnectClient(Configuration(), channel)
+
+      client.addArtifact(URI.create(s"ivy://${main.toString}?repos=$repo"))
+
+      val requests = service.getAndClearLatestAddArtifactRequests()
+      assert(requests.nonEmpty)
+      assert(requests.forall(_.getBatch.getEntriesCount == 0))
+      assert(requests.flatMap(_.getBatch.getArtifactsList.asScala).exists { artifact =>
+        artifact.getName.contains("my.artifactsuite.fallback_mylib-0.1")
+      })
+    }
+
+    service.errorToThrowOnAnalyze = Some(Status.UNAVAILABLE.asRuntimeException())
+    client = new SparkConnectClient(Configuration(retryPolicies = Nil), channel)
+    intercept[Exception] {
+      client.addArtifact(URI.create("ivy://my.artifactsuite.fallback:mylib:0.1"))
+    }
+    assert(service.getAndClearLatestAddArtifactRequests().isEmpty)
+  }
+
   test("Mix of SingleChunkArtifact and chunked artifact") {
     val path1 = artifactFilePath.resolve("smallClassFile.class")
     val file1 = path1.toUri
diff --git a/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/SparkConnectClientSuite.scala b/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/SparkConnectClientSuite.scala
index 6a456a554c5c..b37a0d08d8d6 100644
--- a/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/SparkConnectClientSuite.scala
+++ b/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/SparkConnectClientSuite.scala
@@ -1428,6 +1428,8 @@ class DummySparkConnectService() extends SparkConnectServiceGrpc.SparkConnectSer
   private val sparkConfigs = mutable.Map.empty[String, String]
 
   var errorToThrowOnExecute: Option[Throwable] = None
+  var errorToThrowOnAnalyze: Option[Throwable] = None
+  var serverCapabilities: Seq[String] = Nil
 
   private var errorToThrowOnConfig: Map[String, Throwable] = Map.empty
 
@@ -1501,6 +1503,11 @@ class DummySparkConnectService() extends SparkConnectServiceGrpc.SparkConnectSer
   override def analyzePlan(
       request: AnalyzePlanRequest,
       responseObserver: StreamObserver[AnalyzePlanResponse]): Unit = {
+    errorToThrowOnAnalyze.foreach { error =>
+      errorToThrowOnAnalyze = None
+      responseObserver.onError(error)
+      return
+    }
     // Reply with a dummy response using the same client ID
     val requestSessionId = request.getSessionId
     synchronized {
@@ -1520,11 +1527,17 @@ class DummySparkConnectService() extends SparkConnectServiceGrpc.SparkConnectSer
         case _ => inputPlan = null
       }
     }
-    val response = AnalyzePlanResponse
+    val responseBuilder = AnalyzePlanResponse
       .newBuilder()
       .setSessionId(requestSessionId)
-      .build()
-    responseObserver.onNext(response)
+    if (request.hasSparkVersion) {
+      responseBuilder.setSparkVersion(
+        proto.AnalyzePlanResponse.SparkVersion
+          .newBuilder()
+          .setVersion("test-version")
+          .addAllCapabilities(serverCapabilities.asJava))
+    }
+    responseObserver.onNext(responseBuilder.build())
     responseObserver.onCompleted()
   }
 
diff --git a/sql/connect/common/src/main/protobuf/spark/connect/base.proto b/sql/connect/common/src/main/protobuf/spark/connect/base.proto
index be4ab641c7b5..86bc96386564 100644
--- a/sql/connect/common/src/main/protobuf/spark/connect/base.proto
+++ b/sql/connect/common/src/main/protobuf/spark/connect/base.proto
@@ -283,6 +283,9 @@ message AnalyzePlanResponse {
 
   message SparkVersion {
     string version = 1;
+    // Capabilities supported by the server. Capability names are versioned so clients can
+    // negotiate behavior without inferring it from the Spark version or a configuration value.
+    repeated string capabilities = 2;
   }
 
   message DDLParse {
@@ -679,9 +682,26 @@ message AddArtifactsRequest {
     ArtifactChunk data = 2;
   }
 
+  // A Maven dependency that must be resolved by the server. The URI uses Spark's existing
+  // ivy://group:module:version syntax.
+  message MavenDependency {
+    string uri = 1;
+  }
+
+  // An ordered artifact batch entry used by clients that support server-side Maven resolution.
+  message ArtifactEntry {
+    oneof value {
+      SingleChunkArtifact artifact = 1;
+      MavenDependency maven_dependency = 2;
+    }
+  }
+
   // A number of `SingleChunkArtifact` batched into a single RPC.
   message Batch {
+    // Uploaded artifacts in batches that do not use ordered entries.
     repeated SingleChunkArtifact artifacts = 1;
+    // Ordered mixture of uploaded artifacts and server-resolved Maven dependencies.
+    repeated ArtifactEntry entries = 2;
   }
 
   // Signals the beginning/start of a chunked artifact.
diff --git a/sql/connect/common/src/main/scala/org/apache/spark/sql/connect/client/ArtifactManager.scala b/sql/connect/common/src/main/scala/org/apache/spark/sql/connect/client/ArtifactManager.scala
index 1cfc9f170e89..08bc4f2a8741 100644
--- a/sql/connect/common/src/main/scala/org/apache/spark/sql/connect/client/ArtifactManager.scala
+++ b/sql/connect/common/src/main/scala/org/apache/spark/sql/connect/client/ArtifactManager.scala
@@ -40,7 +40,7 @@ import org.apache.spark.connect.proto.AddArtifactsResponse.ArtifactSummary
 import org.apache.spark.network.util.JavaUtils.sha256Hex
 import org.apache.spark.sql.Artifact
 import org.apache.spark.sql.Artifact.{newCacheArtifact, newIvyArtifacts}
-import org.apache.spark.util.{SparkFileUtils, SparkStringUtils, SparkThreadUtils}
+import org.apache.spark.util.{MavenUtils, SparkFileUtils, SparkStringUtils, SparkThreadUtils}
 
 /**
  * The Artifact Manager is responsible for handling and transferring artifacts from the local
@@ -66,6 +66,22 @@ class ArtifactManager(
   private[this] val classFinders = new CopyOnWriteArrayList[ClassFinder]
   private[this] val stubState = stub.stubState
 
+  private sealed trait ArtifactEntry {
+    def size: Long
+  }
+  private case class UploadedArtifact(artifact: Artifact) extends ArtifactEntry {
+    override def size: Long = artifact.size
+  }
+  private case class MavenDependency(uri: URI) extends ArtifactEntry {
+    override lazy val size: Long = proto.AddArtifactsRequest.ArtifactEntry
+      .newBuilder()
+      .setMavenDependency(
+        proto.AddArtifactsRequest.MavenDependency.newBuilder().setUri(uri.toString))
+      .build()
+      .getSerializedSize
+      .toLong
+  }
+
   /**
    * Register a [[ClassFinder]] for dynamically generated classes.
    */
@@ -99,12 +115,23 @@ class ArtifactManager(
     }
   }
 
+  private def hasRequestedRepositories(uri: URI): Boolean = {
+    val (_, _, repositories) = MavenUtils.parseQueryParams(uri)
+    repositories.split(",").exists(_.trim.nonEmpty)
+  }
+
+  private[client] def isServerSideMavenCandidate(uri: URI): Boolean =
+    uri.getScheme == "ivy" && !hasRequestedRepositories(uri)
+
   /**
    * Add a single artifact to the session.
    *
    * Currently it supports local files with extensions .jar and .class and Apache Ivy URIs
    */
-  def addArtifact(uri: URI): Unit = addArtifacts(parseArtifacts(uri))
+  def addArtifact(uri: URI): Unit = addArtifact(uri, serverSideMavenArtifacts = false)
+
+  private[client] def addArtifact(uri: URI, serverSideMavenArtifacts: Boolean): Unit =
+    addArtifacts(Seq(uri), serverSideMavenArtifacts)
 
   /**
    * Add a single in-memory artifact to the session while preserving the directory structure
@@ -160,7 +187,24 @@ class ArtifactManager(
    *
    * Currently it supports local files with extensions .jar and .class and Apache Ivy URIs
    */
-  def addArtifacts(uris: Seq[URI]): Unit = addArtifacts(uris.flatMap(parseArtifacts))
+  def addArtifacts(uris: Seq[URI]): Unit =
+    addArtifacts(uris, serverSideMavenArtifacts = false)
+
+  private[client] def addArtifacts(uris: Seq[URI], serverSideMavenArtifacts: Boolean): Unit = {
+    if (serverSideMavenArtifacts) {
+      val entries = uris.flatMap { uri =>
+        uri.getScheme match {
+          // Resolve explicit repositories on the client so arbitrary repositories are never
+          // accessed from the server.
+          case "ivy" if !hasRequestedRepositories(uri) => MavenDependency(uri) :: Nil
+          case _ => parseArtifacts(uri).map(UploadedArtifact)
+        }
+      }
+      addArtifactEntries(entries, useOrderedEntries = true)
+    } else {
+      addArtifacts(uris.flatMap(parseArtifacts))
+    }
+  }
 
   private[client] def isCachedArtifact(hash: String): Boolean = {
     val artifactName = s"${Artifact.CACHE_PREFIX}/$hash"
@@ -293,13 +337,19 @@ class ArtifactManager(
    * Add a number of artifacts to the session.
    */
   private[client] def addArtifacts(artifacts: Iterable[Artifact]): Unit = {
-    if (artifacts.isEmpty) {
+    addArtifactEntries(artifacts.map(UploadedArtifact), useOrderedEntries = false)
+  }
+
+  private def addArtifactEntries(
+      entries: Iterable[ArtifactEntry],
+      useOrderedEntries: Boolean): Unit = {
+    if (entries.isEmpty) {
       return
     }
 
     try {
       stubState.retryHandler.retry {
-        addArtifactsImpl(artifacts)
+        addArtifactsImpl(entries, useOrderedEntries)
       }
     } catch {
       case ex: StatusRuntimeException =>
@@ -307,7 +357,9 @@ class ArtifactManager(
     }
   }
 
-  private[client] def addArtifactsImpl(artifacts: Iterable[Artifact]): Unit = {
+  private def addArtifactsImpl(
+      entries: Iterable[ArtifactEntry],
+      useOrderedEntries: Boolean): Unit = {
     val promise = Promise[Seq[ArtifactSummary]]()
     val responseHandler = new StreamObserver[proto.AddArtifactsResponse] {
       private val summaries = mutable.Buffer.empty[ArtifactSummary]
@@ -329,36 +381,33 @@ class ArtifactManager(
       }
     }
     val stream = stub.addArtifacts(responseHandler)
-    val currentBatch = mutable.Buffer.empty[Artifact]
+    val currentBatch = mutable.Buffer.empty[ArtifactEntry]
     var currentBatchSize = 0L
 
-    def addToBatch(dep: Artifact, size: Long): Unit = {
+    def addToBatch(dep: ArtifactEntry): Unit = {
       currentBatch += dep
-      currentBatchSize += size
+      currentBatchSize += dep.size
     }
 
     def writeBatch(): Unit = {
-      addBatchedArtifacts(currentBatch.toSeq, stream)
+      addBatchedArtifacts(currentBatch.toSeq, stream, useOrderedEntries)
       currentBatch.clear()
       currentBatchSize = 0
     }
 
-    artifacts.iterator.foreach { artifact =>
-      val data = artifact.storage
-      val size = data.size
-      if (size > CHUNK_SIZE) {
+    entries.iterator.foreach {
+      case UploadedArtifact(artifact) if artifact.size > CHUNK_SIZE =>
         // Payload can either be a batch OR a single chunked artifact. Write batch if non-empty
         // before chunking current artifact.
         if (currentBatch.nonEmpty) {
           writeBatch()
         }
         addChunkedArtifact(artifact, stream)
-      } else {
-        if (currentBatchSize + size > CHUNK_SIZE) {
+      case entry =>
+        if (currentBatch.nonEmpty && currentBatchSize + entry.size > CHUNK_SIZE) {
           writeBatch()
         }
-        addToBatch(artifact, size)
-      }
+        addToBatch(entry)
     }
     if (currentBatch.nonEmpty) {
       writeBatch()
@@ -376,33 +425,46 @@ class ArtifactManager(
    * single [[proto.AddArtifactsRequest]].
    */
   private def addBatchedArtifacts(
-      artifacts: Seq[Artifact],
-      stream: StreamObserver[proto.AddArtifactsRequest]): Unit = {
+      entries: Seq[ArtifactEntry],
+      stream: StreamObserver[proto.AddArtifactsRequest],
+      useOrderedEntries: Boolean): Unit = {
     val builder = proto.AddArtifactsRequest
       .newBuilder()
       .setUserContext(clientConfig.userContext)
       .setClientType(clientConfig.userAgent)
       .setSessionId(sessionId)
-    artifacts.foreach { artifact =>
-      val in = new CheckedInputStream(artifact.storage.stream, new CRC32)
-      try {
-        val data = proto.AddArtifactsRequest.ArtifactChunk
-          .newBuilder()
-          .setData(ByteString.readFrom(in))
-          .setCrc(in.getChecksum.getValue)
-
+    entries.foreach {
+      case UploadedArtifact(artifact) =>
+        val in = new CheckedInputStream(artifact.storage.stream, new CRC32)
+        try {
+          val data = proto.AddArtifactsRequest.ArtifactChunk
+            .newBuilder()
+            .setData(ByteString.readFrom(in))
+            .setCrc(in.getChecksum.getValue)
+          val uploaded = proto.AddArtifactsRequest.SingleChunkArtifact
+            .newBuilder()
+            .setName(artifact.path.toString)
+            .setData(data)
+          if (useOrderedEntries) {
+            builder.getBatchBuilder.addEntriesBuilder().setArtifact(uploaded)
+          } else {
+            builder.getBatchBuilder.addArtifacts(uploaded)
+          }
+          ()
+        } catch {
+          case NonFatal(e) =>
+            stream.onError(e)
+            throw e
+        } finally {
+          in.close()
+        }
+      case MavenDependency(uri) =>
+        require(useOrderedEntries)
         builder.getBatchBuilder
-          .addArtifactsBuilder()
-          .setName(artifact.path.toString)
-          .setData(data)
-          .build()
-      } catch {
-        case NonFatal(e) =>
-          stream.onError(e)
-          throw e
-      } finally {
-        in.close()
-      }
+          .addEntriesBuilder()
+          .getMavenDependencyBuilder
+          .setUri(uri.toString)
+        ()
     }
     stream.onNext(builder.build())
   }
diff --git a/sql/connect/common/src/main/scala/org/apache/spark/sql/connect/client/SparkConnectClient.scala b/sql/connect/common/src/main/scala/org/apache/spark/sql/connect/client/SparkConnectClient.scala
index 5a3bd0b892c5..e60f4a83b694 100644
--- a/sql/connect/common/src/main/scala/org/apache/spark/sql/connect/client/SparkConnectClient.scala
+++ b/sql/connect/common/src/main/scala/org/apache/spark/sql/connect/client/SparkConnectClient.scala
@@ -43,7 +43,7 @@ import org.apache.spark.sql.connect.RuntimeConfig
 import org.apache.spark.sql.connect.common.ProtoUtils
 import org.apache.spark.sql.connect.common.config.ConnectCommon
 import org.apache.spark.sql.util.CloseableIterator
-import org.apache.spark.util.SparkSystemUtils
+import org.apache.spark.util.{SparkFileUtils, SparkSystemUtils}
 
 /**
  * Conceptually the remote spark session that communicates with the server.
@@ -85,6 +85,25 @@ private[sql] class SparkConnectClient(
 
   private val conf: RuntimeConfig = new RuntimeConfig(this)
 
+  private lazy val serverCapabilities: Set[String] = {
+    val builder = proto.AnalyzePlanRequest
+      .newBuilder()
+      .setSparkVersion(proto.AnalyzePlanRequest.SparkVersion.newBuilder().build())
+      .setUserContext(userContext)
+      .setSessionId(sessionId)
+      .setClientType(userAgent)
+    serverSideSessionId.foreach(builder.setClientObservedServerSideSessionId)
+    bstub
+      .analyzePlan(builder.build())
+      .getSparkVersion
+      .getCapabilitiesList
+      .asScala
+      .toSet
+  }
+
+  private def supportsServerSideMavenArtifacts: Boolean =
+    serverCapabilities.contains(SparkConnectClient.SERVER_SIDE_MAVEN_ARTIFACTS_CAPABILITY)
+
   // Cached plan compression options.
   private var _planCompressionOptions: Option[Option[PlanCompressionOptions]] = None
 
@@ -585,16 +604,20 @@ private[sql] class SparkConnectClient(
   /**
    * Add a single artifact to the client session.
    *
-   * Currently only local files with extensions .jar and .class are supported.
+   * Supports local .jar and .class files and Apache Ivy URIs.
    */
-  def addArtifact(path: String): Unit = artifactManager.addArtifact(path)
+  def addArtifact(path: String): Unit = addArtifact(SparkFileUtils.resolveURI(path))
 
   /**
    * Add a single artifact to the client session.
    *
-   * Currently only local files with extensions .jar and .class are supported.
+   * Supports local .jar and .class files and Apache Ivy URIs.
    */
-  def addArtifact(uri: URI): Unit = artifactManager.addArtifact(uri)
+  def addArtifact(uri: URI): Unit = {
+    val serverSide = artifactManager.isServerSideMavenCandidate(uri) &&
+      supportsServerSideMavenArtifacts
+    artifactManager.addArtifact(uri, serverSide)
+  }
 
   /**
    * Add a single in-memory artifact to the session while preserving the directory structure
@@ -636,9 +659,13 @@ private[sql] class SparkConnectClient(
   /**
    * Add multiple artifacts to the session.
    *
-   * Currently only local files with extensions .jar and .class are supported.
+   * Supports local .jar and .class files and Apache Ivy URIs.
    */
-  def addArtifacts(uri: Seq[URI]): Unit = artifactManager.addArtifacts(uri)
+  def addArtifacts(uri: Seq[URI]): Unit = {
+    val serverSide = uri.exists(artifactManager.isServerSideMavenCandidate) &&
+      supportsServerSideMavenArtifacts
+    artifactManager.addArtifacts(uri, serverSide)
+  }
 
   /**
    * Register a [[ClassFinder]] for dynamically generated classes.
@@ -710,6 +737,8 @@ private final class SparkConnectOperationIdException(val operationId: String)
     extends RuntimeException(s"Spark Connect operation ID: $operationId", null, false, false)
 
 object SparkConnectClient {
+  private[connect] val SERVER_SIDE_MAVEN_ARTIFACTS_CAPABILITY =
+    "serverSideMavenArtifacts.v1"
 
   private[connect] val OPERATION_ID_HEADER = "spark-connect-operation-id"
 
diff --git a/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectAddArtifactsHandler.scala b/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectAddArtifactsHandler.scala
index 78970b747956..91180a17950f 100644
--- a/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectAddArtifactsHandler.scala
+++ b/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectAddArtifactsHandler.scala
@@ -17,21 +17,26 @@
 package org.apache.spark.sql.connect.service
 
 import java.io.File
+import java.net.{URI, URISyntaxException}
 import java.nio.file.{Files, Path, Paths}
+import java.util.concurrent.TimeUnit
 import java.util.zip.{CheckedOutputStream, CRC32}
 
 import scala.collection.mutable
 import scala.util.control.NonFatal
 
+import io.grpc.Context
 import io.grpc.stub.StreamObserver
 
-import org.apache.spark.SparkRuntimeException
+import org.apache.spark.{SparkException, SparkRuntimeException}
 import org.apache.spark.connect.proto
 import org.apache.spark.connect.proto.{AddArtifactsRequest, AddArtifactsResponse}
 import org.apache.spark.connect.proto.AddArtifactsResponse.ArtifactSummary
+import org.apache.spark.sql.Artifact
 import org.apache.spark.sql.artifact.ArtifactManager
 import org.apache.spark.sql.connect.utils.ErrorUtils
 import org.apache.spark.sql.util.ArtifactUtils
+import org.apache.spark.util.RuntimeDependencyResolver.RejectRequestedRepositories
 import org.apache.spark.util.Utils
 
 /**
@@ -44,8 +49,11 @@ class SparkConnectAddArtifactsHandler(val responseObserver: StreamObserver[AddAr
 
   // Temporary directory where artifacts are rebuilt from the bytes sent over the wire.
   protected val stagingDir: Path = Utils.createTempDir().toPath
-  protected val stagedArtifacts: mutable.Buffer[StagedArtifact] =
-    mutable.Buffer.empty[StagedArtifact]
+  private sealed trait PendingArtifact
+  private case class PendingStagedArtifact(artifact: StagedArtifact) extends PendingArtifact
+  private case class PendingMavenDependency(uri: URI) extends PendingArtifact
+  private val pendingArtifacts = mutable.Buffer.empty[PendingArtifact]
+  private val grpcContext = Context.current()
   // If not null, indicates the currently active chunked artifact that is being rebuilt from
   // several [[AddArtifactsRequest]]s.
   private var chunkedArtifact: StagedChunkedArtifact = _
@@ -78,8 +86,22 @@ class SparkConnectAddArtifactsHandler(val responseObserver: StreamObserver[AddAr
         chunkedArtifact = null
       }
     } else if (req.hasBatch) {
-      // Each artifact in the batch is single-chunked.
-      req.getBatch.getArtifactsList.forEach(artifact => writeArtifactToFile(artifact).close())
+      val batch = req.getBatch
+      if (batch.getArtifactsCount > 0 && batch.getEntriesCount > 0) {
+        throw SparkException.internalError(
+          "AddArtifacts batch cannot contain both legacy artifacts and ordered entries")
+      }
+      batch.getArtifactsList.forEach(artifact => writeArtifactToFile(artifact).close())
+      batch.getEntriesList.forEach { entry =>
+        entry.getValueCase match {
+          case proto.AddArtifactsRequest.ArtifactEntry.ValueCase.ARTIFACT =>
+            writeArtifactToFile(entry.getArtifact).close()
+          case proto.AddArtifactsRequest.ArtifactEntry.ValueCase.MAVEN_DEPENDENCY =>
+            addMavenDependency(entry.getMavenDependency)
+          case _ =>
+            throw SparkException.internalError("AddArtifacts ordered entry has no value")
+        }
+      }
     } else {
       throw new UnsupportedOperationException(s"Unsupported data transfer request: $req")
     }
@@ -101,11 +123,87 @@ class SparkConnectAddArtifactsHandler(val responseObserver: StreamObserver[AddAr
     responseObserver.onError(throwable)
   }
 
+  private def addMavenDependency(dependency: proto.AddArtifactsRequest.MavenDependency): Unit = {
+    val value = dependency.getUri
+    val uri =
+      try new URI(value)
+      catch {
+        case _: URISyntaxException => null
+      }
+    if (uri == null || uri.getScheme != "ivy") {
+      throw SparkException.internalError(s"Maven dependency must be an ivy URI: $value")
+    }
+    pendingArtifacts += PendingMavenDependency(uri)
+  }
+
   protected def addStagedArtifactToArtifactManager(artifact: StagedArtifact): Unit = {
     require(holder != null)
     holder.addArtifact(artifact.path, artifact.stagedPath, artifact.fragment)
   }
 
+  protected def resolveMavenDependency(
+      uri: URI,
+      connectTimeoutMs: Int,
+      readTimeoutMs: Int,
+      isCancelled: () => Boolean): Seq[Artifact] = {
+    holder.artifactManager.resolveArtifacts(
+      uri,
+      RejectRequestedRepositories,
+      connectTimeoutMs,
+      readTimeoutMs,
+      isCancelled)
+  }
+
+  private def resolveMavenDependencies(dependencies: Seq[URI]): Map[URI, Seq[Artifact]] = {
+    dependencies.map { uri =>
+      val timeoutCapMs = Option(grpcContext.getDeadline)
+        .map(_.timeRemaining(TimeUnit.MILLISECONDS))
+        .map(math.min(_, Int.MaxValue.toLong))
+        .getOrElse(Int.MaxValue.toLong)
+        .toInt
+      if (timeoutCapMs <= 0) {
+        throw SparkException.internalError(
+          "AddArtifacts deadline expired before Maven resolution")
+      }
+      val connectTimeoutMs = math.min(timeoutCapMs, holder.artifactManager.ivyConnectTimeoutMs)
+      val readTimeoutMs = math.min(timeoutCapMs, holder.artifactManager.ivyReadTimeoutMs)
+      uri -> resolveMavenDependency(
+        uri,
+        connectTimeoutMs,
+        readTimeoutMs,
+        () => grpcContext.isCancelled || Thread.currentThread().isInterrupted)
+    }.toMap
+  }
+
+  private def stageResolvedArtifact(artifact: Artifact): StagedArtifact = {
+    val localFile = artifact.storage match {
+      case file: Artifact.LocalFile => file.path
+      case other =>
+        throw SparkException.internalError(
+          s"Resolved Maven artifact has unexpected storage: ${other.getClass.getName}")
+    }
+    val physicalPath = Files.createTempFile(stagingDir, "resolved-maven-", ".jar")
+    val staged = new StagedArtifact(artifact.path.toString, Some(physicalPath))
+    Utils.tryWithSafeFinally {
+      staged.writeFrom(localFile)
+      staged
+    }(staged.close())
+  }
+
+  private def prepareArtifacts(): Seq[StagedArtifact] = {
+    val dependencies = pendingArtifacts
+      .collect { case PendingMavenDependency(uri) =>
+        uri
+      }
+      .distinct
+      .toSeq
+    val resolved = resolveMavenDependencies(dependencies)
+    pendingArtifacts.flatMap {
+      case PendingStagedArtifact(artifact) => Seq(artifact)
+      case PendingMavenDependency(uri) => resolved(uri).map(stageResolvedArtifact)
+    }.toSeq
+  }
+
   /**
    * Process all the staged artifacts built in this stream.
    *
@@ -114,8 +212,8 @@ class SparkConnectAddArtifactsHandler(val responseObserver: StreamObserver[AddAr
   protected def flushStagedArtifacts(): Seq[ArtifactSummary] = {
     val failedArtifactExceptions = mutable.ListBuffer[SparkRuntimeException]()
 
-    // Non-lazy transformation when using Buffer.
-    val summaries = stagedArtifacts.map { artifact =>
+    // Resolve the complete ordered batch before mutating session state.
+    val summaries = prepareArtifacts().map { artifact =>
       try {
         // We do not store artifacts that fail the CRC. The failure is reported in the artifact
         // summary and it is up to the client to decide whether to retry sending the artifact.
@@ -176,7 +274,7 @@ class SparkConnectAddArtifactsHandler(val responseObserver: StreamObserver[AddAr
   private def writeArtifactToFile(
       artifact: proto.AddArtifactsRequest.SingleChunkArtifact): StagedArtifact = {
     val stagedDep = new StagedArtifact(artifact.getName)
-    stagedArtifacts += stagedDep
+    pendingArtifacts += PendingStagedArtifact(stagedDep)
     stagedDep.write(artifact.getData)
     stagedDep
   }
@@ -189,7 +287,7 @@ class SparkConnectAddArtifactsHandler(val responseObserver: StreamObserver[AddAr
       artifact: proto.AddArtifactsRequest.BeginChunkedArtifact): StagedChunkedArtifact = {
     val stagedChunkedArtifact =
       new StagedChunkedArtifact(artifact.getName, artifact.getNumChunks, artifact.getTotalBytes)
-    stagedArtifacts += stagedChunkedArtifact
+    pendingArtifacts += PendingStagedArtifact(stagedChunkedArtifact)
     stagedChunkedArtifact.write(artifact.getInitialChunk)
     stagedChunkedArtifact
   }
@@ -197,7 +295,7 @@ class SparkConnectAddArtifactsHandler(val responseObserver: StreamObserver[AddAr
   /**
    * Handles rebuilding an artifact from bytes sent over the wire.
    */
-  class StagedArtifact(val name: String) {
+  class StagedArtifact(val name: String, physicalPath: Option[Path] = None) {
     // Workaround to keep the fragment.
     val (canonicalFileName: String, fragment: Option[String]) =
       if (name.startsWith(s"archives${File.separator}")) {
@@ -213,16 +311,28 @@ class SparkConnectAddArtifactsHandler(val responseObserver: StreamObserver[AddAr
       }
 
     val path: Path = Paths.get(canonicalFileName)
-    val stagedPath: Path =
-      try {
-        ArtifactUtils.concatenatePaths(stagingDir, path)
-      } catch {
-        case _: IllegalArgumentException =>
-          throw new SparkRuntimeException(
-            errorClass = "INVALID_ARTIFACT_PATH",
-            messageParameters = Map("name" -> name))
-        case NonFatal(e) => throw e
+    if (path.isAbsolute) {
+      throw new SparkRuntimeException(
+        errorClass = "INVALID_ARTIFACT_PATH",
+        messageParameters = Map("name" -> name))
+    }
+    val stagedPath: Path = physicalPath.getOrElse {
+      val requestedPath =
+        try {
+          ArtifactUtils.concatenatePaths(stagingDir, path)
+        } catch {
+          case _: IllegalArgumentException =>
+            throw new SparkRuntimeException(
+              errorClass = "INVALID_ARTIFACT_PATH",
+              messageParameters = Map("name" -> name))
+          case NonFatal(e) => throw e
+        }
+      if (Files.exists(requestedPath)) {
+        Files.createTempFile(stagingDir, "duplicate-artifact-", ".tmp")
+      } else {
+        requestedPath
       }
+    }
 
     Files.createDirectories(stagedPath.getParent)
 
@@ -250,11 +360,28 @@ class SparkConnectAddArtifactsHandler(val responseObserver: StreamObserver[AddAr
           throw e
       }
 
-      overallChecksum.update(dataChunk.getData.toByteArray)
+      val bytes = dataChunk.getData.toByteArray
+      overallChecksum.update(bytes)
       updateCrc(checksumOut.getChecksum.getValue == dataChunk.getCrc)
       checksumOut.getChecksum.reset()
     }
 
+    def writeFrom(file: Path): Unit = {
+      val in = Files.newInputStream(file)
+      try {
+        val buffer = new Array[Byte](64 * 1024)
+        var read = in.read(buffer)
+        while (read != -1) {
+          checksumOut.write(buffer, 0, read)
+          overallChecksum.update(buffer, 0, read)
+          read = in.read(buffer)
+        }
+        updateCrc(isSuccess = true)
+      } finally {
+        in.close()
+      }
+    }
+
     def close(): Unit = {
       if (artifactSummary == null) {
         checksumOut.close()
diff --git a/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectAnalyzeHandler.scala b/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectAnalyzeHandler.scala
index 49450f938719..d66f38c775f0 100644
--- a/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectAnalyzeHandler.scala
+++ b/sql/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectAnalyzeHandler.scala
@@ -169,6 +169,7 @@ private[connect] class SparkConnectAnalyzeHandler(
           proto.AnalyzePlanResponse.SparkVersion
             .newBuilder()
             .setVersion(session.version)
+            .addCapabilities("serverSideMavenArtifacts.v1")
             .build())
 
       case proto.AnalyzePlanRequest.AnalyzeCase.DDL_PARSE =>
diff --git a/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectServiceSuite.scala b/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectServiceSuite.scala
index bc2cf4a9dba9..2e04eebb2a19 100644
--- a/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectServiceSuite.scala
+++ b/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectServiceSuite.scala
@@ -43,6 +43,7 @@ import org.apache.spark.internal.Logging
 import org.apache.spark.scheduler.{SparkListener, SparkListenerEvent}
 import org.apache.spark.sql.catalyst.expressions.{GenericInternalRow, UnsafeProjection}
 import org.apache.spark.sql.connect.SparkConnectTestUtils
+import org.apache.spark.sql.connect.client.SparkConnectClient
 import org.apache.spark.sql.connect.common.DataTypeProtoConverter
 import org.apache.spark.sql.connect.config.Connect
 import org.apache.spark.sql.connect.dsl.MockRemoteSession
@@ -75,6 +76,18 @@ class SparkConnectServiceSuite
   private def sparkSessionHolder = SparkConnectTestUtils.createDummySessionHolder(spark)
   private def DEFAULT_UUID = UUID.fromString("89ea6117-1f45-4c03-ae27-f47c6aded093")
 
+  test("Spark version advertises server-side Maven resolution") {
+    val request = proto.AnalyzePlanRequest
+      .newBuilder()
+      .setSparkVersion(proto.AnalyzePlanRequest.SparkVersion.newBuilder())
+      .build()
+    val response = new SparkConnectAnalyzeHandler(null).process(request, sparkSessionHolder)
+
+    assert(
+      response.getSparkVersion.getCapabilitiesList.asScala
+        .contains(SparkConnectClient.SERVER_SIDE_MAVEN_ARTIFACTS_CAPABILITY))
+  }
+
   test("Test schema in analyze response") {
     withTable("test") {
       spark.sql("""
diff --git a/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/service/AddArtifactsHandlerSuite.scala b/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/service/AddArtifactsHandlerSuite.scala
index 8094a084d3b6..da0b9ead6f43 100644
--- a/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/service/AddArtifactsHandlerSuite.scala
+++ b/sql/connect/server/src/test/scala/org/apache/spark/sql/connect/service/AddArtifactsHandlerSuite.scala
@@ -17,8 +17,10 @@
 package org.apache.spark.sql.connect.service
 
 import java.io.{FileOutputStream, InputStream}
-import java.nio.file.{Files, Path}
+import java.net.URI
+import java.nio.file.{Files, Path, Paths}
 import java.util.UUID
+import java.util.concurrent.{Callable, Executors, TimeUnit}
 import java.util.jar.{JarEntry, JarOutputStream}
 import java.util.zip.CRC32
 
@@ -29,6 +31,7 @@ import scala.jdk.CollectionConverters._
 
 import com.google.protobuf.ByteString
 import com.google.rpc.ErrorInfo
+import io.grpc.Context
 import io.grpc.Status.Code
 import io.grpc.StatusRuntimeException
 import io.grpc.protobuf.StatusProto
@@ -37,6 +40,7 @@ import io.grpc.stub.StreamObserver
 import org.apache.spark.SparkRuntimeException
 import org.apache.spark.connect.proto
 import org.apache.spark.connect.proto.{AddArtifactsRequest, AddArtifactsResponse}
+import org.apache.spark.sql.Artifact
 import org.apache.spark.sql.connect.ResourceHelper
 import org.apache.spark.sql.test.SharedSparkSession
 import org.apache.spark.util.{ThreadUtils, Utils}
@@ -63,7 +67,9 @@ class AddArtifactsHandlerSuite extends SharedSparkSession with ResourceHelper {
 
   class TestAddArtifactsHandler(
       responseObserver: StreamObserver[AddArtifactsResponse],
-      throwIfArtifactExists: Boolean = false)
+      throwIfArtifactExists: Boolean = false,
+      resolvedMavenArtifacts: Map[URI, Seq[Artifact]] = Map.empty,
+      resolutionDelayMs: Long = 0L)
       extends SparkConnectAddArtifactsHandler(responseObserver) {
 
     // Stop the staged artifacts from being automatically deleted
@@ -71,6 +77,8 @@ class AddArtifactsHandlerSuite extends SharedSparkSession with ResourceHelper {
 
     private val finalArtifacts = mutable.Buffer.empty[String]
     private val artifactChecksums: mutable.Map[String, Long] = mutable.Map.empty
+    private val artifactContents: mutable.Map[String, Array[Byte]] = mutable.Map.empty
+    private val resolutionTimeouts = mutable.Buffer.empty[(Int, Int)]
 
     // Record the artifacts that are sent out for final processing.
     override protected def addStagedArtifactToArtifactManager(artifact: StagedArtifact): Unit = {
@@ -86,9 +94,27 @@ class AddArtifactsHandlerSuite extends SharedSparkSession with ResourceHelper {
 
       finalArtifacts.append(artifact.name)
       artifactChecksums += (artifact.name -> artifact.getCrc)
+      artifactContents += (artifact.name -> Files.readAllBytes(artifact.stagedPath))
+    }
+
+    override protected def resolveMavenDependency(
+        uri: URI,
+        connectTimeoutMs: Int,
+        readTimeoutMs: Int,
+        isCancelled: () => Boolean): Seq[Artifact] = {
+      resolutionTimeouts += ((connectTimeoutMs, readTimeoutMs))
+      if (resolutionDelayMs > 0) {
+        Thread.sleep(resolutionDelayMs)
+      }
+      resolvedMavenArtifacts.getOrElse(
+        uri,
+        super.resolveMavenDependency(uri, connectTimeoutMs, readTimeoutMs, isCancelled))
     }
 
     def getFinalArtifacts: Seq[String] = finalArtifacts.toSeq
+    def getArtifactChecksum(name: String): Long = artifactChecksums(name)
+    def getArtifactContents(name: String): Array[Byte] = artifactContents(name)
+    def getResolutionTimeouts: Seq[(Int, Int)] = resolutionTimeouts.toSeq
     def stagingDirectory: Path = this.stagingDir
     def forceCleanUp(): Unit = super.cleanUpStagedArtifacts()
   }
@@ -315,6 +341,150 @@ class AddArtifactsHandlerSuite extends SharedSparkSession with ResourceHelper {
     }
   }
 
+  test("ordered entries resolve Maven dependencies before registration") {
+    val promise = Promise[AddArtifactsResponse]()
+    val ivyUri = URI.create("ivy://my.connect.lib:mylib:0.1")
+    val resolved = Artifact.newJarArtifact(
+      Paths.get("resolved.jar"),
+      new Artifact.LocalFile(inputFilePath.resolve("smallJar.jar")))
+    val handler = new TestAddArtifactsHandler(
+      new DummyStreamObserver(promise),
+      resolvedMavenArtifacts = Map(ivyUri -> Seq(resolved)))
+    try {
+      def uploaded(name: String, path: Path): proto.AddArtifactsRequest.ArtifactEntry = {
+        val bytes = ByteString.copyFrom(Files.readAllBytes(path))
+        val crc = new CRC32()
+        crc.update(bytes.toByteArray)
+        val artifact = proto.AddArtifactsRequest.SingleChunkArtifact
+          .newBuilder()
+          .setName(name)
+          .setData(
+            proto.AddArtifactsRequest.ArtifactChunk
+              .newBuilder()
+              .setData(bytes)
+              .setCrc(crc.getValue))
+        proto.AddArtifactsRequest.ArtifactEntry.newBuilder().setArtifact(artifact).build()
+      }
+
+      val request = AddArtifactsRequest
+        .newBuilder()
+        .setSessionId(sessionId)
+        .setUserContext(proto.UserContext.newBuilder().setUserId("c1"))
+        .setBatch(
+          proto.AddArtifactsRequest.Batch
+            .newBuilder()
+            .addEntries(uploaded(
+              "classes/smallClassFile.class",
+              inputFilePath.resolve("smallClassFile.class")))
+            .addEntries(
+              proto.AddArtifactsRequest.ArtifactEntry
+                .newBuilder()
+                .setMavenDependency(proto.AddArtifactsRequest.MavenDependency
+                  .newBuilder()
+                  .setUri(ivyUri.toString)))
+            .addEntries(uploaded("jars/smallJar.jar", inputFilePath.resolve("smallJar.jar")))
+            .build())
+        .build()
+
+      handler.onNext(request)
+      assert(handler.getFinalArtifacts.isEmpty)
+      handler.onCompleted()
+
+      val response = ThreadUtils.awaitResult(promise.future, 5.seconds)
+      assert(
+        handler.getFinalArtifacts == Seq(
+          "classes/smallClassFile.class",
+          "jars/resolved.jar",
+          "jars/smallJar.jar"))
+      assert(response.getArtifactsList.asScala.map(_.getName) == handler.getFinalArtifacts)
+      val resolvedBytes = Files.readAllBytes(inputFilePath.resolve("smallJar.jar"))
+      val resolvedCrc = new CRC32()
+      resolvedCrc.update(resolvedBytes)
+      assert(handler.getArtifactContents("jars/resolved.jar").sameElements(resolvedBytes))
+      assert(handler.getArtifactChecksum("jars/resolved.jar") == resolvedCrc.getValue)
+      assert(response.getArtifacts(1).getIsCrcSuccessful)
+    } finally {
+      handler.forceCleanUp()
+    }
+  }
+
+  test("server-side Maven dependencies reject requested repositories") {
+    val error = Promise[Throwable]()
+    val observer = new StreamObserver[AddArtifactsResponse] {
+      override def onNext(value: AddArtifactsResponse): Unit = {}
+      override def onError(throwable: Throwable): Unit = error.success(throwable)
+      override def onCompleted(): Unit = {}
+    }
+    val handler = new TestAddArtifactsHandler(observer)
+    try {
+      val ivyUri = "ivy://my.connect.lib:mylib:0.1?repos=https://example.com/repository"
+      val request = AddArtifactsRequest
+        .newBuilder()
+        .setSessionId(sessionId)
+        .setUserContext(proto.UserContext.newBuilder().setUserId("c1"))
+        .setBatch(
+          proto.AddArtifactsRequest.Batch
+            .newBuilder()
+            .addEntries(
+              proto.AddArtifactsRequest.ArtifactEntry
+                .newBuilder()
+                .setMavenDependency(
+                  proto.AddArtifactsRequest.MavenDependency.newBuilder().setUri(ivyUri))))
+        .build()
+
+      handler.onNext(request)
+      handler.onCompleted()
+
+      val failure = ThreadUtils.awaitResult(error.future, 5.seconds)
+      assert(failure.getMessage.contains("do not allow repositories"))
+      assert(handler.getFinalArtifacts.isEmpty)
+    } finally {
+      handler.forceCleanUp()
+    }
+  }
+
+  test("server-side Maven dependencies share the remaining RPC deadline") {
+    val firstUri = URI.create("ivy://my.connect.first:mylib:0.1")
+    val secondUri = URI.create("ivy://my.connect.second:mylib:0.1")
+    val promise = Promise[AddArtifactsResponse]()
+    val scheduler = Executors.newSingleThreadScheduledExecutor()
+    val context = Context.current().withDeadlineAfter(5, TimeUnit.SECONDS, scheduler)
+    val handler = context.call(new Callable[TestAddArtifactsHandler] {
+      override def call(): TestAddArtifactsHandler = new TestAddArtifactsHandler(
+        new DummyStreamObserver(promise),
+        resolvedMavenArtifacts = Map(firstUri -> Nil, secondUri -> Nil),
+        resolutionDelayMs = 100L)
+    })
+    try {
+      val batch = proto.AddArtifactsRequest.Batch.newBuilder()
+      Seq(firstUri, secondUri).foreach { uri =>
+        batch.addEntries(
+          proto.AddArtifactsRequest.ArtifactEntry
+            .newBuilder()
+            .setMavenDependency(
+              proto.AddArtifactsRequest.MavenDependency.newBuilder().setUri(uri.toString)))
+      }
+      handler.onNext(
+        AddArtifactsRequest
+          .newBuilder()
+          .setSessionId(sessionId)
+          .setUserContext(proto.UserContext.newBuilder().setUserId("c1"))
+          .setBatch(batch)
+          .build())
+      handler.onCompleted()
+      ThreadUtils.awaitResult(promise.future, 5.seconds)
+
+      val timeouts = handler.getResolutionTimeouts
+      assert(timeouts.size == 2)
+      assert(timeouts(1)._1 < timeouts.head._1)
+      assert(timeouts(1)._2 < timeouts.head._2)
+    } finally {
+      context.cancel(null)
+      scheduler.shutdownNow()
+      handler.forceCleanUp()
+    }
+  }
+
   test("Multi chunk artifact") {
     val promise = Promise[AddArtifactsResponse]()
     val handler = new TestAddArtifactsHandler(new DummyStreamObserver(promise))
@@ -426,6 +596,48 @@ class AddArtifactsHandlerSuite extends SharedSparkSession with ResourceHelper {
     }
   }
 
+  test("CRC failure does not reserve an artifact path") {
+    val promise = Promise[AddArtifactsResponse]()
+    val handler = new TestAddArtifactsHandler(new DummyStreamObserver(promise))
+    try {
+      val name = "classes/smallClassFile.class"
+      val bytes =
+        ByteString.copyFrom(Files.readAllBytes(inputFilePath.resolve("smallClassFile.class")))
+      val crc = new CRC32()
+      crc.update(bytes.toByteArray)
+      def entry(checksum: Long): proto.AddArtifactsRequest.ArtifactEntry = {
+        val artifact = proto.AddArtifactsRequest.SingleChunkArtifact
+          .newBuilder()
+          .setName(name)
+          .setData(
+            proto.AddArtifactsRequest.ArtifactChunk
+              .newBuilder()
+              .setData(bytes)
+              .setCrc(checksum))
+        proto.AddArtifactsRequest.ArtifactEntry.newBuilder().setArtifact(artifact).build()
+      }
+      val request = AddArtifactsRequest
+        .newBuilder()
+        .setSessionId(sessionId)
+        .setUserContext(proto.UserContext.newBuilder().setUserId("c1"))
+        .setBatch(
+          proto.AddArtifactsRequest.Batch
+            .newBuilder()
+            .addEntries(entry(crc.getValue + 1))
+            .addEntries(entry(crc.getValue)))
+        .build()
+
+      handler.onNext(request)
+      handler.onCompleted()
+
+      val summaries = ThreadUtils.awaitResult(promise.future, 5.seconds).getArtifactsList.asScala
+      assert(summaries.map(_.getIsCrcSuccessful) == Seq(false, true))
+      assert(handler.getFinalArtifacts == Seq(name))
+    } finally {
+      handler.forceCleanUp()
+    }
+  }
+
   private def createDummyArtifactRequests(name: String): Seq[proto.AddArtifactsRequest] = {
     val bytes = ByteString.EMPTY
     val context = proto.UserContext
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/artifact/ArtifactManager.scala b/sql/core/src/main/scala/org/apache/spark/sql/artifact/ArtifactManager.scala
index 9b22cf4d9dd6..c82acef5818b 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/artifact/ArtifactManager.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/artifact/ArtifactManager.scala
@@ -33,13 +33,14 @@ import org.apache.hadoop.fs.{LocalFileSystem, Path => FSPath}
 
 import org.apache.spark.{JobArtifactSet, JobArtifactState, SparkContext, SparkEnv, SparkException, SparkRuntimeException, SparkUnsupportedOperationException}
 import org.apache.spark.internal.{Logging, LogKeys}
-import org.apache.spark.internal.config.{CONNECT_SCALA_UDF_STUB_PREFIXES, EXECUTOR_USER_CLASS_PATH_FIRST}
+import org.apache.spark.internal.config.{CONNECT_SCALA_UDF_STUB_PREFIXES, EXECUTOR_USER_CLASS_PATH_FIRST, JAR_IVY_CONNECT_TIMEOUT, JAR_IVY_READ_TIMEOUT, JAR_IVY_REPO_PATH, JAR_IVY_SETTING_PATH}
 import org.apache.spark.sql.Artifact
 import org.apache.spark.sql.classic.SparkSession
 import org.apache.spark.sql.internal.{SQLConf, StaticSQLConf}
 import org.apache.spark.sql.util.ArtifactUtils
 import org.apache.spark.storage.{BlockManager, CacheId, StorageLevel}
-import org.apache.spark.util.{ChildFirstURLClassLoader, StubClassLoader, Utils}
+import org.apache.spark.util.{ChildFirstURLClassLoader, RuntimeDependencyResolver, StubClassLoader, Utils}
+import org.apache.spark.util.RuntimeDependencyResolver.{AllowRequestedRepositories, RepositoryPolicy}
 
 /**
  * This class handles the storage of artifacts as well as preparing the artifacts for use.
@@ -70,6 +71,21 @@ class ArtifactManager(session: SparkSession) extends AutoCloseable with Logging
     (ArtifactUtils.concatenatePaths(artifactRootPath, session.sessionUUID),
       s"$artifactRootURI/${session.sessionUUID}")
 
+  private lazy val runtimeDependencyResolver = {
+    val sparkConf = session.sparkContext.getConf
+    val ivyPath = Files.createDirectories(artifactPath.resolve(".ivy-cache")).toString
+    new RuntimeDependencyResolver(
+      ivySettingsPath = sparkConf.get(JAR_IVY_SETTING_PATH),
+      ivyPath = Some(ivyPath),
+      localIvyPath = sparkConf.getOption(JAR_IVY_REPO_PATH.key))
+  }
+
+  private[sql] lazy val ivyConnectTimeoutMs: Int =
+    session.sparkContext.getConf.get(JAR_IVY_CONNECT_TIMEOUT).toInt
+
+  private[sql] lazy val ivyReadTimeoutMs: Int =
+    session.sparkContext.getConf.get(JAR_IVY_READ_TIMEOUT).toInt
+
   // The base directory/URI where all class file artifacts are stored for this `sessionUUID`.
   protected[artifact] val (classDir, replClassURI): (Path, String) =
     (ArtifactUtils.concatenatePaths(artifactPath, "classes"), s"$artifactURI/classes/")
@@ -323,6 +339,21 @@ class ArtifactManager(session: SparkSession) extends AutoCloseable with Logging
     }
   }
 
+  private[sql] def resolveArtifacts(
+      uri: URI,
+      repositoryPolicy: RepositoryPolicy = AllowRequestedRepositories,
+      connectTimeoutMs: Int = ivyConnectTimeoutMs,
+      readTimeoutMs: Int = ivyReadTimeoutMs,
+      isCancelled: () => Boolean = () => false): Seq[Artifact] = {
+    uri.getScheme match {
+      case "ivy" =>
+        runtimeDependencyResolver
+          .resolve(uri, connectTimeoutMs, readTimeoutMs, repositoryPolicy, isCancelled)
+          .map(path => Artifact.newJarArtifact(path.getFileName, new Artifact.LocalFile(path)))
+      case _ => Artifact.parseArtifacts(uri)
+    }
+  }
+
   def classloader: ClassLoader = synchronized {
     cachedClassLoader.getOrElse {
       val loader = buildClassLoader
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/classic/SparkSession.scala b/sql/core/src/main/scala/org/apache/spark/sql/classic/SparkSession.scala
index a35c0aa01530..c3acd83bc7ec 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/classic/SparkSession.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/classic/SparkSession.scala
@@ -730,7 +730,7 @@ class SparkSession private(
   /** @inheritdoc */
   @Experimental
   override def addArtifact(uri: URI): Unit = {
-    artifactManager.addLocalArtifacts(Artifact.parseArtifacts(uri))
+    artifactManager.addLocalArtifacts(artifactManager.resolveArtifacts(uri))
   }
 
   /** @inheritdoc */
@@ -759,7 +759,7 @@ class SparkSession private(
   @Experimental
   @scala.annotation.varargs
   override def addArtifacts(uri: URI*): Unit = {
-    artifactManager.addLocalArtifacts(uri.flatMap(Artifact.parseArtifacts))
+    artifactManager.addLocalArtifacts(uri.flatMap(artifactManager.resolveArtifacts(_)))
   }
 
   /** @inheritdoc */
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/artifact/ArtifactManagerIvySettingsSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/artifact/ArtifactManagerIvySettingsSuite.scala
new file mode 100644
index 000000000000..889e37c6afc8
--- /dev/null
+++ b/sql/core/src/test/scala/org/apache/spark/sql/artifact/ArtifactManagerIvySettingsSuite.scala
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql.artifact
+
+import java.net.URI
+import java.nio.file.Files
+
+import org.apache.spark.SparkConf
+import org.apache.spark.internal.config.{JAR_IVY_CONNECT_TIMEOUT, JAR_IVY_READ_TIMEOUT}
+import org.apache.spark.sql.test.SharedSparkSession
+import org.apache.spark.util.{IvyTestUtils, MavenUtils, Utils}
+import org.apache.spark.util.MavenUtils.MavenCoordinate
+
+class ArtifactManagerIvySettingsSuite extends SharedSparkSession {
+
+  private val ivySettingsDir = Utils.createTempDir(namePrefix = "artifact-ivy-settings")
+  private val coordinate = MavenCoordinate("my.runtime.lib", "mylib", "0.1")
+  private val repository = IvyTestUtils.createLocalRepositoryForTests(
+    coordinate,
+    dependencies = None,
+    rootDir = Some(ivySettingsDir.toPath.resolve("repository").toFile))
+  private val ivySettings = ivySettingsDir.toPath.resolve("ivysettings.xml")
+  Files.writeString(
+    ivySettings,
+    s"""<ivysettings>
+       |  <settings defaultResolver="runtime"/>
+       |  <resolvers>
+       |    <ibiblio name="runtime" m2compatible="true" root="${repository.toURI}"/>
+       |  </resolvers>
+       |</ivysettings>""".stripMargin)
+
+  override protected def sparkConf: SparkConf = {
+    super.sparkConf
+      .set(MavenUtils.JAR_IVY_SETTING_PATH_KEY, ivySettings.toString)
+      .set(JAR_IVY_CONNECT_TIMEOUT, 7000L)
+      .set(JAR_IVY_READ_TIMEOUT, 11000L)
+  }
+
+  override def afterAll(): Unit = {
+    try {
+      super.afterAll()
+    } finally {
+      Utils.deleteRecursively(ivySettingsDir)
+    }
+  }
+
+  test("Ivy timeout configurations have defaults") {
+    val conf = new SparkConf(loadDefaults = false)
+    assert(conf.get(JAR_IVY_CONNECT_TIMEOUT) == 30000L)
+    assert(conf.get(JAR_IVY_READ_TIMEOUT) == 300000L)
+  }
+
+  test("SparkSession artifact APIs use spark.jars.ivySettings") {
+    val ivyUri = URI.create(s"ivy://${coordinate.toString}")
+    val otherSession = spark.newSession()
+
+    spark.addArtifact(ivyUri)
+    val jarName = s"${coordinate.groupId}_${coordinate.artifactId}-${coordinate.version}.jar"
+    assert(Files.exists(spark.artifactManager.artifactPath.resolve("jars").resolve(jarName)))
+    assert(
+      !Files.exists(otherSession.artifactManager.artifactPath.resolve("jars").resolve(jarName)))
+
+    otherSession.addArtifacts(ivyUri)
+
+    assert(Files.exists(
+      otherSession.artifactManager.artifactPath.resolve("jars").resolve(jarName)))
+    assert(
+      spark.artifactManager.artifactPath.resolve(".ivy-cache") !=
+        otherSession.artifactManager.artifactPath.resolve(".ivy-cache"))
+    assert(spark.artifactManager.ivyConnectTimeoutMs == 7000)
+    assert(spark.artifactManager.ivyReadTimeoutMs == 11000)
+  }
+}


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org