You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "holdenk (via GitHub)" <gi...@apache.org> on 2026/08/30 04:28:06 UTC

[PR] [SPARK-59118][SQL] Error when doAs is used and not supported [spark]

holdenk opened a new pull request, #58414:
URL: https://github.com/apache/spark/pull/58414

   ### What changes were proposed in this pull request?
   Add a config param for enabling hard failure on doAs in Thriftserver when we do not support it.
   
   ### Why are the changes needed?
   
   People might expect doAs to run as a different user, which is not supported.
   
   ### Does this PR introduce _any_ user-facing change?
   
   Explicit error unless config flag is disabled.
   
   ### How was this patch tested?
   
   New unit test.
   
   ### Was this patch authored or co-authored using generative AI tooling?
   Yes, claude 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-59118][SQL] Error when doAs is used and not supported [spark]

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #58414:
URL: https://github.com/apache/spark/pull/58414#discussion_r3974251752


##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala:
##########
@@ -146,6 +146,21 @@ object StaticSQLConf {
       .booleanConf
       .createWithDefault(true)
 
+  val HIVE_THRIFT_SERVER_ALLOW_INEFFECTIVE_DOAS =
+    buildStaticConf("spark.sql.hive.thriftServer.allowIneffectiveDoAs")
+      .doc("With hive.server2.enable.doAs=true the Spark Thrift Server impersonates the " +
+        "connecting user for Hive metastore calls, but it executes queries and accesses " +
+        "storage as its own service identity (SPARK-5159), so storage-level permissions are " +
+        "checked against the privileged service principal instead of the impersonated user. " +
+        "Because that can silently grant users access to data they could not read with their " +
+        "own credentials, the server refuses to start when hive.server2.enable.doAs=true and " +
+        "hive.server2.authentication verifies user identities (anything other than " +
+        "NONE/NOSASL), unless this option is set to true to acknowledge the limitation.")
+      .version("5.0.0")
+      .withBindingPolicy(ConfigBindingPolicy.NOT_APPLICABLE)
+      .booleanConf
+      .createWithDefault(false)

Review Comment:
   Defaulting this to `false` means the Thrift Server *refuses to start* for any existing `hive.server2.enable.doAs=true` + verifying-auth (Kerberos/LDAP) deployment once it upgrades to 5.0. The SPARK-5159 rationale is sound, and the escape hatch + migration-guide entry + the 4.x warn-first #58438 are the right scaffolding. The open question is whether *refuse-to-start* is the right default for 5.0: it turns a long-latent, silently-"working" config into an immediate startup outage on upgrade. If the hard-fail default is intended, it's worth a prominent release-note callout (beyond the migration guide) given how many secured HS2 deployments run `doAs=true`; if not, a loud startup warning that only escalates to a hard error under an explicit opt-in would be less disruptive while still closing the security gap.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-59118][SQL] Error when doAs is used and not supported [spark]

Posted by "asf-gitbox-commits (via GitHub)" <gi...@apache.org>.
asf-gitbox-commits closed pull request #58414: [SPARK-59118][SQL] Error when doAs is used and not supported
URL: https://github.com/apache/spark/pull/58414


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-59118][SQL] Error when doAs is used and not supported [spark]

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #58414:
URL: https://github.com/apache/spark/pull/58414#discussion_r3973951443


##########
docs/sql-migration-guide.md:
##########
@@ -22,6 +22,10 @@ license: |
 * Table of contents
 {:toc}
 
+## Upgrading from Spark SQL 4.4 to 5.0
+
+- Since Spark 5.0, the Spark Thrift Server refuses to start when `hive.server2.enable.doAs` is `true` and `hive.server2.authentication` verifies user identities (anything other than `NONE`/`NOSASL`). It impersonates the connecting user for Hive metastore calls, but queries and the storage access they perform still run as the server's own service identity, so storage-level permissions are checked against the service principal rather than the connecting user (SPARK-5159). Spark 4.4 logs a warning for the same configuration. To restore the previous behavior, set `spark.sql.hive.thriftServer.allowIneffectiveDoAs` to `true`. Setting `hive.server2.enable.doAs` to `false` also starts the server, but note that it stops impersonating metastore calls as well, so it is not a no-op.

Review Comment:
   "Spark 4.4 logs a warning for the same configuration" — this PR targets 5.0 and adds no warning anywhere (only the hard failure), so the 4.4 warning must be coming from a separate branch-4.x change. Is that companion already merged to branch-4.x? If it isn't, this note points users upgrading from 4.4 at behavior that doesn't exist there yet.



##########
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2DoAsSuite.scala:
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.hive.thriftserver
+
+import org.apache.hadoop.hive.conf.HiveConf
+import org.apache.hadoop.hive.conf.HiveConf.ConfVars
+import org.apache.hive.service.auth.HiveAuthFactory.AuthTypes
+
+import org.apache.spark.sql.QueryTest
+import org.apache.spark.sql.internal.StaticSQLConf
+import org.apache.spark.sql.test.SharedSparkSession
+
+/**
+ * Tests for the SPARK-59118 startup guard: the Thrift Server refuses to start when
+ * `hive.server2.enable.doAs` is on but impersonation does not reach query execution.
+ */
+class HiveThriftServer2DoAsSuite extends QueryTest with SharedSparkSession {
+
+  private def hiveConf(authType: String, doAs: Boolean): HiveConf = {
+    val conf = new HiveConf()
+    conf.setVar(ConfVars.HIVE_SERVER2_AUTHENTICATION, authType)
+    conf.setBoolVar(ConfVars.HIVE_SERVER2_ENABLE_DOAS, doAs)
+    conf
+  }
+
+  private def check(conf: HiveConf, allowIneffectiveDoAs: Boolean = false): Unit = {
+    HiveThriftServer2.failIfIneffectiveDoAs(conf, allowIneffectiveDoAs)
+  }
+
+  // The auth types that establish a user identity worth impersonating.
+  private val verifyingAuthTypes =
+    AuthTypes.values().filterNot(Set(AuthTypes.NONE, AuthTypes.NOSASL)).map(_.getAuthName)
+
+  test("SPARK-5159 / SPARK-59118 refuse to start when doAs is enabled but not enforced") {
+    val e = intercept[IllegalArgumentException](check(hiveConf("KERBEROS", doAs = true)))
+    assert(e.getMessage.contains(ConfVars.HIVE_SERVER2_ENABLE_DOAS.varname))
+    assert(e.getMessage.contains(StaticSQLConf.HIVE_THRIFT_SERVER_ALLOW_INEFFECTIVE_DOAS.key))
+    assert(e.getMessage.contains("SPARK-5159"))
+  }
+
+  test("SPARK-59118 refuse to start for every auth type that verifies the user") {
+    // Guards against a new AuthTypes value silently landing on the permissive side.
+    assert(verifyingAuthTypes.nonEmpty)
+    verifyingAuthTypes.foreach { authType =>
+      intercept[IllegalArgumentException](check(hiveConf(authType, doAs = true)))
+    }
+  }
+
+  test("SPARK-59118 does not refuse to start when auth type establishes no user identity") {
+    Seq("NONE", "NOSASL", "none", "noSasl").foreach { authType =>
+      check(hiveConf(authType, doAs = true))
+    }
+  }
+
+  test("SPARK-59118 does not refuse to start when doAs is disabled") {
+    verifyingAuthTypes.foreach { authType =>
+      check(hiveConf(authType, doAs = false))
+    }
+  }
+
+  test("SPARK-59118 does not refuse to start on an untouched Hive config") {
+    // Hive defaults doAs to true and auth to NONE, so out-of-the-box servers must still start.
+    check(new HiveConf())
+  }
+
+  test("SPARK-59118 allowIneffectiveDoAs=true permits an otherwise-refused config") {
+    check(hiveConf("KERBEROS", doAs = true), allowIneffectiveDoAs = true)
+  }
+
+  test("SPARK-59118 an unrecognized auth type fails closed") {
+    intercept[IllegalArgumentException](check(hiveConf("NOT_AN_AUTH_TYPE", doAs = true)))

Review Comment:
   The guard's comment calls out that `authType` is null when the auth is explicitly set empty and that the constant-on-left comparison must not NPE. That branch isn't exercised — the closest case here uses a non-null unknown string. A case like `check(hiveConf("", doAs = true))` asserting it fails closed without an NPE would lock in the null-safety you flagged.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-59118][SQL] Error when doAs is used and not supported [spark]

Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on code in PR #58414:
URL: https://github.com/apache/spark/pull/58414#discussion_r3990870570


##########
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala:
##########
@@ -105,6 +106,34 @@ object HiveThriftServer2 extends Logging {
     }
   }
 
+  // Sessions are impersonated for metastore calls, but queries run as the service identity
+  // (SPARK-5159), so storage ACLs are checked against the wrong principal. Refuse rather than
+  // look like we enforce something we do not. Auth types that never establish a user identity
+  // have nothing to impersonate, so they are exempt.
+  private[thriftserver] def failIfIneffectiveDoAs(
+      hiveConf: HiveConf,
+      allowIneffectiveDoAs: Boolean): Unit = {
+    val authType = hiveConf.getVar(ConfVars.HIVE_SERVER2_AUTHENTICATION)
+    val unverifiedAuthTypes = Seq(AuthTypes.NONE, AuthTypes.NOSASL).map(_.getAuthName)
+    // Constant on the left: authType is null when explicitly set empty, and this must not NPE.
+    val authVerifiesUser = !unverifiedAuthTypes.exists(_.equalsIgnoreCase(authType))
+    if (authVerifiesUser && hiveConf.getBoolVar(ConfVars.HIVE_SERVER2_ENABLE_DOAS) &&
+        !allowIneffectiveDoAs) {
+      throw new IllegalArgumentException(

Review Comment:
   This is a user-facing error, so it should go through the error-class framework rather than a raw `IllegalArgumentException` with a prose message (`common/utils/src/main/resources/error/README.md`: "developers should specify a standardized SQLSTATE, an error condition, and message parameters rather than an arbitrary error message"). The sibling startup check in `SparkSQLCLIService.init` already does this via `QueryExecutionErrors.invalidKerberosConfigForHiveServer2Error`. Could you add an error condition to `error-conditions.json` and throw it via `QueryExecutionErrors` / `HiveThriftServerErrors`, so the exception is a `SparkThrowable` with a SQLSTATE and can be asserted with `checkError`?



##########
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala:
##########
@@ -105,6 +106,34 @@ object HiveThriftServer2 extends Logging {
     }
   }
 
+  // Sessions are impersonated for metastore calls, but queries run as the service identity
+  // (SPARK-5159), so storage ACLs are checked against the wrong principal. Refuse rather than
+  // look like we enforce something we do not. Auth types that never establish a user identity
+  // have nothing to impersonate, so they are exempt.
+  private[thriftserver] def failIfIneffectiveDoAs(
+      hiveConf: HiveConf,
+      allowIneffectiveDoAs: Boolean): Unit = {
+    val authType = hiveConf.getVar(ConfVars.HIVE_SERVER2_AUTHENTICATION)
+    val unverifiedAuthTypes = Seq(AuthTypes.NONE, AuthTypes.NOSASL).map(_.getAuthName)
+    // Constant on the left: authType is null when explicitly set empty, and this must not NPE.
+    val authVerifiesUser = !unverifiedAuthTypes.exists(_.equalsIgnoreCase(authType))
+    if (authVerifiesUser && hiveConf.getBoolVar(ConfVars.HIVE_SERVER2_ENABLE_DOAS) &&
+        !allowIneffectiveDoAs) {
+      throw new IllegalArgumentException(
+        s"${ConfVars.HIVE_SERVER2_ENABLE_DOAS.varname} is set to true, but the Spark Thrift " +
+        "Server impersonates the connecting user only for Hive metastore calls: queries and " +

Review Comment:
   "queries and the storage access they perform still run as the server's own service identity" is only true on the executor side. `HiveSessionProxy` wraps `executeStatementAsync` in the session UGI's `doAs`, so `SparkExecuteStatementOperation.runInternal` calls `Utils.getUGI()` while already impersonating the connecting user, and the background action runs under that proxy UGI on the driver. Driver-side `FileSystem` calls (e.g. `InMemoryFileIndex` listing) are therefore impersonated; only executor task I/O runs as the service principal. Since operators will read this text (and the conf doc / migration note, which repeat it) to decide whether to set `allowIneffectiveDoAs=true`, it would be better to say executor-side data access is not impersonated rather than "queries" as a whole.



##########
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala:
##########
@@ -105,6 +106,34 @@ object HiveThriftServer2 extends Logging {
     }
   }
 
+  // Sessions are impersonated for metastore calls, but queries run as the service identity
+  // (SPARK-5159), so storage ACLs are checked against the wrong principal. Refuse rather than
+  // look like we enforce something we do not. Auth types that never establish a user identity
+  // have nothing to impersonate, so they are exempt.
+  private[thriftserver] def failIfIneffectiveDoAs(
+      hiveConf: HiveConf,
+      allowIneffectiveDoAs: Boolean): Unit = {
+    val authType = hiveConf.getVar(ConfVars.HIVE_SERVER2_AUTHENTICATION)
+    val unverifiedAuthTypes = Seq(AuthTypes.NONE, AuthTypes.NOSASL).map(_.getAuthName)

Review Comment:
   The comment above says `NONE`/`NOSASL` "have nothing to impersonate", but that is not what Hive does. `ThriftCLIService.getSessionHandle` opens the session with impersonation whenever `doAs` is true and a username exists, and under `NONE` `getUserName` falls back to the client-supplied `req.getUsername()`. So with `NONE` + `doAs=true` (Hive's out-of-the-box config) metastore calls are impersonated as an unverified, client-chosen name while storage still runs as the service principal, which is the same SPARK-5159 gap this guard describes. Exempting `NONE` may still be the right call (there is no security expectation without a verified identity), but the rationale should say that rather than "nothing to impersonate". Alternatively, keying on `doAs` alone or on `UserGroupInformation.isSecurityEnabled` would avoid a hand-maintained `AuthTypes` list.



##########
project/SparkBuild.scala:
##########
@@ -584,6 +584,8 @@ object SparkParallelTestGrouping {
     "org.apache.spark.sql.SQLQueryTestSuite",
     "org.apache.spark.sql.hive.client.HadoopVersionInfoSuite",
     "org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperationSuite",
+    // Constructs a HiveThriftServer2, which flips HiveConf's static loadHiveServer2Config.

Review Comment:
   `ThriftServerWithSparkContextInBinarySuite` / `InHttpSuite` already construct a `HiveThriftServer2` in-process via `SharedThriftServer.startWithSparkSession`, and they run in the default test group without a dedicated JVM. If the static `loadHiveServer2Config` flip were a real hazard, those suites would need this too. Unless there is an observed failure, I'd drop this entry so we don't fork an extra JVM for a suite of eight short tests.



##########
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala:
##########
@@ -105,6 +106,34 @@ object HiveThriftServer2 extends Logging {
     }
   }
 
+  // Sessions are impersonated for metastore calls, but queries run as the service identity
+  // (SPARK-5159), so storage ACLs are checked against the wrong principal. Refuse rather than
+  // look like we enforce something we do not. Auth types that never establish a user identity
+  // have nothing to impersonate, so they are exempt.
+  private[thriftserver] def failIfIneffectiveDoAs(
+      hiveConf: HiveConf,
+      allowIneffectiveDoAs: Boolean): Unit = {
+    val authType = hiveConf.getVar(ConfVars.HIVE_SERVER2_AUTHENTICATION)
+    val unverifiedAuthTypes = Seq(AuthTypes.NONE, AuthTypes.NOSASL).map(_.getAuthName)
+    // Constant on the left: authType is null when explicitly set empty, and this must not NPE.

Review Comment:
   I don't think `authType` can be null here. `HiveConf.getVar` returns `""` when the value is explicitly set empty and the default `NONE` for an empty `<value/>` in `hive-site.xml` (Hadoop `Configuration.get(name, default)` semantics), and `Configuration.set(name, null)` is rejected. So the constant-on-the-left is harmless but the comment is misleading. Separately, an empty or misspelled auth type is classified as "verifies user" and reported as a doAs problem; following the advice to set `allowIneffectiveDoAs=true` then does not help, because `HiveAuthFactory.getAuthTransFactory` fails with `Unsupported authentication type`. Either correct the comment, or leave invalid auth types to Hive's own error.



##########
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2DoAsSuite.scala:
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.hive.thriftserver
+
+import org.apache.hadoop.hive.conf.HiveConf
+import org.apache.hadoop.hive.conf.HiveConf.ConfVars
+import org.apache.hive.service.auth.HiveAuthFactory.AuthTypes
+
+import org.apache.spark.sql.QueryTest
+import org.apache.spark.sql.internal.StaticSQLConf
+import org.apache.spark.sql.test.SharedSparkSession
+
+/**
+ * Tests for the SPARK-59118 startup guard: the Thrift Server refuses to start when
+ * `hive.server2.enable.doAs` is on but impersonation does not reach query execution.
+ */
+class HiveThriftServer2DoAsSuite extends QueryTest with SharedSparkSession {

Review Comment:
   Only the last test (`init refuses to start the server`) uses `spark`; the other seven call `failIfIneffectiveDoAs` directly on a `HiveConf`, and the branch-4.x sibling in #58438 uses `SparkFunSuite` for the same cases. Could we keep this suite on `SparkFunSuite` and move the single `init()` wiring test into an existing session-backed thrift-server suite? That avoids paying `SharedSparkSession` setup for pure-function tests and keeps the 4.x and master suites aligned.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-59118][SQL] Error when doAs is used and not supported [spark]

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #58414:
URL: https://github.com/apache/spark/pull/58414#discussion_r3973951443


##########
docs/sql-migration-guide.md:
##########
@@ -22,6 +22,10 @@ license: |
 * Table of contents
 {:toc}
 
+## Upgrading from Spark SQL 4.4 to 5.0
+
+- Since Spark 5.0, the Spark Thrift Server refuses to start when `hive.server2.enable.doAs` is `true` and `hive.server2.authentication` verifies user identities (anything other than `NONE`/`NOSASL`). It impersonates the connecting user for Hive metastore calls, but queries and the storage access they perform still run as the server's own service identity, so storage-level permissions are checked against the service principal rather than the connecting user (SPARK-5159). Spark 4.4 logs a warning for the same configuration. To restore the previous behavior, set `spark.sql.hive.thriftServer.allowIneffectiveDoAs` to `true`. Setting `hive.server2.enable.doAs` to `false` also starts the server, but note that it stops impersonating metastore calls as well, so it is not a no-op.

Review Comment:
   "Spark 4.4 logs a warning for the same configuration" — this PR targets 5.0 and adds no warning anywhere (only the hard failure), so the 4.4 warning must be coming from a separate branch-4.x change. Is that companion already merged to branch-4.x? If it isn't, this note points users upgrading from 4.4 at behavior that doesn't exist there yet.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-59118][SQL] Error when doAs is used and not supported [spark]

Posted by "holdenk (via GitHub)" <gi...@apache.org>.
holdenk commented on PR #58414:
URL: https://github.com/apache/spark/pull/58414#issuecomment-5824097203

   Merged for master, the backport remains https://github.com/apache/spark/pull/58438 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org