You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "pan3793 (via GitHub)" <gi...@apache.org> on 2026/09/07 12:49:13 UTC

[PR] [SPARK-57518][SQL][FOLLOWUP] Disable spark.sql.thriftServer.catalogMetadata.enabled by default [spark]

pan3793 opened a new pull request, #58573:
URL: https://github.com/apache/spark/pull/58573

   ### What changes were proposed in this pull request?
   
   Flip the default of `spark.sql.thriftServer.catalogMetadata.enabled` from `true` to `false`. Tests in `SparkMetadataOperationSuite` are updated accordingly: pre-existing tests assert the legacy behavior again, and SPARK-57518 tests that exercise the catalog-aware path now enable the conf explicitly.
   
   ### Why are the changes needed?
   
   The catalog-aware metadata operations shipped in #56627 are incomplete: getTables/getColumns do not route through DSv2 catalogs, and getCatalogs only lists already-loaded catalogs. There are also known correctness issues with the getSchemas implementation, documented in #57468, which has seen no active engagement from the author. Since 4.3.0 is not released yet, it is safer to keep the legacy behavior by default and let users opt in until the follow-up work lands.
   
   ### Does this PR introduce _any_ user-facing change?
   
   Within unreleased branches only: JDBC metadata operations (getCatalogs, getSchemas, getTables, getColumns) fall back to the legacy behavior unless the conf is explicitly enabled.
   
   ### How was this patch tested?
   
   Updated `SparkMetadataOperationSuite`; all 25 tests pass (`build/sbt -Phive-thriftserver 'hive-thriftserver/testOnly *SparkMetadataOperationSuite'`).
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: GLM 5.3


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-57518][SQL][FOLLOWUP] Disable spark.sql.thriftServer.catalogMetadata.enabled by default [spark]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #58573:
URL: https://github.com/apache/spark/pull/58573#issuecomment-5570895206

   cc @HeartSaVioR @cloud-fan @sarutak 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-57518][SQL][FOLLOWUP] Disable spark.sql.thriftServer.catalogMetadata.enabled by default [spark]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #58573:
URL: https://github.com/apache/spark/pull/58573#issuecomment-5603372307

   **Merge Summary:**
   - merged into master https://github.com/apache/spark/commit/1249988e5a2359f1a05d9771851cbf340c13b067
   - merged into branch-4.x https://github.com/apache/spark/commit/e5bf1d6276f243e2ca893a485da53a18093eb57c
   - merged into branch-4.3 https://github.com/apache/spark/commit/e21dd0fe030fbb7bd63bdb0097755931c050f8ab
   
   *Posted by `merge_spark_pr.py`*


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-57518][SQL][FOLLOWUP] Disable spark.sql.thriftServer.catalogMetadata.enabled by default [spark]

Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR commented on PR #58573:
URL: https://github.com/apache/spark/pull/58573#issuecomment-5577957100

   cc. @cloud-fan 
   Would you mind weighing on this change for 4.3.0?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-57518][SQL][FOLLOWUP] Disable spark.sql.thriftServer.catalogMetadata.enabled by default [spark]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #58573:
URL: https://github.com/apache/spark/pull/58573#issuecomment-5597520400

   I'm going to merge this (to master/4.x/4.3) if there is no other feedback at the end of today


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-57518][SQL][FOLLOWUP] Disable spark.sql.thriftServer.catalogMetadata.enabled by default [spark]

Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #58573:
URL: https://github.com/apache/spark/pull/58573#issuecomment-5584017557

   > Would you mind weighing on this change for 4.3.0?
   
   seems to me like we should definitely include this in 4.3.0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-57518][SQL][FOLLOWUP] Disable spark.sql.thriftServer.catalogMetadata.enabled by default [spark]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 closed pull request #58573: [SPARK-57518][SQL][FOLLOWUP] Disable spark.sql.thriftServer.catalogMetadata.enabled by default
URL: https://github.com/apache/spark/pull/58573


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org