You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/12/28 00:19:55 UTC

[GitHub] [spark] Tagar commented on pull request #39115: [SPARK-41563][SQL] Support partition filter in MSCK REPAIR TABLE statement

Tagar commented on PR #39115:
URL: https://github.com/apache/spark/pull/39115#issuecomment-1366276253

   I had a customer that runs MSCK on really huge tables, and it takes them hours to complete that operation. So this looks the same as the 2nd bullet point in @wecharyu's "why are the changes needed?". 
   
   That particular customer uses Athena and want to populate list of newly added partitions to Glue Catalog.
   
   It would be great for MSCK operation to have an optional partitioning clause so it would not rescan whole table, but just those partitions matching the PARTITION pattern. For example,
   
   ```sql
   MSCK TABLE [db_name.]table_name PARTITION '202107%'
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org