You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "siying (via GitHub)" <gi...@apache.org> on 2023/10/09 18:05:58 UTC

[PR] [SPARK-45472][SS] RocksDB State Store Doesn't Need to Recheck checkpoint path existence [spark]

siying opened a new pull request, #43299:
URL: https://github.com/apache/spark/pull/43299

   ### What changes were proposed in this pull request?
   In RocksDBFileManager, we add a variable to indicate that root path is already checked and created if not existing, so that we don't need to recheck the second time.
   
   ### Why are the changes needed?
   Right now, every time RocksDB.load() is called, we check checkpoint directory existence and create it if not. This is relatively expensive and show up in performance profiling.
   
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   
   ### How was this patch tested?
   Existing CI tests to cover it.
   
   ### Was this patch authored or co-authored using generative AI tooling?
   No


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-45472][SS] RocksDB State Store Doesn't Need to Recheck checkpoint path existence [spark]

Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR commented on PR #43299:
URL: https://github.com/apache/spark/pull/43299#issuecomment-1754222412

   Thanks! Merging to master.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-45472][SS] RocksDB State Store Doesn't Need to Recheck checkpoint path existence [spark]

Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR closed pull request #43299: [SPARK-45472][SS] RocksDB State Store Doesn't Need to Recheck checkpoint path existence
URL: https://github.com/apache/spark/pull/43299


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org