[GitHub] [spark] Kimahriman commented on pull request #38880: [SPARK-38277][SS] Clear write batch after RocksDB state store's commit

[GitBox <gi...@apache.org>]

Kimahriman commented on PR #38880:
URL: https://github.com/apache/spark/pull/38880#issuecomment-1335177296

   > Also cc. @Kimahriman I'm sorry if you wanted to take this up.
   
   No problem! I had a very similar PR I was working on before my previous one, and then tested it in production and realized it didn't fix the problem so I never submitted it. 
   
   The only other thing I had done was save off the write batch memory size before clearing it, and used that for the metrics. I believe the metrics are gathered post-commit (not sure if this is always the case but definitely true some of the cases), so the write batch mem size will always be 12 after creating a new one.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org