[GitHub] [flink-benchmarks] Myasuka commented on pull request #23: [FLINK-23399][state] Add a benchmark for rescaling

[GitBox <gi...@apache.org>]

Myasuka commented on pull request #23:
URL: https://github.com/apache/flink-benchmarks/pull/23#issuecomment-1086577485


   @fredia Would you please share the configuration for each state backend test?
   And we can give some analysis on the performance behavior of different state-backends.
   For RocksDB state backend: 
   the rescale-in case need to scan one DB and insert to the init DB.
   the rescale-out case need to call delete range on the init DB (after deleteRange feature merged in FLINK-21321). 
   The process of scan and insert is certainly much longer than writing delete range.
   
   For file system state backend (we make the original data size as the same):
   the rescale-in case need to fill up 100% data into one subtask.
   the rescale-out case need to fill up 50% data into one subtask. That's why we see the result number of rescale-in is twice as  rescale-out.
   
   If could separate the benchmark on checkpoint rescaling from other benchmarks, the longger operation duration would not impact much.
   
   BTW, could you also share the result to prove the performance improvement of FLINK-21321?
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org