You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Arpith P <ar...@gmail.com> on 2020/10/14 16:50:00 UTC
Large state RocksDb backend increases app start time
Hi,
I'm currently storing around 70GB of data in map sate backed by RocksDB
backend . Once I restore an application from savepoint currently the
application takes more than 4mins to start processing events. How can I
speed this up or is there any other recommended approach.
I'm using the following predefined options with RocksDB.
RocksDBStateBackend backend = new RocksDBStateBackend(checkpointDir,
incrementalCheckpoints);
backend.setPredefinedOptions(PredefinedOptions.SPINNING_DISK_OPTIMIZED_HIGH_MEM);
Thanks,
Arpith
Re: Large state RocksDb backend increases app start time
Posted by Yun Tang <my...@live.com>.
Hi Arpith
If you use savepoint to restore RocksDB state, the actual phase is to insert original binary key-value pairs into an empty RocksDB which would be slow if state large. There existed several discussions about the optimizations of this phase [1] [2].
If you want to walk around this issue quickly, you could use incremental checkpoint to restore rocksDB state as it just open the DB with existing sst files instead of loading data. Moreover, rocksDB incremental checkpoint also support the job to change parallelism currently.
[1] https://issues.apache.org/jira/browse/FLINK-17971
[2] https://issues.apache.org/jira/browse/FLINK-17288
Best
Yun Tang
________________________________
From: Arpith P <ar...@gmail.com>
Sent: Thursday, October 15, 2020 0:50
To: user <us...@flink.apache.org>
Subject: Large state RocksDb backend increases app start time
Hi,
I'm currently storing around 70GB of data in map sate backed by RocksDB backend . Once I restore an application from savepoint currently the application takes more than 4mins to start processing events. How can I speed this up or is there any other recommended approach.
I'm using the following predefined options with RocksDB.
RocksDBStateBackend backend = new RocksDBStateBackend(checkpointDir, incrementalCheckpoints);
backend.setPredefinedOptions(PredefinedOptions.SPINNING_DISK_OPTIMIZED_HIGH_MEM);
Thanks,
Arpith