[GitHub] [incubator-pegasus] empiredan commented on a diff in pull request #1009: feat: support to restrict the size of rocksdb info logs

[GitBox <gi...@apache.org>]

empiredan commented on code in PR #1009:
URL: https://github.com/apache/incubator-pegasus/pull/1009#discussion_r900701289


##########
src/server/pegasus_server_impl_init.cpp:
##########
@@ -95,6 +95,41 @@ DSN_DEFINE_uint64(
     "batch-get operation iterate count exceed this threshold will be logged, 0 means no check");
 DSN_TAG_VARIABLE(rocksdb_abnormal_batch_get_count_threshold, FT_MUTABLE);
 
+// In production environment, it has been observed that an instance of rocksdb about 20GB in total
+// size which has run beyond 199 days, generated a big log file sized 96MB, with 492KB for each
+// day.
+//
+// Accordingly, default value for `rocksdb_log_file_time_to_roll` can be set to one day, and
+// that for `rocksdb_keep_log_file_num` can be set to 32, which means log files for recent one
+// month will be reserved.
+//
+// On the other hand, the max size of a log file is also be restricted to 8MB. In practice, the
+// size of logs over a day tends to be less than 1MB; however, once errors are reported very
+// frequently, the log file will grow larger and go far beyond several hundreds of KB.
+DSN_DEFINE_uint64("pegasus.server",
+                  rocksdb_max_log_file_size,
+                  8 * 1024 * 1024,
+                  "specify the maximal size of the info log file: once the log file is larger "
+                  "than this option, a new info log file will be created; if this option is set "
+                  "to 0, all logs will be written to one log file.");
+DSN_TAG_VARIABLE(rocksdb_max_log_file_size, FT_MUTABLE);

Review Comment:
   > it’s hard to say this config is mutable, since for the already running rocksdb instances, this config is immutable. of course we can find some other way to update these configs, but that’s another matter.
   
   Yes,  actually all of these options are also immutable for rocksdb. The only method that reload these options dynamically, is to update these options by flags firstly for each replica server, then kill partitions and reopen them.
   
   However, if the options of lots of partitions should be reload, instead of killing partitions, it's better to update options in configuration file and then restart the replica servers. Thus if it's necessary, we can launch another PR to update options dynamically.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org