You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2021/12/07 08:53:58 UTC
[GitHub] [hive] klcopp commented on a change in pull request #2827: Hive 25737 fix cycle metrics
klcopp commented on a change in pull request #2827:
URL: https://github.com/apache/hive/pull/2827#discussion_r763128180
##########
File path: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
##########
@@ -3190,6 +3190,24 @@ private static void populateLlapDaemonVarsSet(Set<String> llapDaemonVarsSetLocal
"has had a transaction done on it since the last major compaction. So decreasing this\n" +
"value will increase the load on the NameNode."),
+ HIVE_COMPACTOR_INITIATOR_DURATION_UPDATE_INTERVAL("hive.compactor.initiator.duration.update.interval", "1s",
Review comment:
I think the default update interval should be at least 10s, but probably more like 60s. There should be a balance of keeping users updated without burdening the HMS
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
##########
@@ -494,6 +494,18 @@ public static ConfVars getMetaConf(String name) {
12, TimeUnit.HOURS,
"Age of oldest initiated compaction in the compaction queue after which an error will be logged. " +
"Default time unit is: hours"),
+ COMPACTOR_LONG_RUNNING_INITIATOR_THRESHOLD_WARNING(
+ "metastore.compactor.long.running.initiator.threshold.warning",
+ "hive.compactor.long.running.initiator.threshold.warning",
+ 6, TimeUnit.HOURS,
+ "Initiator duration after which a warning will be logged. " +
+ "Default time unit is: hours"),
+ COMPACTOR_LONG_RUNNING_INITIATOR_THRESHOLD_ERROR(
+ "metastore.compactor.long.running.initiator.threshold.error",
+ "hive.compactor.long.running.initiator.threshold.error",
+ 12, TimeUnit.HOURS,
+ "Initiator duration after which an wrror will be logged. " +
Review comment:
nit: spelling
##########
File path: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
##########
@@ -3190,6 +3190,24 @@ private static void populateLlapDaemonVarsSet(Set<String> llapDaemonVarsSetLocal
"has had a transaction done on it since the last major compaction. So decreasing this\n" +
"value will increase the load on the NameNode."),
+ HIVE_COMPACTOR_INITIATOR_DURATION_UPDATE_INTERVAL("hive.compactor.initiator.duration.update.interval", "1s",
+ new TimeValidator(TimeUnit.SECONDS),
+ "Time in seconds that drives the update interval of compaction_initiator_duration metric.\n" +
+ "Smaller value results in a fine grained metric update.\n" +
+ "This updater can be turned off if its value less than or equals to zero.\n"+
+ "In this case the above metric will be update only after the initiator completed one cycle.\n" +
+ "The hive.compactor.initiator.on must be turned on (true) in-order to enable the Initiator,\n" +
+ "otherwise this setting has no effect."),
+
+ HIVE_COMPACTOR_CLEANER_DURATION_UPDATE_INTERVAL("hive.compactor.cleaner.duration.update.interval", "1s",
+ new TimeValidator(TimeUnit.SECONDS),
+ "Time in seconds that drives the update interval of compaction_cleaner_duration metric.\n" +
+ "Smaller value results in a fine grained metric update.\n" +
+ "This updater can be turned off if its value less than or equals to zero.\n"+
+ "In this case the above metric will be update only after the initiator completed one cycle.\n" +
Review comment:
nit: Initiator is mentioned here and in the lines below.
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
##########
@@ -494,6 +494,18 @@ public static ConfVars getMetaConf(String name) {
12, TimeUnit.HOURS,
"Age of oldest initiated compaction in the compaction queue after which an error will be logged. " +
"Default time unit is: hours"),
+ COMPACTOR_LONG_RUNNING_INITIATOR_THRESHOLD_WARNING(
+ "metastore.compactor.long.running.initiator.threshold.warning",
+ "hive.compactor.long.running.initiator.threshold.warning",
+ 6, TimeUnit.HOURS,
+ "Initiator duration after which a warning will be logged. " +
Review comment:
I would say, "Initiator cycle duration" here just to be clear that it's the whole cycle we're talking about
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org