You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2021/11/17 06:08:42 UTC
[GitHub] [hive] pkumarsinha commented on a change in pull request #2789: HIVE-25700. Prevent deletion of Notification Events post restarts.
pkumarsinha commented on a change in pull request #2789:
URL: https://github.com/apache/hive/pull/2789#discussion_r750902924
##########
File path: hcatalog/server-extensions/src/main/java/org/apache/hive/hcatalog/listener/DbNotificationListener.java
##########
@@ -1417,10 +1426,27 @@ private void process(NotificationEvent event, ListenerEvent listenerEvent) throw
setTimeToLive(MetastoreConf.getTimeVar(conf, ttlConf, TimeUnit.SECONDS));
setCleanupInterval(
MetastoreConf.getTimeVar(conf, ConfVars.EVENT_DB_LISTENER_CLEAN_INTERVAL, TimeUnit.MILLISECONDS));
+ setWaitInterval(MetastoreConf
+ .getTimeVar(conf, EVENT_DB_LISTENER_CLEAN_STARTUP_WAIT_INTERVAL, TimeUnit.MILLISECONDS));
}
@Override
public void run() {
+ LOG.info("Wait interval is {}", waitInterval);
+ if (waitInterval > 0) {
+ try {
+ LOG.info("Cleaner Thread Restarted and {} or {} is configured. So cleaner thread will startup post waiting "
+ + "{} ms", EVENT_DB_LISTENER_CLEAN_STARTUP_WAIT_INTERVAL,
+ EVENT_DB_LISTENER_CLEAN_STARTUP_WAIT_INTERVAL.getHiveName(), waitInterval);
+ Thread.sleep(waitInterval);
+ } catch (InterruptedException e) {
+ LOG.error("Failed during the initial wait before start.", e);
+ Thread.currentThread().interrupt();
Review comment:
Also, Interruption to this thread can cause event entry accumulation in the back-end DB. Rather, should we make this thread immune to interruption? Is this only expected to happen during the test run?
##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
##########
@@ -752,6 +752,9 @@ public static ConfVars getMetaConf(String name) {
EVENT_DB_LISTENER_CLEAN_INTERVAL("metastore.event.db.listener.clean.interval",
"hive.metastore.event.db.listener.clean.interval", 7200, TimeUnit.SECONDS,
"sleep interval between each run for cleanup of events from the database listener queue"),
+ EVENT_DB_LISTENER_CLEAN_STARTUP_WAIT_INTERVAL("metastore.event.db.listener.clean.startup.wait.interval",
+ "hive.metastore.event.db.listener.clean.startup.wait.interval", 1, TimeUnit.DAYS,
+ "wait interval post start of Cleaner thread after which the cleaner thread starts to work"),
Review comment:
a) Capitalize "wait".
b) Is there a difference between Cleaner thread start time vs HMS process uptime(not considering time taken to launch a thread which idea would be very minimal)? If not, then referring to process start time would be better to convey the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org