You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2022/04/15 20:00:43 UTC

[GitHub] [hive] kovjanos opened a new pull request, #3215: HIVE-26145: Disable notification cleaner if interval is zero

kovjanos opened a new pull request, #3215:
URL: https://github.com/apache/hive/pull/3215

   ### What changes were proposed in this pull request?
   Make it possible to disable the db-notification cleaner(s) in an instance by configuring interval to zero, in case if there are multiple instances running 
   
   ### Why are the changes needed?
   In a highly concurrent deployment there can be multiple HMS instances running where it would be needed to run cleaner only in one instance to avoid overhead and/or conflict on the notification tables.
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   ### How was this patch tested?
   ...test is pending


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] pvary commented on a diff in pull request #3215: HIVE-26145: Disable notification cleaner if interval is zero

Posted by GitBox <gi...@apache.org>.
pvary commented on code in PR #3215:
URL: https://github.com/apache/hive/pull/3215#discussion_r857500634


##########
hcatalog/server-extensions/src/main/java/org/apache/hive/hcatalog/listener/DbNotificationListener.java:
##########
@@ -174,12 +174,17 @@ public class DbNotificationListener extends TransactionalMetaStoreEventListener
 
   //cleaner is a static object, use static synchronized to make sure its thread-safe
   private static synchronized void init(Configuration conf) throws MetaException {
-    if (cleaner == null) {
+    long freq = MetastoreConf.getTimeVar(conf, MetastoreConf.ConfVars.EVENT_DB_LISTENER_CLEAN_INTERVAL, TimeUnit.MILLISECONDS);
+    if (cleaner == null && freq > 0) {
       cleaner =
           new CleanerThread(conf, RawStoreProxy.getProxy(conf, conf,
               MetastoreConf.getVar(conf, ConfVars.RAW_STORE_IMPL)));
       cleaner.start();
-    }
+      LOG.info("Scheduling notification log cleanup service with " +

Review Comment:
   Nit: Could we use parametrized logging here instead of concatenating the strings?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] github-actions[bot] closed pull request #3215: HIVE-26145: Disable notification cleaner if interval is zero

Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #3215: HIVE-26145: Disable notification cleaner if interval is zero
URL: https://github.com/apache/hive/pull/3215


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] github-actions[bot] commented on pull request #3215: HIVE-26145: Disable notification cleaner if interval is zero

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #3215:
URL: https://github.com/apache/hive/pull/3215#issuecomment-1166150215

   This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
   Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org


[GitHub] [hive] zabetak commented on a diff in pull request #3215: HIVE-26145: Disable notification cleaner if interval is zero

Posted by GitBox <gi...@apache.org>.
zabetak commented on code in PR #3215:
URL: https://github.com/apache/hive/pull/3215#discussion_r857554725


##########
hcatalog/server-extensions/src/main/java/org/apache/hive/hcatalog/listener/DbNotificationListener.java:
##########
@@ -174,12 +174,17 @@ public class DbNotificationListener extends TransactionalMetaStoreEventListener
 
   //cleaner is a static object, use static synchronized to make sure its thread-safe
   private static synchronized void init(Configuration conf) throws MetaException {
-    if (cleaner == null) {
+    long freq = MetastoreConf.getTimeVar(conf, MetastoreConf.ConfVars.EVENT_DB_LISTENER_CLEAN_INTERVAL, TimeUnit.MILLISECONDS);
+    if (cleaner == null && freq > 0) {
       cleaner =

Review Comment:
   The `cleaner` object is accessed in many places in this class and in order to ensure that we are not going to have a NPE we need to review all calls carefully. For instance, it seems that configuration changes (`onConfigChange`) may touch try to update some aspects of the `cleaner` assuming that the latter one is initialized.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org