[GitHub] [hive] deniskuzZ commented on a diff in pull request #3466: HIVE-26419: Use a different pool for DataNucleus' secondary connectio…

[GitBox <gi...@apache.org>]

deniskuzZ commented on code in PR #3466:
URL: https://github.com/apache/hive/pull/3466#discussion_r927895735


##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/PersistenceManagerProvider.java:
##########
@@ -253,8 +253,12 @@ private static PersistenceManagerFactory initPMF(Configuration conf, boolean for
     } else {
       try {
         DataSource ds = (maxPoolSize > 0) ? dsp.create(conf, maxPoolSize) : dsp.create(conf);
+        // The secondary connection factory is used for schema generation, and for value generation operations.
+        // We should use a different pool for the secondary connection factory to avoid resource starvation.
+        // Since DataNucleus uses locks for schema generation and value generation, 2 connections should be sufficient.
+        DataSource ds2 = dsp.create(conf, /* maxPoolSize */ 2);

Review Comment:
   According to the documentation:
   Datastore connections are obtained from up to 2 connection factories. The primary connection factory is used for persistence operations, and optionally for value generation operations. The secondary connection factory is used for schema generation, and optionally for value generation operations.
   Schema generation happens at HMS startup. 
   Where do you see a potential issue with resource starvation?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org