You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "mingmxu (via GitHub)" <gi...@apache.org> on 2023/09/29 22:29:59 UTC

[GitHub] [hive] mingmxu opened a new pull request, #4765: [WIP][HIVE-27756] expose connectionLifetime in HikariCP

mingmxu opened a new pull request, #4765:
URL: https://github.com/apache/hive/pull/4765

   ### What changes were proposed in this pull request?
   In this PR, it adds a new conf `datanucleus.connectionLifetime` for hive-metastore, which would be helpful for users who use Postgres as the backend database, and connect it via PgBouncer.
   
   
   ### Why are the changes needed?
   As the issue explains in https://issues.apache.org/jira/browse/HIVE-27756, without it users have to set a long lifetime in pgbouncer, which impacts the operations on db side.
   
   ### Does this PR introduce _any_ user-facing change?
   NO, the default behavior is retained. 
   
   ### Is the change a dependency upgrade?
   NO
   
   
   ### How was this patch tested?
   test in our own production clusters
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] [HIVE-27756] expose connectionLifetime in HikariCP [hive]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #4765: [HIVE-27756] expose connectionLifetime in HikariCP
URL: https://github.com/apache/hive/pull/4765


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] [HIVE-27756] expose connectionLifetime in HikariCP [hive]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #4765:
URL: https://github.com/apache/hive/pull/4765#issuecomment-1837656467

   This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
   Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] [HIVE-27756] expose connectionLifetime in HikariCP [hive]

Posted by "dengzhhu653 (via GitHub)" <gi...@apache.org>.
dengzhhu653 commented on PR #4765:
URL: https://github.com/apache/hive/pull/4765#issuecomment-1746857214

   nit: we can add `hikaricp.maxLifetime=x`(x is milliseconds) to the configuration of metastore for the same purpose.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] sonarcloud[bot] commented on pull request #4765: [HIVE-27756] expose connectionLifetime in HikariCP

Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4765:
URL: https://github.com/apache/hive/pull/4765#issuecomment-1741674492

   Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4765)
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4765&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4765&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4765&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4765&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4765&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4765&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4765&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4765&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4765&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4765&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4765&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4765&resolved=false&types=CODE_SMELL)
   
   [![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4765&metric=coverage&view=list) No Coverage information  
   [![No Duplication information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/NoDuplicationInfo-16px.png 'No Duplication information')](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4765&metric=duplicated_lines_density&view=list) No Duplication information
   
   ![warning](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning-16px.png 'warning') The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
   Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/)
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] [HIVE-27756] expose connectionLifetime in HikariCP [hive]

Posted by "zabetak (via GitHub)" <gi...@apache.org>.
zabetak commented on PR #4765:
URL: https://github.com/apache/hive/pull/4765#issuecomment-1746879716

   @dengzhhu653 I think I suggested the opposite :) Can you clarify what you mean?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] [HIVE-27756] expose connectionLifetime in HikariCP [hive]

Posted by "dengzhhu653 (via GitHub)" <gi...@apache.org>.
dengzhhu653 commented on PR #4765:
URL: https://github.com/apache/hive/pull/4765#issuecomment-1746898121

   > @dengzhhu653 I think I suggested the opposite :) Can you clarify what you mean?
   
   Sorry, I mean we don't need to add a new property in MetastoreConf, we can directly configure `hikaricp.maxLifetime=x` in hive-site.xml and restart the HMS to refresh the `maxLifetime` of the pool.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

Re: [PR] [HIVE-27756] expose connectionLifetime in HikariCP [hive]

Posted by "zabetak (via GitHub)" <gi...@apache.org>.
zabetak commented on code in PR #4765:
URL: https://github.com/apache/hive/pull/4765#discussion_r1342402547


##########
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java:
##########
@@ -701,6 +701,9 @@ public enum ConfVars {
         "datanucleus.connectionPoolingType", "HikariCP", new StringSetValidator("DBCP",
         "HikariCP", "NONE"),
         "Specify connection pool library for datanucleus"),
+    CONNECTION_POOLING_LIFETIME("datanucleus.connectionLifetime",
+        "datanucleus.connectionLifetime", 1800,
+        "The maximum lifetime, in seconds, of a connection in the pool, used in HikariCP only."),

Review Comment:
   If it is a Hikari only configuration then it shouldn't be part of `MetastoreConf`. Also it is strange to have this under `datanucleus` namespace if it is not a datanucleus configuration.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org