[jira] [Updated] (AMBARI-9736) Ambari Exhausts Connection Pool When Using MySQL Due To Invalid Connections

["Jonathan Hurley (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AMBARI-9736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Hurley updated AMBARI-9736:
------------------------------------
    Attachment: AMBARI-9736.patch

> Ambari Exhausts Connection Pool When Using MySQL Due To Invalid Connections
> ---------------------------------------------------------------------------
>
>                 Key: AMBARI-9736
>                 URL: https://issues.apache.org/jira/browse/AMBARI-9736
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 1.7.0, 2.0.0
>            Reporter: Jonathan Hurley
>            Assignee: Jonathan Hurley
>            Priority: Critical
>             Fix For: 1.7.0, 2.0.0
>
>         Attachments: AMBARI-9736.patch
>
>
> When using MySQL, Ambari appears to try to use invalid connections about about 8 hours of use. 
> {noformat}
> Exception EclipseLink-4002 (Eclipse Persistence Services - 2.4.0.v20120608-r11652): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
> The last packet successfully received from the server was 909,270 milliseconds ago. The last packet sent successfully to the server was 0 milliseconds ago.
> Error Code: 0
> Call: SELECT t0.service_config_id, t0.cluster_id, t0.create_timestamp, t0.group_id, t0.note, t0.service_name, t0.user_name, t0.version FROM serviceconfig t0 WHERE ((t0.cluster_id = ?) AND (t0.create_timestamp = (SELECT MAX(t1.create_timestamp) FROM serviceconfig t1 WHERE (((t1.service_name = t0.service_name) AND (t1.cluster_id = ?)) AND (t1.group_id IS NULL)))))
> bind => 2 parameters bound
> Query: ReadAllQuery(referenceClass=ServiceConfigEntity sql="SELECT t0.service_config_id, t0.cluster_id, t0.create_timestamp, t0.group_id, t0.note, t0.service_name, t0.user_name, t0.version FROM serviceconfig t0 WHERE ((t0.cluster_id = ?) AND (t0.create_timestamp = (SELECT MAX(t1.create_timestamp) FROM serviceconfig t1 WHERE (((t1.service_name = t0.service_name) AND (t1.cluster_id = ?)) AND (t1.group_id IS NULL)))))")
> {noformat}
> EclipseLink does not appear to correctly handle connections that have been invalidated by MySQL after the default timeout time of 8 hours. 
> STR:
> - Set MySQL's timeouts via /etc/my.cnf
> {noformat}
> [mysqld]
> interactive_timeout=120
> wait_timeout=120
> {noformat}
> - Verify the connection timeouts are lowered
> {noformat}
> mysql> SHOW VARIABLES LIKE "%timeout%";
> +----------------------------+----------+
> | Variable_name              | Value    |
> +----------------------------+----------+
> | connect_timeout            | 10       |
> | delayed_insert_timeout     | 300      |
> | innodb_lock_wait_timeout   | 50       |
> | innodb_rollback_on_timeout | OFF      |
> | interactive_timeout        | 120      |
> | lock_wait_timeout          | 31536000 |
> | net_read_timeout           | 30       |
> | net_write_timeout          | 60       |
> | slave_net_timeout          | 3600     |
> | wait_timeout               | 120      |
> +----------------------------+----------+
> 10 rows in set (0.00 sec)
> {noformat}
> - Launch Ambari and wait a few minutes. The exceptions will begin and Ambari will stop working properly. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)