You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by David Day <dd...@developerprogram.com> on 2016/08/24 21:22:09 UTC

Create new MySQL connections if read-only status is detected - AWS RDS failover

Apache Tomcat/7.0.54
CentOS release 6.7
AWS RDS Aurora 5.6.10a

Using RDS in a failover configuration, AWS directs requests to the
write instance with a cluster CNAME e.g.
dbname-cluster.cluster-id.us-west-2.rds.amazonaws.com, and the
read-only instance to it's canonical name e.g.
dbname-us-west-2c.id.us-west-2.rds.amazonaws.com.  Connections are
established to the IP address of the writer.  After a failover the
original writer and reader instances change roles, but the application
continues to send traffic to the original writer, which now has become
read-only.  This results in errors and requires Tomcat to be
restarted.

A confirmed fix is to disable DNS cache by setting
networkaddress.cache.ttl=0 in
$JAVA_HOME/jre/lib/security/java.security.  A downside is the loss of
DNS cache for other network calls from the application. I'm looking
for a method to periodically test the state of the database, and if
determined to be read-only, create new connections.  By that time,
failover will have completed and new requests to the CNAME will then
result to the writer instance.

I've read some on jdbcInterceptors and connection state, but nothing
strikes me as a method to test the database is read-only then take
action.  I'd like help if you know of connection settings I can try,
or feedback if disabling DNS cache seems a reasonable solution.

Cheers
David

-- 
Web: http://www.developerprogram.com
2300 Greenhill Drive, Suite 400
Round Rock, Texas, 78664

*"Developer and Partner Programs Done Right"*

NOTICE: This e-mail message is for the sole use of the intended 
recipient(s) and may contain confidential and privileged information. Any 
unauthorized review, use, disclosure or distribution is prohibited. Nothing 
contained in this message or in any attachment shall constitute a contract 
or electronic signature under the Electronic Signatures in Global and 
National Commerce Act, any version of the Uniform Electronic Transactions 
Act or any other statute governing electronic transactions.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Create new MySQL connections if read-only status is detected - AWS RDS failover

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

David,

On 8/24/16 5:22 PM, David Day wrote:
> Apache Tomcat/7.0.54 CentOS release 6.7 AWS RDS Aurora 5.6.10a
> 
> Using RDS in a failover configuration, AWS directs requests to the 
> write instance with a cluster CNAME e.g. 
> dbname-cluster.cluster-id.us-west-2.rds.amazonaws.com, and the 
> read-only instance to it's canonical name e.g. 
> dbname-us-west-2c.id.us-west-2.rds.amazonaws.com.  Connections are 
> established to the IP address of the writer.  After a failover the 
> original writer and reader instances change roles, but the
> application continues to send traffic to the original writer, which
> now has become read-only.  This results in errors and requires
> Tomcat to be restarted.
> 
> A confirmed fix is to disable DNS cache by setting 
> networkaddress.cache.ttl=0 in 
> $JAVA_HOME/jre/lib/security/java.security.  A downside is the loss
> of DNS cache for other network calls from the application. I'm
> looking for a method to periodically test the state of the
> database, and if determined to be read-only, create new
> connections.  By that time, failover will have completed and new
> requests to the CNAME will then result to the writer instance.
> 
> I've read some on jdbcInterceptors and connection state, but
> nothing strikes me as a method to test the database is read-only
> then take action.  I'd like help if you know of connection settings
> I can try, or feedback if disabling DNS cache seems a reasonable
> solution.

How, in general, would you test to see if the database is in read-only
mode?

- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJXv0iLAAoJEBzwKT+lPKRYQ2MQAKDZ4x0h+qSh7G6SmXhk20Wh
+DETPSVrI6ENHwkJIik+qImEZbpQmkvTM34hIz5t6wPOmqlLSI60alLNNZ2w5dd4
dloUQAuFkkOyEc61ELCmo0+ZscEdDTlaStXIvh9kP/nglobdc5D2qEwW21274tNq
bE1mFZCa4rllxwdGRFdjev9m20ekzS1x8K+IOHekGELPWcoDf4ka6+C1H1bfIkxt
VNWQCQ6ALE60/q4RFeRyFII0eu4glR6YryF2QAZFt2ZYbgBzQM7ENH0Z1O1+fdYa
QYevdyguPeeVgxoxVzelLGDn+OEZanH4va3TyXWPa6lT//gvPNph+vZGIO9+Y+eF
vpZs6LX39ILY5toLpmi5qz+ZL6Q0TsOPOKetDqItuMbPp50tqGYmag/qVdHidnqR
ynIeha1aTUxkKeAlujjYtcYKraJoc7ESxeDfISuNhT3Lo7TlztLZgDBOlcCGHWSK
b0jMpto7cjsk9JvaKwz5tI6TBsiDCC0s6c7R7i130pm54jX0vMCukzTSD78Z4ClS
PyTwaXNUZvjIWcTWPpSToiB9gqsrn1Zw1D/SeDGczvFNqR4fucPnMounUf4Luru0
QdfQ5vYxFf2ePwohXmpScu3z5D7B9NQDkuGecxVNlOx1wX84xxJdzFWkzU71cgdd
R/6N+ZiugFroPNIt20/0
=/FXe
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org