You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by pe...@apache.org on 2005/04/29 22:14:58 UTC
cvs commit: jakarta-tomcat-catalina/webapps/docs cluster-howto.xml
pero 2005/04/29 13:14:58
Modified: webapps/docs cluster-howto.xml
Log:
Not ready, but add some important information about current cluster modul
implementation details.
Revision Changes Path
1.6 +428 -7 jakarta-tomcat-catalina/webapps/docs/cluster-howto.xml
Index: cluster-howto.xml
===================================================================
RCS file: /home/cvs/jakarta-tomcat-catalina/webapps/docs/cluster-howto.xml,v
retrieving revision 1.5
retrieving revision 1.6
diff -u -r1.5 -r1.6
--- cluster-howto.xml 5 Feb 2004 16:05:31 -0000 1.5
+++ cluster-howto.xml 29 Apr 2005 20:14:58 -0000 1.6
@@ -8,6 +8,7 @@
<properties>
<author email="fhanik@apache.org">Filip Hanik</author>
+ <author email="pero@apache.org">Peter Rossbach</author>
<title>Clustering/Session Replication HOW-TO</title>
</properties>
@@ -24,7 +25,9 @@
<li>Uncomment the <code>Valve(ReplicationValve)</code> element in server.xml</li>
<li>If your Tomcat instances are running on the same machine, make sure the <code>tcpListenPort</code>
attribute is unique for each instance.</li>
-<li>Make sure your <code>web.xml</code> has the <code><distributable/></code> element</li>
+<li>Make sure your <code>web.xml</code> has the <code><distributable/></code> element
+ or set at your <code><Context distributable="true" /></code></li>
+<li>Make sure that jvmRoute attribute is set at your Engine <code><Engine name="Catalina" jvmRoute="node1" ></code></li>
</ul>
<p>Load balancing can be achieved through many techniques, as seen in the
<a href="balancer-howto.html">Load Balancing</a> chapter.</p>
@@ -38,8 +41,8 @@
<p>To enable session replication in Tomcat, three different paths can be followed to achieve the exact same thing:</p>
<ol>
- <li>Using session persistence, and saving the session to a shared file system (PersistenceManager)</li>
- <li>Using session persistence, and saving the session to a shared database (JDBCManager)</li>
+ <li>Using session persistence, and saving the session to a shared file system (PersistenceManager + FileStore)</li>
+ <li>Using session persistence, and saving the session to a shared database (PersistenceManager + JDBCStore)</li>
<li>Using in-memory-replication, using the SimpleTcpCluster that ships with Tomcat 5 (server/lib/catalina-cluster.jar)</li>
</ol>
@@ -47,7 +50,9 @@
This is an algorithm that is only efficient when the clusters are small. For large clusters, the next
release will support a primary-secondary session replication where the session will only be stored at one
- or maybe two backup servers.
+ or maybe two backup servers.
+ Currently you can use the domain worker attribute (mod:_jk > 1.2.8) to build cluster partitions
+ with the potential of very scaleable cluster solution.
In order to keep the network traffic down in an all-to-all environment, you can split your cluster
into smaller groups. This can be easily achieved by using different multicast addresses for the different groups.
A very simple setup would look like this:
@@ -66,7 +71,7 @@
<p>What is important to mention here, is that session replication is only the beginning of clustering.
Another popular concept used to implement clusters is farming, ie, you deploy your apps only to one
server, and the cluster will distribute the deployments across the entire cluster.
- This is all capabilities that can go into the next release.</p>
+ This is all capabilities that can go into with the FarmWarDeployer (s. cluster example at <code>server.xml</code>)</p>
<p>In the next section will go deeper into how session replication works and how to configure it.</p>
</section>
@@ -259,20 +264,436 @@
request times, and synchronous replication guarantees the session to be replicated before the request returns.
</p>
<p>
- The parameter "replicationMode" has three different settings: "pooled", "synchronous" and "asynchronous"
+ The parameter "replicationMode" has four different settings: "pooled", "synchronous", "asynchronous" and "fastasyncqueue"
</p>
+
+<section name="Cluster Configuration for ReplicationTransmitter">
+<p>
+List of Attributes<br/>
+<table border="1" cellpadding="5">
+
+ <tr>
+ <th align="center" bgcolor="aqua">Attribute</th>
+ <th align="center" bgcolor="aqua">Description</th>
+ <th align="center" bgcolor="aqua">Default value</th>
+ </tr>
+
+ <tr>
+ <td>replicationMode</td>
+ <td>replication mode (synchronous,pooled.asynchronous,fastasyncqueue)
+ </td>
+ <td><code>pooled</code></td>
+ </tr>
+
+ <tr>
+ <td>processSenderFrequency</td>
+ <td></td>
+ <td><code>2</code></td>
+ </tr>
+
+ <tr>
+ <td>compress</td>
+ <td>compress bytes before sending (consume memory, but reduce network traffic - GZIP)</td>
+ <td><code>false</code></td>
+ </tr>
+
+ <tr>
+ <td>ackTimeout</td>
+ <td>acknowledge timeout</td>
+ <td><code>15000</code></td>
+ </tr>
+
+ <tr>
+ <td>waitForAck</td>
+ <td>Wait for ack after data send</td>
+ <td><code>true</code></td>
+ </tr>
+
+ <tr>
+ <td>autoConnect</td>
+ <td>is sender disabled, fork a new socket</td>
+ <td><code>false</code></td>
+ </tr>
+
+ <tr>
+ <td>doTransmitterProcessingStats</td>
+ <td>create processing time stats</td>
+ <td><code>false</code></td>
+ </tr>
+</table>
+</p>
+<p>
+Example to get statistic information and transfer is compressed<br/>
+<source>
+ <Sender
+ className="org.apache.catalina.cluster.tcp.ReplicationTransmitter"
+ replicationMode="fastasyncqueue"
+ compress="true"
+ doTransmitterProcessingStats="true"
+ ackTimeout="15000"
+ waitForAck="true"
+ autoConnect="false"
+</source>
+</p>
+</section>
+
+<section name="Cluster Configuration for ReplicationTransmitter (fastayncqueue - mode)">
+<p>
+List of Attributes<br/>
+<table border="1" cellpadding="5">
+
+ <tr>
+ <th align="center" bgcolor="aqua">Attribute</th>
+ <th align="center" bgcolor="aqua">Description</th>
+ <th align="center" bgcolor="aqua">Default value</th>
+ </tr>
+
+ <tr>
+ <td>keepAliveTimeout</td>
+ <td>active socket keep alive timeout</td>
+ <td><code>60000</code></td>
+ </tr>
+
+ <tr>
+ <td>keepAliveMaxRequestCount</td>
+ <td>max request over this socket</td>
+ <td><code>-1</code></td>
+ </tr>
+
+ <tr>
+ <td>doProcessingStats</td>
+ <td>create Processing time stats</td>
+ <td><code>false</code></td>
+ </tr>
+
+ <tr>
+ <td>doWaitAckStats</td>
+ <td>create waitAck time stats</td>
+ <td><code>true</code></td>
+ </tr>
+
+ <tr>
+ <td>queueDoStats</td>
+ <td>activated queue stats</td>
+ <td><code>false</code></td>
+ </tr>
+
+ <tr>
+ <td>queueCheckLock</td>
+ <td>check to lost locks</td>
+ <td><code>false</code></td>
+ </tr>
+ <tr>
+ <td>queueAddWaitTimeout</td>
+ <td>queue add wait time (tomcat connector thread waits)</td>
+ <td><code>10000</code></td>
+ </tr>
+ <tr>
+ <td>queueRemoveWaitTimeout</td>
+ <td>queue remove wait time (queue thread waits)</td>
+ <td><code>30000</code></td>
+ </tr>
+
+ <tr>
+ <td>maxQueueLength</td>
+ <td>max queue length (default without limit)</td>
+ <td><code>-1</code></td>
+ </tr>
+
+ <tr>
+ <td>threadPriority</td>
+ <td>change queue thread priority (1-10 ; 5 is normal)</td>
+ <td><code>5</code></td>
+ </tr>
+</table>
+
+</p>
+<p>
+Example to get a lot of statistic information and not wait for ACK<br/>
+<source>
+ <Sender
+ className="org.apache.catalina.cluster.tcp.ReplicationTransmitter"
+ replicationMode="fastasyncqueue"
+ doTransmitterProcessingStats="true"
+ doProcessingStats="true"
+ doWaitAckStats="true"
+ queueTimeWait="true"
+ queueDoStats="true"
+ queueCheckLock="true"
+ waitForAck="false"
+ autoConnect="false"
+ keepAliveTimeout="320000"
+ keepAliveMaxRequestCount="-1"/>
+</source>
+</p>
</section>
+<section name="Cluster Configuration for ReplicationTransmitter ( asynchronous - mode)">
+<p>
+List of Attributes<br/>
+<table border="1" cellpadding="5">
+
+ <tr>
+ <th align="center" bgcolor="aqua">Attribute</th>
+ <th align="center" bgcolor="aqua">Description</th>
+ <th align="center" bgcolor="aqua">Default value</th>
+ </tr>
+
+ <tr>
+ <td>keepAliveTimeout</td>
+ <td>active socket keep alive timeout</td>
+ <td><code>60000</code></td>
+ </tr>
+
+ <tr>
+ <td>keepAliveMaxRequestCount</td>
+ <td>max request over this socket</td>
+ <td><code>-1</code></td>
+ </tr>
+
+ <tr>
+ <td>doProcessingStats</td>
+ <td>create Processing time stats</td>
+ <td><code>false</code></td>
+ </tr>
+
+ <tr>
+ <td>doWaitAckStats</td>
+ <td>create waitAck time stats</td>
+ <td><code>true</code></td>
+ </tr>
+</table>
+</p>
+<p>
+Example to get a processing statistic information and wait for ACK<br/>
+<source>
+ <Sender
+ className="org.apache.catalina.cluster.tcp.ReplicationTransmitter"
+ replicationMode="asynchronous"
+ doProcessingStats="true"
+ doWaitAckStats="true"
+ keepAliveTimeout="320000"
+ keepAliveMaxRequestCount="-1"/>
+</source>
+</p>
+</section>
+
+<section name="Cluster Configuration for ReplicationTransmitter ( synchronous - mode)">
+<p>
+List of Attributes<br/>
+<table border="1" cellpadding="5">
+
+ <tr>
+ <th align="center" bgcolor="aqua">Attribute</th>
+ <th align="center" bgcolor="aqua">Description</th>
+ <th align="center" bgcolor="aqua">Default value</th>
+ </tr>
+
+ <tr>
+ <td>keepAliveTimeout</td>
+ <td>active socket keep alive timeout</td>
+ <td><code>60000</code></td>
+ </tr>
+
+ <tr>
+ <td>keepAliveMaxRequestCount</td>
+ <td>max request over this socket</td>
+ <td><code>-1</code></td>
+ </tr>
+
+ <tr>
+ <td>doProcessingStats</td>
+ <td>create Processing time stats</td>
+ <td><code>false</code></td>
+ </tr>
+
+ <tr>
+ <td>doWaitAckStats</td>
+ <td>create waitAck time stats</td>
+ <td><code>true</code></td>
+ </tr>
+</table>
+
+</p>
+<p>
+Example to get a no processing statistic information, wait for ACK, after 10000 request renew socket and autoconnect before first request is send.<br/>
+<source>
+ <Sender
+ className="org.apache.catalina.cluster.tcp.ReplicationTransmitter"
+ replicationMode="synchronous"
+ autoConnect="true"
+ keepAliveTimeout="-1"
+ keepAliveMaxRequestCount="10000"/>
+</source>
+</p>
+</section>
+
+<section name="Cluster Configuration for ReplicationTransmitter ( pooled - mode)">
+<p>
+List of Attributes<br/>
+<table border="1" cellpadding="5">
+
+ <tr>
+ <th align="center" bgcolor="aqua">Attribute</th>
+ <th align="center" bgcolor="aqua">Description</th>
+ <th align="center" bgcolor="aqua">Default value</th>
+ </tr>
+
+ <tr>
+ <td>keepAliveTimeout</td>
+ <td>active socket keep alive timeout</td>
+ <td><code>60000</code></td>
+ </tr>
+
+ <tr>
+ <td>keepAliveMaxRequestCount</td>
+ <td>max request over this socket</td>
+ <td><code>-1</code></td>
+ </tr>
+
+ <tr>
+ <td>maxPoolSocketLimit</td>
+ <td>max pooled sockets (Sender Sockets)</td>
+ <td><code>25</code></td>
+ </tr>
+</table>
+
+</p>
+<p>
+Example to get a no processing statistic information, wait for ACK, after 10000 request renew socket, only 10 SockerSender available and autoconnect before first request is send.<br/>
+<source>
+ <Sender
+ className="org.apache.catalina.cluster.tcp.ReplicationTransmitter"
+ replicationMode="pooled"
+ autoConnect="true"
+ maxPoolSocketLimit="10"
+ keepAliveTimeout="-1"
+ keepAliveMaxRequestCount="10000"/>
+</source>
+</p>
+</section>
+
+</section>
+
+<section name="Monitoring your Cluster with JMX">
+<p>Monitoring is a very important question when you use a cluster. Some of the cluster objects are JMX MBeans </p>
+<p>Add the following parameter to your startup script with Java 5:
+<source>
+-Dcom.sun.management.jmxremote
+-Dcom.sun.management.jmxremote.port=%my.jmx.port%
+-Dcom.sun.management.jmxremote.ssl=false
+-Dcom.sun.management.jmxremote.authenticate=false
+</source>
+</p>
+<p>Activate JMX with JDK 1.4:
+<ol>
+<li>Install the compat package</li>
+<li>Install the mx4j-tools.jar at common/lib (use the same mx4j version as your tomcat release)</li>
+<li>Configure a MX4J JMX HTTP Adaptor at your AJP Connector<p></p>
+<source>
+<Connector port="${AJP.PORT}" handler.list="mx" mx.enabled="true" mx.httpHost="${JMX.HOST}" mx.httpPort="${JMX.PORT}" protocol="AJP/1.3" />
+</source>
+</li>
+<li>Start your tomcat and look with your browser to http://${JMX.HOST}:${JMX.PORT}</li>
+<li>With the connector parameter <code>mx.authMode="basic" mx.authUser="tomcat" mx.authPassword="strange"</code> you can control the access!</li>
+</ol>
+</p>
+<p>
+List of Cluster Mbeans<br/>
+<table border="1" cellpadding="5">
+
+ <tr>
+ <th align="center" bgcolor="aqua">Name</th>
+ <th align="center" bgcolor="aqua">Description</th>
+ <th align="center" bgcolor="aqua">MBean ObjectName</th>
+ </tr>
+
+ <tr>
+ <td>Cluster</td>
+ <td>The complete cluster element</td>
+ <td><code>type=Cluster,host=${HOST}</code></td>
+ </tr>
+
+ <tr>
+ <td>ClusterSender</td>
+ <td>Configuration and stats of the sender infrastructur</td>
+ <td><code>type=ClusterSender,host=${HOST}</code></td>
+ </tr>
+
+ <tr>
+ <td>ClusterReceiver</td>
+ <td>Configuration and stats of the recevier infrastructur</td>
+ <td><code>type=ClusterReceiver,host=${HOST}</code></td>
+ </tr>
+
+ <tr>
+ <td>IDataSender</td>
+ <td>For every cluster member it exist a sender mbeans.
+ It exists speziall MBeans to all replication modes</td>
+ <td><code>type=IDataSender,host=${HOST},
+ senderAddress=${MEMBER.SENDER.IP},
+ senderPort=${MEMBER.SENDER.PORT}</code></td>
+ </tr>
+
+ <tr>
+ <td>DeltaManager</td>
+ <td>This manager control the sessions and handle session replication </td>
+ <td><code>type=Manager,path=${APP.CONTEXT.PATH},host=${HOST}</code></td>
+ </tr>
+
+ <tr>
+ <td>ReplicationValve</td>
+ <td>This valve control the replication to the backup nodes</td>
+ <td><code>type=Valve,name=ReplicationValve,host=${HOST}</code></td>
+ </tr>
+
+ <tr>
+ <td>JvmRouteBinderValve</td>
+ <td>This is a cluster fallback valve to change the Session ID to the current tomcat jvmroute.</td>
+ <td><code>type=Valve,name=JvmRouteBinderValve,host=${HOST},
+ path=${APP.CONTEXT.PATH}</code></td>
+ </tr>
+
+</table>
+</p>
+</section>
<section name="FAQ">
<p>To be completed once I receive questions about session replication:</p>
<ol>
-<li>Q: What happens when I pull the network cord?<p></p>
+<li>Q: What happens when I pull the network card?<p></p>
A: Well, the other members will remove the instance from the cluster,
but when you insert the cable again, the Tomcat instance might have completely flipped out.
This is because the OS might start going 100% CPU when a multicast message is sent.
There has not yet been a good solution for this, I will let you know when I have come up with one.
+ <p></p>
+ </li>
+<li>Q: How can I activated transparent logging?<p></p>
+
+ A: Use "org.apache.catalina.cluster" as logger category and switch to info, debug or trace as log level.
+ A: Configure the <b>clusterLog</b> attribute (logging category) to get and send and receive message log.
+ <p></p>
+ </li>
+<li>Q: How can I used JMX information to monitor the cluster?<p></p>
+
+ A: Yes, it exists a lot ot usefull information to the cluster as MBeans. With Java 5 you can use the
+ jconsole to look inside the runnnig cluster (s. JMX section above).
+ At fastasyncmode replication mode you can got more information with
+ sender attributes <code>doProcessingStats="true"</code> and <code>queueDoStats="true"</code>.
+ <p></p>
+ </li>
+<li><p></p>Q: Can I pause the message sending?<p></p>
+
+ A: Yes, the async senders can buffer the messages, but make sure the membership ping is active.
+ With fastasyncqueue mode you can limit the max queue size.
+ <p></p>
+ </li>
+<li>Q: Can I at more pooled senders?<p></p>
+
+ A: Yes, with sender attribute <code>maxPoolSocketLimit="40"</code> you can have more than the default
+ <code>25</code> sockets to transfer more parallel messages.
+ <p></p>
</li>
</ol>
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
Re: cvs commit: jakarta-tomcat-catalina/webapps/docs cluster-howto.xml
Posted by Jason Brittain <ja...@brittainweb.org>.
pero@apache.org wrote:
> pero 2005/04/29 13:14:58
>
> Modified: webapps/docs cluster-howto.xml
> Log:
> Not ready, but add some important information about current cluster modul
> implementation details.
>
> Revision Changes Path
> 1.6 +428 -7 jakarta-tomcat-catalina/webapps/docs/cluster-howto.xml
[snip]
> <section name="FAQ">
> <p>To be completed once I receive questions about session replication:</p>
> <ol>
> -<li>Q: What happens when I pull the network cord?<p></p>
> +<li>Q: What happens when I pull the network card?<p></p>
Actually, I think this really is supposed to be "cord", not "card". Maybe
replacing it with the word "cable" would be better?
Cheers.
--
Jason Brittain
---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org