overcoming a message-size limitation in tribes parallel messaging with NioSender

[Vince Stewart <st...@gmail.com>]

hello all,

I am using tribes (1.7.37) for proprietary database replication over remote
connection.
Both sides of the connection run tomcat-embedded Java applications on
Ubuntu.

My choice of tribes for this purpose was encouraged by the assertion
"Tribes takes out the complexity of messaging from the replication module
and becomes a fully independent and highly flexible group communication
module." from the article at
http://tomcat.apache.org/tomcat-6.0-doc/tribes/introduction.html

My database replication system works fine till a user saves an image bigger
than about half megabyte at which time the process fails.
If sending asynchronously the sending thread dies a silent death but when
operating in SEND_OPTIONS_USE_ACK mode an error is thrown referring to
"timeout" issues. I noticed an earlier article reporting a volume-related
timeout problem at:
http://tomcat.10.n6.nabble.com/org-apache-catalina-tribes-ChannelException-Operation-has-timed-out-3000-ms-Faulty-members-tcp-64-88-td4656393.html

The time-out is detected by the ParallelNioSender.sendMessage method which
gives a time limit of 3 seconds for all members of an array of NioSender
objects to finish sending. The "timeout" parameter is declared in
AbstractSender class as (long) 3000. I can not see any configuration
parameter to adjust that value nor any reason to do so; in the rapid-fire
LAN clustering environment such a time limit should not be a problem.

However the time-out detection does not care if a sender object is late to
complete simply because the data for transmission is large or the speed of
transmission is slower than usual; if they cant keep up, they're gone.

I have experimentally added two lines of code to
ParallelNioSender.sendMessage which allows multi-megabyte messages to be
sent at Internet connection speed - so far without problems.
A more optimal approach would additionally generate an errors for any
unacceptably slow connection (a functionality made unnecessary by the
prevailing time out detection system).

I do not reproduce the method code in its entirety here; two lines of new
code are proposed:

    @Override
    public synchronized void sendMessage(Member[] destination,
ChannelMessage msg) throws ChannelException {
-
-
            while ( (remaining>0) && (delta<getTimeout()) ) {
                try {
                remaining -= doLoop(selectTimeout,
getMaxRetryAttempts(),waitForAck,msg);
                // vs // for each non-completed sender, doLoop calls
nioSenderObject.process which calls nioSenderObject.write which writes a
single packet of data
                // vs // those two methods determine if the NioSender
object has finished sending its data and doLoop returns the number of
senders completed during that particular invocation
                // vs // this while-loop might iterate just once if all
sender object data lengths are smaller than the tcp packet size (quite
often the case in my experience).
                }catch (Exception x ) {
                // vs // gets here if at least one connection is not
healthy ..... adds faulty connection message to a ChannelError object to be
thrown after other senders are complete
                // vs // or if all remaining non-complete senders are
faulty the ChannelError object is thrown immediately
                }
-
-               if((Channel.SEND_OPTIONS_IGNORE_TIMEOUT_FOR_HEALTHY_SOCKETS
& msg.getOptions()) == 0){               ///////////////////// new code

                                      delta = System.currentTimeMillis() -
start;
                 }

                                   ////////////////////// new code

            }///end of while block
-
-
  }

plus in Channel interface:
Channel.SEND_OPTIONS_IGNORE_TIMEOUT_FOR_HEALTHY_SOCKETS = 0x0080

the above change will do nothing at all unless the user has manipulated the
message options (int) appropriately:
aTribesAwareObject.aChannelObject.send ( membersArray , byteMessage ,
Channel.SEND_OPTIONS_WHATEVER |
Channel.SEND_OPTIONS_IGNORE_TIMEOUT_FOR_HEALTHY_SOCKETS);

any support for this proposal ?
or
any suggestion to circumvent the problem using alternative functionality
-- 
many thanks, Vince Stewart

Re: overcoming a message-size limitation in tribes parallel messaging with NioSender

[Christopher Schultz <ch...@christopherschultz.net>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Vince,

On 3/5/13 8:15 PM, Vince Stewart wrote:
> My database replication system works fine till a user saves an
> image bigger than about half megabyte at which time the process
> fails.
> 
> [snip]
> 
> However the time-out detection does not care if a sender object is
> late to complete simply because the data for transmission is large
> or the speed of transmission is slower than usual; if they cant
> keep up, they're gone.
> 
> I have experimentally added two lines of code to 
> ParallelNioSender.sendMessage which allows multi-megabyte messages
> to be sent at Internet connection speed - so far without problems.
> 
> [snip]
> 
> any support for this proposal ? or any suggestion to circumvent the
> problem using alternative functionality

You should log a bug in bugzilla and attach your patch in diff -U
format. If this really is a "bug", then mark it as such. Otherwise,
mark it as an "enhancement". Bonus points for the behavior being
configurable if appropriate.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEAREIAAYFAlE22T0ACgkQ9CaO5/Lv0PC6bACeKWk8nWftwD4CiJTTy3DfQMtw
1vAAni2O1qdxYbGP+oPpzDxwbf4CocCB
=Fheo
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org