You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@synapse.apache.org by bvalentine1 <bj...@gmail.com> on 2007/09/12 22:52:01 UTC

Load Balancing: Clients wait 60 seconds after 1 of 3 endpoints shut down

Hello,

We want to load balance requests through Synapse but after shutting down 1
of the 3 endpoints, no more requests are recieved by our endpoints for about
60 seconds.

Is this configurable or a symptom of a bad configuration?

Here is the config:

<definitions xmlns="http://ws.apache.org/ns/synapse">

    <sequence name="main" onError="errorHandler">
        <in>
            <send>
                <endpoint>
                    <loadbalance failover="false">
                        <endpoint>
                            <address
uri="http://localhost:7010/wsProvider/webservice/wsProvider">
                               
<suspendDurationOnFailure>30</suspendDurationOnFailure>
				<timeout>
					<duration>3</duration>
					<action>fault</action>
				</timeout>
                            </address>
                        </endpoint>
						<endpoint>
                            <address
uri="http://localhost:7015/wsProvider/webservice/wsProvider">
                               
<suspendDurationOnFailure>30</suspendDurationOnFailure>
				<timeout>
					<duration>3</duration>
					<action>fault</action>
				</timeout>
                            </address>
                        </endpoint>
			<endpoint>
                            <address
uri="http://localhost:7020/wsProvider/webservice/wsProvider">
                               
<suspendDurationOnFailure>30</suspendDurationOnFailure>
				<timeout>
					<duration>3</duration>
					<action>fault</action>
				</timeout>
                            </address>
                        </endpoint>
                    </loadbalance>
                </endpoint>
            </send>
        </in>

        <out>
            <send/>
        </out>
    </sequence>

    <sequence name="errorHandler">

        <makefault>
            <code value="tns:Receiver"
xmlns:tns="http://www.w3.org/2003/05/soap-envelope"/>
            <reason value="COULDN'T SEND THE MESSAGE TO THE SERVER."/>
        </makefault>

        <header name="To" action="remove"/>
        <property name="RESPONSE" value="true"/>

        <send/>
    </sequence>

</definitions>


Thanks for the help



-- 
View this message in context: http://www.nabble.com/Load-Balancing%3A-Clients-wait-60-seconds-after-1-of-3-endpoints-shut-down-tf4431948.html#a12643882
Sent from the Synapse - User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: synapse-user-unsubscribe@ws.apache.org
For additional commands, e-mail: synapse-user-help@ws.apache.org


Re: Load Balancing: Clients wait 60 seconds after 1 of 3 endpoints shut down

Posted by bvalentine1 <bj...@gmail.com>.

What about the timeout configuration?  Is that not relevant?  I set the
duration to 3 thinking that Synapse would give up on a reply after 3
seconds, but this is not happening.  Actually, I haven't noticed any
difference between when I have the timeout elements in my config and when I
don't.  Am I configuring this correctly?  Are my expectations off?

Thanks for the help.  The speedy replies that I have already received have
been very helpful.


Chathura Ekanayake wrote:
> 
> Hi,
> 
> This delay is introduced by the underlying transport used by Synapse. That
> transport takes about
> 60 seconds to report the failure of the target endpoint. Loadbalancer
> takes
> action to remove the failed
> endpoint and send requests to remaining endpoints only after the failure
> is
> reported.
> Currently there is no way to configure the time taken to detect the
> failure
> at the transport level.
> 
> Although this is not related to the problem, you are using
> failover="false"
> attribute in your load balance
> configuration. This will cause lose of messages, sent to the failed
> endpoint, while it is detected as failed.
> If you want to resend such messages to remaining endpoints, use
> failover="true" or remove the failover
> attribute, which defaults to true.
> 
> Thanks,
> Chathura
> 
> 
> 
> On 9/13/07, bvalentine1 <bj...@gmail.com> wrote:
>>
>>
>> Hello,
>>
>> We want to load balance requests through Synapse but after shutting down
>> 1
>> of the 3 endpoints, no more requests are recieved by our endpoints for
>> about
>> 60 seconds.
>>
>> Is this configurable or a symptom of a bad configuration?
>>
>> Here is the config:
>>
>> <definitions xmlns="http://ws.apache.org/ns/synapse">
>>
>>     <sequence name="main" onError="errorHandler">
>>         <in>
>>             <send>
>>                 <endpoint>
>>                     <loadbalance failover="false">
>>                         <endpoint>
>>                             <address
>> uri="http://localhost:7010/wsProvider/webservice/wsProvider">
>>
>> <suspendDurationOnFailure>30</suspendDurationOnFailure>
>>                                 <timeout>
>>                                         <duration>3</duration>
>>                                         <action>fault</action>
>>                                 </timeout>
>>                             </address>
>>                         </endpoint>
>>                                                 <endpoint>
>>                             <address
>> uri="http://localhost:7015/wsProvider/webservice/wsProvider">
>>
>> <suspendDurationOnFailure>30</suspendDurationOnFailure>
>>                                 <timeout>
>>                                         <duration>3</duration>
>>                                         <action>fault</action>
>>                                 </timeout>
>>                             </address>
>>                         </endpoint>
>>                         <endpoint>
>>                             <address
>> uri="http://localhost:7020/wsProvider/webservice/wsProvider">
>>
>> <suspendDurationOnFailure>30</suspendDurationOnFailure>
>>                                 <timeout>
>>                                         <duration>3</duration>
>>                                         <action>fault</action>
>>                                 </timeout>
>>                             </address>
>>                         </endpoint>
>>                     </loadbalance>
>>                 </endpoint>
>>             </send>
>>         </in>
>>
>>         <out>
>>             <send/>
>>         </out>
>>     </sequence>
>>
>>     <sequence name="errorHandler">
>>
>>         <makefault>
>>             <code value="tns:Receiver"
>> xmlns:tns="http://www.w3.org/2003/05/soap-envelope"/>
>>             <reason value="COULDN'T SEND THE MESSAGE TO THE SERVER."/>
>>         </makefault>
>>
>>         <header name="To" action="remove"/>
>>         <property name="RESPONSE" value="true"/>
>>
>>         <send/>
>>     </sequence>
>>
>> </definitions>
>>
>>
>> Thanks for the help
>>
>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Load-Balancing%3A-Clients-wait-60-seconds-after-1-of-3-endpoints-shut-down-tf4431948.html#a12643882
>> Sent from the Synapse - User mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: synapse-user-unsubscribe@ws.apache.org
>> For additional commands, e-mail: synapse-user-help@ws.apache.org
>>
>>
> 
> 

-- 
View this message in context: http://www.nabble.com/Load-Balancing%3A-Clients-wait-60-seconds-after-1-of-3-endpoints-shut-down-tf4431948.html#a12681074
Sent from the Synapse - User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: synapse-user-unsubscribe@ws.apache.org
For additional commands, e-mail: synapse-user-help@ws.apache.org


Re: Load Balancing: Clients wait 60 seconds after 1 of 3 endpoints shut down

Posted by "Asankha C. Perera" <as...@wso2.com>.
Paul

We need to expose two parameters for the NIO sender.. the defaults are 
10secs to establish a connection and 60secs to decide if a socket 
connection timed out. Although exposed, these would apply system wide. 
Is this something we want to include for 1.1?

asankha

See HttpCoreNIOSender::getClientParameters()
....
            .setIntParameter(HttpConnectionParams.SO_TIMEOUT,
                cfg.getProperty(HttpConnectionParams.SO_TIMEOUT, 60000))
            .setIntParameter(HttpConnectionParams.CONNECTION_TIMEOUT,
                cfg.getProperty(HttpConnectionParams.CONNECTION_TIMEOUT, 
10000))


Paul Fremantle wrote:
> Chathura
>
> How do you reconfigure the transport to have a lower wait time?
>
> Paul
>
> On 9/13/07, Chathura Ekanayake <cc...@gmail.com> wrote:
>   
>> Hi,
>>
>> This delay is introduced by the underlying transport used by Synapse. That
>> transport takes about
>> 60 seconds to report the failure of the target endpoint. Loadbalancer takes
>> action to remove the failed
>> endpoint and send requests to remaining endpoints only after the failure is
>> reported.
>> Currently there is no way to configure the time taken to detect the failure
>> at the transport level.
>>
>> Although this is not related to the problem, you are using failover="false"
>> attribute in your load balance
>> configuration. This will cause lose of messages, sent to the failed
>> endpoint, while it is detected as failed.
>> If you want to resend such messages to remaining endpoints, use
>> failover="true" or remove the failover
>> attribute, which defaults to true.
>>
>> Thanks,
>> Chathura
>>
>>
>>
>> On 9/13/07, bvalentine1 <bj...@gmail.com> wrote:
>>     
>>> Hello,
>>>
>>> We want to load balance requests through Synapse but after shutting down 1
>>> of the 3 endpoints, no more requests are recieved by our endpoints for
>>> about
>>> 60 seconds.
>>>
>>> Is this configurable or a symptom of a bad configuration?
>>>
>>> Here is the config:
>>>
>>> <definitions xmlns="http://ws.apache.org/ns/synapse">
>>>
>>>     <sequence name="main" onError="errorHandler">
>>>         <in>
>>>             <send>
>>>                 <endpoint>
>>>                     <loadbalance failover="false">
>>>                         <endpoint>
>>>                             <address
>>> uri="http://localhost:7010/wsProvider/webservice/wsProvider">
>>>
>>> <suspendDurationOnFailure>30</suspendDurationOnFailure>
>>>                                 <timeout>
>>>                                         <duration>3</duration>
>>>                                         <action>fault</action>
>>>                                 </timeout>
>>>                             </address>
>>>                         </endpoint>
>>>                                                 <endpoint>
>>>                             <address
>>> uri="http://localhost:7015/wsProvider/webservice/wsProvider">
>>>
>>> <suspendDurationOnFailure>30</suspendDurationOnFailure>
>>>                                 <timeout>
>>>                                         <duration>3</duration>
>>>                                         <action>fault</action>
>>>                                 </timeout>
>>>                             </address>
>>>                         </endpoint>
>>>                         <endpoint>
>>>                             <address
>>> uri="http://localhost:7020/wsProvider/webservice/wsProvider">
>>>
>>> <suspendDurationOnFailure>30</suspendDurationOnFailure>
>>>                                 <timeout>
>>>                                         <duration>3</duration>
>>>                                         <action>fault</action>
>>>                                 </timeout>
>>>                             </address>
>>>                         </endpoint>
>>>                     </loadbalance>
>>>                 </endpoint>
>>>             </send>
>>>         </in>
>>>
>>>         <out>
>>>             <send/>
>>>         </out>
>>>     </sequence>
>>>
>>>     <sequence name="errorHandler">
>>>
>>>         <makefault>
>>>             <code value="tns:Receiver"
>>> xmlns:tns="http://www.w3.org/2003/05/soap-envelope"/>
>>>             <reason value="COULDN'T SEND THE MESSAGE TO THE SERVER."/>
>>>         </makefault>
>>>
>>>         <header name="To" action="remove"/>
>>>         <property name="RESPONSE" value="true"/>
>>>
>>>         <send/>
>>>     </sequence>
>>>
>>> </definitions>
>>>
>>>
>>> Thanks for the help
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Load-Balancing%3A-Clients-wait-60-seconds-after-1-of-3-endpoints-shut-down-tf4431948.html#a12643882
>>> Sent from the Synapse - User mailing list archive at Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: synapse-user-unsubscribe@ws.apache.org
>>> For additional commands, e-mail: synapse-user-help@ws.apache.org
>>>
>>>
>>>       
>
>
>   

Re: Load Balancing: Clients wait 60 seconds after 1 of 3 endpoints shut down

Posted by Paul Fremantle <pz...@gmail.com>.
Chathura

How do you reconfigure the transport to have a lower wait time?

Paul

On 9/13/07, Chathura Ekanayake <cc...@gmail.com> wrote:
> Hi,
>
> This delay is introduced by the underlying transport used by Synapse. That
> transport takes about
> 60 seconds to report the failure of the target endpoint. Loadbalancer takes
> action to remove the failed
> endpoint and send requests to remaining endpoints only after the failure is
> reported.
> Currently there is no way to configure the time taken to detect the failure
> at the transport level.
>
> Although this is not related to the problem, you are using failover="false"
> attribute in your load balance
> configuration. This will cause lose of messages, sent to the failed
> endpoint, while it is detected as failed.
> If you want to resend such messages to remaining endpoints, use
> failover="true" or remove the failover
> attribute, which defaults to true.
>
> Thanks,
> Chathura
>
>
>
> On 9/13/07, bvalentine1 <bj...@gmail.com> wrote:
> >
> >
> > Hello,
> >
> > We want to load balance requests through Synapse but after shutting down 1
> > of the 3 endpoints, no more requests are recieved by our endpoints for
> > about
> > 60 seconds.
> >
> > Is this configurable or a symptom of a bad configuration?
> >
> > Here is the config:
> >
> > <definitions xmlns="http://ws.apache.org/ns/synapse">
> >
> >     <sequence name="main" onError="errorHandler">
> >         <in>
> >             <send>
> >                 <endpoint>
> >                     <loadbalance failover="false">
> >                         <endpoint>
> >                             <address
> > uri="http://localhost:7010/wsProvider/webservice/wsProvider">
> >
> > <suspendDurationOnFailure>30</suspendDurationOnFailure>
> >                                 <timeout>
> >                                         <duration>3</duration>
> >                                         <action>fault</action>
> >                                 </timeout>
> >                             </address>
> >                         </endpoint>
> >                                                 <endpoint>
> >                             <address
> > uri="http://localhost:7015/wsProvider/webservice/wsProvider">
> >
> > <suspendDurationOnFailure>30</suspendDurationOnFailure>
> >                                 <timeout>
> >                                         <duration>3</duration>
> >                                         <action>fault</action>
> >                                 </timeout>
> >                             </address>
> >                         </endpoint>
> >                         <endpoint>
> >                             <address
> > uri="http://localhost:7020/wsProvider/webservice/wsProvider">
> >
> > <suspendDurationOnFailure>30</suspendDurationOnFailure>
> >                                 <timeout>
> >                                         <duration>3</duration>
> >                                         <action>fault</action>
> >                                 </timeout>
> >                             </address>
> >                         </endpoint>
> >                     </loadbalance>
> >                 </endpoint>
> >             </send>
> >         </in>
> >
> >         <out>
> >             <send/>
> >         </out>
> >     </sequence>
> >
> >     <sequence name="errorHandler">
> >
> >         <makefault>
> >             <code value="tns:Receiver"
> > xmlns:tns="http://www.w3.org/2003/05/soap-envelope"/>
> >             <reason value="COULDN'T SEND THE MESSAGE TO THE SERVER."/>
> >         </makefault>
> >
> >         <header name="To" action="remove"/>
> >         <property name="RESPONSE" value="true"/>
> >
> >         <send/>
> >     </sequence>
> >
> > </definitions>
> >
> >
> > Thanks for the help
> >
> >
> >
> > --
> > View this message in context:
> > http://www.nabble.com/Load-Balancing%3A-Clients-wait-60-seconds-after-1-of-3-endpoints-shut-down-tf4431948.html#a12643882
> > Sent from the Synapse - User mailing list archive at Nabble.com.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: synapse-user-unsubscribe@ws.apache.org
> > For additional commands, e-mail: synapse-user-help@ws.apache.org
> >
> >
>


-- 
Paul Fremantle
Co-Founder and VP of Technical Sales, WSO2
OASIS WS-RX TC Co-chair

blog: http://pzf.fremantle.org
paul@wso2.com

"Oxygenating the Web Service Platform", www.wso2.com

---------------------------------------------------------------------
To unsubscribe, e-mail: synapse-user-unsubscribe@ws.apache.org
For additional commands, e-mail: synapse-user-help@ws.apache.org


Re: Load Balancing: Clients wait 60 seconds after 1 of 3 endpoints shut down

Posted by Chathura Ekanayake <cc...@gmail.com>.
Hi,

This delay is introduced by the underlying transport used by Synapse. That
transport takes about
60 seconds to report the failure of the target endpoint. Loadbalancer takes
action to remove the failed
endpoint and send requests to remaining endpoints only after the failure is
reported.
Currently there is no way to configure the time taken to detect the failure
at the transport level.

Although this is not related to the problem, you are using failover="false"
attribute in your load balance
configuration. This will cause lose of messages, sent to the failed
endpoint, while it is detected as failed.
If you want to resend such messages to remaining endpoints, use
failover="true" or remove the failover
attribute, which defaults to true.

Thanks,
Chathura



On 9/13/07, bvalentine1 <bj...@gmail.com> wrote:
>
>
> Hello,
>
> We want to load balance requests through Synapse but after shutting down 1
> of the 3 endpoints, no more requests are recieved by our endpoints for
> about
> 60 seconds.
>
> Is this configurable or a symptom of a bad configuration?
>
> Here is the config:
>
> <definitions xmlns="http://ws.apache.org/ns/synapse">
>
>     <sequence name="main" onError="errorHandler">
>         <in>
>             <send>
>                 <endpoint>
>                     <loadbalance failover="false">
>                         <endpoint>
>                             <address
> uri="http://localhost:7010/wsProvider/webservice/wsProvider">
>
> <suspendDurationOnFailure>30</suspendDurationOnFailure>
>                                 <timeout>
>                                         <duration>3</duration>
>                                         <action>fault</action>
>                                 </timeout>
>                             </address>
>                         </endpoint>
>                                                 <endpoint>
>                             <address
> uri="http://localhost:7015/wsProvider/webservice/wsProvider">
>
> <suspendDurationOnFailure>30</suspendDurationOnFailure>
>                                 <timeout>
>                                         <duration>3</duration>
>                                         <action>fault</action>
>                                 </timeout>
>                             </address>
>                         </endpoint>
>                         <endpoint>
>                             <address
> uri="http://localhost:7020/wsProvider/webservice/wsProvider">
>
> <suspendDurationOnFailure>30</suspendDurationOnFailure>
>                                 <timeout>
>                                         <duration>3</duration>
>                                         <action>fault</action>
>                                 </timeout>
>                             </address>
>                         </endpoint>
>                     </loadbalance>
>                 </endpoint>
>             </send>
>         </in>
>
>         <out>
>             <send/>
>         </out>
>     </sequence>
>
>     <sequence name="errorHandler">
>
>         <makefault>
>             <code value="tns:Receiver"
> xmlns:tns="http://www.w3.org/2003/05/soap-envelope"/>
>             <reason value="COULDN'T SEND THE MESSAGE TO THE SERVER."/>
>         </makefault>
>
>         <header name="To" action="remove"/>
>         <property name="RESPONSE" value="true"/>
>
>         <send/>
>     </sequence>
>
> </definitions>
>
>
> Thanks for the help
>
>
>
> --
> View this message in context:
> http://www.nabble.com/Load-Balancing%3A-Clients-wait-60-seconds-after-1-of-3-endpoints-shut-down-tf4431948.html#a12643882
> Sent from the Synapse - User mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: synapse-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: synapse-user-help@ws.apache.org
>
>