You are viewing a plain text version of this content. The canonical link for it is here.
Posted to fx-dev@ws.apache.org by Gilbert Pilz <Gi...@bea.com> on 2006/10/03 21:49:38 UTC
RE: Sandesha 1.0; fix for clients hanging in SandeshaContext.endSequence()
There is a race condition for outbound sequences between the
SenderWorker thread transmitting the TerminateSequence message and the
endSequence() call stopping that thread. It's difficult to figure out
what a good wait value might be because, depending upon how many
messages are queued up, the length of time you might need to wait for
the TerminateSequence to be sent could be arbitrarily long.
I've changed the code to log an INFO-level message when an incomplete
sequence is ignored. If ignoring incomplete sequences causes people
problems then they shouldn't do that.
- gp
> -----Original Message-----
> From: Jaliya Ekanayake [mailto:jnekanayake@gmail.com]
> Sent: Friday, September 29, 2006 8:06 PM
> To: Gilbert Pilz; sandesha-dev@ws.apache.org
> Subject: Re: Sandesha 1.0; fix for clients hanging in
> SandeshaContext.endSequence()
>
> Sure, I have one clarification.
>
> What not allow some time-out before we cleanup. The reason
> for this is as follows; In the one-way messaging case
> SandeshaContext.endSequence() is called just after the
> service requests so if someone use END_SEQUENCE_IGNORE_ACTIVE
> then the sequences will immediately be terminated. It is
> better to set some timestamp to wait.
>
> In addition can you please send the patch as an attachment?
>
> Thanks,
> -Jaliya
>
> ----- Original Message -----
> From: "Gilbert Pilz" <Gi...@bea.com>
> To: <sa...@ws.apache.org>
> Sent: Friday, September 29, 2006 7:14 PM
> Subject: Sandesha 1.0; fix for clients hanging in
> SandeshaContext.endSequence()
>
>
> The story so far: the SandeshaContext.endSequence() method
> waits for all the sequences related to a SandeshaContext to
> "complete" before it returns. Because some implementations of
> WS-RM (200502) do not automatically terminate their offered
> sequence when the sequence that carried that offer is
> terminated (did you get that?), this may cause a Sandesha
> client to hang until the sequence's inactivity timeout expires.
> At this point the client will see an AxisFault that they may
> or may not understand to be benign.
>
> My plan was to fix this by adding a property to the
> SandeshaContext called "EndSeqBehavior". The default value
> for this behavior, "END_SEQUENCE_BLOCK_ON_ACTIVE", causes
> endSequence() to behave exactly as it does today (block on
> any active sequences until they either complete or time out
> due to inactivity). Another value for this behavior,
> "END_SEQUENCE_IGNORE_ACTIVE", causes endSequence() to simply
> ignore any active sequences and clean up. This behavior is ok
> if all you are interested in doing is cleaning up quickly,
> but it could make things difficult for the peers of the
> sequences that are more or less orphaned.
> If they tried to send messages using those sequences they
> would retry and retry and retry until they reached some
> internal limit that told them to stop. This seems rather
> wasteful given the fact that Sandesha knew it was going to
> abandon the sequence, so I thought there should be one more
> value for the EndSeqBehavior property called
> "END_SEQUENCE_FAULT_ACTIVE". This value would cause
> endSequence() to send an unsolicited wsrm:SequenceTerminated
> fault to the sequence peer of any active sequences.
>
> Unfortunately the last value has proven much harder to
> implement than I expected so I thought it would be a good
> idea to at least get the changes for
> END_SEQUENCE_IGNORE_ACTIVE into the tree. Below are two patch
> files; one for org/apache/sandesha/Constants.java and one for
> org/apache/sandesha/SandeshaContext.java.
>
> Jaliya, if you could look these over and get them into the
> tree, I would appreciate that very much. Note that clients
> that are unaware of the EndSeqBehavior property will continue
> to work as they do today w/out recompilation etc. Clients
> that wish to avoid the hanging problem described above simply
> have to call the new
> SandeshaContext.setEndSeqBehavior() method with the
> appropriate value (as defined in Constants).
>
> - gp
>
> Index: src/org/apache/sandesha/Constants.java
> ===================================================================
> --- src/org/apache/sandesha/Constants.java (revision 451449)
> +++ src/org/apache/sandesha/Constants.java (working copy)
> @@ -72,6 +72,10 @@
>
> int SYNCHRONOUS = 0;
>
> + // Flags for controlling the behavior of endSequence()
> in the face
> of active sequences.
> + int END_SEQUENCE_BLOCK_ON_ACTIVE = 0; // default
> + int END_SEQUENCE_IGNORE_ACTIVE = 1; // ignore any active
> seqs
> +
> public interface WSA {
> String NS_ADDRESSING_ANONYMOUS =
> AddressingUtils.getAnonymousRoleURI();
> }
>
> ----------------------------------------
>
> Index: src/org/apache/sandesha/SandeshaContext.java
> ===================================================================
> --- src/org/apache/sandesha/SandeshaContext.java (revision
> 451449)
> +++ src/org/apache/sandesha/SandeshaContext.java (working copy)
> @@ -76,6 +76,7 @@
> private boolean sendOffer;
> private long messageNumber;
> private boolean sync;
> + private int endSeqActiveBehavior;
>
> static int clientListnerPort = -1;
> public final int MIN_PORT = 0;
> @@ -155,6 +156,14 @@
> this.toURL = toURL;
> }
>
> + public int getEndSeqBehavior() {
> + return endSeqActiveBehavior;
> + }
> +
> + public void setEndSeqBehavior(int flag) {
> + this.endSeqActiveBehavior = flag;
> + }
> +
> public SandeshaContext() throws AxisFault {
> initSandeshaAsync();
> }
> @@ -331,33 +340,38 @@
> public final RMReport endSequence() throws AxisFault {
>
> IStorageManager storageManager = new ClientStorageManager();
> - long startingTime = System.currentTimeMillis();
> - long inactivityTimeOut =
> PolicyLoader.getInstance().getInactivityTimeout();
>
> - Iterator ite = callMap.keySet().iterator();
> + if (endSeqActiveBehavior !=
> Constants.END_SEQUENCE_IGNORE_ACTIVE)
> + {
> + long startingTime = System.currentTimeMillis();
> + long inactivityTimeOut =
> PolicyLoader.getInstance().getInactivityTimeout();
>
> - while (ite.hasNext()) {
> - String key = (String) ite.next();
> - Call tempCall = (Call) callMap.get(key);
> - String seqId = (String)
> tempCall.getProperty(Constants.ClientProperties.CALL_KEY);
> - while (!storageManager.isSequenceComplete(seqId)) {
> - try {
> - if (log.isDebugEnabled()) {
> -
> log.debug(Constants.InfomationMessage.WAITING_TO_STOP_CLIENT);
> + Iterator ite = callMap.keySet().iterator();
> +
> + while (ite.hasNext()) {
> + String key = (String) ite.next();
> + Call tempCall = (Call) callMap.get(key);
> + String seqId = (String)
> tempCall.getProperty(Constants.ClientProperties.CALL_KEY);
> + while (!storageManager.isSequenceComplete(seqId)) {
> + try {
> + if (log.isDebugEnabled()) {
> +
> log.debug(Constants.InfomationMessage.WAITING_TO_STOP_CLIENT);
> + }
> +
> Thread.sleep(Constants.CLIENT_WAIT_PERIOD_FOR_COMPLETE);
> + if ((System.currentTimeMillis() -
> startingTime)
> >= inactivityTimeOut) {
> + stopClientByForce();
> + this.report.setError("Inactivity Time Out
> Reached. Sequence not complete");
> + }
> + } catch (InterruptedException e) {
> + log.error("",e);
> }
> -
> Thread.sleep(Constants.CLIENT_WAIT_PERIOD_FOR_COMPLETE);
> - if ((System.currentTimeMillis() -
> startingTime) >=
> inactivityTimeOut) {
> - stopClientByForce();
> - this.report.setError("Inactivity Time Out
> Reached. Sequence not complete");
> - }
> - } catch (InterruptedException e) {
> - log.error("",e);
> }
> }
> }
> + // else - ignore any sequence that are still active
>
> if (this.report.getError() == null) {
> - this.report.setAllAcked(true);
> + this.report.setAllAcked(true);
> }
>
> seqMap.remove(key);
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: sandesha-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: sandesha-dev-help@ws.apache.org
>
>
RE: Sandesha 1.0; fix for clients hanging in SandeshaContext.endSequence()
Posted by Gilbert Pilz <Gi...@bea.com>.
Attached are patches against 452927 . . .
- gp
> -----Original Message-----
> From: Jaliya Ekanayake [mailto:jnekanayake@gmail.com]
> Sent: Tuesday, October 03, 2006 6:43 PM
> To: Gilbert Pilz; sandesha-dev@ws.apache.org
> Subject: Re: Sandesha 1.0; fix for clients hanging in
> SandeshaContext.endSequence()
>
> Hi Gilbert,
>
> Thanks for the patch. I tried to apply it but the version is
> not matching the current svn tree.
> Please update the code and creat the patch.
>
> -Jaliya
>
>
> ----- Original Message -----
> From: "Gilbert Pilz" <Gi...@bea.com>
> To: "Jaliya Ekanayake" <ja...@apache.org>;
> <sa...@ws.apache.org>
> Sent: Tuesday, October 03, 2006 3:49 PM
> Subject: RE: Sandesha 1.0; fix for clients hanging in
> SandeshaContext.endSequence()
>
>
> There is a race condition for outbound sequences between the
> SenderWorker thread transmitting the TerminateSequence message and the
> endSequence() call stopping that thread. It's difficult to
> figure out what a good wait value might be because, depending
> upon how many messages are queued up, the length of time you
> might need to wait for the TerminateSequence to be sent could
> be arbitrarily long.
>
> I've changed the code to log an INFO-level message when an
> incomplete sequence is ignored. If ignoring incomplete
> sequences causes people problems then they shouldn't do that.
>
> - gp
>
>
> > -----Original Message-----
> > From: Jaliya Ekanayake [mailto:jnekanayake@gmail.com]
> > Sent: Friday, September 29, 2006 8:06 PM
> > To: Gilbert Pilz; sandesha-dev@ws.apache.org
> > Subject: Re: Sandesha 1.0; fix for clients hanging in
> > SandeshaContext.endSequence()
> >
> > Sure, I have one clarification.
> >
> > What not allow some time-out before we cleanup. The reason
> for this is
> > as follows; In the one-way messaging case
> > SandeshaContext.endSequence() is called just after the service
> > requests so if someone use END_SEQUENCE_IGNORE_ACTIVE then the
> > sequences will immediately be terminated. It is better to set some
> > timestamp to wait.
> >
> > In addition can you please send the patch as an attachment?
> >
> > Thanks,
> > -Jaliya
> >
> > ----- Original Message -----
> > From: "Gilbert Pilz" <Gi...@bea.com>
> > To: <sa...@ws.apache.org>
> > Sent: Friday, September 29, 2006 7:14 PM
> > Subject: Sandesha 1.0; fix for clients hanging in
> > SandeshaContext.endSequence()
> >
> >
> > The story so far: the SandeshaContext.endSequence() method
> waits for
> > all the sequences related to a SandeshaContext to
> "complete" before it
> > returns. Because some implementations of WS-RM (200502) do not
> > automatically terminate their offered sequence when the
> sequence that
> > carried that offer is terminated (did you get that?), this
> may cause a
> > Sandesha client to hang until the sequence's inactivity timeout
> > expires.
> > At this point the client will see an AxisFault that they may or may
> > not understand to be benign.
> >
> > My plan was to fix this by adding a property to the SandeshaContext
> > called "EndSeqBehavior". The default value for this behavior,
> > "END_SEQUENCE_BLOCK_ON_ACTIVE", causes
> > endSequence() to behave exactly as it does today (block on
> any active
> > sequences until they either complete or time out due to
> inactivity).
> > Another value for this behavior,
> "END_SEQUENCE_IGNORE_ACTIVE", causes
> > endSequence() to simply ignore any active sequences and
> clean up. This
> > behavior is ok if all you are interested in doing is cleaning up
> > quickly, but it could make things difficult for the peers of the
> > sequences that are more or less orphaned.
> > If they tried to send messages using those sequences they
> would retry
> > and retry and retry until they reached some internal limit
> that told
> > them to stop. This seems rather wasteful given the fact
> that Sandesha
> > knew it was going to abandon the sequence, so I thought
> there should
> > be one more value for the EndSeqBehavior property called
> > "END_SEQUENCE_FAULT_ACTIVE". This value would cause
> > endSequence() to send an unsolicited
> wsrm:SequenceTerminated fault to
> > the sequence peer of any active sequences.
> >
> > Unfortunately the last value has proven much harder to
> implement than
> > I expected so I thought it would be a good idea to at least get the
> > changes for END_SEQUENCE_IGNORE_ACTIVE into the tree. Below are two
> > patch files; one for org/apache/sandesha/Constants.java and one for
> > org/apache/sandesha/SandeshaContext.java.
> >
> > Jaliya, if you could look these over and get them into the tree, I
> > would appreciate that very much. Note that clients that are
> unaware of
> > the EndSeqBehavior property will continue to work as they do today
> > w/out recompilation etc. Clients that wish to avoid the hanging
> > problem described above simply have to call the new
> > SandeshaContext.setEndSeqBehavior() method with the
> appropriate value
> > (as defined in Constants).
> >
> > - gp
> >
> > Index: src/org/apache/sandesha/Constants.java
> > ===================================================================
> > --- src/org/apache/sandesha/Constants.java (revision 451449)
> > +++ src/org/apache/sandesha/Constants.java (working copy)
> > @@ -72,6 +72,10 @@
> >
> > int SYNCHRONOUS = 0;
> >
> > + // Flags for controlling the behavior of endSequence()
> > in the face
> > of active sequences.
> > + int END_SEQUENCE_BLOCK_ON_ACTIVE = 0; // default
> > + int END_SEQUENCE_IGNORE_ACTIVE = 1; // ignore
> any active
> > seqs
> > +
> > public interface WSA {
> > String NS_ADDRESSING_ANONYMOUS =
> > AddressingUtils.getAnonymousRoleURI();
> > }
> >
> > ----------------------------------------
> >
> > Index: src/org/apache/sandesha/SandeshaContext.java
> > ===================================================================
> > --- src/org/apache/sandesha/SandeshaContext.java (revision
> > 451449)
> > +++ src/org/apache/sandesha/SandeshaContext.java (working copy)
> > @@ -76,6 +76,7 @@
> > private boolean sendOffer;
> > private long messageNumber;
> > private boolean sync;
> > + private int endSeqActiveBehavior;
> >
> > static int clientListnerPort = -1;
> > public final int MIN_PORT = 0;
> > @@ -155,6 +156,14 @@
> > this.toURL = toURL;
> > }
> >
> > + public int getEndSeqBehavior() {
> > + return endSeqActiveBehavior;
> > + }
> > +
> > + public void setEndSeqBehavior(int flag) {
> > + this.endSeqActiveBehavior = flag;
> > + }
> > +
> > public SandeshaContext() throws AxisFault {
> > initSandeshaAsync();
> > }
> > @@ -331,33 +340,38 @@
> > public final RMReport endSequence() throws AxisFault {
> >
> > IStorageManager storageManager = new
> ClientStorageManager();
> > - long startingTime = System.currentTimeMillis();
> > - long inactivityTimeOut =
> > PolicyLoader.getInstance().getInactivityTimeout();
> >
> > - Iterator ite = callMap.keySet().iterator();
> > + if (endSeqActiveBehavior !=
> > Constants.END_SEQUENCE_IGNORE_ACTIVE)
> > + {
> > + long startingTime = System.currentTimeMillis();
> > + long inactivityTimeOut =
> > PolicyLoader.getInstance().getInactivityTimeout();
> >
> > - while (ite.hasNext()) {
> > - String key = (String) ite.next();
> > - Call tempCall = (Call) callMap.get(key);
> > - String seqId = (String)
> > tempCall.getProperty(Constants.ClientProperties.CALL_KEY);
> > - while (!storageManager.isSequenceComplete(seqId)) {
> > - try {
> > - if (log.isDebugEnabled()) {
> > -
> > log.debug(Constants.InfomationMessage.WAITING_TO_STOP_CLIENT);
> > + Iterator ite = callMap.keySet().iterator();
> > +
> > + while (ite.hasNext()) {
> > + String key = (String) ite.next();
> > + Call tempCall = (Call) callMap.get(key);
> > + String seqId = (String)
> > tempCall.getProperty(Constants.ClientProperties.CALL_KEY);
> > + while (!storageManager.isSequenceComplete(seqId)) {
> > + try {
> > + if (log.isDebugEnabled()) {
> > +
> > log.debug(Constants.InfomationMessage.WAITING_TO_STOP_CLIENT);
> > + }
> > +
> > Thread.sleep(Constants.CLIENT_WAIT_PERIOD_FOR_COMPLETE);
> > + if ((System.currentTimeMillis() -
> > startingTime)
> > >= inactivityTimeOut) {
> > + stopClientByForce();
> > +
> this.report.setError("Inactivity Time Out
> > Reached. Sequence not complete");
> > + }
> > + } catch (InterruptedException e) {
> > + log.error("",e);
> > }
> > -
> > Thread.sleep(Constants.CLIENT_WAIT_PERIOD_FOR_COMPLETE);
> > - if ((System.currentTimeMillis() -
> > startingTime) >=
> > inactivityTimeOut) {
> > - stopClientByForce();
> > - this.report.setError("Inactivity Time Out
> > Reached. Sequence not complete");
> > - }
> > - } catch (InterruptedException e) {
> > - log.error("",e);
> > }
> > }
> > }
> > + // else - ignore any sequence that are still active
> >
> > if (this.report.getError() == null) {
> > - this.report.setAllAcked(true);
> > + this.report.setAllAcked(true);
> > }
> >
> > seqMap.remove(key);
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: sandesha-dev-unsubscribe@ws.apache.org
> > For additional commands, e-mail: sandesha-dev-help@ws.apache.org
> >
> >
>
>
RE: Sandesha 1.0; fix for clients hanging in SandeshaContext.endSequence()
Posted by Gilbert Pilz <Gi...@bea.com>.
Attached are patches against 452927 . . .
- gp
> -----Original Message-----
> From: Jaliya Ekanayake [mailto:jnekanayake@gmail.com]
> Sent: Tuesday, October 03, 2006 6:43 PM
> To: Gilbert Pilz; sandesha-dev@ws.apache.org
> Subject: Re: Sandesha 1.0; fix for clients hanging in
> SandeshaContext.endSequence()
>
> Hi Gilbert,
>
> Thanks for the patch. I tried to apply it but the version is
> not matching the current svn tree.
> Please update the code and creat the patch.
>
> -Jaliya
>
>
> ----- Original Message -----
> From: "Gilbert Pilz" <Gi...@bea.com>
> To: "Jaliya Ekanayake" <ja...@apache.org>;
> <sa...@ws.apache.org>
> Sent: Tuesday, October 03, 2006 3:49 PM
> Subject: RE: Sandesha 1.0; fix for clients hanging in
> SandeshaContext.endSequence()
>
>
> There is a race condition for outbound sequences between the
> SenderWorker thread transmitting the TerminateSequence message and the
> endSequence() call stopping that thread. It's difficult to
> figure out what a good wait value might be because, depending
> upon how many messages are queued up, the length of time you
> might need to wait for the TerminateSequence to be sent could
> be arbitrarily long.
>
> I've changed the code to log an INFO-level message when an
> incomplete sequence is ignored. If ignoring incomplete
> sequences causes people problems then they shouldn't do that.
>
> - gp
>
>
> > -----Original Message-----
> > From: Jaliya Ekanayake [mailto:jnekanayake@gmail.com]
> > Sent: Friday, September 29, 2006 8:06 PM
> > To: Gilbert Pilz; sandesha-dev@ws.apache.org
> > Subject: Re: Sandesha 1.0; fix for clients hanging in
> > SandeshaContext.endSequence()
> >
> > Sure, I have one clarification.
> >
> > What not allow some time-out before we cleanup. The reason
> for this is
> > as follows; In the one-way messaging case
> > SandeshaContext.endSequence() is called just after the service
> > requests so if someone use END_SEQUENCE_IGNORE_ACTIVE then the
> > sequences will immediately be terminated. It is better to set some
> > timestamp to wait.
> >
> > In addition can you please send the patch as an attachment?
> >
> > Thanks,
> > -Jaliya
> >
> > ----- Original Message -----
> > From: "Gilbert Pilz" <Gi...@bea.com>
> > To: <sa...@ws.apache.org>
> > Sent: Friday, September 29, 2006 7:14 PM
> > Subject: Sandesha 1.0; fix for clients hanging in
> > SandeshaContext.endSequence()
> >
> >
> > The story so far: the SandeshaContext.endSequence() method
> waits for
> > all the sequences related to a SandeshaContext to
> "complete" before it
> > returns. Because some implementations of WS-RM (200502) do not
> > automatically terminate their offered sequence when the
> sequence that
> > carried that offer is terminated (did you get that?), this
> may cause a
> > Sandesha client to hang until the sequence's inactivity timeout
> > expires.
> > At this point the client will see an AxisFault that they may or may
> > not understand to be benign.
> >
> > My plan was to fix this by adding a property to the SandeshaContext
> > called "EndSeqBehavior". The default value for this behavior,
> > "END_SEQUENCE_BLOCK_ON_ACTIVE", causes
> > endSequence() to behave exactly as it does today (block on
> any active
> > sequences until they either complete or time out due to
> inactivity).
> > Another value for this behavior,
> "END_SEQUENCE_IGNORE_ACTIVE", causes
> > endSequence() to simply ignore any active sequences and
> clean up. This
> > behavior is ok if all you are interested in doing is cleaning up
> > quickly, but it could make things difficult for the peers of the
> > sequences that are more or less orphaned.
> > If they tried to send messages using those sequences they
> would retry
> > and retry and retry until they reached some internal limit
> that told
> > them to stop. This seems rather wasteful given the fact
> that Sandesha
> > knew it was going to abandon the sequence, so I thought
> there should
> > be one more value for the EndSeqBehavior property called
> > "END_SEQUENCE_FAULT_ACTIVE". This value would cause
> > endSequence() to send an unsolicited
> wsrm:SequenceTerminated fault to
> > the sequence peer of any active sequences.
> >
> > Unfortunately the last value has proven much harder to
> implement than
> > I expected so I thought it would be a good idea to at least get the
> > changes for END_SEQUENCE_IGNORE_ACTIVE into the tree. Below are two
> > patch files; one for org/apache/sandesha/Constants.java and one for
> > org/apache/sandesha/SandeshaContext.java.
> >
> > Jaliya, if you could look these over and get them into the tree, I
> > would appreciate that very much. Note that clients that are
> unaware of
> > the EndSeqBehavior property will continue to work as they do today
> > w/out recompilation etc. Clients that wish to avoid the hanging
> > problem described above simply have to call the new
> > SandeshaContext.setEndSeqBehavior() method with the
> appropriate value
> > (as defined in Constants).
> >
> > - gp
> >
> > Index: src/org/apache/sandesha/Constants.java
> > ===================================================================
> > --- src/org/apache/sandesha/Constants.java (revision 451449)
> > +++ src/org/apache/sandesha/Constants.java (working copy)
> > @@ -72,6 +72,10 @@
> >
> > int SYNCHRONOUS = 0;
> >
> > + // Flags for controlling the behavior of endSequence()
> > in the face
> > of active sequences.
> > + int END_SEQUENCE_BLOCK_ON_ACTIVE = 0; // default
> > + int END_SEQUENCE_IGNORE_ACTIVE = 1; // ignore
> any active
> > seqs
> > +
> > public interface WSA {
> > String NS_ADDRESSING_ANONYMOUS =
> > AddressingUtils.getAnonymousRoleURI();
> > }
> >
> > ----------------------------------------
> >
> > Index: src/org/apache/sandesha/SandeshaContext.java
> > ===================================================================
> > --- src/org/apache/sandesha/SandeshaContext.java (revision
> > 451449)
> > +++ src/org/apache/sandesha/SandeshaContext.java (working copy)
> > @@ -76,6 +76,7 @@
> > private boolean sendOffer;
> > private long messageNumber;
> > private boolean sync;
> > + private int endSeqActiveBehavior;
> >
> > static int clientListnerPort = -1;
> > public final int MIN_PORT = 0;
> > @@ -155,6 +156,14 @@
> > this.toURL = toURL;
> > }
> >
> > + public int getEndSeqBehavior() {
> > + return endSeqActiveBehavior;
> > + }
> > +
> > + public void setEndSeqBehavior(int flag) {
> > + this.endSeqActiveBehavior = flag;
> > + }
> > +
> > public SandeshaContext() throws AxisFault {
> > initSandeshaAsync();
> > }
> > @@ -331,33 +340,38 @@
> > public final RMReport endSequence() throws AxisFault {
> >
> > IStorageManager storageManager = new
> ClientStorageManager();
> > - long startingTime = System.currentTimeMillis();
> > - long inactivityTimeOut =
> > PolicyLoader.getInstance().getInactivityTimeout();
> >
> > - Iterator ite = callMap.keySet().iterator();
> > + if (endSeqActiveBehavior !=
> > Constants.END_SEQUENCE_IGNORE_ACTIVE)
> > + {
> > + long startingTime = System.currentTimeMillis();
> > + long inactivityTimeOut =
> > PolicyLoader.getInstance().getInactivityTimeout();
> >
> > - while (ite.hasNext()) {
> > - String key = (String) ite.next();
> > - Call tempCall = (Call) callMap.get(key);
> > - String seqId = (String)
> > tempCall.getProperty(Constants.ClientProperties.CALL_KEY);
> > - while (!storageManager.isSequenceComplete(seqId)) {
> > - try {
> > - if (log.isDebugEnabled()) {
> > -
> > log.debug(Constants.InfomationMessage.WAITING_TO_STOP_CLIENT);
> > + Iterator ite = callMap.keySet().iterator();
> > +
> > + while (ite.hasNext()) {
> > + String key = (String) ite.next();
> > + Call tempCall = (Call) callMap.get(key);
> > + String seqId = (String)
> > tempCall.getProperty(Constants.ClientProperties.CALL_KEY);
> > + while (!storageManager.isSequenceComplete(seqId)) {
> > + try {
> > + if (log.isDebugEnabled()) {
> > +
> > log.debug(Constants.InfomationMessage.WAITING_TO_STOP_CLIENT);
> > + }
> > +
> > Thread.sleep(Constants.CLIENT_WAIT_PERIOD_FOR_COMPLETE);
> > + if ((System.currentTimeMillis() -
> > startingTime)
> > >= inactivityTimeOut) {
> > + stopClientByForce();
> > +
> this.report.setError("Inactivity Time Out
> > Reached. Sequence not complete");
> > + }
> > + } catch (InterruptedException e) {
> > + log.error("",e);
> > }
> > -
> > Thread.sleep(Constants.CLIENT_WAIT_PERIOD_FOR_COMPLETE);
> > - if ((System.currentTimeMillis() -
> > startingTime) >=
> > inactivityTimeOut) {
> > - stopClientByForce();
> > - this.report.setError("Inactivity Time Out
> > Reached. Sequence not complete");
> > - }
> > - } catch (InterruptedException e) {
> > - log.error("",e);
> > }
> > }
> > }
> > + // else - ignore any sequence that are still active
> >
> > if (this.report.getError() == null) {
> > - this.report.setAllAcked(true);
> > + this.report.setAllAcked(true);
> > }
> >
> > seqMap.remove(key);
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: sandesha-dev-unsubscribe@ws.apache.org
> > For additional commands, e-mail: sandesha-dev-help@ws.apache.org
> >
> >
>
>
Re: Sandesha 1.0; fix for clients hanging in SandeshaContext.endSequence()
Posted by Jaliya Ekanayake <jn...@gmail.com>.
Hi Gilbert,
Thanks for the patch. I tried to apply it but the version is not matching
the current svn tree.
Please update the code and creat the patch.
-Jaliya
----- Original Message -----
From: "Gilbert Pilz" <Gi...@bea.com>
To: "Jaliya Ekanayake" <ja...@apache.org>; <sa...@ws.apache.org>
Sent: Tuesday, October 03, 2006 3:49 PM
Subject: RE: Sandesha 1.0; fix for clients hanging in
SandeshaContext.endSequence()
There is a race condition for outbound sequences between the
SenderWorker thread transmitting the TerminateSequence message and the
endSequence() call stopping that thread. It's difficult to figure out
what a good wait value might be because, depending upon how many
messages are queued up, the length of time you might need to wait for
the TerminateSequence to be sent could be arbitrarily long.
I've changed the code to log an INFO-level message when an incomplete
sequence is ignored. If ignoring incomplete sequences causes people
problems then they shouldn't do that.
- gp
> -----Original Message-----
> From: Jaliya Ekanayake [mailto:jnekanayake@gmail.com]
> Sent: Friday, September 29, 2006 8:06 PM
> To: Gilbert Pilz; sandesha-dev@ws.apache.org
> Subject: Re: Sandesha 1.0; fix for clients hanging in
> SandeshaContext.endSequence()
>
> Sure, I have one clarification.
>
> What not allow some time-out before we cleanup. The reason
> for this is as follows; In the one-way messaging case
> SandeshaContext.endSequence() is called just after the
> service requests so if someone use END_SEQUENCE_IGNORE_ACTIVE
> then the sequences will immediately be terminated. It is
> better to set some timestamp to wait.
>
> In addition can you please send the patch as an attachment?
>
> Thanks,
> -Jaliya
>
> ----- Original Message -----
> From: "Gilbert Pilz" <Gi...@bea.com>
> To: <sa...@ws.apache.org>
> Sent: Friday, September 29, 2006 7:14 PM
> Subject: Sandesha 1.0; fix for clients hanging in
> SandeshaContext.endSequence()
>
>
> The story so far: the SandeshaContext.endSequence() method
> waits for all the sequences related to a SandeshaContext to
> "complete" before it returns. Because some implementations of
> WS-RM (200502) do not automatically terminate their offered
> sequence when the sequence that carried that offer is
> terminated (did you get that?), this may cause a Sandesha
> client to hang until the sequence's inactivity timeout expires.
> At this point the client will see an AxisFault that they may
> or may not understand to be benign.
>
> My plan was to fix this by adding a property to the
> SandeshaContext called "EndSeqBehavior". The default value
> for this behavior, "END_SEQUENCE_BLOCK_ON_ACTIVE", causes
> endSequence() to behave exactly as it does today (block on
> any active sequences until they either complete or time out
> due to inactivity). Another value for this behavior,
> "END_SEQUENCE_IGNORE_ACTIVE", causes endSequence() to simply
> ignore any active sequences and clean up. This behavior is ok
> if all you are interested in doing is cleaning up quickly,
> but it could make things difficult for the peers of the
> sequences that are more or less orphaned.
> If they tried to send messages using those sequences they
> would retry and retry and retry until they reached some
> internal limit that told them to stop. This seems rather
> wasteful given the fact that Sandesha knew it was going to
> abandon the sequence, so I thought there should be one more
> value for the EndSeqBehavior property called
> "END_SEQUENCE_FAULT_ACTIVE". This value would cause
> endSequence() to send an unsolicited wsrm:SequenceTerminated
> fault to the sequence peer of any active sequences.
>
> Unfortunately the last value has proven much harder to
> implement than I expected so I thought it would be a good
> idea to at least get the changes for
> END_SEQUENCE_IGNORE_ACTIVE into the tree. Below are two patch
> files; one for org/apache/sandesha/Constants.java and one for
> org/apache/sandesha/SandeshaContext.java.
>
> Jaliya, if you could look these over and get them into the
> tree, I would appreciate that very much. Note that clients
> that are unaware of the EndSeqBehavior property will continue
> to work as they do today w/out recompilation etc. Clients
> that wish to avoid the hanging problem described above simply
> have to call the new
> SandeshaContext.setEndSeqBehavior() method with the
> appropriate value (as defined in Constants).
>
> - gp
>
> Index: src/org/apache/sandesha/Constants.java
> ===================================================================
> --- src/org/apache/sandesha/Constants.java (revision 451449)
> +++ src/org/apache/sandesha/Constants.java (working copy)
> @@ -72,6 +72,10 @@
>
> int SYNCHRONOUS = 0;
>
> + // Flags for controlling the behavior of endSequence()
> in the face
> of active sequences.
> + int END_SEQUENCE_BLOCK_ON_ACTIVE = 0; // default
> + int END_SEQUENCE_IGNORE_ACTIVE = 1; // ignore any active
> seqs
> +
> public interface WSA {
> String NS_ADDRESSING_ANONYMOUS =
> AddressingUtils.getAnonymousRoleURI();
> }
>
> ----------------------------------------
>
> Index: src/org/apache/sandesha/SandeshaContext.java
> ===================================================================
> --- src/org/apache/sandesha/SandeshaContext.java (revision
> 451449)
> +++ src/org/apache/sandesha/SandeshaContext.java (working copy)
> @@ -76,6 +76,7 @@
> private boolean sendOffer;
> private long messageNumber;
> private boolean sync;
> + private int endSeqActiveBehavior;
>
> static int clientListnerPort = -1;
> public final int MIN_PORT = 0;
> @@ -155,6 +156,14 @@
> this.toURL = toURL;
> }
>
> + public int getEndSeqBehavior() {
> + return endSeqActiveBehavior;
> + }
> +
> + public void setEndSeqBehavior(int flag) {
> + this.endSeqActiveBehavior = flag;
> + }
> +
> public SandeshaContext() throws AxisFault {
> initSandeshaAsync();
> }
> @@ -331,33 +340,38 @@
> public final RMReport endSequence() throws AxisFault {
>
> IStorageManager storageManager = new ClientStorageManager();
> - long startingTime = System.currentTimeMillis();
> - long inactivityTimeOut =
> PolicyLoader.getInstance().getInactivityTimeout();
>
> - Iterator ite = callMap.keySet().iterator();
> + if (endSeqActiveBehavior !=
> Constants.END_SEQUENCE_IGNORE_ACTIVE)
> + {
> + long startingTime = System.currentTimeMillis();
> + long inactivityTimeOut =
> PolicyLoader.getInstance().getInactivityTimeout();
>
> - while (ite.hasNext()) {
> - String key = (String) ite.next();
> - Call tempCall = (Call) callMap.get(key);
> - String seqId = (String)
> tempCall.getProperty(Constants.ClientProperties.CALL_KEY);
> - while (!storageManager.isSequenceComplete(seqId)) {
> - try {
> - if (log.isDebugEnabled()) {
> -
> log.debug(Constants.InfomationMessage.WAITING_TO_STOP_CLIENT);
> + Iterator ite = callMap.keySet().iterator();
> +
> + while (ite.hasNext()) {
> + String key = (String) ite.next();
> + Call tempCall = (Call) callMap.get(key);
> + String seqId = (String)
> tempCall.getProperty(Constants.ClientProperties.CALL_KEY);
> + while (!storageManager.isSequenceComplete(seqId)) {
> + try {
> + if (log.isDebugEnabled()) {
> +
> log.debug(Constants.InfomationMessage.WAITING_TO_STOP_CLIENT);
> + }
> +
> Thread.sleep(Constants.CLIENT_WAIT_PERIOD_FOR_COMPLETE);
> + if ((System.currentTimeMillis() -
> startingTime)
> >= inactivityTimeOut) {
> + stopClientByForce();
> + this.report.setError("Inactivity Time Out
> Reached. Sequence not complete");
> + }
> + } catch (InterruptedException e) {
> + log.error("",e);
> }
> -
> Thread.sleep(Constants.CLIENT_WAIT_PERIOD_FOR_COMPLETE);
> - if ((System.currentTimeMillis() -
> startingTime) >=
> inactivityTimeOut) {
> - stopClientByForce();
> - this.report.setError("Inactivity Time Out
> Reached. Sequence not complete");
> - }
> - } catch (InterruptedException e) {
> - log.error("",e);
> }
> }
> }
> + // else - ignore any sequence that are still active
>
> if (this.report.getError() == null) {
> - this.report.setAllAcked(true);
> + this.report.setAllAcked(true);
> }
>
> seqMap.remove(key);
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: sandesha-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: sandesha-dev-help@ws.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: sandesha-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: sandesha-dev-help@ws.apache.org
Re: Sandesha 1.0; fix for clients hanging in SandeshaContext.endSequence()
Posted by Jaliya Ekanayake <jn...@gmail.com>.
Hi Gilbert,
Thanks for the patch. I tried to apply it but the version is not matching
the current svn tree.
Please update the code and creat the patch.
-Jaliya
----- Original Message -----
From: "Gilbert Pilz" <Gi...@bea.com>
To: "Jaliya Ekanayake" <ja...@apache.org>; <sa...@ws.apache.org>
Sent: Tuesday, October 03, 2006 3:49 PM
Subject: RE: Sandesha 1.0; fix for clients hanging in
SandeshaContext.endSequence()
There is a race condition for outbound sequences between the
SenderWorker thread transmitting the TerminateSequence message and the
endSequence() call stopping that thread. It's difficult to figure out
what a good wait value might be because, depending upon how many
messages are queued up, the length of time you might need to wait for
the TerminateSequence to be sent could be arbitrarily long.
I've changed the code to log an INFO-level message when an incomplete
sequence is ignored. If ignoring incomplete sequences causes people
problems then they shouldn't do that.
- gp
> -----Original Message-----
> From: Jaliya Ekanayake [mailto:jnekanayake@gmail.com]
> Sent: Friday, September 29, 2006 8:06 PM
> To: Gilbert Pilz; sandesha-dev@ws.apache.org
> Subject: Re: Sandesha 1.0; fix for clients hanging in
> SandeshaContext.endSequence()
>
> Sure, I have one clarification.
>
> What not allow some time-out before we cleanup. The reason
> for this is as follows; In the one-way messaging case
> SandeshaContext.endSequence() is called just after the
> service requests so if someone use END_SEQUENCE_IGNORE_ACTIVE
> then the sequences will immediately be terminated. It is
> better to set some timestamp to wait.
>
> In addition can you please send the patch as an attachment?
>
> Thanks,
> -Jaliya
>
> ----- Original Message -----
> From: "Gilbert Pilz" <Gi...@bea.com>
> To: <sa...@ws.apache.org>
> Sent: Friday, September 29, 2006 7:14 PM
> Subject: Sandesha 1.0; fix for clients hanging in
> SandeshaContext.endSequence()
>
>
> The story so far: the SandeshaContext.endSequence() method
> waits for all the sequences related to a SandeshaContext to
> "complete" before it returns. Because some implementations of
> WS-RM (200502) do not automatically terminate their offered
> sequence when the sequence that carried that offer is
> terminated (did you get that?), this may cause a Sandesha
> client to hang until the sequence's inactivity timeout expires.
> At this point the client will see an AxisFault that they may
> or may not understand to be benign.
>
> My plan was to fix this by adding a property to the
> SandeshaContext called "EndSeqBehavior". The default value
> for this behavior, "END_SEQUENCE_BLOCK_ON_ACTIVE", causes
> endSequence() to behave exactly as it does today (block on
> any active sequences until they either complete or time out
> due to inactivity). Another value for this behavior,
> "END_SEQUENCE_IGNORE_ACTIVE", causes endSequence() to simply
> ignore any active sequences and clean up. This behavior is ok
> if all you are interested in doing is cleaning up quickly,
> but it could make things difficult for the peers of the
> sequences that are more or less orphaned.
> If they tried to send messages using those sequences they
> would retry and retry and retry until they reached some
> internal limit that told them to stop. This seems rather
> wasteful given the fact that Sandesha knew it was going to
> abandon the sequence, so I thought there should be one more
> value for the EndSeqBehavior property called
> "END_SEQUENCE_FAULT_ACTIVE". This value would cause
> endSequence() to send an unsolicited wsrm:SequenceTerminated
> fault to the sequence peer of any active sequences.
>
> Unfortunately the last value has proven much harder to
> implement than I expected so I thought it would be a good
> idea to at least get the changes for
> END_SEQUENCE_IGNORE_ACTIVE into the tree. Below are two patch
> files; one for org/apache/sandesha/Constants.java and one for
> org/apache/sandesha/SandeshaContext.java.
>
> Jaliya, if you could look these over and get them into the
> tree, I would appreciate that very much. Note that clients
> that are unaware of the EndSeqBehavior property will continue
> to work as they do today w/out recompilation etc. Clients
> that wish to avoid the hanging problem described above simply
> have to call the new
> SandeshaContext.setEndSeqBehavior() method with the
> appropriate value (as defined in Constants).
>
> - gp
>
> Index: src/org/apache/sandesha/Constants.java
> ===================================================================
> --- src/org/apache/sandesha/Constants.java (revision 451449)
> +++ src/org/apache/sandesha/Constants.java (working copy)
> @@ -72,6 +72,10 @@
>
> int SYNCHRONOUS = 0;
>
> + // Flags for controlling the behavior of endSequence()
> in the face
> of active sequences.
> + int END_SEQUENCE_BLOCK_ON_ACTIVE = 0; // default
> + int END_SEQUENCE_IGNORE_ACTIVE = 1; // ignore any active
> seqs
> +
> public interface WSA {
> String NS_ADDRESSING_ANONYMOUS =
> AddressingUtils.getAnonymousRoleURI();
> }
>
> ----------------------------------------
>
> Index: src/org/apache/sandesha/SandeshaContext.java
> ===================================================================
> --- src/org/apache/sandesha/SandeshaContext.java (revision
> 451449)
> +++ src/org/apache/sandesha/SandeshaContext.java (working copy)
> @@ -76,6 +76,7 @@
> private boolean sendOffer;
> private long messageNumber;
> private boolean sync;
> + private int endSeqActiveBehavior;
>
> static int clientListnerPort = -1;
> public final int MIN_PORT = 0;
> @@ -155,6 +156,14 @@
> this.toURL = toURL;
> }
>
> + public int getEndSeqBehavior() {
> + return endSeqActiveBehavior;
> + }
> +
> + public void setEndSeqBehavior(int flag) {
> + this.endSeqActiveBehavior = flag;
> + }
> +
> public SandeshaContext() throws AxisFault {
> initSandeshaAsync();
> }
> @@ -331,33 +340,38 @@
> public final RMReport endSequence() throws AxisFault {
>
> IStorageManager storageManager = new ClientStorageManager();
> - long startingTime = System.currentTimeMillis();
> - long inactivityTimeOut =
> PolicyLoader.getInstance().getInactivityTimeout();
>
> - Iterator ite = callMap.keySet().iterator();
> + if (endSeqActiveBehavior !=
> Constants.END_SEQUENCE_IGNORE_ACTIVE)
> + {
> + long startingTime = System.currentTimeMillis();
> + long inactivityTimeOut =
> PolicyLoader.getInstance().getInactivityTimeout();
>
> - while (ite.hasNext()) {
> - String key = (String) ite.next();
> - Call tempCall = (Call) callMap.get(key);
> - String seqId = (String)
> tempCall.getProperty(Constants.ClientProperties.CALL_KEY);
> - while (!storageManager.isSequenceComplete(seqId)) {
> - try {
> - if (log.isDebugEnabled()) {
> -
> log.debug(Constants.InfomationMessage.WAITING_TO_STOP_CLIENT);
> + Iterator ite = callMap.keySet().iterator();
> +
> + while (ite.hasNext()) {
> + String key = (String) ite.next();
> + Call tempCall = (Call) callMap.get(key);
> + String seqId = (String)
> tempCall.getProperty(Constants.ClientProperties.CALL_KEY);
> + while (!storageManager.isSequenceComplete(seqId)) {
> + try {
> + if (log.isDebugEnabled()) {
> +
> log.debug(Constants.InfomationMessage.WAITING_TO_STOP_CLIENT);
> + }
> +
> Thread.sleep(Constants.CLIENT_WAIT_PERIOD_FOR_COMPLETE);
> + if ((System.currentTimeMillis() -
> startingTime)
> >= inactivityTimeOut) {
> + stopClientByForce();
> + this.report.setError("Inactivity Time Out
> Reached. Sequence not complete");
> + }
> + } catch (InterruptedException e) {
> + log.error("",e);
> }
> -
> Thread.sleep(Constants.CLIENT_WAIT_PERIOD_FOR_COMPLETE);
> - if ((System.currentTimeMillis() -
> startingTime) >=
> inactivityTimeOut) {
> - stopClientByForce();
> - this.report.setError("Inactivity Time Out
> Reached. Sequence not complete");
> - }
> - } catch (InterruptedException e) {
> - log.error("",e);
> }
> }
> }
> + // else - ignore any sequence that are still active
>
> if (this.report.getError() == null) {
> - this.report.setAllAcked(true);
> + this.report.setAllAcked(true);
> }
>
> seqMap.remove(key);
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: sandesha-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: sandesha-dev-help@ws.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: sandesha-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: sandesha-dev-help@ws.apache.org