You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@xalan.apache.org by "Edward L. Knoll" <ed...@cosd.fedex.com> on 2004/12/17 16:41:56 UTC
Concurrency problem with static Java calls.
Context: Xalan-J, 2.6.0; SAX transformation chain.
We were having a concurrency operation associated with a call to a
static Java method we were making from an XSL stylesheet. We
restructured the Java class, so that the operation could be performed as
a method call on an instance; the static method was changed to create an
instance of the class and make the instance method call.
When we create an object instance and make the instance method call, we
do not have a concurrency problem. If we call the static method, we
have a concurrency problem. I reverified that the problem is still
there with the static method call after restructuring the class.
What ever this problem is, it's tied up with how static Java method
calls are handled from the stylesheet versus instance method calls. The
static method call is reentrant; there should not have been a problem.
The problem appears to be tied up with how the parameters are handled;
in some instances it appears as though parameters from one invocation
are mixed up with another (presumably from another thread). If I were
to hazard a guess, on static Java method calls the parameters are being
maintained in a static structure.
I won't be researching this farther, since will be using the instance
method call. As I've said before, company proprietary policies won't
allow me to provide source. Our test driver is running 10 threads on a
8 CPU platform.
--
Edward L. Knoll Phone (FedEx) : (719)484-2717
e-mail (FedEx) : edward.knoll@fedex.com
e-mail (personal) : edward@elknoll.com
---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org
Re: Concurrency problem with static Java calls.
Posted by Henry Zongaro <zo...@ca.ibm.com>.
Hi, Edward.
"Edward L. Knoll" <ed...@cosd.fedex.com> wrote on 2004-12-20 11:55:43
AM:
> If I'd stopped to think about it, I would not have expected to be able
> to change a result-tree-fragement into a number/integer, but the logic
> appears to work (outside the threading issues). I've included some XSL
> fragments to illustrate:
Just FYI - according to Section 11.1 of XSLT 1.0:[1]
the operations permitted on a result tree fragment are a subset
of those permitted on a node-set. An operation is permitted on
a result tree fragment only if that operation would be permitted
on a string (the operation on the string may involve first
converting the string to a number or boolean)
Thanks,
Henry
[1] http://www.w3.org/TR/xslt#section-Result-Tree-Fragments
------------------------------------------------------------------
Henry Zongaro Xalan development
IBM SWS Toronto Lab T/L 969-6044; Phone +1 905 413-6044
mailto:zongaro@ca.ibm.com
---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org
Re: Concurrency problem with static Java calls.
Posted by "Edward L. Knoll" <ed...@cosd.fedex.com>.
Empirical evidence now seems to be pointing at another possible
explanation. The Java function we are calling takes several parameters
which are either integers or strings. Template scope XSL variables
were being used to establish the values of these parameters and then
these local variables were passed to the Java function. I happened to
notice that while all of the Java function parameters where simple types
(e.g. integers/strings), some of the local variables were technically
result-tree-fragments. I eliminated some of these local variables we
were defining and that seemed to reduce the problem incidence rate,
although I can't actually prove a cause and effect relationship.
If I'd stopped to think about it, I would not have expected to be able
to change a result-tree-fragement into a number/integer, but the logic
appears to work (outside the threading issues). I've included some XSL
fragments to illustrate:
<xsl:variable name='Offset'> <!-- [+-]HHMM portion -->
<xsl:variable name='afterZ' select="substring-after($Input,'Z')" />
<xsl:choose>
<xsl:when test="string-length($afterZ)=5">
<xsl:value-of select="$afterZ" />
</xsl:when>
<xsl:otherwise>
<xsl:value-of select="'+0000'" />
</xsl:otherwise>
</xsl:choose>
</xsl:variable>
<xsl:variable name="Offset_mSec"
select="(number(substring($Offset,2,2)) * 3600000) +
(number(substring($Offset,4,2)) * 60000)" />
<xsl:choose>
<xsl:when test="substring($Offset,1,1) = '-'">
<xsl:value-of select="FastDateFormat:formatDate(
$DateFormatter, $DateFormat,
substring($DateTime,5,2), substring($DateTime,7,2),
substring($DateTime,1,4), substring($DateTime,9,2),
substring($DateTime,11,2), '0', -($Offset_mSec))" />
</xsl:when>
<xsl:otherwise>
<xsl:value-of select="FastDateFormat:formatDate(
$DateFormatter, $DateFormat,
substring($DateTime,5,2), substring($DateTime,7,2),
substring($DateTime,1,4), substring($DateTime,9,2),
substring($DateTime,11,2), '0', $Offset_mSec)" />
</xsl:otherwise>
</xsl:choose>
Edward L. Knoll wrote:
> Spoke to soon. The incident rate dropped a lot, but the problem still
> persistes.
>
> Edward L. Knoll wrote:
>
>> Context: Xalan-J, 2.6.0; SAX transformation chain.
>>
>> We were having a concurrency operation associated with a call to a
>> static Java method we were making from an XSL stylesheet. We
>> restructured the Java class, so that the operation could be performed
>> as a method call on an instance; the static method was changed to
>> create an instance of the class and make the instance method call.
>>
>> When we create an object instance and make the instance method call,
>> we do not have a concurrency problem. If we call the static method,
>> we have a concurrency problem. I reverified that the problem is
>> still there with the static method call after restructuring the class.
>>
>> What ever this problem is, it's tied up with how static Java method
>> calls are handled from the stylesheet versus instance method calls.
>> The static method call is reentrant; there should not have been a
>> problem. The problem appears to be tied up with how the parameters
>> are handled; in some instances it appears as though parameters from
>> one invocation are mixed up with another (presumably from another
>> thread). If I were to hazard a guess, on static Java method calls
>> the parameters are being maintained in a static structure.
>>
>> I won't be researching this farther, since will be using the instance
>> method call. As I've said before, company proprietary policies won't
>> allow me to provide source. Our test driver is running 10 threads on
>> a 8 CPU platform.
>>
>
--
Edward L. Knoll Phone (FedEx) : (719)484-2717
e-mail (FedEx) : edward.knoll@fedex.com
e-mail (personal) : edward@elknoll.com
---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org
Re: Concurrency problems reproduced - what now.
Posted by Brian Minchau <mi...@ca.ibm.com>.
Ed,
please open separate JIRA issues, it is easy to mark one as a duplicate of
the other should that turn out to be the case.
This way each issue can have its own status, rather than tangling them
together. A single zip file, or separate attachements is fine, what ever
is best for you.
----------
Brian Minchau
XSLT Development, IBM Toronto
e-mail: minchau@ca.ibm.com
"Edward L. Knoll"
<ed.knoll@cosd.fe
dex.com> To
xalan-dev@xml.apache.org
01/12/2005 02:58 cc
PM
Subject
Concurrency problems reproduced -
Please respond to what now.
xalan-dev
I've reproduced the three major types of concurrency problems we
encountered with Xalan-Java 2.6.0 with fairly small elements for which I
can supply the source. One problem is an HTML serialization problem
which has likely been resolved, but I will supply anyway, so someone can
verify. The other two problems may turn out to be different aspects of
the same problem, since they both appear to revolve around management of
local variables within templates.
Do you want these packaged as a single defect?
How do you want the source: multiple text file attachments or a single
tar file? The components include a Java class/program, XSL stylesheets,
XML file(s), and the scripts I use to run the tests and cross compare
results to identify the errors.
I still have some cleanup and (re)packaging to do before I submit these,
but these activities should be completed soon.
Thanks,
Ed Knoll
--
Edward L. Knoll Phone (FedEx) : (719)484-2717
e-mail (FedEx) : edward.knoll@fedex.com
e-mail (personal) : edward@elknoll.com
---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org
Concurrency problems reproduced - what now.
Posted by "Edward L. Knoll" <ed...@cosd.fedex.com>.
I've reproduced the three major types of concurrency problems we
encountered with Xalan-Java 2.6.0 with fairly small elements for which I
can supply the source. One problem is an HTML serialization problem
which has likely been resolved, but I will supply anyway, so someone can
verify. The other two problems may turn out to be different aspects of
the same problem, since they both appear to revolve around management of
local variables within templates.
Do you want these packaged as a single defect?
How do you want the source: multiple text file attachments or a single
tar file? The components include a Java class/program, XSL stylesheets,
XML file(s), and the scripts I use to run the tests and cross compare
results to identify the errors.
I still have some cleanup and (re)packaging to do before I submit these,
but these activities should be completed soon.
Thanks,
Ed Knoll
--
Edward L. Knoll Phone (FedEx) : (719)484-2717
e-mail (FedEx) : edward.knoll@fedex.com
e-mail (personal) : edward@elknoll.com
---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org
Re: Concurrency problems with variable management?
Posted by Christine Li <jy...@ca.ibm.com>.
Hi, Edward
Thanks for trying to identify the problem. Once you can reproduce the
problem, please one a bug report and attach a test case to demonstrate the
problem http://nagoya.apache.org/jira/
Thanks,
Christine Li
XSLT Development
IBM Toronto Lab
Tel: (905)413-2601
Email: jycli@ca.ibm.com
"Edward L. Knoll" <ed...@cosd.fedex.com>
12/23/2004 02:49 PM
Please respond to
xalan-dev
To
xalan-dev@xml.apache.org
cc
Subject
Concurrency problems with variable management?
We seem to have isolated a couple of concurrency problems to the
handling of (template scope) variables in two different contexts. I
will include code fragments to illustrate the problems. Corporate
proprietary policies will not allow me to send the entire source or the
data. Within the next couple of weeks it is my intent to attempt to
reproduce both of these problems with simple/benign examples which (if
successful) I will submit with a bug report.
Our thread test driver is running 20 threads on a 8 processor box. Each
thread is performing the same set of 16 different transformations (in a
couple of different orders). Each transformation is performed by a SAX
processing chain. The XSL transforms are loaded into templates. The
classes used in the SAX processing chain are recreated for each
transformation.
In the first example, a local variable is passed to two separate
templates. The value of the IntermediateValue variable changes between
the two xsl:call-template invocations. The incorrect value can be found
in the input XML. We believe the incorrect value is being picked up
from another thread. This error will generally occur once or twice
every 10 runs (of the 20 threads).
<xsl:template match="sep-gmt-post-timestamp" priority="20.0">
<xsl:variable name="IntermediateValue"
select="concat(normalize-space(.),
substring-after(../event-creation-timestamp,'Z'))" />
<xsl:call-template name='TransformField'>
<xsl:with-param name="IntermediateValue"
select="$IntermediateValue" />
<xsl:with-param name="OtherElements">
<xsl:call-template name="FormatDateTime">
<xsl:with-param name="Input">
<xsl:value-of select="$IntermediateValue" />
</xsl:with-param>
</xsl:call-template>
</xsl:with-param>
</xsl:call-template>
</xsl:template>
The other error occurred on almost every test run. A local variable
inside of a recursive template call appeared to be changing. The
"count" was sometimes '1' when it should have been '0'. We totally
eliminated the local variables and the problem when away. I've included
old and new code fragments:
<!--Original/Broken Code-->
<xsl:template name="OutputFieldAndValue">
<xsl:param name="count" />
....
<xsl:variable name="paddingSpaces">
<xsl:value-of select="$num * 4" />
</xsl:variable>
....
<xsl:variable name="calculatedPadding">
<xsl:value-of
select="substring($TranslatedSpaces,1,$paddingSpaces)" />
</xsl:variable>
....
<xsl:for-each select="*">
<xsl:call-template name="OutputFieldAndValue">
<xsl:with-param name="count" select="$count+1" />
</xsl:call-template>
</xsl:for-each>
....
</xsl:template>
<!--New/Working Code-->
<xsl:template name="OutputFieldAndValue">
<xsl:param name="count" select="0" />
<xsl:param name="indent" select="''" />
....
<xsl:for-each select="*">
<xsl:call-template name="OutputFieldAndValue">
<xsl:with-param name="count" select="$count+1" />
<xsl:with-param name="indent"
select="substring($TranslatedSpaces,1,$count*4)" />
</xsl:call-template>
</xsl:for-each>
....
</xsl:template>
--
Edward L. Knoll Phone (FedEx) : (719)484-2717
e-mail (FedEx) : edward.knoll@fedex.com
e-mail (personal) : edward@elknoll.com
---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org
Concurrency problems with variable management?
Posted by "Edward L. Knoll" <ed...@cosd.fedex.com>.
We seem to have isolated a couple of concurrency problems to the
handling of (template scope) variables in two different contexts. I
will include code fragments to illustrate the problems. Corporate
proprietary policies will not allow me to send the entire source or the
data. Within the next couple of weeks it is my intent to attempt to
reproduce both of these problems with simple/benign examples which (if
successful) I will submit with a bug report.
Our thread test driver is running 20 threads on a 8 processor box. Each
thread is performing the same set of 16 different transformations (in a
couple of different orders). Each transformation is performed by a SAX
processing chain. The XSL transforms are loaded into templates. The
classes used in the SAX processing chain are recreated for each
transformation.
In the first example, a local variable is passed to two separate
templates. The value of the IntermediateValue variable changes between
the two xsl:call-template invocations. The incorrect value can be found
in the input XML. We believe the incorrect value is being picked up
from another thread. This error will generally occur once or twice
every 10 runs (of the 20 threads).
<xsl:template match="sep-gmt-post-timestamp" priority="20.0">
<xsl:variable name="IntermediateValue"
select="concat(normalize-space(.),
substring-after(../event-creation-timestamp,'Z'))" />
<xsl:call-template name='TransformField'>
<xsl:with-param name="IntermediateValue"
select="$IntermediateValue" />
<xsl:with-param name="OtherElements">
<xsl:call-template name="FormatDateTime">
<xsl:with-param name="Input">
<xsl:value-of select="$IntermediateValue" />
</xsl:with-param>
</xsl:call-template>
</xsl:with-param>
</xsl:call-template>
</xsl:template>
The other error occurred on almost every test run. A local variable
inside of a recursive template call appeared to be changing. The
"count" was sometimes '1' when it should have been '0'. We totally
eliminated the local variables and the problem when away. I've included
old and new code fragments:
<!--Original/Broken Code-->
<xsl:template name="OutputFieldAndValue">
<xsl:param name="count" />
....
<xsl:variable name="paddingSpaces">
<xsl:value-of select="$num * 4" />
</xsl:variable>
....
<xsl:variable name="calculatedPadding">
<xsl:value-of
select="substring($TranslatedSpaces,1,$paddingSpaces)" />
</xsl:variable>
....
<xsl:for-each select="*">
<xsl:call-template name="OutputFieldAndValue">
<xsl:with-param name="count" select="$count+1" />
</xsl:call-template>
</xsl:for-each>
....
</xsl:template>
<!--New/Working Code-->
<xsl:template name="OutputFieldAndValue">
<xsl:param name="count" select="0" />
<xsl:param name="indent" select="''" />
....
<xsl:for-each select="*">
<xsl:call-template name="OutputFieldAndValue">
<xsl:with-param name="count" select="$count+1" />
<xsl:with-param name="indent"
select="substring($TranslatedSpaces,1,$count*4)" />
</xsl:call-template>
</xsl:for-each>
....
</xsl:template>
--
Edward L. Knoll Phone (FedEx) : (719)484-2717
e-mail (FedEx) : edward.knoll@fedex.com
e-mail (personal) : edward@elknoll.com
---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org
Re: Concurrency problem with static Java calls.
Posted by "Edward L. Knoll" <ed...@cosd.fedex.com>.
Spoke to soon. The incident rate dropped a lot, but the problem still
persistes.
Edward L. Knoll wrote:
> Context: Xalan-J, 2.6.0; SAX transformation chain.
>
> We were having a concurrency operation associated with a call to a
> static Java method we were making from an XSL stylesheet. We
> restructured the Java class, so that the operation could be performed
> as a method call on an instance; the static method was changed to
> create an instance of the class and make the instance method call.
>
> When we create an object instance and make the instance method call,
> we do not have a concurrency problem. If we call the static method,
> we have a concurrency problem. I reverified that the problem is still
> there with the static method call after restructuring the class.
>
> What ever this problem is, it's tied up with how static Java method
> calls are handled from the stylesheet versus instance method calls.
> The static method call is reentrant; there should not have been a
> problem. The problem appears to be tied up with how the parameters are
> handled; in some instances it appears as though parameters from one
> invocation are mixed up with another (presumably from another
> thread). If I were to hazard a guess, on static Java method calls the
> parameters are being maintained in a static structure.
>
> I won't be researching this farther, since will be using the instance
> method call. As I've said before, company proprietary policies won't
> allow me to provide source. Our test driver is running 10 threads on
> a 8 CPU platform.
>
--
Edward L. Knoll Phone (FedEx) : (719)484-2717
e-mail (FedEx) : edward.knoll@fedex.com
e-mail (personal) : edward@elknoll.com
---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org
More concurrency problems ....
Posted by "Edward L. Knoll" <ed...@cosd.fedex.com>.
Another concurrency issue has been isolated to a recursive XSL
template. This template has a (local) variable established using a
count passed to the template. The variable is an indent string; the
parameter is a number used to determine the amount of indent. Very
infrequently the indent string (variable) doesn't match the count
(parameter).
We've also encountered an output discrepancy in serialized XML output;
we've seen this once over a very large number of runs. The was a known
problem associated with HTML serialization and most of those issues
disappeared when we replaced the HTML Serializer with our own. We'll
try that with the XML Serializer.
I'm beginning to feel like I'm herding cats. I'm not sharing any XML
parsers, ContentHandlers, TransformHandlers, or Serializes between
threads. Is there anything else I should be doing to improve concurrent
behavior?
--
Edward L. Knoll Phone (FedEx) : (719)484-2717
e-mail (FedEx) : edward.knoll@fedex.com
e-mail (personal) : edward@elknoll.com
---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org
Re: Concurrency problem with static Java calls.
Posted by Morris Kwan <mk...@ca.ibm.com>.
Hi, Edward
You can open a bug in the JIRA database
(http://nagoya.apache.org/jira/secure/Dashboard.jspa).
Please attach the description here in the bug report. It might be difficult
to diagnose the problem without a testcase. If possible, please also
describe the types of the parameters, in case the problem is related to the
parameter types.
Thanks.
Morris Kwan
XSLT Development
IBM Toronto Lab
Tel: (905)413-3729
Email: mkwan@ca.ibm.com
"Edward L. Knoll"
<ed.knoll@cosd.fe
dex.com> To
xalan-dev@xml.apache.org
12/17/2004 10:41 cc
AM
Subject
Concurrency problem with static
Please respond to Java calls.
xalan-dev
Context: Xalan-J, 2.6.0; SAX transformation chain.
We were having a concurrency operation associated with a call to a
static Java method we were making from an XSL stylesheet. We
restructured the Java class, so that the operation could be performed as
a method call on an instance; the static method was changed to create an
instance of the class and make the instance method call.
When we create an object instance and make the instance method call, we
do not have a concurrency problem. If we call the static method, we
have a concurrency problem. I reverified that the problem is still
there with the static method call after restructuring the class.
What ever this problem is, it's tied up with how static Java method
calls are handled from the stylesheet versus instance method calls. The
static method call is reentrant; there should not have been a problem.
The problem appears to be tied up with how the parameters are handled;
in some instances it appears as though parameters from one invocation
are mixed up with another (presumably from another thread). If I were
to hazard a guess, on static Java method calls the parameters are being
maintained in a static structure.
I won't be researching this farther, since will be using the instance
method call. As I've said before, company proprietary policies won't
allow me to provide source. Our test driver is running 10 threads on a
8 CPU platform.
--
Edward L. Knoll Phone (FedEx) : (719)484-2717
e-mail (FedEx) : edward.knoll@fedex.com
e-mail (personal) : edward@elknoll.com
---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org