You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@xmlbeans.apache.org by "Mark D Henning (JIRA)" <xm...@xml.apache.org> on 2009/06/02 14:28:08 UTC
[jira] Created: (XMLBEANS-410) xsd2inst incorrectly handling
numeric precision
xsd2inst incorrectly handling numeric precision
-----------------------------------------------
Key: XMLBEANS-410
URL: https://issues.apache.org/jira/browse/XMLBEANS-410
Project: XMLBeans
Issue Type: Bug
Components: Tools
Affects Versions: Version 2.4.1
Environment: All OS, all hardware
Reporter: Mark D Henning
Fix For: TBD
as an example, a xs:decimal (totalDig:20, fracDig:0) was creating 1000.00 whic of course, does not validate.
The problem is in the org.apache.xmlbeans.impl.xsd2inst.SampleXmlUtil.java class.
Root cause of the problem is that the programmer was calling the setScale() method on a BigDecimal, as though it modifies the existing BigDecimal object (i.e. BigDecimal bd = new BigDecimal("1000.00"); bd.setScale(1)) setScale does not modify the existing, but returns a new BigDecimal object. Therefore the correct call would be (BigDecimal bd = new BigDecimal("1000.00"); bd = bd.setScale(1)).
The lines included below represent the repair of the formatDecimal() method which is the eronious function.
// We have the number
// Adjust the scale according to the totalDigits and fractionDigits
int digits = 0;
BigDecimal ONE = new BigDecimal(BigInteger.ONE);
for (BigDecimal n = result; n.abs().compareTo(ONE) >= 0; digits++)
n = n.movePointLeft(1);
if (fractionDigits > 0)
if (totalDigits >= 0)
result = result.setScale(Math.max(fractionDigits, totalDigits - digits));
else
result = result.setScale(fractionDigits);
else if (fractionDigits == 0)
result = result.setScale(0);
return result.toString();
}
I would recommend that the code tree be searched for other setScale method calls to see if others need to be fixed. I currently do not have write access to the subversion repository, so I am unable to check this fix in myself.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@xmlbeans.apache.org
For additional commands, e-mail: dev-help@xmlbeans.apache.org
[jira] Updated: (XMLBEANS-410) xsd2inst incorrectly handling
decimal precision
Posted by "Mark D Henning (JIRA)" <xm...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XMLBEANS-410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark D Henning updated XMLBEANS-410:
------------------------------------
Description:
as an example, a xs:decimal (totalDig:20, fracDig:0) was creating 1000.00 whic of course, does not validate.
The problem is in the org.apache.xmlbeans.impl.xsd2inst.SampleXmlUtil.java class.
Root cause of the problem is that the programmer was calling the setScale() method on a BigDecimal, as though it modifies the existing BigDecimal object (i.e. BigDecimal bd = new BigDecimal("1000.00"); bd.setScale(1)) setScale does not modify the existing, but returns a new BigDecimal object. Therefore the correct call would be (BigDecimal bd = new BigDecimal("1000.00"); bd = bd.setScale(1)).
The lines included below represent the repair of the formatDecimal() method which is the eronious function.
// We have the number
// Adjust the scale according to the totalDigits and fractionDigits
int digits = 0;
BigDecimal ONE = new BigDecimal(BigInteger.ONE);
for (BigDecimal n = result; n.abs().compareTo(ONE) >= 0; digits++)
n = n.movePointLeft(1);
if (fractionDigits > 0)
if (totalDigits >= 0)
result = result.setScale(Math.max(fractionDigits, totalDigits - digits));
else
result = result.setScale(fractionDigits);
else if (fractionDigits == 0)
result = result.setScale(0);
return result.toString();
}
I would recommend that the code tree be searched for other setScale method calls to see if others need to be fixed. I currently do not have write access to the subversion repository, so I am unable to check this fix in myself.
I was unsure whether to code this as a minor or major problem. It appears that the difference between the two is the presence of a work-around, for which there is none in this case.
was:
as an example, a xs:decimal (totalDig:20, fracDig:0) was creating 1000.00 whic of course, does not validate.
The problem is in the org.apache.xmlbeans.impl.xsd2inst.SampleXmlUtil.java class.
Root cause of the problem is that the programmer was calling the setScale() method on a BigDecimal, as though it modifies the existing BigDecimal object (i.e. BigDecimal bd = new BigDecimal("1000.00"); bd.setScale(1)) setScale does not modify the existing, but returns a new BigDecimal object. Therefore the correct call would be (BigDecimal bd = new BigDecimal("1000.00"); bd = bd.setScale(1)).
The lines included below represent the repair of the formatDecimal() method which is the eronious function.
// We have the number
// Adjust the scale according to the totalDigits and fractionDigits
int digits = 0;
BigDecimal ONE = new BigDecimal(BigInteger.ONE);
for (BigDecimal n = result; n.abs().compareTo(ONE) >= 0; digits++)
n = n.movePointLeft(1);
if (fractionDigits > 0)
if (totalDigits >= 0)
result = result.setScale(Math.max(fractionDigits, totalDigits - digits));
else
result = result.setScale(fractionDigits);
else if (fractionDigits == 0)
result = result.setScale(0);
return result.toString();
}
I would recommend that the code tree be searched for other setScale method calls to see if others need to be fixed. I currently do not have write access to the subversion repository, so I am unable to check this fix in myself.
Summary: xsd2inst incorrectly handling decimal precision (was: xsd2inst incorrectly handling numeric precision)
I revised the summary to reflect the fact that it was not general numeric precision that was affected, but only xs:decimal space.
> xsd2inst incorrectly handling decimal precision
> -----------------------------------------------
>
> Key: XMLBEANS-410
> URL: https://issues.apache.org/jira/browse/XMLBEANS-410
> Project: XMLBeans
> Issue Type: Bug
> Components: Tools
> Affects Versions: Version 2.4.1
> Environment: All OS, all hardware
> Reporter: Mark D Henning
> Fix For: TBD
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> as an example, a xs:decimal (totalDig:20, fracDig:0) was creating 1000.00 whic of course, does not validate.
> The problem is in the org.apache.xmlbeans.impl.xsd2inst.SampleXmlUtil.java class.
> Root cause of the problem is that the programmer was calling the setScale() method on a BigDecimal, as though it modifies the existing BigDecimal object (i.e. BigDecimal bd = new BigDecimal("1000.00"); bd.setScale(1)) setScale does not modify the existing, but returns a new BigDecimal object. Therefore the correct call would be (BigDecimal bd = new BigDecimal("1000.00"); bd = bd.setScale(1)).
> The lines included below represent the repair of the formatDecimal() method which is the eronious function.
> // We have the number
> // Adjust the scale according to the totalDigits and fractionDigits
> int digits = 0;
> BigDecimal ONE = new BigDecimal(BigInteger.ONE);
> for (BigDecimal n = result; n.abs().compareTo(ONE) >= 0; digits++)
> n = n.movePointLeft(1);
> if (fractionDigits > 0)
> if (totalDigits >= 0)
> result = result.setScale(Math.max(fractionDigits, totalDigits - digits));
> else
> result = result.setScale(fractionDigits);
> else if (fractionDigits == 0)
> result = result.setScale(0);
> return result.toString();
> }
> I would recommend that the code tree be searched for other setScale method calls to see if others need to be fixed. I currently do not have write access to the subversion repository, so I am unable to check this fix in myself.
> I was unsure whether to code this as a minor or major problem. It appears that the difference between the two is the presence of a work-around, for which there is none in this case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@xmlbeans.apache.org
For additional commands, e-mail: dev-help@xmlbeans.apache.org
[jira] Resolved: (XMLBEANS-410) xsd2inst incorrectly handling
decimal precision
Posted by "Cezar Andrei (JIRA)" <xm...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XMLBEANS-410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cezar Andrei resolved XMLBEANS-410.
-----------------------------------
Resolution: Fixed
Fix Version/s: (was: TBD)
Version 2.4.1
Fix with rev 825680. Thanks Mark for the fix.
> xsd2inst incorrectly handling decimal precision
> -----------------------------------------------
>
> Key: XMLBEANS-410
> URL: https://issues.apache.org/jira/browse/XMLBEANS-410
> Project: XMLBeans
> Issue Type: Bug
> Components: Tools
> Affects Versions: Version 2.4.1
> Environment: All OS, all hardware
> Reporter: Mark D Henning
> Fix For: Version 2.4.1
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> as an example, a xs:decimal (totalDig:20, fracDig:0) was creating 1000.00 whic of course, does not validate.
> The problem is in the org.apache.xmlbeans.impl.xsd2inst.SampleXmlUtil.java class.
> Root cause of the problem is that the programmer was calling the setScale() method on a BigDecimal, as though it modifies the existing BigDecimal object (i.e. BigDecimal bd = new BigDecimal("1000.00"); bd.setScale(1)) setScale does not modify the existing, but returns a new BigDecimal object. Therefore the correct call would be (BigDecimal bd = new BigDecimal("1000.00"); bd = bd.setScale(1)).
> The lines included below represent the repair of the formatDecimal() method which is the eronious function.
> // We have the number
> // Adjust the scale according to the totalDigits and fractionDigits
> int digits = 0;
> BigDecimal ONE = new BigDecimal(BigInteger.ONE);
> for (BigDecimal n = result; n.abs().compareTo(ONE) >= 0; digits++)
> n = n.movePointLeft(1);
> if (fractionDigits > 0)
> if (totalDigits >= 0)
> result = result.setScale(Math.max(fractionDigits, totalDigits - digits));
> else
> result = result.setScale(fractionDigits);
> else if (fractionDigits == 0)
> result = result.setScale(0);
> return result.toString();
> }
> I would recommend that the code tree be searched for other setScale method calls to see if others need to be fixed. I currently do not have write access to the subversion repository, so I am unable to check this fix in myself.
> I was unsure whether to code this as a minor or major problem. It appears that the difference between the two is the presence of a work-around, for which there is none in this case.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@xmlbeans.apache.org
For additional commands, e-mail: dev-help@xmlbeans.apache.org