You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-dev@xerces.apache.org by Robert Hairgrove <ev...@hispeed.ch.INVALID> on 2022/08/28 13:24:59 UTC
Xerces 4 and XML Schema 1.1 validation
The platforms I am using are Linux Ubuntu 18.04 with GCC 7.50 and
Windows 10. My C++ application (built with Qt libraries 5.15.5 for
Windows and Linux) links to Xerces-C++ as a library and performs
validation on a XML file used to configure the importing of CSV text
files against an embedded XSD schema file. Having used this setup with
Xerces-C++ version 3.2.3 and the XSD file formatted for XML Schema 1.0,
it works very well. The import of the data is done using the opensource
"libcsv" library (located on GitHub: https://github.com/rgamble/libcsv),
and I am performing validation on a per-field basis in the callback
function using the configuration from the XML file which validates,
among other things, against regular expressions and minimum/maximum
values, etc.
Now the documentation for Xerces-C++ states that validation with XML
Schema 1.1 is possible. I downloaded the sources for the latest version
from GitHub (Xerces version = 4.0.0) and built my application using
that. It appears that even now, one of the most useful new features of
XML Schema 1.1 (assertions) is not working quite the way it should.
I have attached a minimal test case of what I am trying to do: CSV files
can have column headers in their first line, or maybe not. So I have two
sets of column definitions, one for files with headers, and one for
headerless files. The <col> elements in the first set have a 'name'
attribute, which is the column header; the elements in the second set
have an 'ord' attribute which is basically just a sequential integer
telling the application in which order the column appears in the file.
So I cannot define the 'use' attribute in the XSD as "required" for
either attribute, but I wanted to add '<xs:assert
test="exists(@name|@ord)/>' after the list of <col> attributes, as you
can see from the attached example.
I tested this on the site mentioned in the Apache website for Xerces-C++
at https://www.softwarebytes.org/xmlvalidation/. It seems to work quite
well; when I remove one of the "ord" attributes so that neither "name"
nor "ord" appears, I get a validation error: "Assertion evaluation
('exists(@name|@ord)') for element 'col' on schema type
'#AnonType_colcolumnsimportfileroot' did not succeed." When I put it
back, it says it is valid.
On Xerces-C++ 4.0.0, using the same files, it throws no errors, neither
with the apparently valid XML nor the faulty one with the missing "ord"
attribute. I am using the Xerces SAX2XMLReader with both XML and XSD
passed in a memory buffer. However, I might be setting it up wrong? Here
is the relevant part of my parsing code (I left out the code for the
error handler which is correctly implemented; you can refer to the Qt
documentation for the QByteArray and QIODevice classes which set up the
buffers from the data in the file):
<snip>
// fi_xsd and fi_xml are of type QFileInfo:
bool retval = false;
QFile schema(fi_xsd.absoluteFilePath());
QFile xmlfile(fi_xml.absoluteFilePath());
QByteArray ba_xsd, ba_xml;
if (schema.open(QIODevice::ReadOnly)) {
ba_xsd = schema.readAll();
schema.close();
}
if (xmlfile.open(QIODevice::ReadOnly)) {
ba_xml = xmlfile.readAll();
xmlfile.close();
}
try {
xercesc::SAX2XMLReader *parser =
xercesc::XMLReaderFactory::createXMLReader();
xercesc::ErrorHandler *handler = new CErrorHandler();
xercesc::MemBufInputSource
inMemorySchemaSource(reinterpret_cast<const
XMLByte*>(ba_xsd.constData()), ba_xsd.size(), "/schema.xsd");
xercesc::MemBufInputSource inMemoryXMLSource(reinterpret_cast<const
XMLByte*>(ba_xml.constData()), ba_xml.size (), "/xml.xsd");
parser->loadGrammar(inMemorySchemaSource,
xercesc::Grammar::SchemaGrammarType, true);
parser->setFeature(xercesc::XMLUni::fgXercesUseCachedGrammarInParse, true);
parser->setFeature(xercesc::XMLUni::fgSAX2CoreValidation, true);
parser->setFeature(xercesc::XMLUni::fgSAX2CoreNameSpaces, true);
parser->setFeature(xercesc::XMLUni::fgXercesSchemaFullChecking, true);
parser->setProperty(xercesc::XMLUni::fgXercesSchemaExternalNoNameSpaceSchemaLocation,
const_cast<void*>(static_cast<const void*>("")));
parser->setErrorHandler(handler);
parser->parse(inMemoryXMLSource);
if (parser->getErrorCount() != 0) {
std::cout << "ERROR: XML file '" << xml_file_path_ << "' does not
conform to the schema" << std::endl;
} else {
std::cout << "Valid!" << std::endl;
retval = true;
}
}
catch (const xercesc::SAXParseException &) {
std::cout << "Unhandled SAXParseException was caught" << std::endl;
}
catch (const xercesc::SAXException &) {
std::cout << "Unhandled SAXException was caught" << std::endl;
}
return retval;
</snip>
In addition to the bits one has to add to the <xs:schema> element, is
there anything else I have to do to turn on 1.1 validation in Xerces?
Thank you.
RE: Xerces 4 and XML Schema 1.1 validation
Posted by Roger Leigh <rl...@codelibre.net>.
Hi Robert,
I don't think you've overlooked anything here.
Kind regards,
Roger
> -----Original Message-----
> From: Robert Hairgrove <ev...@hispeed.ch.INVALID>
> Sent: 29 August 2022 10:10
> To: c-dev@xerces.apache.org; c-users@xerces.apache.org
> Subject: Re: Xerces 4 and XML Schema 1.1 validation
>
> It looks like I was confused by the statements found here under "Features"
> (https://xerces.apache.org/xerces-c/):
>
> * Conforms to
> o XML 1.0 (Third Edition)
> <http://www.w3.org/TR/2004/REC-xml-20040204/>, W3C
> Recommendation
> o XML 1.1 (First Edition)
> <http://www.w3.org/TR/2004/REC-xml11-20040204/>, W3C
> Recommendation (Note: section 2.13 Normalization Checking has
> not been implemented)
>
> After doing some more research, I realize that I am looking for XSD 1.1
> support (which Xerces-J apparently has, but Xerces-C++ does not) and not
> XML 1.1 which has nothing to do with XSD.
>
> Or did I overlook something else?
RE: Xerces 4 and XML Schema 1.1 validation
Posted by Roger Leigh <rl...@codelibre.net>.
Hi Robert,
I don't think you've overlooked anything here.
Kind regards,
Roger
> -----Original Message-----
> From: Robert Hairgrove <ev...@hispeed.ch.INVALID>
> Sent: 29 August 2022 10:10
> To: c-dev@xerces.apache.org; c-users@xerces.apache.org
> Subject: Re: Xerces 4 and XML Schema 1.1 validation
>
> It looks like I was confused by the statements found here under "Features"
> (https://xerces.apache.org/xerces-c/):
>
> * Conforms to
> o XML 1.0 (Third Edition)
> <http://www.w3.org/TR/2004/REC-xml-20040204/>, W3C
> Recommendation
> o XML 1.1 (First Edition)
> <http://www.w3.org/TR/2004/REC-xml11-20040204/>, W3C
> Recommendation (Note: section 2.13 Normalization Checking has
> not been implemented)
>
> After doing some more research, I realize that I am looking for XSD 1.1
> support (which Xerces-J apparently has, but Xerces-C++ does not) and not
> XML 1.1 which has nothing to do with XSD.
>
> Or did I overlook something else?
Re: Xerces 4 and XML Schema 1.1 validation
Posted by Robert Hairgrove <ev...@hispeed.ch.INVALID>.
It looks like I was confused by the statements found here under
"Features" (https://xerces.apache.org/xerces-c/):
* Conforms to
o XML 1.0 (Third Edition)
<http://www.w3.org/TR/2004/REC-xml-20040204/>, W3C Recommendation
o XML 1.1 (First Edition)
<http://www.w3.org/TR/2004/REC-xml11-20040204/>, W3C
Recommendation (Note: section 2.13 Normalization Checking has
not been implemented)
After doing some more research, I realize that I am looking for XSD 1.1
support (which Xerces-J apparently has, but Xerces-C++ does not) and not
XML 1.1 which has nothing to do with XSD.
Or did I overlook something else?
Re: Xerces 4 and XML Schema 1.1 validation
Posted by Robert Hairgrove <ev...@hispeed.ch.INVALID>.
It looks like I was confused by the statements found here under
"Features" (https://xerces.apache.org/xerces-c/):
* Conforms to
o XML 1.0 (Third Edition)
<http://www.w3.org/TR/2004/REC-xml-20040204/>, W3C Recommendation
o XML 1.1 (First Edition)
<http://www.w3.org/TR/2004/REC-xml11-20040204/>, W3C
Recommendation (Note: section 2.13 Normalization Checking has
not been implemented)
After doing some more research, I realize that I am looking for XSD 1.1
support (which Xerces-J apparently has, but Xerces-C++ does not) and not
XML 1.1 which has nothing to do with XSD.
Or did I overlook something else?
Re: Xerces 4 and XML Schema 1.1 validation
Posted by Robert Hairgrove <ev...@hispeed.ch.INVALID>.
On 8/28/22 18:02, Roger Leigh wrote:
> Hi Robert,
>
> If you compiled without network access, then you'll need to be sure that all of the schema files are accessible, for example by implementing a custom EntityResolver. At a minimum, you'll want to provide your own schemas, plus any schemas they depend upon, e.g. XMLSchema.
>
> I would suggest testing with a working netaccessor before turning it off, to eliminate the possibility that it just can't fetch the schemas, before going further with the custom EntityResolver, in case there are other underlying issues which haven't been identified yet.
>
> By the way, the master branch (4.0.0) is not released yet, and while no issues have been reported to date, you might possibly hit problems using it. I would recommend the xerces-3.2 stable branch or the 3.2.3 stable release.
>
> Kind regards,
> Roger
Hello Roger,
Thank you for taking time to look at this.
I can validate my little test XML file (attached to my first post in
this thread, or did it get removed?) with Xerces-C++ 4.0.0 using XML
Schema 1.0 against my XSD file (also attached to my first post here)
just fine if I remove the bits needed to turn on 1.1, as it were:
- xmlns:vc="http://www.w3.org/2007/XMLSchema-versioning"
- vc:minVersion="1.1"
Setting use="required" for attribute "org" gives me these errors, even
when not connected to the internet:
missing required attribute 'ord' at line 5 column 118
missing required attribute 'ord' at line 6 column 137
missing required attribute 'ord' at line 9 column 108
Changing it back to "optional" says it is valid. I assume that it is
therefore not looking on the internet for XMLSchema.
As you might suppose, for me this is very much a "learning by doing"
experience WRT XML and XML Schema, and I might not see something obvious
that I am doing wrong. But I don't think it has anything to do with
network access, does it? After all, if the library cannot find a schema
which is needed by the XSD file, wouldn't it throw some kind of exception?
Of course, I won't release my product using Xerces 4 until it is
officially released.
Cheers,
Robert
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
Re: Xerces 4 and XML Schema 1.1 validation
Posted by Robert Hairgrove <ev...@hispeed.ch.INVALID>.
On 8/28/22 18:02, Roger Leigh wrote:
> Hi Robert,
>
> If you compiled without network access, then you'll need to be sure that all of the schema files are accessible, for example by implementing a custom EntityResolver. At a minimum, you'll want to provide your own schemas, plus any schemas they depend upon, e.g. XMLSchema.
>
> I would suggest testing with a working netaccessor before turning it off, to eliminate the possibility that it just can't fetch the schemas, before going further with the custom EntityResolver, in case there are other underlying issues which haven't been identified yet.
>
> By the way, the master branch (4.0.0) is not released yet, and while no issues have been reported to date, you might possibly hit problems using it. I would recommend the xerces-3.2 stable branch or the 3.2.3 stable release.
>
> Kind regards,
> Roger
Hello Roger,
Thank you for taking time to look at this.
I can validate my little test XML file (attached to my first post in
this thread, or did it get removed?) with Xerces-C++ 4.0.0 using XML
Schema 1.0 against my XSD file (also attached to my first post here)
just fine if I remove the bits needed to turn on 1.1, as it were:
- xmlns:vc="http://www.w3.org/2007/XMLSchema-versioning"
- vc:minVersion="1.1"
Setting use="required" for attribute "org" gives me these errors, even
when not connected to the internet:
missing required attribute 'ord' at line 5 column 118
missing required attribute 'ord' at line 6 column 137
missing required attribute 'ord' at line 9 column 108
Changing it back to "optional" says it is valid. I assume that it is
therefore not looking on the internet for XMLSchema.
As you might suppose, for me this is very much a "learning by doing"
experience WRT XML and XML Schema, and I might not see something obvious
that I am doing wrong. But I don't think it has anything to do with
network access, does it? After all, if the library cannot find a schema
which is needed by the XSD file, wouldn't it throw some kind of exception?
Of course, I won't release my product using Xerces 4 until it is
officially released.
Cheers,
Robert
RE: Xerces 4 and XML Schema 1.1 validation
Posted by Roger Leigh <rl...@codelibre.net>.
Hi Robert,
If you compiled without network access, then you'll need to be sure that all of the schema files are accessible, for example by implementing a custom EntityResolver. At a minimum, you'll want to provide your own schemas, plus any schemas they depend upon, e.g. XMLSchema.
I would suggest testing with a working netaccessor before turning it off, to eliminate the possibility that it just can't fetch the schemas, before going further with the custom EntityResolver, in case there are other underlying issues which haven't been identified yet.
By the way, the master branch (4.0.0) is not released yet, and while no issues have been reported to date, you might possibly hit problems using it. I would recommend the xerces-3.2 stable branch or the 3.2.3 stable release.
Kind regards,
Roger
> -----Original Message-----
> From: Robert Hairgrove <ev...@hispeed.ch.INVALID>
> Sent: 28 August 2022 15:17
> To: c-dev@xerces.apache.org; c-users@xerces.apache.org
> Subject: Re: Xerces 4 and XML Schema 1.1 validation
>
> BTW, I compiled Xerces-C++ with network access disabled since my users will
> not necessarily be connected to the internet, or will be behind a firewall.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
> For additional commands, e-mail: c-dev-help@xerces.apache.org
RE: Xerces 4 and XML Schema 1.1 validation
Posted by Roger Leigh <rl...@codelibre.net>.
Hi Robert,
If you compiled without network access, then you'll need to be sure that all of the schema files are accessible, for example by implementing a custom EntityResolver. At a minimum, you'll want to provide your own schemas, plus any schemas they depend upon, e.g. XMLSchema.
I would suggest testing with a working netaccessor before turning it off, to eliminate the possibility that it just can't fetch the schemas, before going further with the custom EntityResolver, in case there are other underlying issues which haven't been identified yet.
By the way, the master branch (4.0.0) is not released yet, and while no issues have been reported to date, you might possibly hit problems using it. I would recommend the xerces-3.2 stable branch or the 3.2.3 stable release.
Kind regards,
Roger
> -----Original Message-----
> From: Robert Hairgrove <ev...@hispeed.ch.INVALID>
> Sent: 28 August 2022 15:17
> To: c-dev@xerces.apache.org; c-users@xerces.apache.org
> Subject: Re: Xerces 4 and XML Schema 1.1 validation
>
> BTW, I compiled Xerces-C++ with network access disabled since my users will
> not necessarily be connected to the internet, or will be behind a firewall.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
> For additional commands, e-mail: c-dev-help@xerces.apache.org
Re: Xerces 4 and XML Schema 1.1 validation
Posted by Robert Hairgrove <ev...@hispeed.ch.INVALID>.
BTW, I compiled Xerces-C++ with network access disabled since my users
will not necessarily be connected to the internet, or will be behind a
firewall.
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
Re: Xerces 4 and XML Schema 1.1 validation
Posted by Robert Hairgrove <ev...@hispeed.ch.INVALID>.
BTW, I compiled Xerces-C++ with network access disabled since my users
will not necessarily be connected to the internet, or will be behind a
firewall.