[GitHub] [incubator-daffodil] mbeckerle commented on a change in pull request #395: Add test_nulPattern1

[GitBox <gi...@apache.org>]

mbeckerle commented on a change in pull request #395:
URL: https://github.com/apache/incubator-daffodil/pull/395#discussion_r452865746



##########
File path: daffodil-test/src/test/resources/org/apache/daffodil/section05/facets/NulChars.tdml
##########
@@ -0,0 +1,95 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+-->
+<tdml:testSuite
+  xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/"
+  xmlns:dfdlx="http://www.ogf.org/dfdl/dfdl-1.0/extensions"
+  xmlns:tdml="http://www.ibm.com/xmlns/dfdl/testData"
+  xmlns:ex="http://example.com"
+  xmlns:xs="http://www.w3.org/2001/XMLSchema"
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+  defaultRoundTrip="onePass"
+  defaultValidation="on">
+
+  <tdml:defineSchema name="nullTest">
+    <xs:include schemaLocation="org/apache/daffodil/xsd/DFDLGeneralFormat.dfdl.xsd"/>
+
+    <dfdl:format ref="ex:GeneralFormat"/>
+
+    <xs:element name="root">
+      <xs:complexType>
+        <xs:sequence>
+          <xs:element name="foo" dfdl:lengthKind="pattern" dfdl:lengthPattern="\x{0000}{3,3}">
+            <xs:simpleType>
+              <xs:restriction base="xs:string">
+                <xs:pattern value="&#xE000;*"/>

Review comment:
       So using both E000 and 0000 won't work because you can't say &#x0000; in an XML document of any kind. We would have to interpret the pattern looking for DFDL character entities and allow [&#xE000;%#x0000;]* for DFDL's internal validation, but then we'd have to remove the %#x0000; for Xerces XML Validation. 
   
   Our internal validation needs to take the pattern, and apply the same backward transformation to its characters that we use for XML data. So the internal validation would convert that &#xE000; back into a true NUL character that it is searching for in the infoset. That appears to be the bug. 
   
   Another alternative, I wasn't aware that when we turn Validation On, that *both* our internal validation AND Xerces run. Maybe we should have a Validation="xml" setting that only runs Xerces validation. This is also perhaps a special DFDL-language extension mode, where you can put any XML Schema validations at all, including key-constraints, unique constraints, patterns on hexBinary, all the things DFDL usually disallows, because the validations would be used ONLY for XML validation of an XML infoset. I suppose a few constraints on what you could write for facets would still apply, such as things about single-byte charsets when lengthUnits is bytes and lengthKind is "implicit" for strings. 
   
   There is another ticket for lifting DFDL's restrictions on key/unique constraints, again so that people can write one schema and use it for DFDL parse/unparse, as well as for more comprehensive XML Schema validation. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org