[GitHub] [incubator-daffodil] stevedlawrence commented on a change in pull request #395: Add test_nulPattern1

[GitBox <gi...@apache.org>]

stevedlawrence commented on a change in pull request #395:
URL: https://github.com/apache/incubator-daffodil/pull/395#discussion_r452788392



##########
File path: daffodil-test/src/test/resources/org/apache/daffodil/section05/facets/NulChars.tdml
##########
@@ -0,0 +1,95 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+-->
+<tdml:testSuite
+  xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/"
+  xmlns:dfdlx="http://www.ogf.org/dfdl/dfdl-1.0/extensions"
+  xmlns:tdml="http://www.ibm.com/xmlns/dfdl/testData"
+  xmlns:ex="http://example.com"
+  xmlns:xs="http://www.w3.org/2001/XMLSchema"
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+  defaultRoundTrip="onePass"
+  defaultValidation="on">
+
+  <tdml:defineSchema name="nullTest">
+    <xs:include schemaLocation="org/apache/daffodil/xsd/DFDLGeneralFormat.dfdl.xsd"/>
+
+    <dfdl:format ref="ex:GeneralFormat"/>
+
+    <xs:element name="root">
+      <xs:complexType>
+        <xs:sequence>
+          <xs:element name="foo" dfdl:lengthKind="pattern" dfdl:lengthPattern="\x{0000}{3,3}">
+            <xs:simpleType>
+              <xs:restriction base="xs:string">
+                <xs:pattern value="&#xE000;*"/>

Review comment:
       I suspect the issue here is that the infoset and XML actually have different values during different times of validation. 
   
   Our internal infoset representation has actual NUL characters in them. I think this is correct since some infoset representations (like json) have no problem handling NUL characters. This means the the xE000 pattern will fail when Daffodil performs it's validation. 
   
   When we perform Xerces validation, we generate XML and thus convert those NUL characters to 0xE000. Which will pass the Xerces check.
   
   So the infoset looks different to Daffodil and Xerces, but are using the same pattern. It seems like this can never work unless you change the pattern to something like ($#x0000*|$#xE000), or if we change how we handle XML illegal characters. Or should Daffodil validation do an XML conversion and then validate? I'm not sure what the right behavior is, perhaps this is expected behavior and not a bug?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org