You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by sl...@apache.org on 2019/05/14 12:35:26 UTC
[incubator-daffodil] branch master updated: Ensure fragment bytes
are delivered correctly when dfdl:outputValueCalc is followed by a
different bitOrder
This is an automated email from the ASF dual-hosted git repository.
slawrence pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-daffodil.git
The following commit(s) were added to refs/heads/master by this push:
new 7e2ce3f Ensure fragment bytes are delivered correctly when dfdl:outputValueCalc is followed by a different bitOrder
7e2ce3f is described below
commit 7e2ce3fb07545ac9bc15bf7580a15c32b1e5166a
Author: Steve Lawrence <sl...@apache.org>
AuthorDate: Thu May 9 12:40:29 2019 -0400
Ensure fragment bytes are delivered correctly when dfdl:outputValueCalc is followed by a different bitOrder
When an OVC element is followed by an element with a different bitOrder,
the OVC causes a suspension and the buffered data output stream will
have a different bitOrder. Eventually the suspension will be resolved
and we may need to deliver a fragment byte to the direct data output
stream. We previously did that with putLong, but that uses the
FormatInfo from the suspension with the incorrect bitOrder. And
unfortunately we do not have a FormatInfo associated with the buffered
data output stream with the correct bitOrder.
To resolve this issue, this patch only uses putLong + FormatInfo when we
know the bitOrders of the direct and buffered data output streams must
be the same, i.e. the direct data output stream *does not* end on a byte
boundary. In this case, the FormatInfo from the suspension has the right
information, and we can use putLong to handle the complexities for
combining fragment bytes.
On the other hand, if the direct data output stream *does* end on a byte
boundary, then the bitOrder could be different, so we can't use putLong
since the FormatInfo is wrong. But we don't need to--we can just copy
the fragment byte from buffer to direct since it already ended on a byte
boundary.
DAFFODIL-2125
---
.../io/DirectOrBufferedDataOutputStream.scala | 35 ++++++++++++++----
.../calc_value_properties/outputValueCalc3.tdml | 41 ++++++++++++++++++++++
.../TestOutputValueCalc.scala | 2 ++
3 files changed, 71 insertions(+), 7 deletions(-)
diff --git a/daffodil-io/src/main/scala/org/apache/daffodil/io/DirectOrBufferedDataOutputStream.scala b/daffodil-io/src/main/scala/org/apache/daffodil/io/DirectOrBufferedDataOutputStream.scala
index 077db6b..4d8f031 100644
--- a/daffodil-io/src/main/scala/org/apache/daffodil/io/DirectOrBufferedDataOutputStream.scala
+++ b/daffodil-io/src/main/scala/org/apache/daffodil/io/DirectOrBufferedDataOutputStream.scala
@@ -790,13 +790,34 @@ object DirectOrBufferedDataOutputStream {
val wholeBytesWritten = directDOS.putBytes(ba, 0, byteCount, finfo)
Assert.invariant(byteCount == wholeBytesWritten)
if (nFragBits > 0) {
- val origfrag = bufDOS.fragmentLastByte
- val fragNum =
- if (finfoBitOrder eq BitOrder.MostSignificantBitFirst)
- origfrag >> (8 - nFragBits)
- else
- origfrag
- Assert.invariant(directDOS.putLongUnchecked(fragNum, nFragBits, finfo))
+ if (directDOS.isEndOnByteBoundary) {
+ // We cannot use putLong like below because it's possible that
+ // the fragment byte has a different bitOrder than the finfo
+ // passed in, since that came from a suspension. However, if
+ // the directDOS ended on a byte boundary, that means that its
+ // new fragment byte should be exactly the same as the buffered
+ // DOS fragment byte. So in this case, just copy the frag byte
+ // information from buffered to direct.
+ directDOS.setFragmentLastByte(bufDOS.fragmentLastByte, bufDOS.fragmentLastByteLimit)
+ } else {
+ // If the direct DOS wasn't byte aligned, then we need logic to
+ // write the buffered DOS fragment after the direct DOS
+ // fragment. Fortunately, putLong has all of this logic. Like
+ // above, the call to putLong potentially uses the wrong finfo
+ // since it may have come from a suspension. However, all that
+ // putLong really uses from the finfo is the bitOrder. And
+ // because the directDOS isn't byte aligned we know it must
+ // have the same bitOrder as the buffered DOS. So even though
+ // it could be the wrong format info, it's safe to use in this
+ // case.
+ val origfrag = bufDOS.fragmentLastByte
+ val fragNum =
+ if (finfoBitOrder eq BitOrder.MostSignificantBitFirst)
+ origfrag >> (8 - nFragBits)
+ else
+ origfrag
+ Assert.invariant(directDOS.putLongUnchecked(fragNum, nFragBits, finfo))
+ }
}
//
// bufDOS contents have now been output into directDOS
diff --git a/daffodil-test/src/test/resources/org/apache/daffodil/section17/calc_value_properties/outputValueCalc3.tdml b/daffodil-test/src/test/resources/org/apache/daffodil/section17/calc_value_properties/outputValueCalc3.tdml
index 301c11c..42fea4a 100644
--- a/daffodil-test/src/test/resources/org/apache/daffodil/section17/calc_value_properties/outputValueCalc3.tdml
+++ b/daffodil-test/src/test/resources/org/apache/daffodil/section17/calc_value_properties/outputValueCalc3.tdml
@@ -90,6 +90,26 @@
</xs:complexType>
</xs:element>
+ <xs:element name="ovcBitOrderChange">
+ <xs:complexType>
+ <xs:sequence>
+ <xs:element name="zero" type="xs:int" dfdl:lengthKind="explicit" dfdl:length="8"
+ dfdl:bitOrder="mostSignificantBitFirst" dfdl:byteOrder="bigEndian"
+ dfdl:outputValueCalc="{ ../body/zero }" />
+ <xs:element name="body">
+ <xs:complexType>
+ <xs:sequence>
+ <xs:element name="a" type="xs:int" dfdl:lengthKind="explicit" dfdl:length="4"/>
+ <xs:element name="b" type="xs:int" dfdl:lengthKind="explicit" dfdl:length="4"
+ dfdl:outputValueCalc="{ ../zero }"/>
+ <xs:element name="zero" type="xs:int" dfdl:inputValueCalc="{ 0 }"/>
+ </xs:sequence>
+ </xs:complexType>
+ </xs:element>
+ </xs:sequence>
+ </xs:complexType>
+ </xs:element>
+
</tdml:defineSchema>
<tdml:unparserTestCase name="rHexBinaryLSBF1" root="rHexBinary" model="s" roundTrip="onePass">
@@ -144,6 +164,27 @@
</tdml:dfdlInfoset>
</tdml:infoset>
</tdml:unparserTestCase>
+
+ <tdml:unparserTestCase name="ovc_bitOrderChange" root="ovcBitOrderChange" model="s" roundTrip="onePass">
+ <tdml:document>
+ <tdml:documentPart type="byte">
+ 00 02
+ </tdml:documentPart>
+ </tdml:document>
+
+ <tdml:infoset>
+ <tdml:dfdlInfoset>
+ <ex:ovcBitOrderChange>
+ <zero>0</zero>
+ <body>
+ <a>2</a>
+ <b>0</b>
+ <zero>0</zero>
+ </body>
+ </ex:ovcBitOrderChange>
+ </tdml:dfdlInfoset>
+ </tdml:infoset>
+ </tdml:unparserTestCase>
</tdml:testSuite>
diff --git a/daffodil-test/src/test/scala/org/apache/daffodil/section17/calc_value_properties/TestOutputValueCalc.scala b/daffodil-test/src/test/scala/org/apache/daffodil/section17/calc_value_properties/TestOutputValueCalc.scala
index 00e6607..7a722ff 100644
--- a/daffodil-test/src/test/scala/org/apache/daffodil/section17/calc_value_properties/TestOutputValueCalc.scala
+++ b/daffodil-test/src/test/scala/org/apache/daffodil/section17/calc_value_properties/TestOutputValueCalc.scala
@@ -80,4 +80,6 @@ class TestOutputValueCalc {
@Test def test_ovcHexBinaryLSBF1() { runner3.runOneTest("rHexBinaryLSBF1") }
@Test def test_ovcHexBinaryLSBF2() { runner3.runOneTest("rHexBinaryLSBF2") }
@Test def test_ovcStringLSBF1() { runner3.runOneTest("rStringLSBF1") }
+
+ @Test def test_ovcBitOrderChange() { runner3.runOneTest("ovc_bitOrderChange") }
}