You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by mb...@apache.org on 2018/01/05 14:43:25 UTC
[incubator-daffodil] 02/02: Revised daffodil-io module to require
passing in a FormatInfo object.
This is an automated email from the ASF dual-hosted git repository.
mbeckerle pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-daffodil.git
commit a1d881b13d26e47040c0302b2874cc78f0de27bf
Author: Mike Beckerle <mb...@tresys.com>
AuthorDate: Wed May 31 09:44:57 2017 -0400
Revised daffodil-io module to require passing in a FormatInfo object.
This is just one part in addressing this major issue of daffodil
schema compiler slowness/speed.
It also fixes DAFFODIL-1843 (bit order and OVC interaction bug)
The FormatInfo object is used to obtain the very commonly used format
properties like byteOrder, bitOrder, etc. This eliminates the need to
call setters that set these properties, eliminates the need to
save/restore them on backtracking, and when evaluating unparser
suspensions. Instead, the daffodil runtime just passes an object
(the PState/UState) which contains this information - getting it from
the ERD of an element most likely, or if runtime defined, from an
Evaluatable object which caches the property value on the infoset
element.
The state/finfo has a ref to the processor which must be set so that
FormatInfo can obtain the right ERD/MGRD from that.
THis change eliminates all the "change" processors
(e.g., EncodingChangeParser, ByteOrderChangeParser, and
BitOrderChangeParser and
unparser variants thereof) the insertion of which uses compile-time
context-upward analysis. This should help with increasing sharing and
avoiding duplicated work and data structures in the schema compiler
(DAFFODIL-1444)
There is some overhead in this change - performance impact will have to
be measured. It should reduce backtracking overheads to compensate
perhaps.
Unparser runtime was improved also to remove unneeded DOS splits.
We were splitting the DOS even for evaluations of just expressions -
things that couldn't write to the DOS at all. So we were creating
lots of known zero-length buffering DOS for no reason.
Improved scaladoc and comments to the suspension-related code
areas.
It seems our test coverage of LE_MSBF is thin at the unit test level.
Two TDML tests moved from scala-debug to scala-new as they work now.
Enhance TDML runner for config, defaultConfig, in files or embedded.
Note: Entity replacer/cooker for fill byte (FillByteCooker) isn't
behaving
correctly. See comments in DAFFODIL-1646.
DAFFODIL-1444, DAFFODIL-1843, DAFFODIL-1868
---
daffodil-cli/README_How_to_Debug.txt | 15 +
.../scala/edu/illinois/ncsa/daffodil/Main.scala | 11 +-
.../ncsa/daffodil/debugger/TestCLIDebugger.scala | 9 +-
.../illinois/ncsa/daffodil/dsom/ChoiceGroup.scala | 5 +-
.../ncsa/daffodil/dsom/DFDLAnnotation.scala | 5 -
.../illinois/ncsa/daffodil/dsom/ElementBase.scala | 13 +-
.../RawCommonRuntimeValuedPropertiesMixin.scala | 3 +
.../ncsa/daffodil/dsom/RuntimePropertyMixins.scala | 59 ++-
.../illinois/ncsa/daffodil/dsom/SchemaSet.scala | 5 +-
.../ncsa/daffodil/dsom/SequenceGroup.scala | 5 +-
.../ncsa/daffodil/dsom/TermEncodingMixin.scala | 14 +-
.../ncsa/daffodil/grammar/BitOrderMixin.scala | 49 +-
.../ncsa/daffodil/grammar/ByteOrderMixin.scala | 37 --
.../daffodil/grammar/ElementBaseGrammarMixin.scala | 4 +-
.../daffodil/grammar/EncodingChangeMixin.scala | 73 ---
.../daffodil/grammar/ModelGroupGrammarMixin.scala | 2 +-
.../ncsa/daffodil/grammar/TermGrammarMixin.scala | 5 +-
.../grammar/primitives/BitOrderChange.scala | 71 ---
.../grammar/primitives/ByteOrderChange.scala | 54 ---
.../grammar/primitives/ElementCombinator.scala | 8 +-
.../grammar/primitives/EncodingChange.scala | 60 ---
.../primitives/PrimitivesBinaryNumber.scala | 5 +
.../primitives/PrimitivesElementKinds.scala | 16 +-
.../grammar/primitives/PrimitivesFraming.scala | 12 +-
.../grammar/primitives/PrimitivesLengthKind.scala | 12 +-
.../grammar/primitives/PrimitivesNil.scala | 2 +-
.../grammar/primitives/PrimitivesTextNumber.scala | 8 +-
.../daffodil/grammar/primitives/RepPrims.scala | 2 +-
.../ncsa/daffodil/dsom/TestBinaryInput_01.scala | 168 ++++---
.../daffodil/io/ByteBufferDataInputStream.scala | 262 ++++-------
.../daffodil/io/CharBufferDataOutputStream.scala | 88 ----
.../ncsa/daffodil/io/DataInputStream.scala | 53 +--
.../daffodil/io/DataInputStreamImplMixin.scala | 16 +-
.../ncsa/daffodil/io/DataOutputStream.scala | 46 +-
.../daffodil/io/DataOutputStreamImplMixin.scala | 506 +++++++++------------
.../ncsa/daffodil/io/DataStreamCommon.scala | 43 +-
.../daffodil/io/DataStreamCommonImplMixin.scala | 55 +--
.../io/DirectOrBufferedDataOutputStream.scala | 68 ++-
.../edu/illinois/ncsa/daffodil/io/FormatInfo.scala | 128 ++++++
.../io/StringDataInputStreamForUnparse.scala | 50 +-
.../daffodil/processors/charset/CharsetUtils.scala | 80 +++-
.../ncsa/daffodil/io/FormatInfoForUnitTest.scala | 82 ++++
.../io/TestByteBufferDataInputStream.scala | 168 ++++---
.../io/TestByteBufferDataInputStream2.scala | 23 +-
.../io/TestByteBufferDataInputStream3Bit.scala | 66 +--
.../io/TestByteBufferDataInputStream4.scala | 25 +-
.../io/TestByteBufferDataInputStream5.scala | 24 +-
.../io/TestByteBufferDataInputStream6.scala | 64 +--
.../io/TestByteBufferDataInputStream7.scala | 50 +-
.../ncsa/daffodil/io/TestDataOutputStream.scala | 280 +++++-------
.../ncsa/daffodil/io/TestDataOutputStream2.scala | 46 +-
.../ncsa/daffodil/io/TestDataOutputStream3.scala | 296 +++++-------
.../ncsa/daffodil/io/TestDataOutputStream4.scala | 66 +--
.../io/TestDirectOrBufferedDataOutputStream.scala | 41 +-
.../edu/illinois/ncsa/daffodil/util/Maybe.scala | 3 +-
.../src/main/scala/passera/unsigned/ULong.scala | 1 +
.../unparsers/BinaryBooleanUnparsers.scala | 9 +-
.../unparsers/BinaryNumberUnparsers.scala | 23 +-
.../unparsers/BitOrderChangeUnparser.scala | 76 ----
.../unparsers/ByteOrderChangeUnparser.scala | 49 --
.../unparsers/ConvertTextBooleanUnparser.scala | 2 +-
.../unparsers/ConvertTextCalendarUnparser.scala | 2 +-
.../unparsers/ConvertTextNumberUnparser.scala | 6 +-
.../processors/unparsers/DelimitedUnparsers.scala | 8 +-
.../processors/unparsers/DelimiterUnparsers.scala | 5 +-
.../unparsers/ElementKindUnparsers.scala | 23 +-
.../processors/unparsers/ElementUnparser.scala | 9 +-
.../unparsers/EncodingChangeUnparser.scala | 50 --
.../unparsers/ExpressionEvaluatingUnparsers.scala | 10 +
.../processors/unparsers/FramingUnparsers.scala | 31 +-
.../unparsers/HexBinaryLengthUnparser.scala | 19 +-
.../unparsers/NilEmptyCombinatorUnparsers.scala | 18 +-
.../unparsers/OptionalInfixSepUnparser.scala | 6 +-
.../processors/unparsers/SpecifiedLength2.scala | 52 ++-
.../unparsers/SpecifiedLengthUnparsers.scala | 6 +-
.../unparsers/StringLengthUnparsers.scala | 11 +-
.../daffodil/debugger/InteractiveDebugger.scala | 32 +-
.../daffodil/dpath/SuspendableExpression.scala | 32 +-
.../ncsa/daffodil/infoset/InfosetImpl.scala | 1 +
.../ncsa/daffodil/infoset/InfosetInputter.scala | 2 +-
.../daffodil/processors/EncodingRuntimeData.scala | 32 +-
.../ncsa/daffodil/processors/EvByteOrder.scala | 34 +-
.../ncsa/daffodil/processors/EvEncoding.scala | 6 +-
.../ncsa/daffodil/processors/ProcessorBases.scala | 14 +-
.../daffodil/processors/ProcessorStateBases.scala | 166 ++++++-
.../ncsa/daffodil/processors/Runtime.scala | 23 +-
.../ncsa/daffodil/processors/RuntimeData.scala | 81 +++-
.../daffodil/processors/SuspendableOperation.scala | 10 +-
.../ncsa/daffodil/processors/Suspension.scala | 176 ++++---
.../daffodil/processors/dfa/CreatePaddingDFA.scala | 6 +-
.../ncsa/daffodil/processors/dfa/Registers.scala | 16 +-
.../processors/dfa/TextDelimitedParser.scala | 45 +-
.../processors/dfa/TextDelimitedUnparser.scala | 18 +-
.../processors/dfa/TextPaddingParser.scala | 5 +-
.../ncsa/daffodil/processors/dfa/TextParser.scala | 11 +-
.../processors/parsers/AssertPatternParsers.scala | 2 +-
.../processors/parsers/BinaryBooleanParsers.scala | 2 +-
.../processors/parsers/BinaryNumberParsers.scala | 18 +-
.../processors/parsers/BitOrderChange1.scala | 59 ---
.../processors/parsers/ByteOrderChange1.scala | 54 ---
.../processors/parsers/DelimitedParsers.scala | 6 +-
.../processors/parsers/DelimiterParsers.scala | 10 +-
.../processors/parsers/ElementCombinator1.scala | 5 +-
.../processors/parsers/ElementKindParsers.scala | 11 +-
.../processors/parsers/EncodingChange1.scala | 51 ---
.../processors/parsers/FramingParsers.scala | 14 +-
.../parsers/HexBinaryLengthParsers.scala | 6 +-
.../parsers/NilEmptyCombinatorParsers.scala | 10 +-
.../daffodil/processors/parsers/NilParsers.scala | 2 +-
.../parsers/OptionalInfixSepParser.scala | 6 +-
.../ncsa/daffodil/processors/parsers/PState.scala | 97 +++-
.../ncsa/daffodil/processors/parsers/Parser.scala | 77 +---
.../daffodil/processors/parsers/PrimParser.scala | 10 +
.../processors/parsers/PrimitivesDateTime1.scala | 2 +-
.../processors/parsers/PrimitivesTextNumber1.scala | 80 ++--
.../daffodil/processors/parsers/RepParsers.scala | 4 +-
.../parsers/SpecifiedLengthParsers.scala | 6 +-
.../processors/parsers/StringLengthParsers.scala | 10 +-
.../processors/parsers/TextBooleanParser.scala | 2 +-
.../processors/parsers/ZonedTextParsers.scala | 2 +-
.../processors/unparsers/RepUnparsers.scala | 12 +-
.../daffodil/processors/unparsers/UState.scala | 116 ++++-
.../processors/unparsers/UnparseError.scala | 2 +-
.../daffodil/processors/unparsers/Unparser.scala | 71 ++-
.../illinois/ncsa/daffodil/tdml/TDMLRunner.scala | 73 ++-
.../src/test/resources/testConfigFile.xml | 6 +
.../ncsa/daffodil/tdml/TestTDMLRunner2.scala | 14 +-
.../ncsa/daffodil/tdml/TestTDMLRunnerConfig.scala | 201 ++++++++
.../section00/general/testUnparserBitOrderOVC.tdml | 36 +-
.../section06/entities/charClassEntities.dfdl.xsd | 39 +-
.../PropertiesNotDefined.dfdl.xsd | 3 +-
.../ncsa/daffodil/usertests/DefaultProperties.xsd | 1 +
.../general/TestUnparserGeneralDebug.scala | 3 -
.../section00/general/TestUnparserGeneral2.scala | 36 ++
eclipse-projects/cli-test/.classpath | 2 +-
eclipse-projects/cli-test/.project | 5 +
eclipse-projects/cli/.classpath | 2 +-
eclipse-projects/core-test/.classpath | 2 +-
eclipse-projects/core/.classpath | 2 +-
eclipse-projects/io-lib/.classpath | 2 +-
eclipse-projects/io-test/.classpath | 2 +-
eclipse-projects/japi-test/.classpath | 2 +-
eclipse-projects/japi/.classpath | 2 +-
eclipse-projects/propgen/.classpath | 2 +-
eclipse-projects/runtime1-test/.classpath | 2 +-
eclipse-projects/runtime1-unparser-test/.classpath | 2 +-
eclipse-projects/runtime1-unparser/.classpath | 2 +-
eclipse-projects/runtime1/.classpath | 2 +-
eclipse-projects/sapi-test/.classpath | 2 +-
eclipse-projects/sapi/.classpath | 2 +-
eclipse-projects/tdml-test/.classpath | 2 +-
eclipse-projects/tdml/.classpath | 2 +-
eclipse-projects/test-ibm1/.classpath | 2 +-
eclipse-projects/test-stdLayout/.classpath | 2 +-
eclipse-projects/test/.classpath | 2 +-
eclipse-projects/tutorials/.classpath | 2 +-
156 files changed, 2801 insertions(+), 2892 deletions(-)
diff --git a/daffodil-cli/README_How_to_Debug.txt b/daffodil-cli/README_How_to_Debug.txt
new file mode 100644
index 0000000..d6a37dd
--- /dev/null
+++ b/daffodil-cli/README_How_to_Debug.txt
@@ -0,0 +1,15 @@
+Because the CLI tests spawn a sub-shell and exec the daffodil program
+you have to 'sbt stage' first, and even then you can't set a breakpoint.
+
+If you want to run a test in the Eclipse debugger instead of by
+spawning a shell, here's the idiom.
+
+ // tests all synthesize a command line into val cmd
+ val cmd = .....
+ // Instead of running the rest of the test, do these two lines:
+ val args = cmd.split(' ').tail // cmdline minus the daffodil program name.
+ Main.run(args) // Run Main directly, so we can debug.
+
+ Note that this doesn't do the expect stuff. You have to type the
+commands at the console pane, and inspect the output to see if it is
+what you want.
\ No newline at end of file
diff --git a/daffodil-cli/src/main/scala/edu/illinois/ncsa/daffodil/Main.scala b/daffodil-cli/src/main/scala/edu/illinois/ncsa/daffodil/Main.scala
index 3de1b26..b7d3ed9 100644
--- a/daffodil-cli/src/main/scala/edu/illinois/ncsa/daffodil/Main.scala
+++ b/daffodil-cli/src/main/scala/edu/illinois/ncsa/daffodil/Main.scala
@@ -43,7 +43,6 @@ import java.io.FileInputStream
import java.io.ByteArrayInputStream
import java.io.File
import java.nio.channels.Channels
-import java.nio.charset.StandardCharsets
import java.net.URI
import scala.xml.SAXParseException
import org.rogach.scallop
@@ -878,13 +877,12 @@ object Main extends Logging {
outputter match {
case sxml: ScalaXMLInfosetOutputter => writer.write(sxml.getResult.toString)
case jdom: JDOMInfosetOutputter => writer.write(
- new org.jdom2.output.XMLOutputter().outputString(jdom.getResult)
- )
+ new org.jdom2.output.XMLOutputter().outputString(jdom.getResult))
case w3cdom: W3CDOMInfosetOutputter => {
val tf = TransformerFactory.newInstance()
val transformer = tf.newTransformer()
- val result = new StreamResult(writer)
- val source = new DOMSource(w3cdom.getResult)
+ val result = new StreamResult(writer)
+ val source = new DOMSource(w3cdom.getResult)
transformer.transform(source, result)
}
case _ => // do nothing
@@ -908,8 +906,7 @@ object Main extends Logging {
val dis = ps.dataInputStream
val hasMoreData = dis.isDefinedForLength(1) // do we have even 1 more bit?
if (hasMoreData) {
- dis.setDecoder(StandardCharsets.ISO_8859_1.newDecoder())
- val maybeString = dis.getSomeString(processor.getTunables.maxFieldContentLengthInBytes)
+ val maybeString = dis.getSomeString(processor.getTunables.maxFieldContentLengthInBytes, ps)
val lengthInBytes = if (maybeString.isEmpty) 0 else maybeString.get.length
if (lengthInBytes > 0)
log(LogLevel.Warning, "Left over data. Consumed %s bit(s) with %s bit(s) remaining.", loc.bitPos1b - 1, (lengthInBytes * 8))
diff --git a/daffodil-cli/src/test/scala/edu/illinois/ncsa/daffodil/debugger/TestCLIDebugger.scala b/daffodil-cli/src/test/scala/edu/illinois/ncsa/daffodil/debugger/TestCLIDebugger.scala
index f13798e..b469927 100644
--- a/daffodil-cli/src/test/scala/edu/illinois/ncsa/daffodil/debugger/TestCLIDebugger.scala
+++ b/daffodil-cli/src/test/scala/edu/illinois/ncsa/daffodil/debugger/TestCLIDebugger.scala
@@ -237,7 +237,7 @@ class TestCLIdebugger {
shell.expect(contains("debug"))
shell.sendLine("info data")
- shell.expect(contains("0,1,2,3,4,5,6"))
+ shell.expect(contains("0~,~1~,~2~,~3~,~4~,~5~,~6~"))
// shell.sendLine("set dataLength 5")
// shell.sendLine("info data")
@@ -245,7 +245,7 @@ class TestCLIdebugger {
shell.sendLine("set dataLength -938")
shell.sendLine("info data")
- shell.expect(contains("0,1,2,3,4,5,6"))
+ shell.expect(contains("0~,~1~,~2~,~3~,~4~,~5~,~6~"))
// shell.sendLine("set wrapLength 2")
// shell.sendLine("info data")
@@ -285,6 +285,7 @@ class TestCLIdebugger {
try {
val cmd = String.format("%s -d parse -s %s -r matrix %s", Util.binPath, testSchemaFile, testInputFile)
+
shell.sendLine(cmd)
shell.expect(contains("(debug)"))
@@ -302,7 +303,7 @@ class TestCLIdebugger {
shell.sendLine("enable display 1")
shell.sendLine("step")
- shell.expect(contains("</tns:matrix>"))
+ shell.expect(contains("</tns:cell>"))
shell.sendLine("delete display 1")
shell.sendLine("step")
@@ -895,7 +896,7 @@ class TestCLIdebugger {
shell.sendLine("display info data")
shell.sendLine("step")
shell.expect(contains("│")) // (0 to 0)
- shell.expect(contains("0,1,2,3,4,5,6"))
+ shell.expect(contains("0~,~1~,~2~,~3~,~4~,~5~,~6~"))
shell.sendLine("break cell")
shell.sendLine("condition 1 dfdl:occursIndex() eq 5")
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/ChoiceGroup.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/ChoiceGroup.scala
index ec774c7..f7916ae 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/ChoiceGroup.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/ChoiceGroup.scala
@@ -274,7 +274,10 @@ abstract class ChoiceTermBase( final override val xml: Node,
couldHaveText,
alignmentValueInBits,
hasNoSkipRegions,
- optIgnoreCase)
+ optIgnoreCase,
+ maybeFillByteEv,
+ maybeCheckByteAndBitOrderEv,
+ maybeCheckBitOrderAndCharset)
}
}
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/DFDLAnnotation.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/DFDLAnnotation.scala
index bc107bc..bc5be2c 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/DFDLAnnotation.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/DFDLAnnotation.scala
@@ -50,11 +50,6 @@ import edu.illinois.ncsa.daffodil.schema.annotation.props.FindPropertyMixin
* and Import objects which represent those statements in a schema,
* the proxy DFDLSchemaFile object, etc.
*
- * This change lets us share more easily, also hoist a base
- * SchemaComponentBase over into daffodil-lib, which lets some of this
- * shared code, specifcally stuff about errors and diagnostics,
- * migrate over to daffodil-lib as well where it can be tied a
- * bit more tightly to the OOLAG library there.
*/
abstract class DFDLAnnotation(xmlArg: Node, annotatedSCArg: AnnotatedSchemaComponent)
extends SchemaComponent
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/ElementBase.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/ElementBase.scala
index a55dbbc..b35a40c 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/ElementBase.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/ElementBase.scala
@@ -559,7 +559,12 @@ trait ElementBase
//
false, // !isReferencedByExpressions, // assume it is always to be referenced by expressions
optTruncateSpecifiedLengthString,
- if (isOutputValueCalc) Some(ovcCompiledExpression) else None)
+ if (isOutputValueCalc) Some(ovcCompiledExpression) else None,
+ maybeBinaryFloatRepEv,
+ maybeByteOrderEv,
+ maybeFillByteEv,
+ maybeCheckByteAndBitOrderEv,
+ maybeCheckBitOrderAndCharset)
newERD
}
@@ -621,6 +626,12 @@ trait ElementBase
} else {
representation
}
+ rep match {
+ case Representation.Binary =>
+ byteOrderEv // ensure defined
+ case _ =>
+ charsetEv // ensure defined
+ }
rep
}
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/RawCommonRuntimeValuedPropertiesMixin.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/RawCommonRuntimeValuedPropertiesMixin.scala
index ada0983..b675f91 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/RawCommonRuntimeValuedPropertiesMixin.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/RawCommonRuntimeValuedPropertiesMixin.scala
@@ -42,6 +42,9 @@ trait RawCommonRuntimeValuedPropertiesMixin
protected final lazy val encodingRaw = requireProperty(optionEncodingRaw)
protected final lazy val optionOutputNewLineRaw = findPropertyOption("outputNewLine")
protected final lazy val outputNewLineRaw = requireProperty(optionOutputNewLineRaw)
+
+ protected final lazy val optionFillByteRaw = findPropertyOption("fillByte")
+ protected final lazy val fillByteRaw = requireProperty(optionFillByteRaw)
}
trait RawDelimitedRuntimeValuedPropertiesMixin
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/RuntimePropertyMixins.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/RuntimePropertyMixins.scala
index a5104bd..984b22a 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/RuntimePropertyMixins.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/RuntimePropertyMixins.scala
@@ -78,6 +78,7 @@ import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.NilKind
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.TextTrimKind
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.TextPadKind
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.YesNo
+import edu.illinois.ncsa.daffodil.processors.CheckEncodingEv
/*
* These are the DFDL properties which can have their values come
@@ -117,12 +118,10 @@ trait TermRuntimeValuedPropertiesMixin
ev
}
- final lazy val charsetEv =
- if (maybeCharsetEv.isDefined) maybeCharsetEv.get
- else {
- findProperty("encoding")
- Assert.invariantFailed("the findProperty above is supposed to cause SDE.")
- }
+ final lazy val charsetEv = {
+ if (maybeCharsetEv.isEmpty) encodingRaw // required property
+ maybeCharsetEv.get
+ }
final lazy val maybeCharsetEv =
if (optionEncodingRaw.isDefined) {
@@ -132,12 +131,22 @@ trait TermRuntimeValuedPropertiesMixin
} else
Nope
- final lazy val fillByteEv = {
- val ev = new FillByteEv(fillByte, charsetEv, termRuntimeData)
+ final lazy val checkEncodingEv = {
+ val ev = new CheckEncodingEv(termRuntimeData, alignmentValueInBits, charsetEv)
ev.compile()
ev
}
+ final lazy val maybeFillByteEv = {
+ if (optionFillByteRaw.isDefined) {
+ val ev = new FillByteEv(fillByte, charsetEv, termRuntimeData)
+ ev.compile()
+ One(ev)
+ } else {
+ Nope
+ }
+ }
+
/*
* Property OutputNewLine
*/
@@ -252,11 +261,20 @@ trait ElementRuntimeValuedPropertiesMixin
ExpressionCompilers.String.compile(qn, NodeInfo.NonEmptyString, byteOrderRaw)
}.value
- final lazy val byteOrderEv = LV('byteOrderEv) {
- val ev = new ByteOrderEv(byteOrderExpr, elementRuntimeData)
- ev.compile()
- ev
- }.value
+ final lazy val byteOrderEv = {
+ if (maybeByteOrderEv.isEmpty) byteOrderRaw // must be defined
+ maybeByteOrderEv.get
+ }
+
+ final lazy val maybeByteOrderEv = {
+ if (optionByteOrderRaw.isDefined) {
+ val ev = new ByteOrderEv(byteOrderExpr, elementRuntimeData)
+ ev.compile()
+ One(ev)
+ } else {
+ Nope
+ }
+ }
protected final lazy val lengthExpr = {
val qn = this.qNameForProperty("length")
@@ -610,9 +628,18 @@ trait SimpleTypeRuntimeValuedPropertiesMixin
}.value
final lazy val binaryFloatRepEv = {
- val ev = new BinaryFloatRepEv(binaryFloatRepExpr, erd)
- ev.compile()
- ev
+ if (maybeBinaryFloatRepEv.isEmpty) binaryFloatRepRaw // property is required
+ maybeBinaryFloatRepEv.get
+ }
+
+ final lazy val maybeBinaryFloatRepEv = {
+ if (optionBinaryFloatRepRaw.isDefined) {
+ val ev = new BinaryFloatRepEv(binaryFloatRepExpr, erd)
+ ev.compile()
+ One(ev)
+ } else {
+ Nope
+ }
}
private lazy val textBooleanTrueRepExpr = LV('textBooleanTrueRep) {
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/SchemaSet.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/SchemaSet.scala
index 23583b4..98c3f02 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/SchemaSet.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/SchemaSet.scala
@@ -81,10 +81,13 @@ final class SchemaSet(
val validateDFDLSchemas: Boolean,
checkAllTopLevelArg: Boolean,
parent: SchemaComponent,
- override val tunable: DaffodilTunables)
+ tunableArg: DaffodilTunables)
extends SchemaComponentImpl(<schemaSet/>, parent) // a fake schema component
with SchemaSetIncludesAndImportsMixin {
+ override def tunable =
+ tunableArg
+
requiredEvaluations(isValid)
if (checkAllTopLevel) {
requiredEvaluations(checkForDuplicateTopLevels())
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/SequenceGroup.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/SequenceGroup.scala
index cdc68e1..de82849 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/SequenceGroup.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/SequenceGroup.scala
@@ -254,7 +254,10 @@ abstract class SequenceTermBase(
couldHaveText,
alignmentValueInBits,
hasNoSkipRegions,
- optIgnoreCase)
+ optIgnoreCase,
+ maybeFillByteEv,
+ maybeCheckByteAndBitOrderEv,
+ maybeCheckBitOrderAndCharset)
}
}
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/TermEncodingMixin.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/TermEncodingMixin.scala
index 52beb0e..2a88633 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/TermEncodingMixin.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/dsom/TermEncodingMixin.scala
@@ -98,7 +98,7 @@ trait TermEncodingMixin extends KnownEncodingMixin { self: Term =>
}
lazy val encodingInfo =
- new EncodingRuntimeData(termRuntimeData, charsetEv, schemaFileLocation, optionUTF16Width, defaultEncodingErrorPolicy,
+ new EncodingRuntimeData(termRuntimeData, charsetEv, checkEncodingEv, schemaFileLocation, optionUTF16Width, defaultEncodingErrorPolicy,
summaryEncoding, isKnownEncoding, isScannable, knownEncodingAlignmentInBits)
/**
@@ -202,18 +202,6 @@ trait TermEncodingMixin extends KnownEncodingMixin { self: Term =>
}
/**
- * Returns true if this term either cannot conflict because it has no textual
- * aspects, or if it couldHaveText then the encoding must be same.
- */
- // private def hasCompatibleEncoding(t2: Term): Boolean = {
- // Assert.usage(isKnownEncoding)
- // Assert.usage(t2.isKnownEncoding)
- // if (!couldHaveText) true
- // else if (!t2.couldHaveText) true
- // else this.knownEncodingCharset == t2.knownEncodingCharset
- // }
-
- /**
* no alignment properties that would explicitly create
* a need to align in a way that is not on a suitable boundary
* for a character.
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/BitOrderMixin.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/BitOrderMixin.scala
index 1e25272..1a2b4b6 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/BitOrderMixin.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/BitOrderMixin.scala
@@ -33,8 +33,11 @@
package edu.illinois.ncsa.daffodil.grammar
import edu.illinois.ncsa.daffodil.dsom.Term
import edu.illinois.ncsa.daffodil.equality.TypeEqual
-import edu.illinois.ncsa.daffodil.grammar.primitives.BitOrderChange
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
+import edu.illinois.ncsa.daffodil.processors.CheckByteAndBitOrderEv
+import edu.illinois.ncsa.daffodil.processors.CheckBitOrderAndCharsetEv
+import edu.illinois.ncsa.daffodil.util.Maybe
+import edu.illinois.ncsa.daffodil.dsom.{ Binary, NoText }
trait BitOrderMixin extends GrammarMixin with ByteOrderAnalysisMixin { self: Term =>
@@ -50,6 +53,12 @@ trait BitOrderMixin extends GrammarMixin with ByteOrderAnalysisMixin { self: Ter
final lazy val defaultBitOrder = optDefaultBitOrder.getOrElse(BitOrder.MostSignificantBitFirst)
+ /**
+ * Conservatively determines if this term is known to have
+ * the same bit order as the previous thing.
+ *
+ * If uncertain, returns false.
+ */
final protected lazy val isKnownSameBitOrder: Boolean = {
val res =
if (enclosingTerm.isEmpty) false // root needs bit order
@@ -74,13 +83,41 @@ trait BitOrderMixin extends GrammarMixin with ByteOrderAnalysisMixin { self: Ter
protected final lazy val needsBitOrderChange = {
enclosingTerm.isEmpty || (
optionBitOrder.isDefined &&
- thereIsAByteOrderDefined && // if there is no byte order, then there's no need for bit order. The two go together. An all-textual format doesn't need either one.
- (!isKnownSameBitOrder ||
- (isArray && !hasUniformBitOrderThroughout)))
+ thereIsAByteOrderDefined && // if there is no byte order, then there's no need for bit order. The two go together. An all-textual format doesn't need either one.
+ (!isKnownSameBitOrder ||
+ (isArray && !hasUniformBitOrderThroughout)))
}
- protected final lazy val bitOrderChange = prod("bitOrderChange", needsBitOrderChange) {
- BitOrderChange(this)
+ lazy val maybeCheckByteAndBitOrderEv = {
+ //
+ // TODO: Performance: could be improved, as there are situations where byteOrder
+ // is defined, but still we know it will not be used and this could
+ // be Nope in those cases also. An example would be a 100% text-only item.
+ //
+ if (!isRepresented || !optionByteOrderRaw.isDefined)
+ Maybe.Nope
+ else {
+ val checkByteAndBitOrder = {
+ val ev = new CheckByteAndBitOrderEv(termRuntimeData, defaultBitOrder)
+ ev.compile()
+ ev
+ }
+ Maybe(checkByteAndBitOrder)
+ }
+ }
+
+ lazy val maybeCheckBitOrderAndCharset = {
+ val se = summaryEncoding
+ if (!isRepresented || se == NoText || se == Binary)
+ Maybe.Nope
+ else {
+ val checkBitOrderAndCharset = {
+ val ev = new CheckBitOrderAndCharsetEv(termRuntimeData, defaultBitOrder, charsetEv)
+ ev.compile()
+ ev
+ }
+ Maybe(checkBitOrderAndCharset)
+ }
}
}
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/ByteOrderMixin.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/ByteOrderMixin.scala
index 3abd85c..84d6167 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/ByteOrderMixin.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/ByteOrderMixin.scala
@@ -34,8 +34,6 @@ package edu.illinois.ncsa.daffodil.grammar
import edu.illinois.ncsa.daffodil.dsom._
import edu.illinois.ncsa.daffodil.equality._; object ENoWarn { EqualitySuppressUnusedImportWarning() }
-import edu.illinois.ncsa.daffodil.grammar.primitives.ByteOrderChange
-import edu.illinois.ncsa.daffodil.grammar.primitives.ByteOrderChange
import edu.illinois.ncsa.daffodil.schema.annotation.props.NotFound
import edu.illinois.ncsa.daffodil.schema.annotation.props.Found
@@ -49,39 +47,4 @@ trait ByteOrderAnalysisMixin extends GrammarMixin { self: Term =>
}
}
- protected lazy val isKnownSameByteOrder: Boolean = {
- val optPrior = nearestPriorPhysicalTermSatisfying(term =>
- term.isInstanceOf[ElementBase] &&
- term.thereIsAByteOrderDefined).asInstanceOf[Option[ElementBase]]
- val optThis = this match { case e: ElementBase => Some(e); case _ => None }
- val optPriorByteOrder = optPrior.flatMap { _.byteOrderEv.optConstant }
- val optThisByteOrder = optThis.flatMap { _.byteOrderEv.optConstant }
- val res =
- if (optThisByteOrder.isDefined &&
- optPriorByteOrder.isDefined)
- optThisByteOrder =:= optPriorByteOrder
- else
- false
- res
- }
-
- protected lazy val hasUniformByteOrderThroughout: Boolean = termChildren.map { t =>
- t.thereIsAByteOrderDefined && t.isKnownSameByteOrder && t.hasUniformByteOrderThroughout
- }.forall(x => x)
-
-}
-
-trait ByteOrderMixin extends ByteOrderAnalysisMixin { self: ElementBase =>
-
- protected final lazy val byteOrderChange =
- prod("byteOrderChange",
- enclosingTerm.isEmpty || (
- (thereIsAByteOrderDefined &&
- !isKnownSameByteOrder) || // need to change on the way in
- (isArray && !hasUniformByteOrderThroughout))) { // need to change because of repetition
- // (when we start next iteration, it's not the same as when we started first iteration)
- // THis will SDE if there is no byte order defined for the array element (might only be byteOrder on things within the array)
- // So we're artificially requiring byte order on all arrays that do not have a uniform byte order.
- ByteOrderChange(this)
- }
}
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/ElementBaseGrammarMixin.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/ElementBaseGrammarMixin.scala
index 1bc9909..5172c2f 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/ElementBaseGrammarMixin.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/ElementBaseGrammarMixin.scala
@@ -147,7 +147,6 @@ import edu.illinois.ncsa.daffodil.schema.annotation.props.Found
trait ElementBaseGrammarMixin
extends InitiatedTerminatedMixin
with AlignedMixin
- with ByteOrderMixin
with HasStatementsGrammarMixin
with PaddingInfoMixin { self: ElementBase =>
@@ -1015,8 +1014,7 @@ trait ElementBaseGrammarMixin
LeadingSkipRegion(this) ~ AlignmentFill(this) ~ PrefixLength(this)
}
- private lazy val elementIOPropertiesChange = byteOrderChange ~ termIOPropertiesChange
- private lazy val elementLeftFraming = elementIOPropertiesChange ~ alignAndSkipFraming
+ private lazy val elementLeftFraming = alignAndSkipFraming
private lazy val elementRightFraming = prod("elementRightFraming") { TrailingSkipRegion(this) }
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/EncodingChangeMixin.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/EncodingChangeMixin.scala
deleted file mode 100644
index d59abfe..0000000
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/EncodingChangeMixin.scala
+++ /dev/null
@@ -1,73 +0,0 @@
-/* Copyright (c) 2012-2014 Tresys Technology, LLC. All rights reserved.
- *
- * Developed by: Tresys Technology, LLC
- * http://www.tresys.com
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy of
- * this software and associated documentation files (the "Software"), to deal with
- * the Software without restriction, including without limitation the rights to
- * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
- * of the Software, and to permit persons to whom the Software is furnished to do
- * so, subject to the following conditions:
- *
- * 1. Redistributions of source code must retain the above copyright notice,
- * this list of conditions and the following disclaimers.
- *
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimers in the
- * documentation and/or other materials provided with the distribution.
- *
- * 3. Neither the names of Tresys Technology, nor the names of its contributors
- * may be used to endorse or promote products derived from this Software
- * without specific prior written permission.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE
- * SOFTWARE.
- */
-
-package edu.illinois.ncsa.daffodil.grammar
-
-import edu.illinois.ncsa.daffodil.processors._
-import edu.illinois.ncsa.daffodil.dsom.Term
-import edu.illinois.ncsa.daffodil.equality._
-import edu.illinois.ncsa.daffodil.grammar.primitives.EncodingChange
-
-trait EncodingChangeMixin extends GrammarMixin { self: Term =>
-
- private lazy val enclosingEncoding = enclosingTerm.map(_.encodingEv)
- private lazy val priorSiblingEncoding = priorSibling.map(_.encodingEv)
- private lazy val encodingBefore: Evaluatable[String] = priorSiblingEncoding.getOrElse(enclosingEncoding.getOrElse(encodingEv))
-
- protected lazy val isKnownSameEncoding: Boolean = {
- if (enclosingTerm.isEmpty) false // root always gets a encoding change.
- else {
- val thisEncoding = encodingEv
- val otherEncoding = encodingBefore
- if (thisEncoding.isConstant && otherEncoding.isConstant &&
- thisEncoding.optConstant.get =:= otherEncoding.optConstant.get) {
- if (priorSiblingEncoding.isDefined) {
- // in addition to having the same encoding as the prior sibling, all
- // children of that prior sibling must have the same encoding.
- // Otherwise, a child could change the encoding and it might not be
- // the same as ours.
- priorSibling.get.hasUniformEncodingThroughout
- } else {
- true
- }
- } else {
- false
- }
- }
- }
-
- protected lazy val hasUniformEncodingThroughout: Boolean = termChildren.map { t => t.isKnownSameEncoding && t.hasUniformEncodingThroughout }.forall(x => x)
-
- protected final lazy val encodingChange = prod("encodingChange",
- !isKnownSameEncoding ||
- (isArray && !hasUniformEncodingThroughout)) { EncodingChange(this) }
-}
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/ModelGroupGrammarMixin.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/ModelGroupGrammarMixin.scala
index 66c8865..e8b03ef 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/ModelGroupGrammarMixin.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/ModelGroupGrammarMixin.scala
@@ -50,7 +50,7 @@ trait ModelGroupGrammarMixin
with GroupCommonAGMixin { self: ModelGroup =>
private lazy val groupLeftFraming = prod("groupLeftFraming") {
- termIOPropertiesChange ~ LeadingSkipRegion(this) ~ AlignmentFill(this)
+ LeadingSkipRegion(this) ~ AlignmentFill(this)
}
private lazy val groupRightFraming = prod("groupRightFraming") { TrailingSkipRegion(this) }
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/TermGrammarMixin.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/TermGrammarMixin.scala
index 281f50e..fa46716 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/TermGrammarMixin.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/TermGrammarMixin.scala
@@ -46,8 +46,7 @@ import edu.illinois.ncsa.daffodil.grammar.primitives.Separator
trait TermGrammarMixin
extends AlignedMixin
- with BitOrderMixin
- with EncodingChangeMixin { self: Term =>
+ with BitOrderMixin { self: Term =>
override protected final def grammarContext = this
@@ -62,8 +61,6 @@ trait TermGrammarMixin
protected lazy val hasEncoding = optionEncodingRaw.isDefined
- protected lazy val termIOPropertiesChange = prod("ioPropertiesChange") { bitOrderChange ~ encodingChange }
-
// TODO: replace dfdlScopeBegin and dfdlScopeEnd with a single Combinator.
protected final lazy val dfdlScopeBegin = prod("dfdlScopeBegin", newVarStarts.length > 0) {
newVarStarts.fold(mt) { _ ~ _ }
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/BitOrderChange.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/BitOrderChange.scala
deleted file mode 100644
index 62899b1..0000000
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/BitOrderChange.scala
+++ /dev/null
@@ -1,71 +0,0 @@
-/* Copyright (c) 2012-2014 Tresys Technology, LLC. All rights reserved.
- *
- * Developed by: Tresys Technology, LLC
- * http://www.tresys.com
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy of
- * this software and associated documentation files (the "Software"), to deal with
- * the Software without restriction, including without limitation the rights to
- * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
- * of the Software, and to permit persons to whom the Software is furnished to do
- * so, subject to the following conditions:
- *
- * 1. Redistributions of source code must retain the above copyright notice,
- * this list of conditions and the following disclaimers.
- *
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimers in the
- * documentation and/or other materials provided with the distribution.
- *
- * 3. Neither the names of Tresys Technology, nor the names of its contributors
- * may be used to endorse or promote products derived from this Software
- * without specific prior written permission.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE
- * SOFTWARE.
- */
-
-package edu.illinois.ncsa.daffodil.grammar.primitives
-
-import edu.illinois.ncsa.daffodil.grammar.Terminal
-import edu.illinois.ncsa.daffodil.dsom.Term
-import edu.illinois.ncsa.daffodil.processors.unparsers.BitOrderChangeUnparser
-import edu.illinois.ncsa.daffodil.util.Maybe._
-import edu.illinois.ncsa.daffodil.dsom.ElementBase
-import edu.illinois.ncsa.daffodil.processors.parsers.BitOrderChangeParser
-import edu.illinois.ncsa.daffodil.processors.CheckBitOrderAndCharsetEv
-import edu.illinois.ncsa.daffodil.processors.CheckByteAndBitOrderEv
-
-/**
- * Changes bit order to what the term specifies it is.
- */
-
-case class BitOrderChange(t: Term) extends Terminal(t, true) {
-
- val maybeByteOrder = t match {
- case eb: ElementBase => One(eb.byteOrderEv)
- case _ => Nope
- }
-
- lazy val checkByteAndBitOrder = {
- val ev = new CheckByteAndBitOrderEv(t.termRuntimeData, t.defaultBitOrder, maybeByteOrder)
- ev.compile()
- ev
- }
-
- lazy val checkBitOrderAndCharset = {
- val ev = new CheckBitOrderAndCharsetEv(t.termRuntimeData, t.defaultBitOrder, t.charsetEv)
- ev.compile()
- ev
- }
-
- override lazy val parser = new BitOrderChangeParser(t.termRuntimeData, t.defaultBitOrder, checkByteAndBitOrder, checkBitOrderAndCharset)
-
- override lazy val unparser = new BitOrderChangeUnparser(t.termRuntimeData, t.defaultBitOrder, checkByteAndBitOrder, checkBitOrderAndCharset)
-
-}
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/ByteOrderChange.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/ByteOrderChange.scala
deleted file mode 100644
index ed7f26d..0000000
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/ByteOrderChange.scala
+++ /dev/null
@@ -1,54 +0,0 @@
-/* Copyright (c) 2012-2014 Tresys Technology, LLC. All rights reserved.
- *
- * Developed by: Tresys Technology, LLC
- * http://www.tresys.com
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy of
- * this software and associated documentation files (the "Software"), to deal with
- * the Software without restriction, including without limitation the rights to
- * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
- * of the Software, and to permit persons to whom the Software is furnished to do
- * so, subject to the following conditions:
- *
- * 1. Redistributions of source code must retain the above copyright notice,
- * this list of conditions and the following disclaimers.
- *
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimers in the
- * documentation and/or other materials provided with the distribution.
- *
- * 3. Neither the names of Tresys Technology, nor the names of its contributors
- * may be used to endorse or promote products derived from this Software
- * without specific prior written permission.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE
- * SOFTWARE.
- */
-
-package edu.illinois.ncsa.daffodil.grammar.primitives
-
-import edu.illinois.ncsa.daffodil.grammar.Terminal
-import edu.illinois.ncsa.daffodil.dsom.ElementBase
-import edu.illinois.ncsa.daffodil.processors.unparsers.ByteOrderChangeUnparser
-import edu.illinois.ncsa.daffodil.util.Maybe._
-import edu.illinois.ncsa.daffodil.processors.parsers.ByteOrderChangeParser
-import edu.illinois.ncsa.daffodil.processors.CheckByteAndBitOrderEv
-
-case class ByteOrderChange(e: ElementBase) extends Terminal(e, true) {
-
- lazy val checkByteAndBitOrder = {
- val ev = new CheckByteAndBitOrderEv(e.erd, e.defaultBitOrder, One(e.byteOrderEv))
- ev.compile()
- ev
- }
-
- override lazy val parser = new ByteOrderChangeParser(e.erd, e.byteOrderEv, checkByteAndBitOrder)
-
- override lazy val unparser = new ByteOrderChangeUnparser(e.erd, e.byteOrderEv, checkByteAndBitOrder)
-
-}
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/ElementCombinator.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/ElementCombinator.scala
index 8df509f..3b41270 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/ElementCombinator.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/ElementCombinator.scala
@@ -159,8 +159,7 @@ case class ElementUnused(ctxt: ElementBase)
ctxt.maybeUnparseTargetLengthInBitsEv.get,
ctxt.maybeLengthEv,
ctxt.maybeCharsetEv,
- ctxt.maybeLiteralNilEv,
- ctxt.fillByteEv)
+ ctxt.maybeLiteralNilEv)
}
case class OnlyPadding(ctxt: ElementBase)
@@ -237,7 +236,6 @@ case class RightFill(ctxt: ElementBase)
ctxt.maybeLengthEv,
ctxt.maybeCharsetEv,
ctxt.maybeLiteralNilEv,
- ctxt.fillByteEv,
unparsingPadChar)
}
@@ -268,7 +266,7 @@ case class CaptureContentLengthStart(ctxt: ElementBase)
// improve this check so that this unparser can be optimized out if there
// will not be any suspensions.
if (ctxt.isReferencedByContentLengthUnparserExpressions ||
- (ctxt.maybeFixedLengthInBits.isDefined && ctxt.couldHaveSuspensions))
+ (ctxt.maybeFixedLengthInBits.isDefined && ctxt.couldHaveSuspensions))
new CaptureStartOfContentLengthUnparser(ctxt.erd)
else
new NadaUnparser(ctxt.erd)
@@ -293,7 +291,7 @@ case class CaptureContentLengthEnd(ctxt: ElementBase)
// improve this check so that this unparser can be optimized out if there
// will not be any suspensions.
if (ctxt.isReferencedByContentLengthUnparserExpressions ||
- (ctxt.maybeFixedLengthInBits.isDefined && ctxt.couldHaveSuspensions))
+ (ctxt.maybeFixedLengthInBits.isDefined && ctxt.couldHaveSuspensions))
new CaptureEndOfContentLengthUnparser(ctxt.erd, ctxt.maybeFixedLengthInBits)
else
new NadaUnparser(ctxt.erd)
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/EncodingChange.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/EncodingChange.scala
deleted file mode 100644
index f27460b..0000000
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/EncodingChange.scala
+++ /dev/null
@@ -1,60 +0,0 @@
-/* Copyright (c) 2012-2014 Tresys Technology, LLC. All rights reserved.
- *
- * Developed by: Tresys Technology, LLC
- * http://www.tresys.com
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy of
- * this software and associated documentation files (the "Software"), to deal with
- * the Software without restriction, including without limitation the rights to
- * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
- * of the Software, and to permit persons to whom the Software is furnished to do
- * so, subject to the following conditions:
- *
- * 1. Redistributions of source code must retain the above copyright notice,
- * this list of conditions and the following disclaimers.
- *
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimers in the
- * documentation and/or other materials provided with the distribution.
- *
- * 3. Neither the names of Tresys Technology, nor the names of its contributors
- * may be used to endorse or promote products derived from this Software
- * without specific prior written permission.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE
- * SOFTWARE.
- */
-
-package edu.illinois.ncsa.daffodil.grammar.primitives
-
-import edu.illinois.ncsa.daffodil.dsom.Term
-import edu.illinois.ncsa.daffodil.grammar.Terminal
-import edu.illinois.ncsa.daffodil.processors.unparsers.EncodingChangeUnparser
-import edu.illinois.ncsa.daffodil.processors.CheckBitOrderAndCharsetEv
-import edu.illinois.ncsa.daffodil.processors.parsers.EncodingChangeParser
-import edu.illinois.ncsa.daffodil.processors.CheckEncodingEv
-
-case class EncodingChange(t: Term) extends Terminal(t, true) {
-
- lazy val checkBitOrderAndCharset = {
- val ev = new CheckBitOrderAndCharsetEv(t.termRuntimeData, t.defaultBitOrder, t.charsetEv)
- ev.compile()
- ev
- }
-
- lazy val checkEncoding = {
- val ev = new CheckEncodingEv(t.termRuntimeData, t.alignmentValueInBits, t.charsetEv)
- ev.compile()
- ev
- }
-
- override lazy val parser = new EncodingChangeParser(t.termRuntimeData, checkBitOrderAndCharset, checkEncoding)
-
- override lazy val unparser = new EncodingChangeUnparser(t.termRuntimeData, checkBitOrderAndCharset, checkEncoding)
-
-}
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesBinaryNumber.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesBinaryNumber.scala
index b3bb3f0..92ac625 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesBinaryNumber.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesBinaryNumber.scala
@@ -49,6 +49,7 @@ import edu.illinois.ncsa.daffodil.processors.unparsers.BinaryDoubleUnparser
import edu.illinois.ncsa.daffodil.processors.unparsers.BinaryFloatUnparser
class BinaryIntegerRuntimeLength(val e: ElementBase, signed: Boolean) extends Terminal(e, true) {
+
override lazy val parser = new BinaryIntegerRuntimeLengthParser(e.elementRuntimeData, signed, e.lengthEv, e.lengthUnits)
override lazy val unparser: Unparser = new BinaryIntegerRuntimeLengthUnparser(e.elementRuntimeData, signed, e.lengthEv, e.lengthUnits)
@@ -64,6 +65,7 @@ class BinaryIntegerKnownLength(val e: ElementBase, signed: Boolean, lengthInBits
}
class BinaryDecimalRuntimeLength(val e: ElementBase) extends Terminal(e, true) {
+
override lazy val parser = new BinaryDecimalRuntimeLengthParser(e.elementRuntimeData, e.decimalSigned, e.binaryDecimalVirtualPoint, e.lengthEv, e.lengthUnits)
override lazy val unparser: Unparser = new BinaryDecimalRuntimeLengthUnparser(e.elementRuntimeData, e.decimalSigned, e.binaryDecimalVirtualPoint, e.lengthEv, e.lengthUnits)
@@ -71,18 +73,21 @@ class BinaryDecimalRuntimeLength(val e: ElementBase) extends Terminal(e, true) {
}
class BinaryDecimalKnownLength(val e: ElementBase, lengthInBits: Long) extends Terminal(e, true) {
+
override lazy val parser = new BinaryDecimalKnownLengthParser(e.elementRuntimeData, e.decimalSigned, e.binaryDecimalVirtualPoint, lengthInBits.toInt)
override lazy val unparser: Unparser = new BinaryDecimalKnownLengthUnparser(e.elementRuntimeData, e.decimalSigned, e.binaryDecimalVirtualPoint, lengthInBits.toInt)
}
class BinaryFloat(val e: ElementBase) extends Terminal(e, true) {
+
override lazy val parser = new BinaryFloatParser(e.elementRuntimeData)
override lazy val unparser: Unparser = new BinaryFloatUnparser(e.elementRuntimeData)
}
class BinaryDouble(val e: ElementBase) extends Terminal(e, true) {
+
override lazy val parser = new BinaryDoubleParser(e.elementRuntimeData)
override lazy val unparser: Unparser = new BinaryDoubleUnparser(e.elementRuntimeData)
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesElementKinds.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesElementKinds.scala
index b2a94ef..0671e65 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesElementKinds.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesElementKinds.scala
@@ -73,7 +73,7 @@ case class DelimiterStackCombinatorSequence(sq: SequenceTermBase, body: Gram) ex
lazy val parser: DaffodilParser = new DelimiterStackParser((pInit.toList ++ pSep.toList ++ pTerm.toList).toArray, sq.runtimeData, body.parser)
- override lazy val unparser: DaffodilUnparser = new DelimiterStackUnparser(uInit, uSep, uTerm, sq.runtimeData, body.unparser)
+ override lazy val unparser: DaffodilUnparser = new DelimiterStackUnparser(uInit, uSep, uTerm, sq.termRuntimeData, body.unparser)
}
case class DelimiterStackCombinatorChoice(ch: ChoiceTermBase, body: Gram) extends Terminal(ch, !body.isEmpty) {
@@ -85,7 +85,7 @@ case class DelimiterStackCombinatorChoice(ch: ChoiceTermBase, body: Gram) extend
lazy val parser: DaffodilParser = new DelimiterStackParser((pInit.toList ++ pTerm.toList).toArray, ch.runtimeData, body.parser)
- override lazy val unparser: DaffodilUnparser = new DelimiterStackUnparser(uInit, None, uTerm, ch.runtimeData, body.unparser)
+ override lazy val unparser: DaffodilUnparser = new DelimiterStackUnparser(uInit, None, uTerm, ch.termRuntimeData, body.unparser)
}
case class DelimiterStackCombinatorElement(e: ElementBase, body: Gram) extends Terminal(e, !body.isEmpty) {
@@ -95,9 +95,9 @@ case class DelimiterStackCombinatorElement(e: ElementBase, body: Gram) extends T
lazy val uInit = if (e.initiatorParseEv.isKnownNonEmpty) One(e.initiatorUnparseEv) else Nope
lazy val uTerm = if (e.terminatorParseEv.isKnownNonEmpty) One(e.terminatorUnparseEv) else Nope
- lazy val parser: DaffodilParser = new DelimiterStackParser((pInit.toList ++ pTerm.toList).toArray, e.runtimeData, body.parser)
+ lazy val parser: DaffodilParser = new DelimiterStackParser((pInit.toList ++ pTerm.toList).toArray, e.termRuntimeData, body.parser)
- override lazy val unparser: DaffodilUnparser = new DelimiterStackUnparser(uInit, None, uTerm, e.runtimeData, body.unparser)
+ override lazy val unparser: DaffodilUnparser = new DelimiterStackUnparser(uInit, None, uTerm, e.termRuntimeData, body.unparser)
}
case class DynamicEscapeSchemeCombinatorElement(e: ElementBase, body: Gram) extends Terminal(e, !body.isEmpty) {
@@ -108,9 +108,9 @@ case class DynamicEscapeSchemeCombinatorElement(e: ElementBase, body: Gram) exte
Assert.invariant(schemeParseOpt.isDefined && !schemeParseOpt.get.isConstant)
Assert.invariant(schemeUnparseOpt.isDefined && !schemeUnparseOpt.get.isConstant)
- lazy val parser: DaffodilParser = new DynamicEscapeSchemeParser(schemeParseOpt.get, e.runtimeData, body.parser)
+ lazy val parser: DaffodilParser = new DynamicEscapeSchemeParser(schemeParseOpt.get, e.termRuntimeData, body.parser)
- override lazy val unparser: DaffodilUnparser = new DynamicEscapeSchemeUnparser(schemeUnparseOpt.get, e.runtimeData, body.unparser)
+ override lazy val unparser: DaffodilUnparser = new DynamicEscapeSchemeUnparser(schemeUnparseOpt.get, e.termRuntimeData, body.unparser)
}
case class ComplexTypeCombinator(ct: ComplexTypeBase, body: Gram) extends Terminal(ct.elementDecl, !body.isEmpty) {
@@ -182,7 +182,7 @@ case class ChoiceCombinator(ch: ChoiceTermBase, alternatives: Seq[Gram]) extends
lazy val parser: DaffodilParser = {
if (!ch.isDirectDispatch) {
val folded = alternatives.map { gf => gf }.foldRight(EmptyGram.asInstanceOf[Gram]) { _ | _ }
- new ChoiceCombinatorParser(ch.runtimeData, folded.parser)
+ new ChoiceCombinatorParser(ch.termRuntimeData, folded.parser)
} else {
val dispatchBranchKeyValueTuples = alternatives.flatMap { alt =>
val keyTerm = alt.context.asInstanceOf[Term]
@@ -206,7 +206,7 @@ case class ChoiceCombinator(ch: ChoiceTermBase, alternatives: Seq[Gram]) extends
val dispatchBranchKeyMap = dispatchBranchKeyValueTuples.toMap.mapValues(_.parser)
val serializableMap = dispatchBranchKeyMap.map(identity)
- new ChoiceDispatchCombinatorParser(ch.runtimeData, ch.choiceDispatchKeyEv, serializableMap)
+ new ChoiceDispatchCombinatorParser(ch.termRuntimeData, ch.choiceDispatchKeyEv, serializableMap)
}
}
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesFraming.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesFraming.scala
index 0aa511b..e72b9b3 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesFraming.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesFraming.scala
@@ -51,8 +51,8 @@ abstract class SkipRegion(e: Term, skipLengthInBits: Int, propName: String) exte
e.schemaDefinitionUnless(skipLengthInBits < e.tunable.maxSkipLengthInBytes * 8,
"Property %s %s(bits) is larger than limit %s(bits).", propName, skipLengthInBits, e.tunable.maxSkipLengthInBytes * 8)
- final lazy val parser: Parser = new SkipRegionParser(skipLengthInBits, e.runtimeData)
- final lazy val unparser: Unparser = new SkipRegionUnparser(skipLengthInBits, e.runtimeData, e.fillByteEv)
+ final lazy val parser: Parser = new SkipRegionParser(skipLengthInBits, e.termRuntimeData)
+ final lazy val unparser: Unparser = new SkipRegionUnparser(skipLengthInBits, e.termRuntimeData)
}
case class LeadingSkipRegion(e: Term) extends SkipRegion(e, e.leadingSkipInBits, "leadingSkip")
@@ -74,8 +74,8 @@ case class AlignmentFill(e: Term) extends Terminal(e, !e.isKnownToBeAligned) {
private val alignment = e.alignmentValueInBits
- lazy val parser: Parser = new AlignmentFillParser(alignment, e.runtimeData)
- lazy val unparser: Unparser = new AlignmentFillUnparser(alignment, e.runtimeData, e.fillByteEv)
+ lazy val parser: Parser = new AlignmentFillParser(alignment, e.termRuntimeData)
+ lazy val unparser: Unparser = new AlignmentFillUnparser(alignment, e.termRuntimeData)
}
case class MandatoryTextAlignment(e: Term, alignmentInBits: Int, forDelimiter: Boolean) extends Terminal(e,
@@ -85,6 +85,6 @@ case class MandatoryTextAlignment(e: Term, alignmentInBits: Int, forDelimiter: B
!e.isKnownToBeTextAligned) {
Assert.invariant(alignmentInBits > 0)
- lazy val parser: Parser = new MandatoryTextAlignmentParser(alignmentInBits, e.runtimeData)
- lazy val unparser: Unparser = new MandatoryTextAlignmentUnparser(alignmentInBits, e.runtimeData, e.fillByteEv)
+ lazy val parser: Parser = new MandatoryTextAlignmentParser(alignmentInBits, e.termRuntimeData)
+ lazy val unparser: Unparser = new MandatoryTextAlignmentUnparser(alignmentInBits, e.termRuntimeData)
}
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesLengthKind.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesLengthKind.scala
index 9f4a9e1..91c9ddc 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesLengthKind.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesLengthKind.scala
@@ -69,7 +69,7 @@ case class HexBinarySpecifiedLength(e: ElementBase) extends Terminal(e, true) {
override lazy val parser: DaffodilParser = new HexBinarySpecifiedLengthParser(e.elementRuntimeData, e.elementLengthInBitsEv)
- override lazy val unparser: DaffodilUnparser = new HexBinarySpecifiedLengthUnparser(e.elementRuntimeData, e.unparseTargetLengthInBitsEv, e.fillByteEv)
+ override lazy val unparser: DaffodilUnparser = new HexBinarySpecifiedLengthUnparser(e.elementRuntimeData, e.unparseTargetLengthInBitsEv)
}
case class StringOfSpecifiedLength(e: ElementBase) extends Terminal(e, true) with Padded {
@@ -177,8 +177,7 @@ abstract class HexBinaryDelimited(e: ElementBase)
override lazy val unparser: DaffodilUnparser = new HexBinaryMinLengthInBytesUnparser(
e.minLength.longValue,
- e.elementRuntimeData,
- e.fillByteEv)
+ e.elementRuntimeData)
}
case class HexBinaryDelimitedEndOfData(e: ElementBase)
@@ -192,8 +191,7 @@ case class HexBinaryEndOfBitLimit(e: ElementBase) extends Terminal(e, true) {
override lazy val unparser: DaffodilUnparser = new HexBinaryMinLengthInBytesUnparser(
e.minLength.longValue,
- e.elementRuntimeData,
- e.fillByteEv)
+ e.elementRuntimeData)
}
case class LiteralNilDelimitedEndOfData(eb: ElementBase)
@@ -223,7 +221,7 @@ case class PrefixLength(e: ElementBase) extends UnimplementedPrimitive(e, e.leng
class OptionalInfixSep(term: Term, sep: => Gram, guard: Boolean = true) extends Terminal(term, guard) {
- lazy val parser: DaffodilParser = new OptionalInfixSepParser(term.runtimeData, sep.parser)
+ lazy val parser: DaffodilParser = new OptionalInfixSepParser(term.termRuntimeData, sep.parser)
- override lazy val unparser: DaffodilUnparser = new OptionalInfixSepUnparser(term.runtimeData, sep.unparser)
+ override lazy val unparser: DaffodilUnparser = new OptionalInfixSepUnparser(term.termRuntimeData, sep.unparser)
}
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesNil.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesNil.scala
index b2f38ca..f3ee777 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesNil.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesNil.scala
@@ -64,7 +64,7 @@ case class LiteralCharacterNilOfSpecifiedLength(e: ElementBase)
e.ignoreCaseBool,
e.elementRuntimeData)
- override lazy val unparser = new NadaUnparser(e.runtimeData)
+ override lazy val unparser = new NadaUnparser(e.termRuntimeData)
}
case class LogicalNilValue(e: ElementBase) extends UnimplementedPrimitive(e, e.isNillable)
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesTextNumber.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesTextNumber.scala
index 64a9d40..e406c43 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesTextNumber.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/PrimitivesTextNumber.scala
@@ -71,9 +71,9 @@ import edu.illinois.ncsa.daffodil.processors.parsers.Parser
case class ConvertTextCombinator(e: ElementBase, value: Gram, converter: Gram)
extends Terminal(e, !(value.isEmpty || converter.isEmpty)) {
- lazy val parser = new ConvertTextCombinatorParser(e.runtimeData, value.parser, converter.parser)
+ lazy val parser = new ConvertTextCombinatorParser(e.termRuntimeData, value.parser, converter.parser)
- override lazy val unparser = new ConvertTextCombinatorUnparser(e.runtimeData, value.unparser, converter.unparser)
+ override lazy val unparser = new ConvertTextCombinatorUnparser(e.termRuntimeData, value.unparser, converter.unparser)
}
abstract class ConvertTextNumberPrim[S](e: ElementBase)
@@ -144,7 +144,7 @@ abstract class ConvertTextNumberPrim[S](e: ElementBase)
e.textStandardExponentRepEv.isConstant)
val nff = if (isConstant) {
- new NumberFormatFactoryStatic[S](e.runtimeData, h,
+ new NumberFormatFactoryStatic[S](e.termRuntimeData, h,
decSep,
groupSep,
e.textStandardExponentRepEv,
@@ -156,7 +156,7 @@ abstract class ConvertTextNumberPrim[S](e: ElementBase)
roundingMode,
roundingIncrement)
} else {
- new NumberFormatFactoryDynamic[S](e.runtimeData, h,
+ new NumberFormatFactoryDynamic[S](e.termRuntimeData, h,
decSep,
groupSep,
e.textStandardExponentRepEv,
diff --git a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/RepPrims.scala b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/RepPrims.scala
index 0e2a355..cc1d036 100644
--- a/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/RepPrims.scala
+++ b/daffodil-core/src/main/scala/edu/illinois/ncsa/daffodil/grammar/primitives/RepPrims.scala
@@ -148,6 +148,6 @@ case class OccursCountExpression(e: ElementBase)
extends Terminal(e, true) {
override lazy val parser = new OccursCountExpressionParser(e.occursCountEv, e.elementRuntimeData)
- override lazy val unparser = new NadaUnparser(e.runtimeData) // DFDL v1.0 Section 16.1.4 - The occursCount expression is not evaluated.
+ override lazy val unparser = new NadaUnparser(e.termRuntimeData) // DFDL v1.0 Section 16.1.4 - The occursCount expression is not evaluated.
}
diff --git a/daffodil-core/src/test/scala/edu/illinois/ncsa/daffodil/dsom/TestBinaryInput_01.scala b/daffodil-core/src/test/scala/edu/illinois/ncsa/daffodil/dsom/TestBinaryInput_01.scala
index 278e487..b0a316d 100644
--- a/daffodil-core/src/test/scala/edu/illinois/ncsa/daffodil/dsom/TestBinaryInput_01.scala
+++ b/daffodil-core/src/test/scala/edu/illinois/ncsa/daffodil/dsom/TestBinaryInput_01.scala
@@ -41,46 +41,66 @@ import edu.illinois.ncsa.daffodil.io.ByteBufferDataInputStream
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
import java.nio.ByteBuffer
import edu.illinois.ncsa.daffodil.io.DataInputStream
+import edu.illinois.ncsa.daffodil.io.FormatInfo
+import java.nio.charset.CharsetEncoder
+import java.nio.charset.CharsetDecoder
+import edu.illinois.ncsa.daffodil.util.Maybe
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BinaryFloatRep
+import edu.illinois.ncsa.daffodil.util.MaybeInt
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.EncodingErrorPolicy
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.UTF16Width
// Do no harm number 16 of 626 fail in regression, 154 in total of 797
class TestBinaryInput_01 {
+ class FakeFormatInfo(val bitOrder: BitOrder, val byteOrder: ByteOrder) extends FormatInfo {
+ def encoder: CharsetEncoder = ???
+ def decoder: CharsetDecoder = ???
+ def reportingDecoder: CharsetDecoder = ???
+ def replacingDecoder: CharsetDecoder = ???
+ def fillByte: Byte = ???
+
+ def binaryFloatRep: BinaryFloatRep = ???
+ def maybeCharWidthInBits: MaybeInt = ???
+ def maybeUTF16Width: Maybe[UTF16Width] = ???
+ def encodingMandatoryAlignmentInBits: Int = ???
+ def encodingErrorPolicy: EncodingErrorPolicy = ???
+ }
+
var startOver: DataInputStream.Mark = null
var dis: DataInputStream = null
def fromBytes(ba: Array[Byte], byteOrd: ByteOrder, bitOrd: BitOrder, bitStartPos: Int = 0) = {
dis = ByteBufferDataInputStream(ba, bitStartPos)
- dis.setBitOrder(bitOrd)
- dis.setByteOrder(byteOrd)
startOver = dis.mark("TestBinaryInput_01")
- dis
+ val finfo = new FakeFormatInfo(bitOrd, byteOrd)
+ (dis, finfo)
}
def fromString(str: String, byteOrd: ByteOrder, bitOrd: BitOrder, bitStartPos: Int = 0) = {
dis = ByteBufferDataInputStream(ByteBuffer.wrap(str.getBytes("utf-8")), bitStartPos)
- dis.setBitOrder(bitOrd)
- dis.setByteOrder(byteOrd)
startOver = dis.mark("TestBinaryInput_01")
- dis
+ val finfo = new FakeFormatInfo(bitOrd, byteOrd)
+ (dis, finfo)
}
@After def shutDown {
dis.discard(startOver)
}
- def getLong(dis: DataInputStream, offset: Int, len: Int) = {
+ def getLong(finfo: FormatInfo, dis: DataInputStream, offset: Int, len: Int) = {
dis.reset(startOver)
startOver = dis.mark("TestBinaryInput_01")
- if (offset > 0) dis.skip(offset)
- dis.getUnsignedLong(len).toLong
+ if (offset > 0) dis.skip(offset, finfo)
+ dis.getUnsignedLong(len, finfo).toLong
}
- def getBigInt(dis: DataInputStream, offset: Int, len: Int) = {
+ def getBigInt(finfo: FormatInfo, dis: DataInputStream, offset: Int, len: Int) = {
dis.reset(startOver)
startOver = dis.mark("TestBinaryInput_01")
- if (offset > 0) dis.skip(offset)
- dis.getSignedBigInt(len)
+ if (offset > 0) dis.skip(offset, finfo)
+ dis.getSignedBigInt(len, finfo)
}
/*** DFDL-334 ***/
@@ -91,208 +111,208 @@ class TestBinaryInput_01 {
val LE = ByteOrder.LittleEndian
@Test def testOneBit1() {
- val dis = fromBytes(Misc.bits2Bytes(Seq("00000011")), BE, msbFirst)
+ val (dis, finfo) = fromBytes(Misc.bits2Bytes(Seq("00000011")), BE, msbFirst)
- val bit1 = getLong(dis, 6, 2)
+ val bit1 = getLong(finfo, dis, 6, 2)
assertEquals(3, bit1)
- val bit2 = getLong(dis, 4, 2)
+ val bit2 = getLong(finfo, dis, 4, 2)
assertEquals(0, bit2)
}
@Test def testOneBit2() {
- val dis = fromBytes(Misc.bits2Bytes(Seq("11000000")), BE, msbFirst)
+ val (dis, finfo) = fromBytes(Misc.bits2Bytes(Seq("11000000")), BE, msbFirst)
- val bit1 = getLong(dis, 0, 2)
+ val bit1 = getLong(finfo, dis, 0, 2)
assertEquals(3, bit1)
- val bit2 = getLong(dis, 2, 2)
+ val bit2 = getLong(finfo, dis, 2, 2)
assertEquals(0, bit2)
}
@Test def testOneBit3() {
- val dis = fromBytes(Misc.bits2Bytes(Seq("00000011")), BE, msbFirst)
+ val (dis, finfo) = fromBytes(Misc.bits2Bytes(Seq("00000011")), BE, msbFirst)
- val n = getLong(dis, 6, 2)
+ val n = getLong(finfo, dis, 6, 2)
assertEquals(3, n)
}
@Test
def testBufferBitExtraction() {
- val dis = fromString("3", BE, msbFirst)
+ val (dis, finfo) = fromString("3", BE, msbFirst)
- assertEquals(3, getLong(dis, 1, 3))
+ assertEquals(3, getLong(finfo, dis, 1, 3))
}
@Test
def testBufferBitExtractionShift() {
- val dis = fromString("3", BE, msbFirst)
+ val (dis, finfo) = fromString("3", BE, msbFirst)
- assertEquals(12, getLong(dis, 2, 4))
+ assertEquals(12, getLong(finfo, dis, 2, 4))
}
@Test
def testBufferLeastSignificantBitExtractionShift() {
- val dis = fromString("4", BE, msbFirst)
+ val (dis, finfo) = fromString("4", BE, msbFirst)
- assertEquals(4, getLong(dis, 5, 3))
+ assertEquals(4, getLong(finfo, dis, 5, 3))
}
// Verify aligned byte/short/int/long/bigint extraction
@Test
def testBufferByteBigEndianExtraction() {
- val dis = fromString("3", BE, msbFirst)
+ val (dis, finfo) = fromString("3", BE, msbFirst)
- assertEquals(51, getLong(dis, 0, 8))
+ assertEquals(51, getLong(finfo, dis, 0, 8))
}
@Test
def testBufferByteLittleEndianExtraction() {
- val dis = fromString("3", LE, msbFirst)
+ val (dis, finfo) = fromString("3", LE, msbFirst)
- assertEquals(51, getLong(dis, 0, 8))
+ assertEquals(51, getLong(finfo, dis, 0, 8))
}
@Test
def testBufferShortBigEndianExtraction() {
- val dis = fromString("Om", BE, msbFirst, 0)
- assertEquals(20333, getLong(dis, 0, 16))
+ val (dis, finfo) = fromString("Om", BE, msbFirst, 0)
+ assertEquals(20333, getLong(finfo, dis, 0, 16))
}
@Test
def testBufferShortLittleEndianExtraction() {
- val dis = fromString("Om", LE, msbFirst, 0)
- assertEquals(27983, getLong(dis, 0, 16))
+ val (dis, finfo) = fromString("Om", LE, msbFirst, 0)
+ assertEquals(27983, getLong(finfo, dis, 0, 16))
}
@Test
def testBufferIntBigEndianExtraction() {
- val dis = fromString("Help", BE, msbFirst, 0)
- assertEquals(1214606448, getLong(dis, 0, 32))
+ val (dis, finfo) = fromString("Help", BE, msbFirst, 0)
+ assertEquals(1214606448, getLong(finfo, dis, 0, 32))
}
@Test
def testBufferIntLittleEndianExtraction() {
- val dis = fromString("Help", LE, msbFirst, 0)
- assertEquals(1886152008, getLong(dis, 0, 32))
+ val (dis, finfo) = fromString("Help", LE, msbFirst, 0)
+ assertEquals(1886152008, getLong(finfo, dis, 0, 32))
}
@Test
def testBufferLongBigEndianExtraction() {
- val dis = fromString("Harrison", BE, msbFirst, 0)
- assertEquals(BigInt(5215575679192756078L), getBigInt(dis, 0, 64))
+ val (dis, finfo) = fromString("Harrison", BE, msbFirst, 0)
+ assertEquals(BigInt(5215575679192756078L), getBigInt(finfo, dis, 0, 64))
}
@Test
def testBufferLongLittleEndianExtraction() {
- val dis = fromString("Harrison", LE, msbFirst, 0)
- assertEquals(BigInt(7957705963315814728L), getBigInt(dis, 0, 64))
+ val (dis, finfo) = fromString("Harrison", LE, msbFirst, 0)
+ assertEquals(BigInt(7957705963315814728L), getBigInt(finfo, dis, 0, 64))
}
@Test
def testBufferBigIntBigEndianExtraction() {
- val dis = fromString("Something in the way she moves, ", BE, msbFirst, 0)
- val bigInt = getBigInt(dis, 0, 256)
+ val (dis, finfo) = fromString("Something in the way she moves, ", BE, msbFirst, 0)
+ val bigInt = getBigInt(finfo, dis, 0, 256)
assertEquals(BigInt("37738841482167102822784581157237036764884875846207476558974346160344516471840"),
bigInt)
}
@Test
def testBufferBigIntLittleEndianExtraction() {
- val dis = fromString("Something in the way she moves, ", LE, msbFirst, 0)
+ val (dis, finfo) = fromString("Something in the way she moves, ", LE, msbFirst, 0)
assertEquals(BigInt("14552548861771956163454220823873430243364312915206513831353612029437431082835"),
- getBigInt(dis, 0, 256))
+ getBigInt(finfo, dis, 0, 256))
}
// Aligned but not full string
@Test
def testBufferPartialIntBigEndianExtraction() {
- val dis = fromString("SBT", BE, msbFirst, 0)
- assertEquals(5456468, getLong(dis, 0, 24))
+ val (dis, finfo) = fromString("SBT", BE, msbFirst, 0)
+ assertEquals(5456468, getLong(finfo, dis, 0, 24))
}
@Test
def testBufferPartialIntLittleEndianExtraction() {
- val dis = fromString("SBT", LE, msbFirst, 0)
- assertEquals(5522003, getLong(dis, 0, 24))
+ val (dis, finfo) = fromString("SBT", LE, msbFirst, 0)
+ assertEquals(5522003, getLong(finfo, dis, 0, 24))
}
// Non-Aligned 1 Byte or less
@Test
def testBufferBitNumberBigEndianExtraction() {
- val dis = fromString("3", BE, msbFirst)
+ val (dis, finfo) = fromString("3", BE, msbFirst)
- assertEquals(3, getLong(dis, 1, 3))
+ assertEquals(3, getLong(finfo, dis, 1, 3))
}
@Test
def testBufferBitNumberLittleEndianExtraction() {
- val dis = fromString("3", LE, msbFirst)
+ val (dis, finfo) = fromString("3", LE, msbFirst)
- assertEquals(3, getLong(dis, 1, 3))
+ assertEquals(3, getLong(finfo, dis, 1, 3))
}
@Test
def testBufferBitByteBigEndianExtraction() {
- val dis = fromString("3>", BE, msbFirst, 0)
- assertEquals(204, getLong(dis, 2, 8))
+ val (dis, finfo) = fromString("3>", BE, msbFirst, 0)
+ assertEquals(204, getLong(finfo, dis, 2, 8))
}
@Test
def testBufferBitByteLittleEndianExtraction() {
- val dis = fromString("3>", LE, msbFirst, 0)
- assertEquals(0xCC, getLong(dis, 2, 8))
+ val (dis, finfo) = fromString("3>", LE, msbFirst, 0)
+ assertEquals(0xCC, getLong(finfo, dis, 2, 8))
}
// Non-Aligned multi-byte
@Test
def testBufferPartialInt22At0BigEndianExtraction() {
- val dis = fromString("SBT", BE, msbFirst, 0)
- assertEquals(1364117, getLong(dis, 0, 22))
+ val (dis, finfo) = fromString("SBT", BE, msbFirst, 0)
+ assertEquals(1364117, getLong(finfo, dis, 0, 22))
}
@Test
def testBufferPartialInt22At0LittleEndianExtraction() {
- val dis = fromString("SBT", LE, msbFirst, 0)
- assertEquals(0x154253, getLong(dis, 0, 22))
+ val (dis, finfo) = fromString("SBT", LE, msbFirst, 0)
+ assertEquals(0x154253, getLong(finfo, dis, 0, 22))
// Corrected. Was 544253, but that omits shifting the most significant byte left 2
// because the field is only 22 long.
}
@Test
def testBufferPartialInt22At2BigEndianExtraction() {
- val dis = fromString("SBT", BE, msbFirst, 0)
- assertEquals(0x134254, getLong(dis, 2, 22))
+ val (dis, finfo) = fromString("SBT", BE, msbFirst, 0)
+ assertEquals(0x134254, getLong(finfo, dis, 2, 22))
}
@Test
def testBufferPartialInt22At2LittleEndianExtraction() {
- val dis = fromString("SBT", LE, msbFirst, 0)
- assertEquals(0x14094d, getLong(dis, 2, 22)) // Corrected.
+ val (dis, finfo) = fromString("SBT", LE, msbFirst, 0)
+ assertEquals(0x14094d, getLong(finfo, dis, 2, 22)) // Corrected.
// Was 0x50094d, but that omits shifting the most significant byte >> 2 to
// add the bits on the most significant side, not the least significant side.
}
@Test def testOneBit1LSBFirst() {
- val dis = fromBytes(Misc.bits2Bytes(Seq("01100000")), LE, lsbFirst)
+ val (dis, finfo) = fromBytes(Misc.bits2Bytes(Seq("01100000")), LE, lsbFirst)
- val bit1 = getLong(dis, 5, 2)
+ val bit1 = getLong(finfo, dis, 5, 2)
assertEquals(3, bit1)
- val bit2 = getLong(dis, 4, 2)
+ val bit2 = getLong(finfo, dis, 4, 2)
assertEquals(2, bit2)
}
@Test def testOneBit2LSBFirst() {
- val dis = fromBytes(Misc.bits2Bytes(Seq("01010000")), LE, lsbFirst)
+ val (dis, finfo) = fromBytes(Misc.bits2Bytes(Seq("01010000")), LE, lsbFirst)
- val bit1 = getLong(dis, 5, 2)
+ val bit1 = getLong(finfo, dis, 5, 2)
assertEquals(2, bit1)
- val bit2 = getLong(dis, 4, 2)
+ val bit2 = getLong(finfo, dis, 4, 2)
assertEquals(1, bit2)
}
@Test def testOneBit3LSBFirst() {
- val dis = fromBytes(BigInt(0xE4567A).toByteArray, LE, lsbFirst)
+ val (dis, finfo) = fromBytes(BigInt(0xE4567A).toByteArray, LE, lsbFirst)
- val bit1 = getLong(dis, 13, 12)
+ val bit1 = getLong(finfo, dis, 13, 12)
assertEquals(0x2B7, bit1)
}
}
diff --git a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/ByteBufferDataInputStream.scala b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/ByteBufferDataInputStream.scala
index 4afe9bf..ebf42af 100644
--- a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/ByteBufferDataInputStream.scala
+++ b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/ByteBufferDataInputStream.scala
@@ -32,13 +32,11 @@
package edu.illinois.ncsa.daffodil.io
-import edu.illinois.ncsa.daffodil.util.MaybeInt
import edu.illinois.ncsa.daffodil.util.MaybeChar
import edu.illinois.ncsa.daffodil.util.MaybeULong
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.UTF16Width
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.EncodingErrorPolicy
import java.nio.charset.CharsetDecoder
import java.nio.charset.StandardCharsets
import java.nio.charset.CodingErrorAction
@@ -57,6 +55,7 @@ import edu.illinois.ncsa.daffodil.equality._
import edu.illinois.ncsa.daffodil.util.Pool
import edu.illinois.ncsa.daffodil.util.MStackOf
import edu.illinois.ncsa.daffodil.api.DataStreamLimits
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.EncodingErrorPolicy
/**
* Factory for creating this type of DataInputStream
@@ -75,18 +74,18 @@ object ByteBufferDataInputStream {
* The initialBitPos0b is mostly there for testing purposes - to create small unit tests that
* start with non-byte alignment and such.
*/
- def apply(byteBuffer: ByteBuffer, initialBitPos0b: Long): DataInputStream =
+ def apply(byteBuffer: ByteBuffer, initialBitPos0b: Long): ByteBufferDataInputStream =
new ByteBufferDataInputStream(byteBuffer, initialBitPos0b)
/**
* Mostly this is for unit testing
*/
- def apply(byteArray: Array[Byte], initialBitPos0b: Long = 0): DataInputStream = apply(ByteBuffer.wrap(byteArray), initialBitPos0b)
+ def apply(byteArray: Array[Byte], initialBitPos0b: Long = 0): ByteBufferDataInputStream = apply(ByteBuffer.wrap(byteArray), initialBitPos0b)
/**
* File becomes a direct mapped byte buffer.
*/
- def apply(file: java.io.File, initialBitPos0b: Long): DataInputStream = {
+ def apply(file: java.io.File, initialBitPos0b: Long): ByteBufferDataInputStream = {
Assert.usage(file.length() < Int.MaxValue)
val path = file.toPath()
val channel = FileChannel.open(path)
@@ -97,7 +96,7 @@ object ByteBufferDataInputStream {
* If it is a FileInputStream, get its channel so we can mmap it.
* Otherwise copy into a byte array.
*/
- def apply(in: InputStream, initialBitPos0b: Long): DataInputStream = {
+ def apply(in: InputStream, initialBitPos0b: Long): ByteBufferDataInputStream = {
in match {
case fis: java.io.FileInputStream => {
val fc = fis.getChannel()
@@ -119,7 +118,7 @@ object ByteBufferDataInputStream {
/**
* If it's a file channel, mmap it. Otherwise treat like an InputStream
*/
- def apply(channel: java.nio.channels.ReadableByteChannel, initialBitPos0b: Long): DataInputStream = {
+ def apply(channel: java.nio.channels.ReadableByteChannel, initialBitPos0b: Long): ByteBufferDataInputStream = {
channel match {
case fc: FileChannel => {
val bbuf = fc.map(FileChannel.MapMode.READ_ONLY, 0L, fc.size())
@@ -184,7 +183,6 @@ private[io] class TLState {
*/
final class MarkState(initialBitPos0b: Long)
extends DataStreamCommonState with DataInputStream.Mark {
-
def defaultCodingErrorAction = ByteBufferDataInputStream.defaultCodingErrorAction
def bitPos0b: Long = (savedBytePosition0b << 3) + bitOffset0b
@@ -206,29 +204,19 @@ final class MarkState(initialBitPos0b: Long)
var maybeBitLimitOffset0b: MaybeULong = MaybeULong(0)
- var decoder: CharsetDecoder = {
- val dec = StandardCharsets.UTF_8.newDecoder()
- dec.onMalformedInput(defaultCodingErrorAction)
- dec.onUnmappableCharacter(defaultCodingErrorAction)
- dec
- }
- var codingErrorAction: CodingErrorAction = defaultCodingErrorAction
-
var adaptedRegexMatchBufferLimit: Int = 0
val charIteratorState = new CharIteratorState
+
// any members added here must be added to assignFrom below.
def assignFrom(other: MarkState): Unit = {
super.assignFrom(other)
- // this.dis = other.dis
this.savedBytePosition0b = other.savedBytePosition0b
this.savedByteLimit0b = other.savedByteLimit0b
this.bitOffset0b = other.bitOffset0b
this.savedByteOrder = other.savedByteOrder
this.maybeBitLimitOffset0b = other.maybeBitLimitOffset0b
- this.decoder = other.decoder
- this.codingErrorAction = other.codingErrorAction
this.adaptedRegexMatchBufferLimit = other.adaptedRegexMatchBufferLimit
this.charIteratorState.assignFrom(other.charIteratorState)
}
@@ -348,96 +336,34 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
bitLimit0b = MaybeULong(calcBitLimit0b)
}
- def setByteOrder(byteOrder: ByteOrder): Unit = {
- // threadCheck()
- byteOrder match {
- case ByteOrder.BigEndian => data.order(java.nio.ByteOrder.BIG_ENDIAN)
- case ByteOrder.LittleEndian => data.order(java.nio.ByteOrder.LITTLE_ENDIAN)
- }
- st.savedByteOrder = data.order()
- }
-
- def byteOrder: ByteOrder = {
- data.order match {
- case java.nio.ByteOrder.BIG_ENDIAN => ByteOrder.BigEndian
- case java.nio.ByteOrder.LITTLE_ENDIAN => ByteOrder.LittleEndian
- }
- }
-
- def getDecoder = st.decoder
-
- def setDecoder(decoder: CharsetDecoder): Unit = {
- // threadCheck()
- if (st.decoder == decoder) return
- st.decoder = decoder
- st.priorEncoding = decoder.charset()
- st.decoder.onMalformedInput(st.codingErrorAction)
- st.decoder.onUnmappableCharacter(st.codingErrorAction)
- val cs = decoder.charset()
- if (cs == StandardCharsets.UTF_16 || cs == StandardCharsets.UTF_16BE || cs == StandardCharsets.UTF_16LE) {
- st.encodingMandatoryAlignmentInBits = 8
- if (st.maybeUTF16Width.isDefined && st.maybeUTF16Width.get == UTF16Width.Fixed) {
- st.maybeCharWidthInBits = MaybeInt(16)
- } else {
- st.maybeCharWidthInBits = MaybeInt.Nope
- }
- } else {
- cs match {
- case decoderWithBits: NonByteSizeCharset => {
- st.encodingMandatoryAlignmentInBits = 1
- st.maybeCharWidthInBits = MaybeInt(decoderWithBits.bitWidthOfACodeUnit)
- }
- case _ => {
- st.encodingMandatoryAlignmentInBits = 8
- val encoder = cs.newEncoder()
- val maxBytes = encoder.maxBytesPerChar()
- if (maxBytes == encoder.averageBytesPerChar()) {
- st.maybeCharWidthInBits = MaybeInt((maxBytes * 8).toInt)
- } else {
- st.maybeCharWidthInBits = MaybeInt.Nope
- }
- }
- }
- }
- }
-
- def setEncodingErrorPolicy(eep: EncodingErrorPolicy): Unit = {
- // threadCheck()
- st.codingErrorAction = eep match {
- case EncodingErrorPolicy.Replace => CodingErrorAction.REPLACE
- case EncodingErrorPolicy.Error => CodingErrorAction.REPORT
- }
- st.decoder.onMalformedInput(st.codingErrorAction)
- st.decoder.onUnmappableCharacter(st.codingErrorAction)
- ()
- }
-
- def getByteArray(bitLengthFrom1: Int): Array[Byte] = {
+ def getByteArray(bitLengthFrom1: Int, finfo: FormatInfo): Array[Byte] = {
// threadCheck()
if (!isDefinedForLength(bitLengthFrom1)) throw DataInputStream.NotEnoughDataException(bitLengthFrom1)
val arraySize = (bitLengthFrom1 + 7) / 8
val array = new Array[Byte](arraySize)
- fillByteArray(array, bitLengthFrom1)
+ fillByteArray(array, bitLengthFrom1, finfo)
array
}
- // accepts a preallocated array and a bitLength. Reads the specified number
- // of bits and stores them in the array in big endian byte order and most
- // significant bit first bit order. The most significant byte is stored in
- // the zero'th index in the array. This means that if the array is larger
- // than the number of bytes needed for the specified number of bits, the
- // trailing bytes will be untouched and should be ignored by the caller.
- private def fillByteArray(array: Array[Byte], bitLengthFrom1: Int): Unit = {
+ /**
+ * Accepts a preallocated array and a bitLength. Reads the specified number
+ * of bits and stores them in the array in big endian byte order and most
+ * significant bit first bit order. The most significant byte is stored in
+ * the zero'th index in the array. This means that if the array is larger
+ * than the number of bytes needed for the specified number of bits, the
+ * trailing bytes will be untouched and should be ignored by the caller.
+ */
+ private def fillByteArray(array: Array[Byte], bitLengthFrom1: Int, finfo: FormatInfo): Unit = {
if (isAligned(8) && bitLengthFrom1 % 8 == 0) {
- fillByteArrayAlignedNoFragment(array, bitLengthFrom1)
+ fillByteArrayAlignedNoFragment(array, bitLengthFrom1, finfo)
} else {
- fillByteArrayUnalignedOrFragment(array, bitLengthFrom1)
+ fillByteArrayUnalignedOrFragment(array, bitLengthFrom1, finfo)
}
Assert.invariant(bitPos0b <= bitLimit0b.get)
}
- private def fillByteArrayAlignedNoFragment(array: Array[Byte], bitLengthFrom1: Int): Unit = {
+ private def fillByteArrayAlignedNoFragment(array: Array[Byte], bitLengthFrom1: Int, finfo: FormatInfo): Unit = {
// threadCheck()
Assert.usage(isAligned(8))
Assert.usage(bitLengthFrom1 % 8 == 0)
@@ -447,7 +373,7 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
val bytesToFill = bitLengthFrom1 / 8
Assert.invariant(array.size >= bytesToFill)
- if (byteOrder == ByteOrder.BigEndian && st.bitOrder == BitOrder.MostSignificantBitFirst) {
+ if (finfo.byteOrder == ByteOrder.BigEndian && finfo.bitOrder == BitOrder.MostSignificantBitFirst) {
// bits & bytes are already in order, read them straight into the array
data.get(array, 0, bytesToFill)
} else {
@@ -464,7 +390,7 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
setBitPos0b(savedBitPos0b + bitLengthFrom1)
}
- private def fillByteArrayUnalignedOrFragment(array: Array[Byte], bitLengthFrom1: Int): Unit = {
+ private def fillByteArrayUnalignedOrFragment(array: Array[Byte], bitLengthFrom1: Int, finfo: FormatInfo): Unit = {
// threadCheck()
Assert.usage(!isAligned(8) || bitLengthFrom1 % 8 != 0)
@@ -473,7 +399,7 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
val savedDataLimit = data.limit()
val savedBitPos0b = bitPos0b
data.limit(data.position() + nBytesNeeded)
-
+
val bytesToFill = (bitLengthFrom1 + 7) / 8
Assert.invariant(array.size >= bytesToFill)
val nFragmentBits = bitLengthFrom1 % 8
@@ -497,8 +423,8 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
val fragmentByte =
if (nFragmentBits <= bitsLeftInPriorByte) {
// all nFragmentBits can come from from the priror byte
- val composedByte =
- st.bitOrder match {
+ val composedByte =
+ finfo.bitOrder match {
case BitOrder.MostSignificantBitFirst => ((priorByte << st.bitOffset0b) & 0xFF) >>> (8 - nFragmentBits)
case BitOrder.LeastSignificantBitFirst => ((priorByte << (bitsLeftInPriorByte - nFragmentBits)) & 0xFF) >>> (8 - nFragmentBits)
}
@@ -508,7 +434,7 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
val bitsToGetFromCur = nFragmentBits - bitsLeftInPriorByte
val curByte = Bits.asUnsignedByte(data.get())
val composedByte =
- st.bitOrder match {
+ finfo.bitOrder match {
case BitOrder.MostSignificantBitFirst => {
val priorContribution = ((priorByte << st.bitOffset0b) & 0xFF) >>> (st.bitOffset0b - bitsToGetFromCur)
val curContribution = curByte >>> (8 - bitsToGetFromCur)
@@ -523,7 +449,7 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
priorByte = curByte
composedByte
}
- if (byteOrder =:= ByteOrder.LittleEndian) {
+ if (finfo.byteOrder =:= ByteOrder.LittleEndian) {
array(bytesToFill - 1 - i) = Bits.asSignedByte(fragmentByte)
} else {
array(i) = Bits.asSignedByte(fragmentByte)
@@ -531,8 +457,8 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
i += 1
}
- val newByteOffset =
- if (byteOrder =:= ByteOrder.BigEndian && nFragmentBits > 0) {
+ val newByteOffset =
+ if (finfo.byteOrder =:= ByteOrder.BigEndian && nFragmentBits > 0) {
addFragmentByte()
(st.bitOffset0b + nFragmentBits) % 8
} else {
@@ -540,12 +466,12 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
}
if (newByteOffset == 0) {
- byteOrder match {
+ finfo.byteOrder match {
case ByteOrder.BigEndian => {
// we just parsed a bigEndian fragment byte and it put us back on a byte
// boundary, so we just need to read the rest of the full aligned bytes
data.get(array, 1, bytesToFill - 1)
- }
+ }
case ByteOrder.LittleEndian => {
// we're starting on a byte boundary, so lets just consume all the
// whole bytes quickly, reversing positions in the array. We'll not
@@ -572,18 +498,18 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
// then we only want to fill the number of full bytes, and then we'll
// consume that last fragment byte afterwards
val stopBytePosition =
- if (byteOrder =:= ByteOrder.BigEndian) bytesToFill
+ if (finfo.byteOrder =:= ByteOrder.BigEndian) bytesToFill
else (bitLengthFrom1 / 8)
// consume full bytes
while (i < stopBytePosition) {
val curByte = Bits.asUnsignedByte(data.get())
val composedByte =
- st.bitOrder match {
- case BitOrder.MostSignificantBitFirst => ((priorByte << priorShift) & 0xFF) | ((curByte >>> curShift) & 0xFF)
+ finfo.bitOrder match {
+ case BitOrder.MostSignificantBitFirst => ((priorByte << priorShift) & 0xFF) | ((curByte >>> curShift) & 0xFF)
case BitOrder.LeastSignificantBitFirst => ((priorByte >>> priorShift) & 0xFF) | ((curByte << curShift) & 0xFF)
}
- if (byteOrder =:= ByteOrder.LittleEndian) {
+ if (finfo.byteOrder =:= ByteOrder.LittleEndian) {
array(bytesToFill - 1 - i) = Bits.asSignedByte(composedByte)
} else {
array(i) = Bits.asSignedByte(composedByte)
@@ -593,7 +519,7 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
}
}
- if (byteOrder =:= ByteOrder.LittleEndian && nFragmentBits > 0) {
+ if (finfo.byteOrder =:= ByteOrder.LittleEndian && nFragmentBits > 0) {
addFragmentByte()
}
@@ -602,8 +528,9 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
setBitPos0b(savedBitPos0b + bitLengthFrom1)
}
- def getBinaryDouble(): Double = {
+ def getBinaryDouble(finfo: FormatInfo): Double = {
Assert.usage(isAligned(8)) // aligned, so bitOrder doesn't matter
+ data.order(finfo.jByteOrder)
val db = data.asDoubleBuffer() // note: byte order is inherited from data
if (db.remaining() < 1) throw DataInputStream.NotEnoughDataException(64)
else {
@@ -613,9 +540,10 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
}
}
- def getBinaryFloat(): Float = {
+ def getBinaryFloat(finfo: FormatInfo): Float = {
Assert.usage(isAligned(8)) // aligned, so bitOrder doesn't matter
- val db = data.asFloatBuffer() // note: byte order is inherited from data's current
+ data.order(finfo.jByteOrder)
+ val db = data.asFloatBuffer()
if (db.remaining() < 1) throw DataInputStream.NotEnoughDataException(32)
else {
val d = db.get()
@@ -657,25 +585,25 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
*
* No allocation of objects occurs when calling this.
*/
- def getSignedLong(bitLengthFrom1To64: Int): Long = {
+ def getSignedLong(bitLengthFrom1To64: Int, finfo: FormatInfo): Long = {
// threadCheck()
Assert.usage(bitLengthFrom1To64 >= 1)
Assert.usage(bitLengthFrom1To64 <= 64)
- val res = getUnsignedLong(bitLengthFrom1To64)
+ val res = getUnsignedLong(bitLengthFrom1To64, finfo)
signExtend(res.longValue, bitLengthFrom1To64)
}
- def getUnsignedLong(bitLengthFrom1To64: Int): ULong = {
+ def getUnsignedLong(bitLengthFrom1To64: Int, finfo: FormatInfo): ULong = {
Assert.usage(bitLengthFrom1To64 >= 1)
Assert.usage(bitLengthFrom1To64 <= 64)
if (!isDefinedForLength(bitLengthFrom1To64)) throw DataInputStream.NotEnoughDataException(bitLengthFrom1To64)
- val numBytes = (bitLengthFrom1To64 + 7) / 8
+ val numBytes = (bitLengthFrom1To64 + 7) / 8
// will result in the first numBytes the long array filled in
- fillByteArray(longArray, bitLengthFrom1To64)
+ fillByteArray(longArray, bitLengthFrom1To64, finfo)
var res = Bits.asUnsignedByte(longArray(0)).toLong
var i = 1
@@ -688,14 +616,14 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
ULong(res)
}
- def getSignedBigInt(bitLengthFrom1: Int): BigInt = {
+ def getSignedBigInt(bitLengthFrom1: Int, finfo: FormatInfo): BigInt = {
// threadCheck()
Assert.usage(bitLengthFrom1 >= 1)
if (bitLengthFrom1 <= 64) {
- BigInt(getSignedLong(bitLengthFrom1))
+ BigInt(getSignedLong(bitLengthFrom1, finfo))
} else {
- val allBytes = getByteArray(bitLengthFrom1)
+ val allBytes = getByteArray(bitLengthFrom1, finfo)
val fragmentLength = bitLengthFrom1 % 8
if (fragmentLength > 0) {
// if there is a fragment byte, we need to sign extend the rest of the
@@ -708,9 +636,9 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
}
}
- def getUnsignedBigInt(bitLengthFrom1: Int): BigInt = {
+ def getUnsignedBigInt(bitLengthFrom1: Int, finfo: FormatInfo): BigInt = {
Assert.usage(bitLengthFrom1 >= 1)
- val bytes = getByteArray(bitLengthFrom1)
+ val bytes = getByteArray(bitLengthFrom1, finfo)
BigInt(1, bytes)
}
@@ -726,7 +654,7 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
else true
}
- def skip(nBits: Long): Boolean = {
+ def skip(nBits: Long, finfo: FormatInfo): Boolean = {
// threadCheck()
Assert.usage(nBits <= Int.MaxValue)
if (!this.isDefinedForLength(nBits)) return false
@@ -765,7 +693,6 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
data.limit(st.savedByteLimit0b)
data.order(st.savedByteOrder)
markPool.returnToPool(current)
- charIterator.reset() // this also resets the decoder.
bitLimit0b = MaybeULong(calcBitLimit0b)
}
@@ -785,12 +712,12 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
markPool.finalCheck
}
- private def stringLengthInbits(nBytesConsumed: Long, nCharsTransferred: Long): Long = {
+ private def stringLengthInbits(nBytesConsumed: Long, nCharsTransferred: Long, finfo: FormatInfo): Long = {
// threadCheck()
//
// There are two ways length is determined.
//
- if (st.maybeCharWidthInBits.isEmpty) {
+ if (finfo.maybeCharWidthInBits.isEmpty) {
// we don't have a character width in bits
// that would be because the encoding is variable width
// like utf-8 or utf-16BE/LE with dfdl:utf16Width='variable', or shift-JIS
@@ -802,7 +729,7 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
// number of bytes consumed.
8 * nBytesConsumed
} else {
- val charWidthInBits = st.maybeCharWidthInBits.get
+ val charWidthInBits = finfo.maybeCharWidthInBits.get
// we have a character width in bits, and it is fixed
//
// Hence, we can determine the number of bits consumed by way of
@@ -811,7 +738,7 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
}
}
- def decodeIt(decoder: CharsetDecoder, cb: java.nio.CharBuffer): CoderResult = {
+ def decodeIt(decoder: CharsetDecoder, cb: java.nio.CharBuffer, finfo: FormatInfo): CoderResult = {
var cr: CoderResult = null
var nCharsTransferred: Int = 0
var nBytesConsumed: Int = 0
@@ -831,7 +758,7 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
//
if (cbRemainingBefore == 1 && nCharsTransferred == 0 && nBytesConsumed == 0 && cr.isOverflow()) {
Assert.invariant(bbRemainingBefore >= 4)
- Assert.invariant(st.decoder.charset() == StandardCharsets.UTF_8)
+ Assert.invariant(finfo.decoder.charset() == StandardCharsets.UTF_8)
val firstByte = data.get(data.position())
Assert.invariant(firstByte == 0xF0.toByte) // F0 means 3 more bytes for a total of 4
//
@@ -891,17 +818,16 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
cr
}
- def fillCharBuffer(cb: java.nio.CharBuffer): MaybeULong = {
+ def fillCharBuffer(cb: java.nio.CharBuffer, finfo: FormatInfo): MaybeULong = {
// threadCheck()
- if (!align(st.encodingMandatoryAlignmentInBits)) return MaybeULong.Nope
+ if (!align(finfo.encodingMandatoryAlignmentInBits, finfo)) return MaybeULong.Nope
//
// Corner case stuff for utf-8 and surrogate pairs.
//
- if (st.decoder.charset() == StandardCharsets.UTF_8) {
+ if (finfo.decoder.charset() == StandardCharsets.UTF_8) {
if (st.maybeTrailingSurrogateForUTF8.isDefined &&
- st.priorBitPos == bitPos0b &&
- st.priorEncoding == StandardCharsets.UTF_8) {
+ st.priorBitPos == bitPos0b) {
// We're utf-8, the prior character was a leading surrogate,
// we're at the same location we were when we produced the
// leading surrogate. So produce the trailing.
@@ -916,23 +842,21 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
// irrespective of whether the user wants REPLACE semantics
// we need REPORT semantics here.
//
- st.decoder.onMalformedInput(CodingErrorAction.REPORT)
- st.decoder.onUnmappableCharacter(CodingErrorAction.REPORT)
var cr: CoderResult = null
var nCharsTransferred: Int = 0
var nBytesConsumed: Int = 0
- st.decoder.reset()
- val decoder = st.decoder match {
+ finfo.reportingDecoder.reset()
+ val decoder = finfo.reportingDecoder match {
case decoderWithBits: NonByteSizeCharsetDecoder => {
decoderWithBits.setInitialBitOffset(st.bitOffset0b)
decoderWithBits.setFinalByteBitLimitOffset0b(st.maybeBitLimitOffset0b)
decoderWithBits
}
- case _ => st.decoder
+ case x => x
}
val cbRemainingBefore = cb.remaining()
val bbRemainingBefore = data.remaining()
- cr = decodeIt(decoder, cb)
+ cr = decodeIt(decoder, cb, finfo)
nCharsTransferred = cbRemainingBefore - cb.remaining()
nBytesConsumed = bbRemainingBefore - data.remaining()
@@ -968,19 +892,17 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
//
if (nCharsTransferred == 0) {
// we got an error on the very first character
- if (st.codingErrorAction == CodingErrorAction.REPORT)
+ if (finfo.encodingErrorPolicy eq EncodingErrorPolicy.Error)
cr.throwException()
else {
- Assert.invariant(st.codingErrorAction == CodingErrorAction.REPLACE)
+ Assert.invariant(finfo.encodingErrorPolicy eq EncodingErrorPolicy.Replace)
//
// it's possible bytes were consumed. Rewind those.
setBitPos0b(bitPos0bBefore)
- st.decoder.onMalformedInput(CodingErrorAction.REPLACE)
- st.decoder.onUnmappableCharacter(CodingErrorAction.REPLACE)
val cbRemainingBefore = cb.remaining()
val bbRemainingBefore = data.remaining()
- cr = decodeIt(decoder, cb)
+ cr = decodeIt(finfo.replacingDecoder, cb, finfo)
nCharsTransferred = cbRemainingBefore - cb.remaining()
nBytesConsumed = bbRemainingBefore - data.remaining()
}
@@ -1000,13 +922,13 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
MaybeULong.Nope
} else {
Assert.invariant(nBytesConsumed == bbRemainingBefore - data.remaining())
- val lengthInBits = stringLengthInbits(nBytesConsumed, nCharsTransferred)
+ val lengthInBits = stringLengthInbits(nBytesConsumed, nCharsTransferred, finfo)
setBitPos0b(bitPos0bBefore + lengthInBits)
MaybeULong(nCharsTransferred)
}
}
- def skipChars(nChars: Long): Boolean = {
+ def skipChars(nChars: Long, finfo: FormatInfo): Boolean = {
// threadCheck()
Assert.usage(nChars <= skipCharBuf.capacity())
skipCharBuf.clear
@@ -1014,7 +936,7 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
var maybeN: MaybeULong = MaybeULong(0L)
var total = 0L
while (maybeN.isDefined && total < nChars) {
- maybeN = fillCharBuffer(skipCharBuf)
+ maybeN = fillCharBuffer(skipCharBuf, finfo)
if (maybeN.isDefined) {
total += maybeN.get.toLong
}
@@ -1051,7 +973,7 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
isMatchDone
}
- def lookingAt(matcher: java.util.regex.Matcher, initialRegexMatchLimitInChars: Long = limits.defaultInitialRegexMatchLimitInChars): Boolean = {
+ def lookingAt(matcher: java.util.regex.Matcher, finfo: FormatInfo, initialRegexMatchLimitInChars: Long = limits.defaultInitialRegexMatchLimitInChars): Boolean = {
// threadCheck()
val initialBitPos0b = bitPos0b
var isMatchDone = false
@@ -1077,7 +999,7 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
//
regexMatchBuffer.position(positionAfterLastFill)
regexMatchBuffer.limit(math.max(limitAfterLastFill, regexMatchBuffer.limit))
- val ml = fillCharBuffer(regexMatchBuffer)
+ val ml = fillCharBuffer(regexMatchBuffer, finfo)
val bufPosAfter = regexMatchBuffer.position()
val limitAfter = regexMatchBuffer.limit()
positionAfterLastFill = bufPosAfter
@@ -1132,13 +1054,13 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
st.adaptedRegexMatchBufferLimit = newLimit // set the prior value as our adapted length
}
val nBitsConsumed: Long =
- if (this.isFixedWidthEncoding) {
+ if (finfo.isFixedWidthEncoding) {
// that means the characters are fixed width
- Assert.notYetImplemented(if (st.maybeUTF16Width.isDefined) st.maybeUTF16Width.get == UTF16Width.Variable else false)
+ Assert.notYetImplemented(if (finfo.maybeUTF16Width.isDefined) finfo.maybeUTF16Width.get == UTF16Width.Variable else false)
// TODO: utf16 width variable just changes the way we count chars slightly. Have to scan the string
// for surrogate pairs, and subtract 1 from the length in chars for each pair found.
// not bothering for now.
- val bitsPerChar = st.maybeCharWidthInBits.get
+ val bitsPerChar = finfo.maybeCharWidthInBits.get
nCharsInMatch * bitsPerChar
} else {
//
@@ -1171,7 +1093,7 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
// In other words, if hitEnd is false after an attempted match, then we don't need to
// keep pushing ahead decoding (and substituting for errors) into the data.
//
- val isFilled = fillCharBufferLoop(lengthDeterminationBuffer)
+ val isFilled = fillCharBufferLoop(lengthDeterminationBuffer, finfo)
Assert.invariant(isFilled)
val nbits = bitPos0b - initialBitPos0b
nbits
@@ -1182,7 +1104,11 @@ final class ByteBufferDataInputStream private (var data: ByteBuffer, initialBitP
}
private val charIterator = new BBDISCharIterator(st, this)
- def asIteratorChar: DataInputStream.CharIterator = charIterator
+ def asIteratorChar: DataInputStream.CharIterator = {
+ val ci = charIterator
+ ci.reset()
+ ci
+ }
def pastData(nBytesRequested: Int): ByteBuffer = {
// threadCheck()
@@ -1240,9 +1166,23 @@ class BBDISCharIterator(st: MarkState, dis: ByteBufferDataInputStream)
extends DataInputStream.CharIterator
with ThreadCheckMixin {
+ private var finfo_ : FormatInfo = _
+
+ def finfo = {
+ if (finfo_ eq null) Assert.usageError("must setFormatInfo before invoking interator.")
+ finfo_
+ }
+
+ def setFormatInfo(finfo: FormatInfo): Unit = {
+ finfo_ = finfo
+ }
+
def reset() {
st.charIteratorState.clear()
- st.decoder.reset()
+ if (finfo_ ne null) {
+ finfo.decoder.reset()
+ finfo_ = null
+ }
}
/**
@@ -1255,12 +1195,12 @@ class BBDISCharIterator(st: MarkState, dis: ByteBufferDataInputStream)
// threadCheck()
val ist = st.charIteratorState
val dataBitPosBefore0b = dis.bitPos0b
- if (!dis.align(st.encodingMandatoryAlignmentInBits)) return false
+ if (!dis.align(finfo.encodingMandatoryAlignmentInBits, finfo)) return false
ist.bitPos0bAtLastFetch = dis.bitPos0b // keep track of where we start trying to fetch a character
ist.cb.clear()
ist.cb.limit(1)
- st.decoder.reset()
- val maybeNumChars = dis.fillCharBuffer(ist.cb) // throws if a decode error and encodingErrorPolicy="error"
+ finfo.decoder.reset()
+ val maybeNumChars = dis.fillCharBuffer(ist.cb, finfo) // throws if a decode error and encodingErrorPolicy="error"
if (maybeNumChars.isEmpty) {
// Couldn't get one character
ist.deltaBits = 0
diff --git a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/CharBufferDataOutputStream.scala b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/CharBufferDataOutputStream.scala
deleted file mode 100644
index 09e30ee..0000000
--- a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/CharBufferDataOutputStream.scala
+++ /dev/null
@@ -1,88 +0,0 @@
-/* Copyright (c) 2016 Tresys Technology, LLC. All rights reserved.
- *
- * Developed by: Tresys Technology, LLC
- * http://www.tresys.com
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy of
- * this software and associated documentation files (the "Software"), to deal with
- * the Software without restriction, including without limitation the rights to
- * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
- * of the Software, and to permit persons to whom the Software is furnished to do
- * so, subject to the following conditions:
- *
- * 1. Redistributions of source code must retain the above copyright notice,
- * this list of conditions and the following disclaimers.
- *
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimers in the
- * documentation and/or other materials provided with the distribution.
- *
- * 3. Neither the names of Tresys Technology, nor the names of its contributors
- * may be used to endorse or promote products derived from this Software
- * without specific prior written permission.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE
- * SOFTWARE.
- */
-
-package edu.illinois.ncsa.daffodil.io
-
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
-import passera.unsigned.ULong
-import java.nio.ByteBuffer
-import edu.illinois.ncsa.daffodil.exceptions.Assert
-import java.nio.CharBuffer
-import edu.illinois.ncsa.daffodil.util.Misc
-
-/**
- * Used to implement unparsing when length is specified in lengthUnits 'characters' when
- * the encoding is variable width (e.g., utf-8)
- */
-final class CharBufferDataOutputStream
- extends DataOutputStream with DataOutputStreamImplMixin {
-
- override def id = 0
-
- private[io] def isBuffering: Boolean = true
-
- private def doNotUse = Assert.usageError("Not to be called on " + Misc.getNameFromClass(this))
-
- protected def getJavaOutputStream(): java.io.OutputStream = doNotUse
- protected def setJavaOutputStream(newOutputStream: java.io.OutputStream): Unit = doNotUse
-
- private var target: CharBuffer = null
-
- def setCharBuffer(cb: CharBuffer) {
- target = cb
- }
-
- private def notToBeUsed = Assert.usageError("not to be used")
-
- override def putCharBuffer(cb: java.nio.CharBuffer): Long = {
- Assert.usage(target != null)
- val numCharsTransferred = cb.read(target)
- numCharsTransferred
- }
-
- override def putBigInt(bigInt: BigInt, bitLengthFrom1: Int, signed: Boolean): Boolean = notToBeUsed
- override def putByteArray(ba: Array[Byte], bitLengthFrom1: Int): Boolean = notToBeUsed
- override def putBytes(ba: Array[Byte]) = notToBeUsed
- override def putULong(unsignedLong: ULong, bitLengthFrom1To64: Int): Boolean = notToBeUsed
- override def putLong(signedLong: Long, bitLengthFrom1To64: Int): Boolean = notToBeUsed
- override def putBinaryFloat(float: Float): Boolean = notToBeUsed
- override def putBinaryDouble(double: Double): Boolean = notToBeUsed
- final override protected def putLong_BE_MSBFirst(signedLong: Long, bitLengthFrom1To64: Int): Boolean = notToBeUsed
- final override protected def putLong_LE_MSBFirst(signedLong: Long, bitLengthFrom1To64: Int): Boolean = notToBeUsed
- final override protected def putLong_LE_LSBFirst(signedLong: Long, bitLengthFrom1To64: Int): Boolean = notToBeUsed
- override def skip(nBits: Long): Boolean = notToBeUsed
- override def futureData(nBytesRequested: Int): ByteBuffer = notToBeUsed
- override def pastData(nBytesRequested: Int): ByteBuffer = notToBeUsed
- override def setByteOrder(byteOrder: ByteOrder): Unit = notToBeUsed
- override def byteOrder: ByteOrder = notToBeUsed
- override def validateFinalStreamState { /* do nothing */ }
-}
diff --git a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataInputStream.scala b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataInputStream.scala
index 5d26f4a..c14a4bb 100644
--- a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataInputStream.scala
+++ b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataInputStream.scala
@@ -33,7 +33,6 @@
package edu.illinois.ncsa.daffodil.io
import java.nio.CharBuffer
-import java.nio.charset.CharsetDecoder
import java.util.regex.Matcher
import edu.illinois.ncsa.daffodil.util.Maybe
import passera.unsigned.ULong
@@ -160,6 +159,10 @@ object DataInputStream {
*/
trait CharIterator extends Iterator[Char] {
+ def reset(): Unit
+
+ def setFormatInfo(fi: FormatInfo): Unit
+
/**
* Looks ahead 1 into the data. Returns -1.toChar if there is no
* character to peek to.
@@ -222,30 +225,6 @@ trait DataInputStream
setBitLimit0b(newLimit)
}
- /*
- * Setters for all the text and binary characteristics.
- * <p>
- * These are set rather than passed to various operations because
- * they will, most often, be very slow changing relative to the
- * operations that actually perform data motion.
- * <p>
- * If any of these are not set as part of initialization,
- * then IllegalStateException is
- * thrown when any of the methods that access data are called, or
- * any of the mark/reset/discard methods are called.
- */
-
- /**
- * Sets the character set decoder.
- * <p>
- * If the `charWidthInBits` is less
- * than 8, then the decoder must be an instance of
- * `NonByteSizeCharsetEncoderDecoder` (a trait mixed into such decoders)
- * so that an initial bit offset can be set if necessary.
- */
- def getDecoder: CharsetDecoder
- def setDecoder(decoder: CharsetDecoder): Unit
-
/**
* Returns a mark value. It saves the current state of the input stream such
* that it can be restored by calling reset(mark).
@@ -319,7 +298,7 @@ trait DataInputStream
* If the bitLengthFrom1 is not a multiple of 8, the final byte will be
* padded with zeros to make a full byte.
*/
- def getByteArray(bitLengthFrom1: Int): Array[Byte]
+ def getByteArray(bitLengthFrom1: Int, finfo: FormatInfo): Array[Byte]
/**
* Returns a long integer containing the bits between the current bit position
@@ -339,7 +318,7 @@ trait DataInputStream
* to check if sufficient bits are available. Alternatively one can catch the exception,
* but that is likely less performant.
*/
- def getUnsignedLong(bitLengthFrom1To64: Int): ULong
+ def getUnsignedLong(bitLengthFrom1To64: Int, finfo: FormatInfo): ULong
/**
* Similar, but returns a negative value if the most-significant bit of the
@@ -360,7 +339,7 @@ trait DataInputStream
* to check if sufficient bits are available. Alternatively one can catch the exception,
* but that is likely less performant.
*/
- def getSignedLong(bitLengthFrom1To64: Int): Long
+ def getSignedLong(bitLengthFrom1To64: Int, finfo: FormatInfo): Long
/**
* Constructs a big integer from the data. The current bit order and byte order are used.
@@ -377,7 +356,7 @@ trait DataInputStream
* It is recommended that getUnsignedLong be used for any bit length 64 or less, as
* that method does not require a heap allocated object to represent the value.
*/
- def getUnsignedBigInt(bitLengthFrom1: Int): BigInt
+ def getUnsignedBigInt(bitLengthFrom1: Int, finfo: FormatInfo): BigInt
/**
* Constructs a big integer from the data. The current bit order and byte order are used.
@@ -397,7 +376,7 @@ trait DataInputStream
* It is recommended that getSignedLong be used for any bit length 64 or less, as
* that method does not require a heap allocated object to represent the value.
*/
- def getSignedBigInt(bitLengthFrom1: Int): BigInt
+ def getSignedBigInt(bitLengthFrom1: Int, finfo: FormatInfo): BigInt
/**
* Float and Double
@@ -408,8 +387,8 @@ trait DataInputStream
* Consider first calling isDefinedForLength before calling this in order to avoid the
* possibility of a throw.
*/
- def getBinaryFloat(): Float
- def getBinaryDouble(): Double
+ def getBinaryFloat(finfo: FormatInfo): Float
+ def getBinaryDouble(finfo: FormatInfo): Double
/**
* Fill a charBuffer with characters.
@@ -468,7 +447,7 @@ trait DataInputStream
* Implementation Note: a specialized 4-bit encoding which maps 4 bits to
* 0-9A-F can be used to treat packed decimal representations like text strings.
*/
- def fillCharBuffer(cb: CharBuffer): MaybeULong
+ def fillCharBuffer(cb: CharBuffer, finfo: FormatInfo): MaybeULong
/**
* Returns One(string) if nChars are available, Nope otherwise.
@@ -476,7 +455,7 @@ trait DataInputStream
* Throws a CharacterCodingException if the encoding error policy is 'error'
* and a decode error is detected within nChars.
*/
- def getString(nChars: Long): Maybe[String]
+ def getString(nChars: Long, finfo: FormatInfo): Maybe[String]
/**
* Returns One(string) if any (up to nChars) are available, Nope otherwise.
@@ -484,14 +463,14 @@ trait DataInputStream
* Throws a CharacterCodingException if the encoding error policy is 'error'
* and a decode error is detected within nChars.
*/
- def getSomeString(nChars: Long): Maybe[String]
+ def getSomeString(nChars: Long, finfo: FormatInfo): Maybe[String]
/**
* Skips N characters and returns true, adjusting the bitPos0b based on
* parsing them. Returns false if there is not enough data
* to skip all N characters.
*/
- def skipChars(nChars: Long): Boolean
+ def skipChars(nChars: Long, finfo: FormatInfo): Boolean
/**
* Matches a regex Matcher against a prefix of the data stream.
@@ -545,7 +524,7 @@ trait DataInputStream
* strings by the underlying Matcher and regular expression API upon which this is
* built.
*/
- def lookingAt(matcher: Matcher, initialRegexMatchLimitInChars: Long = limits.defaultInitialRegexMatchLimitInChars): Boolean
+ def lookingAt(matcher: Matcher, finfo: FormatInfo, initialRegexMatchLimitInChars: Long = limits.defaultInitialRegexMatchLimitInChars): Boolean
/**
* As characters are iterated, the underlying bit position changes.
diff --git a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataInputStreamImplMixin.scala b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataInputStreamImplMixin.scala
index e92b1ba..21e2718 100644
--- a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataInputStreamImplMixin.scala
+++ b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataInputStreamImplMixin.scala
@@ -49,31 +49,31 @@ trait DataInputStreamImplMixin extends DataInputStream
* way fillCharBuffer works to return early when decode errors are
* encountered.
*/
- protected final def fillCharBufferLoop(cb: CharBuffer): Boolean = {
+ protected final def fillCharBufferLoop(cb: CharBuffer, finfo: FormatInfo): Boolean = {
var maybeN: MaybeULong = MaybeULong(0)
var total: Long = 0
val nChars = cb.remaining
while (maybeN.isDefined && total < nChars) {
- maybeN = fillCharBuffer(cb)
+ maybeN = fillCharBuffer(cb, finfo)
if (maybeN.isDefined) total += maybeN.get.toLong
}
total == nChars
}
- final def getString(nChars: Long): Maybe[String] = {
+ final def getString(nChars: Long, finfo: FormatInfo): Maybe[String] = {
withLocalCharBuffer { lcb =>
val cb = lcb.getBuf(nChars)
- val gotAll = fillCharBufferLoop(cb)
+ val gotAll = fillCharBufferLoop(cb, finfo)
val res = if (!gotAll) Nope
else Maybe(cb.flip.toString)
res
}
}
- final def getSomeString(nChars: Long): Maybe[String] = {
+ final def getSomeString(nChars: Long, finfo: FormatInfo): Maybe[String] = {
withLocalCharBuffer { lcb =>
val cb = lcb.getBuf(nChars)
- fillCharBufferLoop(cb)
+ fillCharBufferLoop(cb, finfo)
if (cb.position() == 0) Nope
else Maybe(cb.flip.toString)
// TODO: Performance - we need to copy here. Consider return type of Maybe[StringBuilder]
@@ -94,10 +94,10 @@ trait DataInputStreamImplMixin extends DataInputStream
res
}
- final override def align(bitAlignment1b: Int): Boolean = {
+ final override def align(bitAlignment1b: Int, finfo: FormatInfo): Boolean = {
if (isAligned(bitAlignment1b)) return true
val deltaBits = bitAlignment1b - (bitPos0b % bitAlignment1b)
- skip(deltaBits)
+ skip(deltaBits, finfo)
}
final override def remainingBits = {
diff --git a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataOutputStream.scala b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataOutputStream.scala
index 77071c3..51bba7a 100644
--- a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataOutputStream.scala
+++ b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataOutputStream.scala
@@ -33,7 +33,6 @@
package edu.illinois.ncsa.daffodil.io
import passera.unsigned.ULong
-import java.nio.charset.CharsetEncoder
import java.nio.CharBuffer
import edu.illinois.ncsa.daffodil.util.MaybeULong
import edu.illinois.ncsa.daffodil.util.Logging
@@ -129,14 +128,6 @@ trait DataOutputStream extends DataStreamCommon
def resetMaybeRelBitLimit0b(savedBitLimit0b: MaybeULong): Unit
- def encoder: CharsetEncoder
- def setEncoder(encoder: CharsetEncoder): Unit
-
- /**
- * Sets the fill byte. Only the least significant 8 bits of the argument are used.
- */
- def setFillByte(fillByte: Int): Unit
-
/**
* If bitLengthFrom1To64 bits are available to be written before bitLimit0b (if defined) is encountered,
* then this writes the bitLengthFrom1To64 least significant bits of the long using the
@@ -146,7 +137,7 @@ trait DataOutputStream extends DataStreamCommon
*
* It is a usage error if bitLengthFrom1To64 is not in the range 1 to 64 inclusive.
*/
- def putLong(signedLong: Long, bitLengthFrom1To64: Int): Boolean
+ def putLong(signedLong: Long, bitLengthFrom1To64: Int, finfo: FormatInfo): Boolean
/**
* If bitLengthFrom1To64 bits are available to be written before bitLimit0b (if defined) is encountered,
@@ -157,7 +148,7 @@ trait DataOutputStream extends DataStreamCommon
*
* It is a usage error if bitLengthFrom1To64 is not in the range 1 to 64 inclusive.
*/
- def putULong(unsignedLong: ULong, bitLengthFrom1To64: Int): Boolean
+ def putULong(unsignedLong: ULong, bitLengthFrom1To64: Int, finfo: FormatInfo): Boolean
/**
* If bitLengthFrom1 bits are available to be written before bitLimit0b (if
@@ -174,7 +165,7 @@ trait DataOutputStream extends DataStreamCommon
* It is a usage error if bitLengthFrom1 is not greater than or equal to 1.
*
*/
- def putBigInt(bigInt: BigInt, bitLengthFrom1: Int, signed: Boolean): Boolean
+ def putBigInt(bigInt: BigInt, bitLengthFrom1: Int, signed: Boolean, finfo: FormatInfo): Boolean
/**
* If bitLengthFrom1 bits are available to be written before bitLimit0b (if
@@ -188,8 +179,7 @@ trait DataOutputStream extends DataStreamCommon
* It is a usage error if bitLengthFrom1 is not greater than or equal to 1.
*
*/
- def putByteArray(ba: Array[Byte], bitLengthFrom1: Int): Boolean
-
+ def putByteArray(ba: Array[Byte], bitLengthFrom1: Int, finfo: FormatInfo): Boolean
/**
* Float and Double
@@ -198,38 +188,20 @@ trait DataOutputStream extends DataStreamCommon
* <p>
* Returns false if there are not 32 bits or 64 bits (respectively) available.
*/
- def putBinaryFloat(v: Float): Boolean
- def putBinaryDouble(v: Double): Boolean
-
- /**
- * Returns number of bytes transferred. Stops when the bitLimit is
- * encountered if one is defined.
- */
- def putBytes(ba: Array[Byte]): Long
-
- /**
- * Returns number of bits transferred. The last byte of the byte buffer
- * can be a fragment byte.
- */
- def putBitBuffer(bb: java.nio.ByteBuffer, lengthInBits: Long): Long
-
- /**
- * Returns number of bits transferred. The last byte of the byte buffer
- * can be a fragment byte.
- */
- def putBits(ba: Array[Byte], byteStartOffset0b: Int, lengthInBits: Long): Long
+ def putBinaryFloat(v: Float, finfo: FormatInfo): Boolean
+ def putBinaryDouble(v: Double, finfo: FormatInfo): Boolean
/**
* Returns number of characters transferred. Stops when the bitLimit is
* encountered if one is defined.
*/
- def putString(str: String): Long
- def putCharBuffer(cb: CharBuffer): Long
+ def putString(str: String, finfo: FormatInfo): Long
+ def putCharBuffer(cb: CharBuffer, finfo: FormatInfo): Long
/**
* close-out this output stream. No more writing to this after.
*/
- def setFinished(): Unit
+ def setFinished(finfo: FormatInfo): Unit
def isFinished: Boolean
}
diff --git a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataOutputStreamImplMixin.scala b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataOutputStreamImplMixin.scala
index 5bb8817..10190a7 100644
--- a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataOutputStreamImplMixin.scala
+++ b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataOutputStreamImplMixin.scala
@@ -33,16 +33,10 @@
package edu.illinois.ncsa.daffodil.io
import edu.illinois.ncsa.daffodil.util.Maybe
-import edu.illinois.ncsa.daffodil.util.MaybeInt
import edu.illinois.ncsa.daffodil.util.Maybe.One
import edu.illinois.ncsa.daffodil.util.Maybe.Nope
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.UTF16Width
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.EncodingErrorPolicy
-import java.nio.charset.CharsetEncoder
-import java.nio.charset.StandardCharsets
-import java.nio.charset.CodingErrorAction
import passera.unsigned.ULong
import java.nio.ByteBuffer
import edu.illinois.ncsa.daffodil.exceptions.Assert
@@ -65,58 +59,6 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
with DataStreamCommonImplMixin
with LocalBufferMixin {
- def defaultCodingErrorAction: CodingErrorAction = CodingErrorAction.REPLACE
-
- private val initialEncoder = {
- val enc = StandardCharsets.UTF_8.newEncoder
- enc.onMalformedInput(defaultCodingErrorAction)
- enc.onUnmappableCharacter(defaultCodingErrorAction)
- enc
- }
-
- private var fillByte_ : Int = 0
- def fillByte = fillByte_
-
- /**
- * The fillLong is a long that has been pre-filled with the fill byte.
- *
- * If the fillByte changes, then the fillLong is updated to track it.
- */
- private var fillLong_ : Long = 0L
- def fillLong = fillLong_
-
- def setFillByte(newFillByte: Int) {
- Assert.usage(isWritable)
- Assert.usage(newFillByte <= 255 && newFillByte >= 0)
- slamFillByte(newFillByte)
- }
-
- /**
- * slamFillByte assigns the fill byte but doesn't do the invariant checking.
- * It is for use from contexts where you need to set the fill byte, but the
- * usual invariants aren't expected to hold.
- */
- def slamFillByte(newFillByte: Int): Unit = {
- fillByte_ = newFillByte
- fillLong_ = {
- if (fillByte_ == 0) 0L
- else {
- var fl: Long = 0L
- var i = 0
- while (i < 8) {
- fl = fl << 8
- fl = fl | fillByte_
- i += 1
- }
- fl
- }
- }
- }
-
- protected final var encoder_ : CharsetEncoder = initialEncoder
-
- protected final var codingErrorAction: CodingErrorAction = defaultCodingErrorAction
-
/**
* Relative bit position zero based.
*
@@ -225,7 +167,7 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
* doesn't change.
*
* This value allows stored relative offsets (stored as the start/ends
- * of content and value lenght regions) to still be meaningful even though
+ * of content and value length regions) to still be meaningful even though
* we have collapsed the stream into a direct one.
*/
private var maybeAbsolutizedRelativeStartingBitPosInBits_ = MaybeULong.Nope
@@ -281,22 +223,10 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
*/
var bitStartOffset0b: Int = 0
- final def remainingBits: MaybeULong = {
+ final override def remainingBits: MaybeULong = {
if (maybeRelBitLimit0b.isEmpty) MaybeULong.Nope else MaybeULong(maybeRelBitLimit0b.get - relBitPos0b.toLong)
}
- private var byteOrder_ : ByteOrder = ByteOrder.BigEndian
-
- def setByteOrder(byteOrder: ByteOrder): Unit = {
- Assert.usage(isWritable)
- byteOrder_ = byteOrder
- }
-
- def byteOrder: ByteOrder = {
- Assert.usage(isReadable)
- byteOrder_
- }
-
var debugOutputStream: Maybe[ByteArrayOutputStream] = Nope
/**
@@ -316,6 +246,15 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
private var fragmentLastByteLimit_ : Int = 0
def fragmentLastByteLimit = fragmentLastByteLimit_
+ private var maybeFragmentBitOrder: Maybe[BitOrder] = Nope
+ def fragmentBitOrder = {
+ Assert.usage(maybeFragmentBitOrder.isDefined)
+ this.maybeFragmentBitOrder.value
+ }
+ def setFragmentBitOrder(bo: BitOrder) {
+ maybeFragmentBitOrder = One(bo)
+ }
+
def setFragmentLastByte(newFragmentByte: Int, nBitsInUse: Int) {
Assert.usage(nBitsInUse >= 0 && nBitsInUse <= 7)
Assert.usage(newFragmentByte >= 0 && newFragmentByte <= 255) // no bits above first byte are in use.
@@ -340,7 +279,7 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
@inline private[io] final def isDead = { _dosState =:= Uninitialized }
@inline override final def isFinished = { _dosState =:= Finished }
- @inline override def setFinished() { _dosState = Finished }
+ // @inline override def setFinished(finfo: FormatInfo) { _dosState = Finished }
@inline private[io] final def isActive = { _dosState =:= Active }
@inline private[io] final def isReadOnly = { isFinished && isBuffering }
@inline private[io] final def isWritable = {
@@ -353,66 +292,6 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
}
@inline private[io] final def isReadable = { !isDead }
- final def encoder = {
- Assert.usage(isReadable)
- encoder_
- }
-
- final def setEncoder(encoder: CharsetEncoder): Unit = {
- Assert.usage(isWritable)
- if (this.encoder_ == encoder) return
- encoder_ = encoder
- encoder.onMalformedInput(codingErrorAction)
- encoder.onUnmappableCharacter(codingErrorAction)
- val cs = encoder.charset()
-
- if (cs == StandardCharsets.UTF_16 || cs == StandardCharsets.UTF_16BE || cs == StandardCharsets.UTF_16LE) {
- if (maybeUTF16Width.isDefined && maybeUTF16Width.get == UTF16Width.Fixed) {
- maybeCharWidthInBits = MaybeInt(16)
- encodingMandatoryAlignmentInBits = 8
- } else {
- maybeCharWidthInBits = MaybeInt.Nope
- encodingMandatoryAlignmentInBits = 8
- }
- } else {
- cs match {
- case encoderWithBits: NonByteSizeCharset => {
- maybeCharWidthInBits = MaybeInt(encoderWithBits.bitWidthOfACodeUnit)
- encodingMandatoryAlignmentInBits = 1
- }
- case _ => {
- val maxBytes = encoder.maxBytesPerChar()
- if (maxBytes == encoder.averageBytesPerChar()) {
- maybeCharWidthInBits = MaybeInt((maxBytes * 8).toInt)
- encodingMandatoryAlignmentInBits = 8
- } else {
- maybeCharWidthInBits = MaybeInt.Nope
- encodingMandatoryAlignmentInBits = 8
- }
- }
- }
- }
- }
-
- final def encodingErrorPolicy = {
- Assert.usage(isReadable)
- codingErrorAction match {
- case CodingErrorAction.REPLACE => EncodingErrorPolicy.Replace
- case CodingErrorAction.REPORT => EncodingErrorPolicy.Error
- }
- }
-
- final def setEncodingErrorPolicy(eep: EncodingErrorPolicy): Unit = {
- Assert.usage(isWritable)
- codingErrorAction = eep match {
- case EncodingErrorPolicy.Replace => CodingErrorAction.REPLACE
- case EncodingErrorPolicy.Error => CodingErrorAction.REPORT
- }
- encoder.onMalformedInput(codingErrorAction)
- encoder.onUnmappableCharacter(codingErrorAction)
- ()
- }
-
protected def setJavaOutputStream(newOutputStream: java.io.OutputStream): Unit
protected def getJavaOutputStream(): java.io.OutputStream
@@ -422,20 +301,12 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
protected def assignFrom(other: DataOutputStreamImplMixin) {
Assert.usage(isWritable)
super.assignFrom(other)
- this.fillByte_ = other.fillByte_
- this.fillLong_ = other.fillLong_
- // DO NOT SET THIS IT IS SET IN THE CALLER ; this.encoder = other.encoder
- // DO NOT SET THIS IT IS SET IN THE CALLER ; this.codingErrorAction = other.codingErrorAction
this.maybeAbsStartingBitPos0b_ = other.maybeAbsStartingBitPos0b_
this.maybeAbsolutizedRelativeStartingBitPosInBits_ = other.maybeAbsolutizedRelativeStartingBitPosInBits_
this.relBitPos0b_ = other.relBitPos0b_
this.maybeAbsBitLimit0b = other.maybeAbsBitLimit0b
this.maybeRelBitLimit0b_ = other.maybeRelBitLimit0b_
- this.byteOrder_ = other.byteOrder_
this.debugOutputStream = other.debugOutputStream
- //this.setFragmentLastByte(other.fragmentLastByte, other.fragmentLastByteLimit)
- this.setEncoder(other.encoder)
- this.setEncodingErrorPolicy(other.encodingErrorPolicy)
}
/**
@@ -460,140 +331,145 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
}
}
- def putBigInt(bigInt: BigInt, bitLengthFrom1: Int, signed: Boolean): Boolean = {
+ final override def putBigInt(bigInt: BigInt, bitLengthFrom1: Int, signed: Boolean, finfo: FormatInfo): Boolean = {
Assert.usage(isWritable)
Assert.usage(bitLengthFrom1 > 0)
Assert.usage(signed || (!signed && bigInt >= 0))
- if (bitLengthFrom1 <= 64) {
- // output as a long
- val longInt = bigInt.toLong
- putLong(longInt, bitLengthFrom1)
- } else {
- // Note that we do not need to distinguish between signed and unsigned
- // here. Checks should have been performend earlier to ensure that the
- // bigInt value matches the signed/unsignedness. So we just need to
- // convert this to a byte array, ensure that the number of bits in that
- // bit array fit the bit length, and put the array.
- val array = bigInt.toByteArray
- val numWholeBytesNeeded = (bitLengthFrom1 + 7) / 8
-
- val maybeArrayToPut =
- if (array.size > numWholeBytesNeeded) {
- // This is the only case where we care about signedness. If the
- // BigInt is a positive value and the most significant bit is a 1,
- // then toByteArray will create one extra byte that is all zeros so
- // that the two's complement isn't negative. In this case, the array
- // size is 1 greater than numWholeBytesNeeded and the most
- // significant byte is zero. For a signed type (e.g. xs:integer),
- // having this extra byte, regardless of its value, means the BigInt
- // value was too big to fit into bitLengthFrom1 bits. However, if
- // this is a unsigned type (e.g. xs:nonNegativeInteger), this extra
- // byte can be zero and still fit into bitLengthFrom1. So if this is
- // the case, then just chop off that byte and write the array.
- // Otherwise, this results in Nope which ends up returning false.
- if (!signed && (array.size == numWholeBytesNeeded + 1) && array(0) == 0) {
- One(array.tail)
- } else {
- Nope
- }
- } else if (array.size < numWholeBytesNeeded) {
- // This bigInt value can definitely fit in the number of bits,
- // however, we need to pad it up to numWholeBytesNeed. We may need to
- // sign extend with the most significant bit of most significant byte
- // of the array
- val paddedArray = new Array[Byte](numWholeBytesNeeded)
- val numPaddingBytes = numWholeBytesNeeded - array.size
-
- array.copyToArray(paddedArray, numPaddingBytes)
- val sign = 0x80 & array(0)
- if (sign > 0) {
- // The most significant bit of the most significant byte was 1. That
- // means this was a negative number and these padding bytes must be
- // sign extended
- Assert.invariant(bigInt < 0)
- var i = 0
- while (i < numPaddingBytes) {
- paddedArray(i) = 0xFF.toByte
- i += 1
+ val res = {
+ if (bitLengthFrom1 <= 64) {
+ // output as a long
+ val longInt = bigInt.toLong
+ putLongChecked(longInt, bitLengthFrom1, finfo)
+ } else {
+ // Note that we do not need to distinguish between signed and unsigned
+ // here. Checks should have been performend earlier to ensure that the
+ // bigInt value matches the signed/unsignedness. So we just need to
+ // convert this to a byte array, ensure that the number of bits in that
+ // bit array fit the bit length, and put the array.
+ val array = bigInt.toByteArray
+ val numWholeBytesNeeded = (bitLengthFrom1 + 7) / 8
+
+ val maybeArrayToPut =
+ if (array.size > numWholeBytesNeeded) {
+ // This is the only case where we care about signedness. If the
+ // BigInt is a positive value and the most significant bit is a 1,
+ // then toByteArray will create one extra byte that is all zeros so
+ // that the two's complement isn't negative. In this case, the array
+ // size is 1 greater than numWholeBytesNeeded and the most
+ // significant byte is zero. For a signed type (e.g. xs:integer),
+ // having this extra byte, regardless of its value, means the BigInt
+ // value was too big to fit into bitLengthFrom1 bits. However, if
+ // this is a unsigned type (e.g. xs:nonNegativeInteger), this extra
+ // byte can be zero and still fit into bitLengthFrom1. So if this is
+ // the case, then just chop off that byte and write the array.
+ // Otherwise, this results in Nope which ends up returning false.
+ if (!signed && (array.size == numWholeBytesNeeded + 1) && array(0) == 0) {
+ One(array.tail)
+ } else {
+ Nope
}
- }
- One(paddedArray)
- } else {
- // We got the right amount of bytes, however, it is possible that
- // more significant bits were set that can fit in bitLengthFrom1.
- // Determine if that is the case.
- val fragBits = bitLengthFrom1 % 8
- if (fragBits == 0) {
- // no frag bits, so the most significant bit is fine and no sign
- // extending is needed to be checked
- One(array)
+ } else if (array.size < numWholeBytesNeeded) {
+ // This bigInt value can definitely fit in the number of bits,
+ // however, we need to pad it up to numWholeBytesNeed. We may need to
+ // sign extend with the most significant bit of most significant byte
+ // of the array
+ val paddedArray = new Array[Byte](numWholeBytesNeeded)
+ val numPaddingBytes = numWholeBytesNeeded - array.size
+
+ array.copyToArray(paddedArray, numPaddingBytes)
+ val sign = 0x80 & array(0)
+ if (sign > 0) {
+ // The most significant bit of the most significant byte was 1. That
+ // means this was a negative number and these padding bytes must be
+ // sign extended
+ Assert.invariant(bigInt < 0)
+ var i = 0
+ while (i < numPaddingBytes) {
+ paddedArray(i) = 0xFF.toByte
+ i += 1
+ }
+ }
+ One(paddedArray)
} else {
- val shifted = array(0) >> (fragBits - 1) // shift off the bits we don't care about (with sign extend shift)
- val signBit = shifted & 0x1 // get the most significant bit
- val signExtendedBits = shifted >> 1 // shift off the sign bit
-
- // At this point, signExtendedBits should be all 1's (i.e. -1) if
- // the sign bit was 1, or all zeros (i.e. 0) if the sign bit was 0.
- // If this isn't the case, then there were non-signed extended bits
- // above our most significant bit, and so this BigInt was too big
- // for the number of bits. If this is the case, result in a Nope
- // which ends up returning false.
- if ((signBit == 1 && signExtendedBits != -1) || (signBit == 0 && signExtendedBits != 0)) {
- // error
- Nope
- } else {
- // The most significant byte is properly sign extended for the
- // for the number of bits, so the array is good
+ // We got the right amount of bytes, however, it is possible that
+ // more significant bits were set that can fit in bitLengthFrom1.
+ // Determine if that is the case.
+ val fragBits = bitLengthFrom1 % 8
+ if (fragBits == 0) {
+ // no frag bits, so the most significant bit is fine and no sign
+ // extending is needed to be checked
One(array)
+ } else {
+ val shifted = array(0) >> (fragBits - 1) // shift off the bits we don't care about (with sign extend shift)
+ val signBit = shifted & 0x1 // get the most significant bit
+ val signExtendedBits = shifted >> 1 // shift off the sign bit
+
+ // At this point, signExtendedBits should be all 1's (i.e. -1) if
+ // the sign bit was 1, or all zeros (i.e. 0) if the sign bit was 0.
+ // If this isn't the case, then there were non-signed extended bits
+ // above our most significant bit, and so this BigInt was too big
+ // for the number of bits. If this is the case, result in a Nope
+ // which ends up returning false.
+ if ((signBit == 1 && signExtendedBits != -1) || (signBit == 0 && signExtendedBits != 0)) {
+ // error
+ Nope
+ } else {
+ // The most significant byte is properly sign extended for the
+ // for the number of bits, so the array is good
+ One(array)
+ }
}
}
- }
- if (maybeArrayToPut.isDefined) {
- // at this point, we have an array that is of the right size and properly
- // sign extended. It is also BE MSBF, so we can put it just like we would
- // put a hexBinary array
- putByteArray(maybeArrayToPut.get, bitLengthFrom1)
- } else {
- false
+ if (maybeArrayToPut.isDefined) {
+ // at this point, we have an array that is of the right size and properly
+ // sign extended. It is also BE MSBF, so we can put it just like we would
+ // put a hexBinary array
+ putByteArray(maybeArrayToPut.get, bitLengthFrom1, finfo)
+ } else {
+ false
+ }
}
}
+ res
}
- def putByteArray(array: Array[Byte], bitLengthFrom1: Int): Boolean = {
+ final override def putByteArray(array: Array[Byte], bitLengthFrom1: Int, finfo: FormatInfo): Boolean = {
// this is to be used for an array generated by getByteArray. Thus, this
// array is expected by to BigEndian MSBF. It must be transformed into an
// array that the other putBytes/Bits/etc functions can accept
Assert.usage(bitLengthFrom1 >= 1)
Assert.usage(isWritable)
- if (maybeRelBitLimit0b.isDefined && maybeRelBitLimit0b.get < (relBitPos0b + bitLengthFrom1)) {
- false
- } else {
- if (bitOrder =:= BitOrder.MostSignificantBitFirst) {
- val fragBits = bitLengthFrom1 % 8
- if (byteOrder =:= ByteOrder.LittleEndian) {
- // MSBF & LE
- if (fragBits > 0) {
- array(0) = Bits.asSignedByte((array(0) << (8 - fragBits)) & 0xFF)
+ val res =
+ if (maybeRelBitLimit0b.isDefined && maybeRelBitLimit0b.get < (relBitPos0b + bitLengthFrom1)) {
+ false
+ } else {
+ if (finfo.bitOrder =:= BitOrder.MostSignificantBitFirst) {
+ val fragBits = bitLengthFrom1 % 8
+ if (finfo.byteOrder =:= ByteOrder.LittleEndian) {
+ // MSBF & LE
+ if (fragBits > 0) {
+ array(0) = Bits.asSignedByte((array(0) << (8 - fragBits)) & 0xFF)
+ }
+ Bits.reverseBytes(array)
+ } else {
+ // MSBF & BE
+ if (fragBits > 0) {
+ Bits.shiftLeft(array, 8 - fragBits)
+ }
}
- Bits.reverseBytes(array)
} else {
- // MSBF & BE
- if (fragBits > 0) {
- Bits.shiftLeft(array, 8 - fragBits)
- }
+ // LSBF & LE
+ Bits.reverseBytes(array)
}
- } else {
- // LSBF & LE
- Bits.reverseBytes(array)
- }
- val bits = putBits(array, 0, bitLengthFrom1)
- Assert.invariant(bits == bitLengthFrom1)
- true
- }
+ val bits = putBits(array, 0, bitLengthFrom1, finfo)
+ Assert.invariant(bits == bitLengthFrom1)
+ true
+ }
+ res
}
private def exceedsBitLimit(lengthInBytes: Long): Boolean = {
@@ -603,11 +479,15 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
else false
}
- def putBits(ba: Array[Byte], byteStartOffset0b: Int, lengthInBits: Long): Long = {
+ /**
+ * Returns number of bits transferred. The last byte of the byte buffer
+ * can be a fragment byte.
+ */
+ private[io] def putBits(ba: Array[Byte], byteStartOffset0b: Int, lengthInBits: Long, finfo: FormatInfo): Long = {
Assert.usage((ba.length - byteStartOffset0b) * 8 >= lengthInBits)
val nWholeBytes = (lengthInBits / 8).toInt
val nFragBits = (lengthInBits % 8).toInt
- val nBytesWritten = putBytes(ba, byteStartOffset0b, nWholeBytes)
+ val nBytesWritten = putBytes(ba, byteStartOffset0b, nWholeBytes, finfo)
val nBitsWritten = nBytesWritten * 8
if (nBytesWritten < nWholeBytes) {
nBytesWritten * 8
@@ -615,11 +495,11 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
val isFragWritten =
if (nFragBits > 0) {
var fragByte: Long = ba(byteStartOffset0b + nWholeBytes)
- if (bitOrder == BitOrder.MostSignificantBitFirst) {
+ if (finfo.bitOrder == BitOrder.MostSignificantBitFirst) {
// we need to shift the bits. We want the most significant bits of the byte
fragByte = fragByte >>> (8 - nFragBits)
}
- putLong(fragByte, nFragBits)
+ putLongChecked(fragByte, nFragBits, finfo)
} else
true
if (isFragWritten) nBitsWritten + nFragBits
@@ -627,7 +507,11 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
}
}
- def putBytes(ba: Array[Byte], byteStartOffset0b: Int, lengthInBytes: Int): Long = {
+ /**
+ * Returns number of bytes transferred. Stops when the bitLimit is
+ * encountered if one is defined.
+ */
+ final def putBytes(ba: Array[Byte], byteStartOffset0b: Int, lengthInBytes: Int, finfo: FormatInfo): Long = {
Assert.usage(isWritable)
if (isEndOnByteBoundary) {
val nBytes =
@@ -645,21 +529,25 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
// the data currently ends with some bits in the fragment byte.
//
// rather than duplicate all this shifting logic here, we're going to output
- // each byte separately as an 8-bit chunk using putLong
+ // each byte separately as an 8-bit chunk using putLongUnchecked
//
var i = 0
var continue = true
while ((i < lengthInBytes) && continue) {
- continue = putLong(Bits.asUnsignedByte(ba(i)), 8) // returns false if we hit the limit.
+ continue = putLongUnchecked(Bits.asUnsignedByte(ba(i)), 8, finfo) // returns false if we hit the limit.
if (continue) i += 1
}
i
}
}
- def putBytes(ba: Array[Byte]): Long = putBytes(ba, 0, ba.length)
+ /**
+ * Returns number of bytes transferred. Stops when the bitLimit is
+ * encountered if one is defined.
+ */
+ private[io] def putBytes(ba: Array[Byte], finfo: FormatInfo): Long = putBytes(ba, 0, ba.length, finfo)
- def putBitBuffer(bb: java.nio.ByteBuffer, lengthInBits: Long): Long = {
+ private[io] def putBitBuffer(bb: java.nio.ByteBuffer, lengthInBits: Long, finfo: FormatInfo): Long = {
Assert.usage(bb.remaining() * 8 >= lengthInBits)
val nWholeBytes = (lengthInBits / 8).toInt
val nFragBits = (lengthInBits % 8).toInt
@@ -671,7 +559,7 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
Assert.usage(bb.remaining() == numBytesForLengthInBits)
if (nFragBits > 0) bb.limit(bb.limit - 1) // last byte is the frag byte
- val nBytesWritten = putByteBuffer(bb) // output all but the frag byte if there is one.
+ val nBytesWritten = putByteBuffer(bb, finfo) // output all but the frag byte if there is one.
val nBitsWritten = nBytesWritten * 8
if (nBytesWritten < nWholeBytes) {
nBytesWritten * 8
@@ -680,11 +568,11 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
if (nFragBits > 0) {
bb.limit(bb.limit + 1)
var fragByte: Long = Bits.asUnsignedByte(bb.get(bb.limit - 1))
- if (bitOrder == BitOrder.MostSignificantBitFirst) {
+ if (finfo.bitOrder eq BitOrder.MostSignificantBitFirst) {
// we need to shift the bits. We want the most significant bits of the byte
fragByte = fragByte >>> (8 - nFragBits)
}
- putLong(fragByte, nFragBits)
+ putLongChecked(fragByte, nFragBits, finfo)
} else
true
if (isFragWritten) nBitsWritten + nFragBits
@@ -692,11 +580,11 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
}
}
- private def putByteBuffer(bb: java.nio.ByteBuffer): Long = {
+ private def putByteBuffer(bb: java.nio.ByteBuffer, finfo: FormatInfo): Long = {
Assert.usage(isWritable)
val nTransferred =
if (bb.hasArray) {
- putBytes(bb.array, bb.arrayOffset + bb.position, bb.remaining())
+ putBytes(bb.array, bb.arrayOffset + bb.position, bb.remaining(), finfo)
} else {
if (isEndOnByteBoundary) {
val lengthInBytes = bb.remaining
@@ -714,7 +602,7 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
var continue = true
val limit = bb.remaining()
while ((i < limit) && continue) {
- continue = putLong(Bits.asUnsignedByte(bb.get(i)), 8) // returns false if we hit the limit.
+ continue = putLongChecked(Bits.asUnsignedByte(bb.get(i)), 8, finfo) // returns false if we hit the limit.
if (continue) i += 1
}
i
@@ -723,7 +611,7 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
nTransferred
}
- final def putString(str: String): Long = {
+ final def putString(str: String, finfo: FormatInfo): Long = {
Assert.usage(isWritable)
// must respect bitLimit0b if defined
// must not get encoding errors until a char is being written to the output
@@ -732,11 +620,11 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
val cb = lcb.getBuf(str.length)
cb.append(str) // one copying hop
cb.flip
- putCharBuffer(cb)
+ putCharBuffer(cb, finfo)
}
}
- def putCharBuffer(cb: java.nio.CharBuffer): Long = {
+ final def putCharBuffer(cb: java.nio.CharBuffer, finfo: FormatInfo): Long = {
Assert.usage(isWritable)
val debugCBString = if (areLogging(LogLevel.Debug)) cb.toString() else ""
@@ -761,7 +649,7 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
// in cb to below 1/4 the max number of bytes in a bytebuffer, because the maxBytesPerChar
// is 4, even though it is going to be on average much closer to 1 or 2.
//
- val nBytes = math.ceil(cb.remaining * encoder.maxBytesPerChar()).toLong
+ val nBytes = math.ceil(cb.remaining * finfo.encoder.maxBytesPerChar()).toLong
val bb = lbb.getBuf(nBytes)
// Note that encode reads the charbuffer and encodes the character into a
// local byte buffer. The contents of the local byte buffer are then
@@ -769,7 +657,7 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
// bitPosiition/bitLimit/fragment bytes. So because we encode to a
// temporary output buffer, the encode loop does not need to be aware of
// bit position/limit.
- val cr = encoder.encode(cb, bb, true)
+ val cr = finfo.encoder.encode(cb, bb, true)
cr match {
case CoderResult.UNDERFLOW => //ok. Normal termination
case CoderResult.OVERFLOW => Assert.invariantFailed("byte buffer wasn't big enough to accomodate the string")
@@ -778,19 +666,17 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
}
bb.flip
- val bitsToWrite = encoder match {
+ val bitsToWrite = finfo.encoder match {
case encoderWithBits: NonByteSizeCharsetEncoder => encoderWithBits.bitWidthOfACodeUnit * nToTransfer
case _ => bb.remaining * 8
}
- if (putBitBuffer(bb, bitsToWrite) == 0) 0 else nToTransfer
+ if (putBitBuffer(bb, bitsToWrite, finfo) == 0) 0 else nToTransfer
}
log(LogLevel.Debug, "Wrote string '%s' to %s", debugCBString, this)
res
}
- def putULong(unsignedLong: ULong, bitLengthFrom1To64: Int): Boolean = putLong(unsignedLong.longValue, bitLengthFrom1To64)
-
- def putLong(signedLong: Long, bitLengthFrom1To64: Int): Boolean = {
+ protected final def putLongChecked(signedLong: Long, bitLengthFrom1To64: Int, finfo: FormatInfo): Boolean = {
Assert.usage(bitLengthFrom1To64 >= 1 && bitLengthFrom1To64 <= 64)
Assert.usage(isWritable)
//
@@ -800,11 +686,22 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
if (maybeRelBitLimit0b.isDefined) {
if (maybeRelBitLimit0b.get < relBitPos0b + bitLengthFrom1To64) return false
}
+
+ putLongUnchecked(signedLong, bitLengthFrom1To64, finfo)
+ }
+
+ /**
+ * Use when calling from a put/write operation that has already checked the
+ * length limit.
+ *
+ * Assumed to be called from inner loops, so should be fast as possible.
+ */
+ protected final def putLongUnchecked(signedLong: Long, bitLengthFrom1To64: Int, finfo: FormatInfo): Boolean = {
//
// based on bit and byte order, dispatch to specific code
//
val res =
- if (byteOrder eq ByteOrder.BigEndian) {
+ if (finfo.byteOrder eq ByteOrder.BigEndian) {
//
// You would think this invariant would hold
//
@@ -819,11 +716,11 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
//
putLong_BE_MSBFirst(signedLong, bitLengthFrom1To64)
} else {
- Assert.invariant(byteOrder eq ByteOrder.LittleEndian)
- if (bitOrder eq BitOrder.MostSignificantBitFirst) {
+ Assert.invariant(finfo.byteOrder eq ByteOrder.LittleEndian)
+ if (finfo.bitOrder eq BitOrder.MostSignificantBitFirst) {
putLong_LE_MSBFirst(signedLong, bitLengthFrom1To64)
} else {
- Assert.invariant(bitOrder eq BitOrder.LeastSignificantBitFirst)
+ Assert.invariant(finfo.bitOrder eq BitOrder.LeastSignificantBitFirst)
putLong_LE_LSBFirst(signedLong, bitLengthFrom1To64)
}
}
@@ -848,26 +745,47 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
private val unionIntBuffer = unionByteBuffer.asIntBuffer()
protected val unionLongBuffer = unionByteBuffer.asLongBuffer()
- def putBinaryFloat(v: Float): Boolean = {
+ final override def putBinaryFloat(v: Float, finfo: FormatInfo): Boolean = {
unionFloatBuffer.put(0, v)
val i = unionIntBuffer.get(0)
- putLong(i, 32)
+ val res = putLongChecked(i, 32, finfo)
+ res
}
- def putBinaryDouble(v: Double): Boolean = {
+ final override def putBinaryDouble(v: Double, finfo: FormatInfo): Boolean = {
unionDoubleBuffer.put(0, v)
val l = unionLongBuffer.get(0)
- putLong(l, 64)
+ val res = putLongChecked(l, 64, finfo)
+ res
}
- def skip(nBits: Long): Boolean = {
+ /**
+ * Used when we have to fill in things that are larger or smaller than a byte
+ */
+ private def fillLong(fillByte: Byte) = {
+ var fl: Long = 0L
+ val fb = fillByte.toInt & 0xFF
+ fl = (fl << 8) + fb
+ fl = (fl << 8) + fb
+ fl = (fl << 8) + fb
+ fl = (fl << 8) + fb
+ fl = (fl << 8) + fb
+ fl = (fl << 8) + fb
+ fl = (fl << 8) + fb
+ fl = (fl << 8) + fb
+ fl
+ }
+
+ final override def skip(nBits: Long, finfo: FormatInfo): Boolean = {
Assert.usage(isWritable)
if (maybeRelBitLimit0b.isDefined) {
val lim = maybeRelBitLimit0b.getULong
if (relBitPos0b + ULong(nBits) > lim) return false
}
if (nBits <= 64) {
- putLong(fillLong, nBits.toInt)
+ val fb = finfo.fillByte
+ val lng = fillLong(fb)
+ putLongUnchecked(lng, nBits.toInt, finfo)
} else {
// more than 64 bits to skip
//
@@ -887,7 +805,7 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
if (fragmentLastByteLimit > 0) {
// there is a fragment byte.
val numRemainingFragmentBits = 8 - fragmentLastByteLimit
- val isInitialFragDone = putLong(fillByte, numRemainingFragmentBits)
+ val isInitialFragDone = putLongUnchecked(finfo.fillByte, numRemainingFragmentBits, finfo)
Assert.invariant(isInitialFragDone)
nBitsRemaining -= numRemainingFragmentBits
Assert.invariant(fragmentLastByteLimit == 0) // no longer is a fragment byte on the end
@@ -899,7 +817,7 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
while (nBytes > 0) {
nBytes -= 1
setRelBitPos0b(relBitPos0b + ULong(8))
- realStream.write(fillByte)
+ realStream.write(finfo.fillByte)
nBitsRemaining -= 8
}
//
@@ -907,19 +825,19 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
//
Assert.invariant(nBitsRemaining < 8)
if (nBitsRemaining > 0) {
- val isFinalFragDone = putLong(fillByte, nBitsRemaining.toInt)
+ val isFinalFragDone = putLongUnchecked(finfo.fillByte, nBitsRemaining.toInt, finfo)
Assert.invariant(isFinalFragDone)
}
true
}
}
- def futureData(nBytesRequested: Int): ByteBuffer = {
+ final override def futureData(nBytesRequested: Int): ByteBuffer = {
Assert.usage(isReadable)
ByteBuffer.allocate(0)
}
- def pastData(nBytesRequested: Int): ByteBuffer = {
+ final override def pastData(nBytesRequested: Int): ByteBuffer = {
Assert.usage(isReadable)
if (!areDebugging) throw new IllegalStateException("Must be debugging.")
Assert.usage(nBytesRequested >= 0)
@@ -940,7 +858,7 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
setMaybeRelBitLimit0b(savedBitLimit0b, true)
}
- def validateFinalStreamState {
+ final override def validateFinalStreamState {
// nothing to validate
}
@@ -955,11 +873,11 @@ trait DataOutputStreamImplMixin extends DataStreamCommonState
}
}
- final override def align(bitAlignment1b: Int): Boolean = {
+ final override def align(bitAlignment1b: Int, finfo: FormatInfo): Boolean = {
if (isAligned(bitAlignment1b)) true
else {
val deltaBits = bitAlignment1b - (maybeAbsBitPos0b.get % bitAlignment1b)
- skip(deltaBits)
+ skip(deltaBits, finfo)
}
}
diff --git a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataStreamCommon.scala b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataStreamCommon.scala
index 35b6db4..d8ab340 100644
--- a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataStreamCommon.scala
+++ b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataStreamCommon.scala
@@ -34,14 +34,8 @@ package edu.illinois.ncsa.daffodil.io
import java.nio.ByteBuffer
-import edu.illinois.ncsa.daffodil.api.DataStreamLimits
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BinaryFloatRep
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.EncodingErrorPolicy
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.UTF16Width
-import edu.illinois.ncsa.daffodil.util.Maybe
import edu.illinois.ncsa.daffodil.util.MaybeULong
+import edu.illinois.ncsa.daffodil.api.DataStreamLimits
/**
* This is an interface trait, and it defines methods shared by
@@ -67,37 +61,6 @@ trait DataStreamCommon {
*/
def setLimits(newLimits: DataStreamLimits): Unit
- // def setEncodingMandatoryAlignment(bitAlignment: Int): Unit
- def setEncodingErrorPolicy(eep: EncodingErrorPolicy): Unit
-
- /**
- * Use Nope for variable-width encodings.
- */
- def setMaybeUTF16Width(maybeUTF16Width: Maybe[UTF16Width]): Unit
- def setBinaryFloatRep(binaryFloatRep: BinaryFloatRep): Unit
-
- /*
- * Note that when character encodings are not byte-centric (e.g., 7, 6, 5, or 4 bits)
- * then the bit order *is* used by the character decoding to determine which
- * side of a byte is first.
- */
- def setBitOrder(bitOrder: BitOrder): Unit
-
- /* Note that the byte order for UTF-16 and UTF-32 encodings is
- * not taken from this setByteOrder call, but by use of the
- * UTF-16BE, UTF-16LE, UTF-32BE and UTF-32LE encodings, or
- * by use of the dfdl:unicodeByteOrderMark property.
- * <p>
- * Note that even when character encodings are not byte-centric (e.g., 7, 6, 5, or 4 bits)
- * and a character can span a byte boundary, we STILL don't need byteOrder
- * to decode characters. Just bitOrder.
- *
- * This is because bitOrder alone specifies which end of byte N is adjacent to which
- * end of byte N+1.
- */
- def setByteOrder(byteOrder: ByteOrder): Unit
- def byteOrder: ByteOrder
-
/**
* Returns number of bits remaining (if a limit is defined). Nope if not defined.
*/
@@ -125,7 +88,7 @@ trait DataStreamCommon {
* the alignment.
*/
- def align(bitAlignment1b: Int): Boolean
+ def align(bitAlignment1b: Int, finfo: FormatInfo): Boolean
/**
* For assertion checking really. Optimizations should remove the need for most
@@ -145,7 +108,7 @@ trait DataStreamCommon {
* Advances the bit position by nBits. If nBits aren't available this
* returns false. Otherwise it returns true.
*/
- def skip(nBits: Long): Boolean
+ def skip(nBits: Long, finfo: FormatInfo): Boolean
/**
* Debugging flag. If set then performance may be reduced, but
diff --git a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataStreamCommonImplMixin.scala b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataStreamCommonImplMixin.scala
index a97f2bf..285f206 100644
--- a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataStreamCommonImplMixin.scala
+++ b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DataStreamCommonImplMixin.scala
@@ -31,26 +31,31 @@
*/
package edu.illinois.ncsa.daffodil.io
-
-import edu.illinois.ncsa.daffodil.exceptions.Assert
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BinaryFloatRep
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
-import edu.illinois.ncsa.daffodil.util.Maybe
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.UTF16Width
-import java.nio.charset.StandardCharsets
-import java.nio.charset.Charset
-import edu.illinois.ncsa.daffodil.util.MaybeInt
import edu.illinois.ncsa.daffodil.util.MaybeChar
import edu.illinois.ncsa.daffodil.util.Logging
import edu.illinois.ncsa.daffodil.api.DataStreamLimits
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
+import edu.illinois.ncsa.daffodil.util.Maybe
+import edu.illinois.ncsa.daffodil.util.Maybe._
+import edu.illinois.ncsa.daffodil.exceptions.Assert
trait DataStreamCommonState {
- var binaryFloatRep: BinaryFloatRep = BinaryFloatRep.Ieee
- var bitOrder: BitOrder = BitOrder.MostSignificantBitFirst
- var maybeCharWidthInBits: MaybeInt = MaybeInt.Nope
- var encodingMandatoryAlignmentInBits: Int = 8
- var maybeUTF16Width: Maybe[UTF16Width] = Maybe(UTF16Width.Fixed)
+ /**
+ * Keeps track of the bitOrder of the last operation on the
+ * data stream.
+ */
+ private var maybePriorBitOrder_ : Maybe[BitOrder] = Maybe.Nope
+
+ def setPriorBitOrder(pbo: BitOrder) {
+ maybePriorBitOrder_ = One(pbo)
+ }
+
+ def priorBitOrder: BitOrder = {
+ Assert.usage(maybePriorBitOrder_.isDefined)
+ maybePriorBitOrder_.value
+ }
+
var debugging: Boolean = false
var limits_ : DataStreamLimits = BBSLimits
//
@@ -65,25 +70,17 @@ trait DataStreamCommonState {
// aka a surrogate-pair.
//
var maybeTrailingSurrogateForUTF8: MaybeChar = MaybeChar.Nope
- var priorEncoding: Charset = StandardCharsets.UTF_8
var priorBitPos: Long = 0L
def resetUTF8SurrogatePairCapture {
- this.maybeTrailingSurrogateForUTF8 = MaybeChar.Nope
this.priorBitPos = -1
}
def assignFrom(other: DataStreamCommonState): Unit = {
- this.binaryFloatRep = other.binaryFloatRep
- this.bitOrder = other.bitOrder
- this.maybeCharWidthInBits = other.maybeCharWidthInBits
- this.encodingMandatoryAlignmentInBits = other.encodingMandatoryAlignmentInBits
- this.maybeUTF16Width = other.maybeUTF16Width
this.debugging = other.debugging
this.limits_ = other.limits_
- this.maybeTrailingSurrogateForUTF8 = other.maybeTrailingSurrogateForUTF8
- this.priorEncoding = other.priorEncoding
this.priorBitPos = other.priorBitPos
+ this.maybePriorBitOrder_ = other.maybePriorBitOrder_
}
}
@@ -95,18 +92,6 @@ trait DataStreamCommonImplMixin extends DataStreamCommon with Logging {
protected def cst: DataStreamCommonState
- final override def setBinaryFloatRep(binaryFloatRep: BinaryFloatRep): Unit = {
- Assert.invariant(binaryFloatRep == BinaryFloatRep.Ieee)
- cst.binaryFloatRep = binaryFloatRep
- }
-
- final override def setBitOrder(bitOrder: BitOrder): Unit = { cst.bitOrder = bitOrder }
- // final override def setCharWidthInBits(charWidthInBits: Maybe[Int]): Unit = { cst.maybeCharWidthInBits = charWidthInBits }
- // final override def setEncodingMandatoryAlignment(bitAlignment: Int): Unit = { cst.encodingMandatoryAlignmentInBits = bitAlignment }
- final override def setMaybeUTF16Width(maybeUTF16Width: Maybe[UTF16Width]): Unit = { cst.maybeUTF16Width = maybeUTF16Width }
-
- final def isFixedWidthEncoding = cst.maybeCharWidthInBits.isDefined
-
final override def limits: DataStreamLimits = cst.limits_
final override def setLimits(newLimits: DataStreamLimits) {
diff --git a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DirectOrBufferedDataOutputStream.scala b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DirectOrBufferedDataOutputStream.scala
index 365b70b..7a9cfd2 100644
--- a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DirectOrBufferedDataOutputStream.scala
+++ b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/DirectOrBufferedDataOutputStream.scala
@@ -66,6 +66,12 @@ private[io] class ByteArrayOutputStreamWithGetBuf() extends java.io.ByteArrayOut
}
super.reset()
}
+
+ def hexDump = {
+ (0 to (count - 1)).map { i => "%2x".format(buf(i).toInt & 0xFF) }.mkString(".")
+ }
+
+ override def toString = hexDump
}
/**
@@ -95,6 +101,16 @@ final class DirectOrBufferedDataOutputStream private[io] (var splitFrom: DirectO
extends DataOutputStreamImplMixin {
type ThisType = DirectOrBufferedDataOutputStream
+ override def putULong(unsignedLong: ULong, bitLengthFrom1To64: Int, finfo: FormatInfo): Boolean = {
+ val res = putLongChecked(unsignedLong.longValue, bitLengthFrom1To64, finfo)
+ res
+ }
+
+ override def putLong(signedLong: Long, bitLengthFrom1To64: Int, finfo: FormatInfo) = {
+ val res = putLongChecked(signedLong.longValue, bitLengthFrom1To64, finfo)
+ res
+ }
+
/**
* Must be val, as split-from will get reset to null as streams
* are morphed into direct streams.
@@ -235,8 +251,7 @@ final class DirectOrBufferedDataOutputStream private[io] (var splitFrom: DirectO
val absLargerLimit =
math.max(
if (mabl.isDefined) mabl.get else 0L,
- if (maybeAbsBitLimit0b.isDefined) maybeAbsBitLimit0b.get else 0L
- )
+ if (maybeAbsBitLimit0b.isDefined) maybeAbsBitLimit0b.get else 0L)
if (mabl.isDefined || maybeAbsBitLimit0b.isDefined) {
val newRelLimit = absLargerLimit - this.maybeAbsStartingBitPos0b.get
this.setMaybeRelBitLimit0b(MaybeULong(newRelLimit))
@@ -253,7 +268,7 @@ final class DirectOrBufferedDataOutputStream private[io] (var splitFrom: DirectO
Assert.invariant(isDirect)
}
- override def setFinished() {
+ override def setFinished(finfo: FormatInfo) {
Assert.usage(!isFinished)
// if we are direct, and there's a buffer following this one
//
@@ -275,7 +290,7 @@ final class DirectOrBufferedDataOutputStream private[io] (var splitFrom: DirectO
first.setAbsStartingBitPos0b(dabp)
}
- DirectOrBufferedDataOutputStream.deliverBufferContent(directStream, first) // from first, into direct stream's buffers
+ DirectOrBufferedDataOutputStream.deliverBufferContent(directStream, first, finfo) // from first, into direct stream's buffers
// so now the first one is an EMPTY not necessarily a finished buffered DOS
//
first.convertToDirect(directStream) // first is now the direct stream
@@ -302,7 +317,8 @@ final class DirectOrBufferedDataOutputStream private[io] (var splitFrom: DirectO
if (directStream.cst.fragmentLastByteLimit > 0) {
// must not omit the fragment byte on the end.
directStream.getJavaOutputStream().write(directStream.cst.fragmentLastByte)
- directStream.cst.setFragmentLastByte(0, 0) // zero out so we don't end up thinking it is still there.
+ // zero out so we don't end up thinking it is still there
+ directStream.cst.setFragmentLastByte(0, 0)
}
directStream.setDOSState(Uninitialized) // not just finished. We're dead now.
} else {
@@ -393,6 +409,9 @@ final class DirectOrBufferedDataOutputStream private[io] (var splitFrom: DirectO
}
}
+ /**
+ * Always writes out at least 1 bit.
+ */
final override protected def putLong_BE_MSBFirst(signedLong: Long, bitLengthFrom1To64: Int): Boolean = {
// Note: we don't have to check for bit limit. That check was already done.
//
@@ -473,6 +492,7 @@ final class DirectOrBufferedDataOutputStream private[io] (var splitFrom: DirectO
}
final override protected def putLong_LE_MSBFirst(signedLong: Long, bitLengthFrom1To64: Int): Boolean = {
+
// Note: we don't have to check for bit limit. That check was already done.
//
// LE_MSBF is most complicated of all.
@@ -520,9 +540,9 @@ final class DirectOrBufferedDataOutputStream private[io] (var splitFrom: DirectO
val nFragBitsAfter = fragmentLastByteLimit + nBitsOfFragToBeFilled // this can be 8 if we're going to fill all of the frag.
// Now get the bits that will go into the frag, from the least significant (first) byte.
- val newFragBitsMask = 0x80.toByte >> (nBitsOfFragToBeFilled - 1)
+ val newFragBitsMask = (0x80.toByte >> (nBitsOfFragToBeFilled - 1)) & 0xFF
val LSByte = unionByteBuffer.get(0)
- val bitsToGoIntoFragInPosition = ((LSByte & newFragBitsMask) >>> fragmentLastByteLimit).toInt
+ val bitsToGoIntoFragInPosition = (((LSByte & newFragBitsMask) & 0xFF) >>> fragmentLastByteLimit).toInt
val newFragByte = Bits.asUnsignedByte((fragmentLastByte | bitsToGoIntoFragInPosition).toByte)
Assert.invariant(newFragByte <= 255 && newFragByte >= 0)
@@ -574,6 +594,7 @@ final class DirectOrBufferedDataOutputStream private[io] (var splitFrom: DirectO
}
final override protected def putLong_LE_LSBFirst(signedLong: Long, bitLengthFrom1To64: Int): Boolean = {
+
// Note: we don't have to check for bit limit. That check was already done.
//
// Interestingly, LE_LSBF is slightly simpler than BE_MSBF as we don't have to shift bytes to get the
@@ -681,16 +702,14 @@ final class DirectOrBufferedDataOutputStream private[io] (var splitFrom: DirectO
*
* All calls to setFinished should, somewhere, be surrounded by a catch of this.
*/
-class BitOrderChangeException(directDOS: DirectOrBufferedDataOutputStream,
- bufDOS: DirectOrBufferedDataOutputStream) extends Exception with ThinThrowable {
+class BitOrderChangeException(directDOS: DirectOrBufferedDataOutputStream, finfo: FormatInfo) extends Exception with ThinThrowable {
override def getMessage() = {
- "Data output stream %s with bitOrder '%s'%scannot be populated from %s with bitOrder '%s'.".format(
+ "Data output stream %s with bitOrder '%s' which is not on a byte boundary (%s bits past last byte boundary), cannot be populated with bitOrder '%s'.".format(
directDOS,
- directDOS.bitOrder,
- (if (!directDOS.isEndOnByteBoundary) " not on a byte boundary, " else ""),
- bufDOS,
- bufDOS.bitOrder)
+ directDOS.priorBitOrder,
+ directDOS.fragmentLastByteLimit,
+ finfo.bitOrder)
}
}
@@ -703,20 +722,25 @@ object DirectOrBufferedDataOutputStream {
* Delivers the bits of bufDOS into directDOS's output stream. Deals with the possibility that
* the directDOS ends with a fragment byte, or the bufDOS does, or both.
*/
- private def deliverBufferContent(directDOS: DirectOrBufferedDataOutputStream, bufDOS: DirectOrBufferedDataOutputStream) {
+ private def deliverBufferContent(directDOS: DirectOrBufferedDataOutputStream,
+ bufDOS: DirectOrBufferedDataOutputStream,
+ finfo: FormatInfo) {
Assert.invariant(bufDOS.isBuffering)
Assert.invariant(!directDOS.isBuffering)
val ba = bufDOS.bufferingJOS.getBuf
val bufferNBits = bufDOS.relBitPos0b // don't have to subtract a starting offset. It's always zero in buffered case.
- if (bufDOS.cst.bitOrder ne directDOS.cst.bitOrder) {
- if (!directDOS.isEndOnByteBoundary) {
+ val finfoBitOrder = finfo.bitOrder // bit order we are supposed to write with
+ val priorBitOrder = directDOS.cst.priorBitOrder // bit order that the directDOS had at last successful unparse. (prior is set after each unparser)
+ if (finfoBitOrder ne priorBitOrder) {
+ if ((bufferNBits > ULong.Zero) &&
+ !directDOS.isEndOnByteBoundary) {
//
// If the bit order changes, it has to be on a byte boundary
// It's simply not meaningful for it to change otherwise.
//
- throw new BitOrderChangeException(directDOS, bufDOS)
+ throw new BitOrderChangeException(directDOS, finfo)
}
}
@@ -737,22 +761,22 @@ object DirectOrBufferedDataOutputStream {
if (directDOS.isEndOnByteBoundary && bufDOS.isEndOnByteBoundary) {
val nBytes = (bufferNBits / 8).toInt
- val nBytesPut = directDOS.putBytes(ba, 0, nBytes)
+ val nBytesPut = directDOS.putBytes(ba, 0, nBytes, finfo)
Assert.invariant(nBytesPut == nBytes)
} else {
val nFragBits = bufDOS.fragmentLastByteLimit
val byteCount = bufDOS.bufferingJOS.getCount()
- val wholeBytesWritten = directDOS.putBytes(ba, 0, byteCount)
+ val wholeBytesWritten = directDOS.putBytes(ba, 0, byteCount, finfo)
Assert.invariant(byteCount == wholeBytesWritten)
if (nFragBits > 0) {
val origfrag = bufDOS.fragmentLastByte
val fragNum =
- if (bufDOS.bitOrder eq BitOrder.MostSignificantBitFirst)
+ if (finfoBitOrder eq BitOrder.MostSignificantBitFirst)
origfrag >> (8 - nFragBits)
else
origfrag
- Assert.invariant(directDOS.putLong(fragNum, nFragBits)) // FAILS when doing this for suspensions due to assertions about consistent bit/byte order.
+ Assert.invariant(directDOS.putLongUnchecked(fragNum, nFragBits, finfo))
}
//
// bufDOS contents have now been output into directDOS
diff --git a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/FormatInfo.scala b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/FormatInfo.scala
new file mode 100644
index 0000000..4d07a76
--- /dev/null
+++ b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/FormatInfo.scala
@@ -0,0 +1,128 @@
+/*
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package edu.illinois.ncsa.daffodil.io
+
+import java.nio.charset.CharsetDecoder
+import java.nio.charset.CharsetEncoder
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BinaryFloatRep
+import edu.illinois.ncsa.daffodil.util.MaybeInt
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.EncodingErrorPolicy
+import edu.illinois.ncsa.daffodil.util.Maybe
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.UTF16Width
+
+/**
+ * Abstract interface to obtain format properties or values derived from
+ * properties.
+ *
+ * This includes anything the I/O layer needs, which includes properties that
+ * can be runtime-valued expressions, or that depend on such.
+ *
+ * By passing in an object that provides quick access to these, we avoid the
+ * need to have setters/getters that call setters that change state in the I/O layer.
+ */
+trait FormatInfo {
+
+ /**
+ * Returns a charset encoder for this encoding configured for the
+ * `dfdl:encodingErrorPolicy`. This is the same as either the `reportingEncoder`
+ * or the `replacingEncoder`.
+ */
+ def encoder: CharsetEncoder
+
+ /**
+ * Returns a charset decoder for this encoding configured for the
+ * `dfdl:encodingErrorPolicy`. This is the same as either the `reportingDecoder`
+ * or the `replacingDecoder`.
+ */
+ def decoder: CharsetDecoder
+
+ /**
+ * Returns a decoder configured for `dfdl:encodingErrorPolicy="report"`
+ * regardless of the value of that property.
+ */
+ def reportingDecoder: CharsetDecoder
+
+ /**
+ * Returns decoder configured for `dfdl:encodingErrorPolicy="replace"`
+ * regardless of the value of that property.
+ */
+ def replacingDecoder: CharsetDecoder
+
+ /**
+ * Returns true if encoding is fixed width meaning has the
+ * same number of bits in each character representation.
+ */
+ final def isFixedWidthEncoding: Boolean = maybeCharWidthInBits.isDefined
+
+ /**
+ * Returns ByteOrder
+ */
+ def byteOrder: ByteOrder
+
+ /**
+ * Returns ByteOrder as a java.nio.ByteOrder constant.
+ */
+ final def jByteOrder = byteOrder match {
+ case ByteOrder.BigEndian => java.nio.ByteOrder.BIG_ENDIAN
+ case _ => java.nio.ByteOrder.LITTLE_ENDIAN
+ }
+
+ /**
+ * Returns bit order. If text, this is the bit order for the character set
+ * encoding. If binary, this is the bitOrder property value.
+ */
+ def bitOrder: BitOrder
+
+ /**
+ * Returns the fillByte value.
+ *
+ * Note: This has to be obtained from the FormatInfo
+ * because the `dfdl:fillByte` property can be specified as a character,
+ * which depends then on the `dfdl:encoding` which can be computed via
+ * a runtime expression.
+ */
+ def fillByte: Byte
+
+ /**
+ * Returns the BinaryFloatRep
+ */
+ def binaryFloatRep: BinaryFloatRep
+
+ /**
+ * Returns MaybeInt.Nope for variable-width encodings or no encoding defined.
+ * Returns MaybeInt(n) for fixed width encodings.
+ */
+ def maybeCharWidthInBits: MaybeInt
+
+ /**
+ * Returns `Nope` if `dfdl:utf16Width` is not defined.
+ * Returns `One(w: UTF16Width)` if it is defined.
+ */
+ def maybeUTF16Width: Maybe[UTF16Width]
+
+ /**
+ * Returns 8 if the `dfdl:encoding` is given by a runtime-valued expression.
+ * Returns 8 for all ICU-based encodings.
+ * Returns 1 to 7 (inclusive) for encodings 1 to 7 bits wide respectively.
+ */
+ def encodingMandatoryAlignmentInBits: Int
+
+ /**
+ * Provides the `dfdl:encodingErrorPolicy` as an EncodingErrorPolicy enum.
+ */
+ def encodingErrorPolicy: EncodingErrorPolicy
+}
diff --git a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/StringDataInputStreamForUnparse.scala b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/StringDataInputStreamForUnparse.scala
index 35352f7..727daee 100644
--- a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/StringDataInputStreamForUnparse.scala
+++ b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/io/StringDataInputStreamForUnparse.scala
@@ -32,11 +32,8 @@
package edu.illinois.ncsa.daffodil.io
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.EncodingErrorPolicy
import edu.illinois.ncsa.daffodil.util.Misc
import edu.illinois.ncsa.daffodil.exceptions.Assert
-import java.nio.charset.StandardCharsets
import edu.illinois.ncsa.daffodil.util.MaybeULong
/**
@@ -48,28 +45,20 @@ final class StringDataInputStreamForUnparse
extends DataInputStreamImplMixin {
import DataInputStream._
- final protected val cst = new AnyRef with DataStreamCommonState
+ override final protected val cst = new AnyRef with DataStreamCommonState
var str: String = null
var dis: DataInputStream = null
- private val utf8Decoder = StandardCharsets.UTF_8.newDecoder()
-
- // TODO: Performance - we could keep a BBDIS around, and "reset" it by
- // putting it onto a byte buffer that we fill from the string. That way
- // we wouldn't have to allocate a BBDIS unless the string requires a bigger
- // byte buffer.
- //
- def reset(str: String) {
+ def reset(str: String, finfo: FormatInfo) {
this.str = str
- val ba = str.getBytes(StandardCharsets.UTF_8)
+ val ba = str.getBytes(finfo.decoder.charset())
dis = ByteBufferDataInputStream(ba)
- dis.setDecoder(utf8Decoder)
}
private def doNotUse = Assert.usageError("Not to be called on " + Misc.getNameFromClass(this))
- def asIteratorChar: DataInputStream.CharIterator = {
+ override def asIteratorChar: DataInputStream.CharIterator = {
Assert.usage(dis != null, "Must call reset(str) before any other method.")
dis.asIteratorChar
}
@@ -77,34 +66,27 @@ final class StringDataInputStreamForUnparse
override def bitLimit0b = dis.bitLimit0b
override def bitPos0b: Long = dis.bitPos0b
override def discard(mark: DataInputStream.Mark): Unit = dis.discard(mark)
- override def fillCharBuffer(cb: java.nio.CharBuffer) = dis.fillCharBuffer(cb)
+ override def fillCharBuffer(cb: java.nio.CharBuffer, finfo: FormatInfo) = dis.fillCharBuffer(cb, finfo)
override def futureData(nBytesRequested: Int): java.nio.ByteBuffer = doNotUse
- override def getBinaryDouble(): Double = doNotUse
- override def getBinaryFloat(): Float = doNotUse
- override def getSignedBigInt(bitLengthFrom1: Int): BigInt = doNotUse
- override def getSignedLong(bitLengthFrom1To64: Int): Long = doNotUse
- override def getUnsignedBigInt(bitLengthFrom1: Int): BigInt = doNotUse
- override def getUnsignedLong(bitLengthFrom1To64: Int): passera.unsigned.ULong = doNotUse
- override def getByteArray(bitLengthFrom1: Int): Array[Byte] = doNotUse
- override def lookingAt(matcher: java.util.regex.Matcher, initialRegexMatchLimitInChars: Long): Boolean =
- dis.lookingAt(matcher, initialRegexMatchLimitInChars)
+ override def getBinaryDouble(finfo: FormatInfo): Double = doNotUse
+ override def getBinaryFloat(finfo: FormatInfo): Float = doNotUse
+ override def getSignedBigInt(bitLengthFrom1: Int, finfo: FormatInfo): BigInt = doNotUse
+ override def getSignedLong(bitLengthFrom1To64: Int, finfo: FormatInfo): Long = doNotUse
+ override def getUnsignedBigInt(bitLengthFrom1: Int, finfo: FormatInfo): BigInt = doNotUse
+ override def getUnsignedLong(bitLengthFrom1To64: Int, finfo: FormatInfo): passera.unsigned.ULong = doNotUse
+ override def getByteArray(bitLengthFrom1: Int, finfo: FormatInfo): Array[Byte] = doNotUse
+ override def lookingAt(matcher: java.util.regex.Matcher, finfo: FormatInfo, initialRegexMatchLimitInChars: Long): Boolean =
+ dis.lookingAt(matcher, finfo, initialRegexMatchLimitInChars)
override def mark(requestorID: String): DataInputStream.Mark = dis.mark(requestorID)
override def markPos = dis.markPos
override def pastData(nBytesRequested: Int): java.nio.ByteBuffer = doNotUse
override def reset(mark: DataInputStream.Mark): Unit = dis.reset(mark)
override def resetPos(m: MarkPos) = dis.resetPos(m)
override def setBitLimit0b(bitLimit0b: MaybeULong): Boolean = doNotUse
- override def setByteOrder(byteOrder: ByteOrder): Unit = doNotUse
- override def byteOrder: ByteOrder = doNotUse
- // override def setCharWidthInBits(charWidthInBits: MaybeInt): Unit = doNotUse
override def setDebugging(setting: Boolean): Unit = doNotUse
- override def getDecoder = doNotUse
- override def setDecoder(decoder: java.nio.charset.CharsetDecoder): Unit = doNotUse
- override def setEncodingErrorPolicy(eep: EncodingErrorPolicy): Unit = doNotUse
- // override def setEncodingMandatoryAlignment(bitAlignment: Int): Unit = doNotUse
override def isDefinedForLength(length: Long): Boolean = doNotUse
- override def skip(nBits: Long): Boolean = doNotUse
- override def skipChars(nChars: Long): Boolean = getString(nChars).isDefined
+ override def skip(nBits: Long, finfo: FormatInfo): Boolean = doNotUse
+ override def skipChars(nChars: Long, finfo: FormatInfo): Boolean = getString(nChars, finfo).isDefined
override def resetBitLimit0b(savedBitLimit0b: MaybeULong): Unit = doNotUse
override def validateFinalStreamState {} // does nothing
}
diff --git a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/processors/charset/CharsetUtils.scala b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/processors/charset/CharsetUtils.scala
index ff6745e..8016940 100644
--- a/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/processors/charset/CharsetUtils.scala
+++ b/daffodil-io/src/main/scala/edu/illinois/ncsa/daffodil/processors/charset/CharsetUtils.scala
@@ -138,6 +138,16 @@ object CharsetUtils {
}
+sealed abstract class CoderInfo(val encodingMandatoryAlignmentInBits: Int, val maybeCharWidthInBits: MaybeInt)
+
+case class DecoderInfo(coder: CharsetDecoder, replacingCoder: CharsetDecoder, reportingCoder: CharsetDecoder,
+ encodingMandatoryAlignmentInBitsArg: Int, maybeCharWidthInBitsArg: MaybeInt)
+ extends CoderInfo(encodingMandatoryAlignmentInBitsArg, maybeCharWidthInBitsArg)
+
+case class EncoderInfo(coder: CharsetEncoder, replacingCoder: CharsetEncoder, reportingCoder: CharsetEncoder,
+ encodingMandatoryAlignmentInBitsArg: Int, maybeCharWidthInBitsArg: MaybeInt)
+ extends CoderInfo(encodingMandatoryAlignmentInBitsArg, maybeCharWidthInBitsArg)
+
trait EncoderDecoderMixin
extends LocalBufferMixin {
/**
@@ -150,27 +160,69 @@ trait EncoderDecoderMixin
* might be allocated objects. Use of java hash maps insures this does not allocate
* except when adding a new not-seen-before encoder or decoder.
*/
- private lazy val decoderCache = new java.util.HashMap[Charset, CharsetDecoder]
- private lazy val encoderCache = new java.util.HashMap[Charset, CharsetEncoder]
- def getDecoder(charset: Charset): CharsetDecoder = {
+ private lazy val decoderCache = new java.util.HashMap[Charset, DecoderInfo]
+ private lazy val encoderCache = new java.util.HashMap[Charset, EncoderInfo]
+
+ private def derivations(charset: Charset) = {
+ val tuple = charset match {
+ case nbsc: NonByteSizeCharset => {
+ val encodingMandatoryAlignmentInBits = 1
+ val maybeCharWidthInBits = MaybeInt(nbsc.bitWidthOfACodeUnit)
+ (encodingMandatoryAlignmentInBits, maybeCharWidthInBits)
+ }
+ case _ => {
+ val encodingMandatoryAlignmentInBits = 8
+ val encoder = charset.newEncoder()
+ val maxBytes = encoder.maxBytesPerChar()
+ if (maxBytes == encoder.averageBytesPerChar()) {
+ val maybeCharWidthInBits = MaybeInt((maxBytes * 8).toInt)
+ (encodingMandatoryAlignmentInBits, maybeCharWidthInBits)
+ } else {
+ val maybeCharWidthInBits = MaybeInt.Nope
+ (encodingMandatoryAlignmentInBits, maybeCharWidthInBits)
+ }
+ }
+ }
+ tuple
+ }
+
+ def getDecoder(charset: Charset) = getDecoderInfo(charset).coder
+ def getDecoderInfo(charset: Charset) = {
// threadCheck()
- var decoder = decoderCache.get(charset)
- if (decoder eq null) {
- decoder = charset.newDecoder()
- decoderCache.put(charset, decoder)
+ var entry = decoderCache.get(charset)
+ if (entry eq null) {
+ val coder = charset.newDecoder()
+ val replacingCoder = charset.newDecoder()
+ val reportingCoder = charset.newDecoder()
+ replacingCoder.onMalformedInput(CodingErrorAction.REPLACE)
+ replacingCoder.onUnmappableCharacter(CodingErrorAction.REPLACE)
+ reportingCoder.onMalformedInput(CodingErrorAction.REPORT)
+ reportingCoder.onUnmappableCharacter(CodingErrorAction.REPORT)
+ val (encodingMandatoryAlignmentInBits, maybeCharWidthInBits) = derivations(charset)
+ entry = DecoderInfo(coder, replacingCoder, reportingCoder, encodingMandatoryAlignmentInBits, maybeCharWidthInBits)
+ decoderCache.put(charset, entry)
}
- decoder
+ entry
}
- def getEncoder(charset: Charset): CharsetEncoder = {
+ def getEncoder(charset: Charset) = getEncoderInfo(charset).coder
+ def getEncoderInfo(charset: Charset) = {
// threadCheck()
- var encoder = encoderCache.get(charset)
- if (encoder eq null) {
- encoder = charset.newEncoder()
- encoderCache.put(charset, encoder)
+ var entry = encoderCache.get(charset)
+ if (entry eq null) {
+ val coder = charset.newEncoder()
+ val replacingCoder = charset.newEncoder()
+ val reportingCoder = charset.newEncoder()
+ replacingCoder.onMalformedInput(CodingErrorAction.REPLACE)
+ replacingCoder.onUnmappableCharacter(CodingErrorAction.REPLACE)
+ reportingCoder.onMalformedInput(CodingErrorAction.REPORT)
+ reportingCoder.onUnmappableCharacter(CodingErrorAction.REPORT)
+ val (encodingMandatoryAlignmentInBits, maybeCharWidthInBits) = derivations(charset)
+ entry = EncoderInfo(coder, replacingCoder, reportingCoder, encodingMandatoryAlignmentInBits, maybeCharWidthInBits)
+ encoderCache.put(charset, entry)
}
- encoder
+ entry
}
/**
diff --git a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/FormatInfoForUnitTest.scala b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/FormatInfoForUnitTest.scala
new file mode 100644
index 0000000..fee4a94
--- /dev/null
+++ b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/FormatInfoForUnitTest.scala
@@ -0,0 +1,82 @@
+package edu.illinois.ncsa.daffodil.io
+
+import java.nio.charset.Charset
+import java.nio.charset.CodingErrorAction
+import java.nio.charset.CharsetEncoder
+import java.nio.charset.CharsetDecoder
+import edu.illinois.ncsa.daffodil.util.Maybe
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BinaryFloatRep
+import edu.illinois.ncsa.daffodil.util.MaybeInt
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.UTF16Width
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.EncodingErrorPolicy
+
+object FormatInfoForUnitTest {
+ def apply() = {
+ val obj = new FormatInfoForUnitTest()
+ obj.init()
+ obj
+ }
+}
+
+class FormatInfoForUnitTest private ()
+ extends FormatInfo {
+ var priorEncoding: Charset = java.nio.charset.StandardCharsets.UTF_8
+
+ var encoder: CharsetEncoder = priorEncoding.newEncoder()
+ var decoder: CharsetDecoder = priorEncoding.newDecoder()
+ var reportingDecoder: CharsetDecoder = _
+
+ var replacingDecoder: CharsetDecoder = _
+
+ var byteOrder: ByteOrder = ByteOrder.BigEndian
+ var bitOrder: BitOrder = BitOrder.MostSignificantBitFirst
+ var fillByte: Byte = 0x00.toByte
+ var binaryFloatRep: BinaryFloatRep = BinaryFloatRep.Ieee
+ var maybeCharWidthInBits: MaybeInt = MaybeInt.Nope
+ var maybeUTF16Width: Maybe[UTF16Width] = Maybe.Nope
+ var encodingMandatoryAlignmentInBits: Int = 8
+ var encodingErrorPolicy: EncodingErrorPolicy = EncodingErrorPolicy.Replace
+
+ def reset(cs: Charset): Unit = {
+ priorEncoding = cs
+ init()
+ }
+
+ def init(): Unit = {
+ encoder = priorEncoding.newEncoder()
+ encoder.onMalformedInput(CodingErrorAction.REPLACE)
+ encoder.onUnmappableCharacter(CodingErrorAction.REPLACE)
+ decoder = priorEncoding.newDecoder()
+ decoder.onMalformedInput(CodingErrorAction.REPLACE)
+ decoder.onUnmappableCharacter(CodingErrorAction.REPLACE)
+ reportingDecoder = {
+ val d = priorEncoding.newDecoder()
+ d.onMalformedInput(CodingErrorAction.REPORT)
+ d.onUnmappableCharacter(CodingErrorAction.REPORT)
+ d
+ }
+ replacingDecoder = {
+ val d = priorEncoding.newDecoder()
+ d.onMalformedInput(CodingErrorAction.REPLACE)
+ d.onUnmappableCharacter(CodingErrorAction.REPLACE)
+ d
+ }
+ priorEncoding match {
+ case decoderWithBits: NonByteSizeCharset => {
+ encodingMandatoryAlignmentInBits = 1
+ maybeCharWidthInBits = MaybeInt(decoderWithBits.bitWidthOfACodeUnit)
+ }
+ case _ => {
+ encodingMandatoryAlignmentInBits = 8
+ val maxBytes = encoder.maxBytesPerChar()
+ if (maxBytes == encoder.averageBytesPerChar()) {
+ maybeCharWidthInBits = MaybeInt((maxBytes * 8).toInt)
+ } else {
+ maybeCharWidthInBits = MaybeInt.Nope
+ }
+ }
+ }
+ }
+}
diff --git a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream.scala b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream.scala
index 69c8931..aec214a 100644
--- a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream.scala
+++ b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream.scala
@@ -32,21 +32,29 @@
package edu.illinois.ncsa.daffodil.io
-import org.junit.Test
-import org.junit.Assert._
import java.nio.ByteBuffer
-import passera.unsigned.ULong
import java.nio.CharBuffer
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.EncodingErrorPolicy
-import edu.illinois.ncsa.daffodil.Implicits._
import java.nio.charset.MalformedInputException
-import java.util.regex.Pattern
import java.nio.charset.StandardCharsets
-import edu.illinois.ncsa.daffodil.util.Misc
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
+import java.util.regex.Pattern
+
+import scala.BigInt
+import scala.math.BigInt.int2bigInt
+
+import org.junit.Assert.assertEquals
+import org.junit.Assert.assertFalse
+import org.junit.Assert.assertTrue
+import org.junit.Test
+
+import edu.illinois.ncsa.daffodil.Implicits.intercept
+import edu.illinois.ncsa.daffodil.equality.TypeEqual
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
-import edu.illinois.ncsa.daffodil.equality._
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.EncodingErrorPolicy
import edu.illinois.ncsa.daffodil.util.MaybeULong
+import edu.illinois.ncsa.daffodil.util.Misc
+import passera.unsigned.ULong
+import java.nio.charset.CodingErrorAction
class TestByteBufferDataInputStream {
val tenDigits = "1234567890"
@@ -54,6 +62,8 @@ class TestByteBufferDataInputStream {
val twentyDigits = tenDigits * 2
val twenty = twentyDigits.getBytes("utf-8")
+ val finfo = FormatInfoForUnitTest()
+
implicit class ExactTypeEqual[L](left: L) {
def assertEqualsTyped[R, D >: L <: R, D2 >: R <: L](right: R): Unit = assertEquals(left, right)
}
@@ -82,7 +92,7 @@ class TestByteBufferDataInputStream {
@Test def testBitAndBytePos1 {
val dis = ByteBufferDataInputStream(ten)
- val arr = dis.getByteArray(8)
+ val arr = dis.getByteArray(8, finfo)
assertEqualsTyped[Long](1, arr.size)
assertEqualsTyped[Long](0x31.toByte, arr(0))
assertEqualsTyped[Long](8, dis.bitPos0b)
@@ -94,7 +104,7 @@ class TestByteBufferDataInputStream {
@Test def testBitAndBytePos10 {
val dis = ByteBufferDataInputStream(ten)
- val arr = dis.getByteArray(80)
+ val arr = dis.getByteArray(80, finfo)
assertEqualsTyped[Long](10, arr.size)
assertEqualsTyped[Long](0x30.toByte, arr(9))
assertEqualsTyped[Long](80, dis.bitPos0b)
@@ -107,7 +117,7 @@ class TestByteBufferDataInputStream {
@Test def testBitAndBytePosNotEnoughData1 {
val dis = ByteBufferDataInputStream(ten)
intercept[DataInputStream.NotEnoughDataException] {
- dis.getByteArray(81)
+ dis.getByteArray(81, finfo)
}
assertEqualsTyped[Long](0, dis.bitPos0b)
assertEqualsTyped[Long](80, dis.bitLimit0b.get)
@@ -115,7 +125,7 @@ class TestByteBufferDataInputStream {
@Test def testBitAndBytePosMoreThanEnoughData1 {
val dis = ByteBufferDataInputStream(twenty)
- val arr = dis.getByteArray(80)
+ val arr = dis.getByteArray(80, finfo)
assertEqualsTyped[Long](10, arr.size)
assertEqualsTyped[Long](0x30.toByte, arr(9))
assertEqualsTyped[Long](80, dis.bitPos0b)
@@ -127,7 +137,7 @@ class TestByteBufferDataInputStream {
@Test def testBitLengthLimit1 {
val dis = ByteBufferDataInputStream(twenty)
val isLimitOk = dis.withBitLengthLimit(80) {
- val arr = dis.getByteArray(80)
+ val arr = dis.getByteArray(80, finfo)
assertEqualsTyped[Long](80, dis.bitLimit0b.get)
assertEqualsTyped[Long](80, dis.bitPos0b)
assertEqualsTyped[Long](10, arr.size)
@@ -142,7 +152,7 @@ class TestByteBufferDataInputStream {
@Test def testBinaryDouble1 {
val dis = ByteBufferDataInputStream("123".getBytes("utf-8"))
intercept[DataInputStream.NotEnoughDataException] {
- dis.getBinaryDouble()
+ dis.getBinaryDouble(finfo)
}
assertEqualsTyped[Long](0, dis.bitPos0b)
}
@@ -150,7 +160,7 @@ class TestByteBufferDataInputStream {
@Test def testBinaryDouble2 {
val dis = ByteBufferDataInputStream(twenty)
val expected = ByteBuffer.wrap(twenty).asDoubleBuffer().get()
- val d = dis.getBinaryDouble()
+ val d = dis.getBinaryDouble(finfo)
assertEqualsTyped(expected, d, 0.0)
}
@@ -158,23 +168,23 @@ class TestByteBufferDataInputStream {
val dis = ByteBufferDataInputStream(twenty)
dis.setBitLimit1b(MaybeULong(63)) // not enough bits
intercept[DataInputStream.NotEnoughDataException] {
- dis.getBinaryDouble()
+ dis.getBinaryDouble(finfo)
}
}
@Test def testBinaryDouble4 {
val dis = ByteBufferDataInputStream(twenty)
- dis.getByteArray(8)
+ dis.getByteArray(8, finfo)
dis.setBitLimit1b(MaybeULong(71)) // not enough bits
intercept[DataInputStream.NotEnoughDataException] {
- dis.getBinaryDouble()
+ dis.getBinaryDouble(finfo)
}
}
@Test def testBinaryFloat1 {
val dis = ByteBufferDataInputStream("123".getBytes("utf-8"))
intercept[DataInputStream.NotEnoughDataException] {
- dis.getBinaryFloat()
+ dis.getBinaryFloat(finfo)
}
assertEqualsTyped[Long](0, dis.bitPos0b)
}
@@ -182,7 +192,7 @@ class TestByteBufferDataInputStream {
@Test def testBinaryFloat2 {
val dis = ByteBufferDataInputStream(twenty)
val expected = ByteBuffer.wrap(twenty).asFloatBuffer().get()
- val d = dis.getBinaryFloat()
+ val d = dis.getBinaryFloat(finfo)
assertEqualsTyped(expected, d, 0.0)
}
@@ -191,17 +201,17 @@ class TestByteBufferDataInputStream {
dis.setBitLimit1b(MaybeULong(31)) // not enough bits
assertFalse(dis.isDefinedForLength(32))
intercept[DataInputStream.NotEnoughDataException] {
- dis.getBinaryFloat()
+ dis.getBinaryFloat(finfo)
}
}
@Test def testBinaryFloat4 {
val dis = ByteBufferDataInputStream(twenty)
- dis.getByteArray(8)
+ dis.getByteArray(8, finfo)
dis.setBitLimit1b(MaybeULong(39)) // not enough bits
assertFalse(dis.isDefinedForLength(32))
intercept[DataInputStream.NotEnoughDataException] {
- dis.getBinaryFloat()
+ dis.getBinaryFloat(finfo)
}
}
@@ -209,7 +219,7 @@ class TestByteBufferDataInputStream {
val dis = ByteBufferDataInputStream(twenty)
dis.setBitLimit1b(MaybeULong(1)) // 1b so 1 means no data
intercept[DataInputStream.NotEnoughDataException] {
- dis.getSignedLong(1)
+ dis.getSignedLong(1, finfo)
}
}
@@ -217,7 +227,7 @@ class TestByteBufferDataInputStream {
val dis = ByteBufferDataInputStream(twenty)
dis.setBitLimit1b(MaybeULong(63))
intercept[DataInputStream.NotEnoughDataException] {
- dis.getSignedLong(64)
+ dis.getSignedLong(64, finfo)
}
}
@@ -225,10 +235,10 @@ class TestByteBufferDataInputStream {
val dis = ByteBufferDataInputStream(twenty)
// buffer has 0x3132 in first 16 bits
// binary that is 00110001 00110010
- var sl = dis.getSignedLong(1)
+ var sl = dis.getSignedLong(1, finfo)
assertEqualsTyped[Long](0L, sl)
assertEqualsTyped[Long](1, dis.bitPos0b)
- sl = dis.getSignedLong(9)
+ sl = dis.getSignedLong(9, finfo)
// those bits are 0110001 00 which is 0x0C4 and sign bit is 0 (positive)
assertEqualsTyped[Long](0x0C4L, sl)
assertEqualsTyped[Long](10, dis.bitPos0b)
@@ -236,29 +246,29 @@ class TestByteBufferDataInputStream {
@Test def testSignedLong4 {
val dis = ByteBufferDataInputStream(twenty)
- var ml = dis.getSignedLong(1)
+ var ml = dis.getSignedLong(1, finfo)
assertEqualsTyped[Long](1, dis.bitPos0b)
- ml = dis.getSignedLong(64)
+ ml = dis.getSignedLong(64, finfo)
assertEqualsTyped[Long](65, dis.bitPos0b)
assertEqualsTyped[Long](0x3132333435363738L << 1, ml)
}
@Test def testSignedLong5 {
val dis = ByteBufferDataInputStream(List(0xC1, 0xC2, 0xC3, 0xC4, 0xC5, 0xC6, 0xC7, 0xC8, 0xC9, 0xC0).map { _.toByte }.toArray)
- var ml = dis.getSignedLong(1)
+ var ml = dis.getSignedLong(1, finfo)
assertEqualsTyped[Long](1, dis.bitPos0b)
assertEqualsTyped[Long](1, ml)
- ml = dis.getSignedLong(64)
+ ml = dis.getSignedLong(64, finfo)
assertEqualsTyped[Long](65, dis.bitPos0b)
assertEqualsTyped[Long]((0xC1C2C3C4C5C6C7C8L << 1) + (0xC9 >>> 7), ml)
}
@Test def testSignedLong6 {
val dis = ByteBufferDataInputStream(List(0xC1, 0xC2, 0xC3, 0xC4, 0xC5, 0xC6, 0xC7, 0xC8, 0xC9, 0xC0).map { _.toByte }.toArray)
- var ml = dis.getSignedLong(1)
+ var ml = dis.getSignedLong(1, finfo)
assertEqualsTyped[Long](1, dis.bitPos0b)
assertEqualsTyped[Long](1, ml)
- ml = dis.getSignedLong(32)
+ ml = dis.getSignedLong(32, finfo)
assertEqualsTyped[Long](33, dis.bitPos0b)
val expected = (((0xC1C2C3C4C5L >> 7) & 0xFFFFFFFFL) << 32) >> 32 // sign extend
assertEqualsTyped[Long](expected, ml)
@@ -266,10 +276,10 @@ class TestByteBufferDataInputStream {
@Test def testSignedLong7 {
val dis = ByteBufferDataInputStream(List(0xC1, 0xC2, 0xC3, 0xC4, 0xC5).map { _.toByte }.toArray)
- var ml = dis.getSignedLong(2)
+ var ml = dis.getSignedLong(2, finfo)
assertEqualsTyped[Long](2, dis.bitPos0b)
assertEqualsTyped[Long](-1, ml)
- ml = dis.getSignedLong(32)
+ ml = dis.getSignedLong(32, finfo)
assertEqualsTyped[Long](34, dis.bitPos0b)
val expected = (0xC1C2C3C4C5L >> 6) & 0xFFFFFFFFL // will be positive, no sign extend
assertEqualsTyped[Long](expected, ml)
@@ -277,7 +287,7 @@ class TestByteBufferDataInputStream {
@Test def testUnsignedLong1 {
val dis = ByteBufferDataInputStream(List(0xC1, 0xC2, 0xC3, 0xC4, 0xC5).map { _.toByte }.toArray)
- val ml = dis.getUnsignedLong(32)
+ val ml = dis.getUnsignedLong(32, finfo)
assertEqualsTyped[Long](32, dis.bitPos0b)
val expected = ULong(0xC1C2C3C4L)
assertEqualsTyped[ULong](expected, ml)
@@ -285,9 +295,9 @@ class TestByteBufferDataInputStream {
@Test def testUnsignedLong2 {
val dis = ByteBufferDataInputStream(List(0xA5, 0xA5, 0xA5, 0xA5, 0xA5).map { _.toByte }.toArray)
- dis.getSignedLong(1)
+ dis.getSignedLong(1, finfo)
assertEqualsTyped[Long](1, dis.bitPos0b)
- val ml = dis.getUnsignedLong(32)
+ val ml = dis.getUnsignedLong(32, finfo)
assertEqualsTyped[Long](33, dis.bitPos0b)
val expected = ULong(0x4b4b4b4bL)
assertEqualsTyped[ULong](expected, ml)
@@ -295,7 +305,7 @@ class TestByteBufferDataInputStream {
@Test def testUnsignedLong3 {
val dis = ByteBufferDataInputStream(List(0xFF).map { _.toByte }.toArray)
- val ml = dis.getUnsignedLong(1)
+ val ml = dis.getUnsignedLong(1, finfo)
assertEqualsTyped[Long](1, dis.bitPos0b)
val expected = ULong(1)
assertEqualsTyped[ULong](expected, ml)
@@ -303,7 +313,7 @@ class TestByteBufferDataInputStream {
@Test def testSignedBigInt1 {
val dis = ByteBufferDataInputStream(List(0xFF).map { _.toByte }.toArray)
- val ml = dis.getSignedBigInt(1)
+ val ml = dis.getSignedBigInt(1, finfo)
assertEqualsTyped[Long](1, dis.bitPos0b)
val expected = BigInt(1)
assertTrue(expected =:= ml)
@@ -311,7 +321,7 @@ class TestByteBufferDataInputStream {
@Test def testSignedBigInt2 {
val dis = ByteBufferDataInputStream(List(0xC1, 0xC2, 0xC3, 0xC4, 0xC5).map { _.toByte }.toArray)
- val ml = dis.getSignedBigInt(40)
+ val ml = dis.getSignedBigInt(40, finfo)
assertEqualsTyped[Long](40, dis.bitPos0b)
val expected = BigInt(0xFFFFFFC1C2C3C4C5L)
assertEqualsTyped[BigInt](expected, ml)
@@ -319,7 +329,7 @@ class TestByteBufferDataInputStream {
@Test def testUnsignedBigInt1 {
val dis = ByteBufferDataInputStream(List(0xFF).map { _.toByte }.toArray)
- val ml = dis.getUnsignedBigInt(2)
+ val ml = dis.getUnsignedBigInt(2, finfo)
assertEqualsTyped(2, dis.bitPos0b)
val expected = BigInt(3)
assertEqualsTyped[BigInt](expected, ml)
@@ -327,7 +337,7 @@ class TestByteBufferDataInputStream {
@Test def testUnsignedBigInt2 {
val dis = ByteBufferDataInputStream(List(0xC1, 0xC2, 0xC3, 0xC4, 0xC5).map { _.toByte }.toArray)
- val ml = dis.getUnsignedBigInt(40)
+ val ml = dis.getUnsignedBigInt(40, finfo)
assertEqualsTyped(40, dis.bitPos0b)
val expected = BigInt(0xC1C2C3C4C5L)
assertEqualsTyped[BigInt](expected, ml)
@@ -337,9 +347,10 @@ class TestByteBufferDataInputStream {
val dat = "7766554433221100ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbbaa9 [...]
val dats = dat.sliding(2, 2).toList.flatMap { Misc.hex2Bytes(_) }.toArray
val dis = ByteBufferDataInputStream(dats)
- dis.setByteOrder(ByteOrder.LittleEndian)
- dis.setBitOrder(BitOrder.LeastSignificantBitFirst)
- val mbi = dis.getUnsignedBigInt(dat.length * 4)
+ val finfo = FormatInfoForUnitTest()
+ finfo.byteOrder = ByteOrder.LittleEndian
+ finfo.bitOrder = BitOrder.LeastSignificantBitFirst
+ val mbi = dis.getUnsignedBigInt(dat.length * 4, finfo)
val expected = BigInt(dat.reverse, 16)
val expectedHex = "%x".format(expected)
val actualHex = "%x".format(mbi)
@@ -360,10 +371,11 @@ class TestByteBufferDataInputStream {
assertEqualsTyped[Long](361, dat.length)
assertEqualsTyped[Long](0xE0.toByte, dat.head)
val dis = ByteBufferDataInputStream(dat, 5)
- dis.setByteOrder(ByteOrder.LittleEndian)
- dis.setBitOrder(BitOrder.LeastSignificantBitFirst)
+ val finfo = FormatInfoForUnitTest()
+ finfo.byteOrder = ByteOrder.LittleEndian
+ finfo.bitOrder = BitOrder.LeastSignificantBitFirst
val nBits = (expectedHex.length * 4)
- val mbi = dis.getUnsignedBigInt(nBits)
+ val mbi = dis.getUnsignedBigInt(nBits, finfo)
val actual = mbi
val actualHex = "%x".format(actual)
val expectedHexNoLeadingZeros = "%x".format(expected)
@@ -374,19 +386,19 @@ class TestByteBufferDataInputStream {
val dis = ByteBufferDataInputStream(List(0xC1, 0xC2, 0xC3, 0xC4, 0xC5).map { _.toByte }.toArray)
assertTrue(dis.isAligned(1))
assertTrue(dis.isAligned(43))
- dis.getSignedLong(1)
- dis.align(4)
+ dis.getSignedLong(1, finfo)
+ dis.align(4, finfo)
assertEqualsTyped[Long](4, dis.bitPos0b)
assertTrue(dis.isAligned(2))
assertFalse(dis.isAligned(8))
- dis.skip(3)
+ dis.skip(3, finfo)
assertEqualsTyped[Long](7, dis.bitPos0b)
}
@Test def testFillCharBuffer1 {
val dis = ByteBufferDataInputStream("1".getBytes())
val cb = CharBuffer.allocate(1)
- val ml = dis.fillCharBuffer(cb)
+ val ml = dis.fillCharBuffer(cb, finfo)
assertTrue(ml.isDefined)
assertEqualsTyped[Long](1, ml.get.toLong)
assertEqualsTyped[Long](8, dis.bitPos0b)
@@ -396,7 +408,7 @@ class TestByteBufferDataInputStream {
@Test def testFillCharBuffer2 {
val dis = ByteBufferDataInputStream("年月日".getBytes("utf-8"))
val cb = CharBuffer.allocate(3)
- val ml = dis.fillCharBuffer(cb)
+ val ml = dis.fillCharBuffer(cb, finfo)
assertTrue(ml.isDefined)
assertEqualsTyped[Long](3, ml.get.toLong)
assertEqualsTyped[Char]('年', cb.get(0))
@@ -409,7 +421,7 @@ class TestByteBufferDataInputStream {
val dis = ByteBufferDataInputStream("年月日".getBytes("utf-8"))
dis.setBitLimit0b(MaybeULong(8 * 6))
val cb = CharBuffer.allocate(3)
- val ml = dis.fillCharBuffer(cb)
+ val ml = dis.fillCharBuffer(cb, finfo)
assertTrue(ml.isDefined)
assertEqualsTyped[Long](2, ml.get.toLong)
assertEqualsTyped[Char]('年', cb.get(0))
@@ -423,9 +435,10 @@ class TestByteBufferDataInputStream {
val data = List(0xFF.toByte).toArray ++ "年月日".getBytes("utf-8")
assertEqualsTyped(10, data.length)
val dis = ByteBufferDataInputStream(data)
- dis.setEncodingErrorPolicy(EncodingErrorPolicy.Replace)
+ val finfo = FormatInfoForUnitTest()
+ finfo.decoder = finfo.replacingDecoder
val cb = CharBuffer.allocate(20)
- val ml = dis.fillCharBuffer(cb)
+ val ml = dis.fillCharBuffer(cb, finfo)
assertTrue(ml.isDefined)
assertEqualsTyped[Long](4, ml.get.toLong)
assertEqualsTyped[Char](unicodeReplacementCharacter, cb.get(0))
@@ -439,16 +452,17 @@ class TestByteBufferDataInputStream {
val badByte = List(0xFF.toByte).toArray
val data = "abc".getBytes("utf-8") ++ badByte ++ "123".getBytes("utf-8") ++ badByte ++ badByte ++ "drm".getBytes("utf-8")
val dis = ByteBufferDataInputStream(data)
- dis.setEncodingErrorPolicy(EncodingErrorPolicy.Replace)
+ val finfo = FormatInfoForUnitTest()
+ finfo.decoder = finfo.replacingDecoder
val cb = CharBuffer.allocate(20)
- var ml = dis.fillCharBuffer(cb)
+ var ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEqualsTyped[Long](3, ml.get.toLong)
var str: String = Misc.csToString(cb)
assertEqualsTyped[String]("abc", str)
cb.clear
- ml = dis.fillCharBuffer(cb)
+ ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEqualsTyped(9, ml.get)
@@ -461,9 +475,12 @@ class TestByteBufferDataInputStream {
val badByte = List(0xFF.toByte).toArray
val data = "abc".getBytes("utf-8") ++ badByte ++ "123".getBytes("utf-8") ++ badByte ++ badByte ++ "drm".getBytes("utf-8")
val dis = ByteBufferDataInputStream(data)
- dis.setEncodingErrorPolicy(EncodingErrorPolicy.Error)
+ val finfo = FormatInfoForUnitTest()
+ finfo.encodingErrorPolicy = EncodingErrorPolicy.Error
+ finfo.decoder.onMalformedInput(CodingErrorAction.REPORT)
+ finfo.decoder.onUnmappableCharacter(CodingErrorAction.REPORT)
val cb = CharBuffer.allocate(20)
- var ml = dis.fillCharBuffer(cb)
+ var ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEqualsTyped[Long](3, ml.get.toLong)
@@ -471,7 +488,7 @@ class TestByteBufferDataInputStream {
assertEqualsTyped("abc", str)
cb.clear
val e = intercept[MalformedInputException] {
- ml = dis.fillCharBuffer(cb)
+ ml = dis.fillCharBuffer(cb, finfo)
}
assertEqualsTyped[Long](1, e.getInputLength())
}
@@ -479,6 +496,7 @@ class TestByteBufferDataInputStream {
@Test def testCharIterator1 {
val dis = ByteBufferDataInputStream("年月日".getBytes("utf-8"))
val iter = dis.asIteratorChar
+ iter.setFormatInfo(finfo)
dis.setBitLimit0b(MaybeULong(8 * 6))
assertTrue(iter.hasNext)
assertEqualsTyped[Long](0, dis.bitPos0b)
@@ -495,8 +513,13 @@ class TestByteBufferDataInputStream {
val badByte = List(0xFF.toByte).toArray
val data = "abc".getBytes("utf-8") ++ badByte ++ "123".getBytes("utf-8") ++ badByte ++ badByte ++ "drm".getBytes("utf-8")
val dis = ByteBufferDataInputStream(data)
- dis.setEncodingErrorPolicy(EncodingErrorPolicy.Error)
+ val finfo = FormatInfoForUnitTest()
+ finfo.encodingErrorPolicy = EncodingErrorPolicy.Error
+ finfo.decoder.onMalformedInput(CodingErrorAction.REPORT)
+ finfo.decoder.onUnmappableCharacter(CodingErrorAction.REPORT)
+ finfo.encodingErrorPolicy = EncodingErrorPolicy.Error
val iter = dis.asIteratorChar
+ iter.setFormatInfo(finfo)
dis.setBitLimit0b(MaybeULong(8 * 6))
assertTrue(iter.hasNext)
assertEqualsTyped[Long](0, dis.bitPos0b)
@@ -517,7 +540,7 @@ class TestByteBufferDataInputStream {
val dis = ByteBufferDataInputStream(data)
val pattern = Pattern.compile("a")
val matcher = pattern.matcher("")
- val isMatch = dis.lookingAt(matcher)
+ val isMatch = dis.lookingAt(matcher, finfo)
assertTrue(isMatch)
assertEqualsTyped[Long](0, matcher.start)
assertEqualsTyped[Long](1, matcher.end)
@@ -624,7 +647,7 @@ class TestByteBufferDataInputStream {
val dis = ByteBufferDataInputStream(data)
val pattern = Pattern.compile("a*b+c")
val matcher = pattern.matcher("")
- val isMatch = dis.lookingAt(matcher)
+ val isMatch = dis.lookingAt(matcher, finfo)
assertTrue(isMatch)
assertEqualsTyped[Long](0, matcher.start)
assertEqualsTyped[Long](3, matcher.end)
@@ -642,7 +665,7 @@ class TestByteBufferDataInputStream {
val dis = ByteBufferDataInputStream(data)
val pattern = Pattern.compile(".*")
val matcher = pattern.matcher("")
- val isMatch = dis.lookingAt(matcher)
+ val isMatch = dis.lookingAt(matcher, finfo)
assertTrue(isMatch)
assertEqualsTyped[Long](0, matcher.start)
assertEqualsTyped[Long](14, matcher.end)
@@ -661,7 +684,7 @@ class TestByteBufferDataInputStream {
val dis = ByteBufferDataInputStream(data)
val pattern = Pattern.compile(".{1,13}")
val matcher = pattern.matcher("")
- val isMatch = dis.lookingAt(matcher)
+ val isMatch = dis.lookingAt(matcher, finfo)
assertTrue(isMatch)
assertEqualsTyped[Long](0, matcher.start)
assertEqualsTyped[Long](13, matcher.end)
@@ -680,10 +703,11 @@ class TestByteBufferDataInputStream {
val data3 = dataString3.getBytes(enc)
val data = data1 ++ badByte ++ data3
val dis = ByteBufferDataInputStream(data)
- dis.setDecoder(StandardCharsets.UTF_16BE.newDecoder())
+ val finfo = FormatInfoForUnitTest()
+ finfo.reset(StandardCharsets.UTF_16BE)
val pattern = Pattern.compile(".{1,13}")
val matcher = pattern.matcher("")
- val isMatch = dis.lookingAt(matcher)
+ val isMatch = dis.lookingAt(matcher, finfo)
assertTrue(isMatch)
assertEqualsTyped[Long](0, matcher.start)
assertEqualsTyped[Long](13, matcher.end)
@@ -709,7 +733,7 @@ class TestByteBufferDataInputStream {
) # end lookahead """
val pattern = Pattern.compile(regex)
val matcher = pattern.matcher("")
- val isMatch = dis.lookingAt(matcher)
+ val isMatch = dis.lookingAt(matcher, finfo)
assertFalse(isMatch)
assertTrue(matcher.hitEnd)
}
diff --git a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream2.scala b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream2.scala
index 68c8545..ea6435d 100644
--- a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream2.scala
+++ b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream2.scala
@@ -43,11 +43,12 @@ class TestByteBufferDataInputStream2 {
val ten = tenDigits.getBytes("utf-8")
val twentyDigits = tenDigits * 2
val twenty = twentyDigits.getBytes("utf-8")
+ val finfo = FormatInfoForUnitTest()
@Test def testMark1 {
val dis = ByteBufferDataInputStream(ten)
val m1 = dis.mark("testMark1")
- var arr = dis.getByteArray(80)
+ var arr = dis.getByteArray(80, finfo)
assertEquals(10, arr.size)
assertEquals(0x31.toByte, arr(0))
assertEquals(80, dis.bitPos0b)
@@ -56,7 +57,7 @@ class TestByteBufferDataInputStream2 {
assertEquals(81, dis.bitLimit1b.get)
assertEquals(10, dis.bytePos0b)
dis.reset(m1)
- arr = dis.getByteArray(80)
+ arr = dis.getByteArray(80, finfo)
assertEquals(10, arr.size)
assertEquals(0x30.toByte, arr(9))
assertEquals(80, dis.bitPos0b)
@@ -70,13 +71,15 @@ class TestByteBufferDataInputStream2 {
val dis = ByteBufferDataInputStream(twenty)
var m1: DataInputStream.Mark = null
dis.withBitLengthLimit(5 * 8) {
- val arr = dis.getByteArray(5 * 8)
+ val arr = dis.getByteArray(5 * 8, finfo)
assertEquals(5, arr.size)
m1 = dis.mark("testMark2")
- assertFalse(dis.asIteratorChar.hasNext)
+ val iter = dis.asIteratorChar
+ iter.setFormatInfo(finfo)
+ assertFalse(iter.hasNext)
}
dis.setBitLimit0b(MaybeULong(10 * 8))
- var arr = dis.getByteArray(5 * 8)
+ var arr = dis.getByteArray(5 * 8, finfo)
dis.mark("testMark2b")
assertEquals(5, arr.size)
assertEquals(0x30.toByte, arr(4))
@@ -88,7 +91,7 @@ class TestByteBufferDataInputStream2 {
dis.reset(m1)
assertFalse(dis.isDefinedForLength(1))
dis.asInstanceOf[ByteBufferDataInputStream].resetBitLimit0b(MaybeULong(10 * 8))
- arr = dis.getByteArray(5 * 8)
+ arr = dis.getByteArray(5 * 8, finfo)
assertEquals(5, arr.size)
assertEquals(0x30.toByte, arr(4))
assertEquals(80, dis.bitPos0b)
@@ -101,12 +104,14 @@ class TestByteBufferDataInputStream2 {
@Test def testMark3 {
val dis = ByteBufferDataInputStream(twenty).asInstanceOf[ByteBufferDataInputStream]
dis.setBitLimit0b(MaybeULong(5 * 8))
- var arr = dis.getByteArray(5 * 8)
+ var arr = dis.getByteArray(5 * 8, finfo)
assertEquals(5, arr.size)
val m1 = dis.mark("testMark3")
- assertFalse(dis.asIteratorChar.hasNext)
+ val iter = dis.asIteratorChar
+ iter.setFormatInfo(finfo)
+ assertFalse(iter.hasNext)
dis.resetBitLimit0b(MaybeULong(10 * 8))
- arr = dis.getByteArray(5 * 8)
+ arr = dis.getByteArray(5 * 8, finfo)
val m2 = dis.mark("testMark3b")
assertEquals(5, arr.size)
assertEquals(0x30.toByte, arr(4))
diff --git a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream3Bit.scala b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream3Bit.scala
index 7dc2905..e2a58d5 100644
--- a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream3Bit.scala
+++ b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream3Bit.scala
@@ -49,6 +49,8 @@ import edu.illinois.ncsa.daffodil.processors.charset.OctalLSBF3BitCharset
*/
class TestByteBufferDataInputStream3Bit {
+ val finfo = FormatInfoForUnitTest()
+
/**
* Helper class for creating example data that is unaligned.
*/
@@ -124,9 +126,11 @@ class TestByteBufferDataInputStream3Bit {
*/
@Test def testFillCharBufferOne3BitChar {
val dis = ByteBufferDataInputStream(BitsBitte.enc("01234567"))
- dis.setDecoder(OctalLSBF3BitCharset.newDecoder)
+ val cs = OctalLSBF3BitCharset
+ val finfo = FormatInfoForUnitTest()
+ finfo.reset(cs)
val cb = CharBuffer.allocate(1)
- val ml = dis.fillCharBuffer(cb)
+ val ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEquals(1, ml.get)
@@ -138,9 +142,10 @@ class TestByteBufferDataInputStream3Bit {
val dat = "01234567"
val cs = OctalLSBF3BitCharset
val dis = ByteBufferDataInputStream(BitsBitte.enc(dat))
- dis.setDecoder(cs.newDecoder)
+ val finfo = FormatInfoForUnitTest()
+ finfo.reset(cs)
val cb = CharBuffer.allocate(8)
- val ml = dis.fillCharBuffer(cb)
+ val ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEquals(8, ml.get)
@@ -153,10 +158,11 @@ class TestByteBufferDataInputStream3Bit {
val cs = OctalLSBF3BitCharset
val bytes = BitsBitte.toBytes(BitsBitte.rtl(BitsBitte.rtl("11"), BitsBitte.encode3(dat)))
val dis = ByteBufferDataInputStream(bytes)
- dis.setDecoder(cs.newDecoder)
+ val finfo = FormatInfoForUnitTest()
+ finfo.reset(cs)
val cb = CharBuffer.allocate(8)
- dis.skip(2)
- val ml = dis.fillCharBuffer(cb)
+ dis.skip(2, finfo)
+ val ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEquals(8, ml.get)
@@ -169,11 +175,12 @@ class TestByteBufferDataInputStream3Bit {
val cs = OctalLSBF3BitCharset
val bytes = BitsBitte.toBytes(BitsBitte.rtl(BitsBitte.rtl("11"), BitsBitte.encode3(dat)))
val dis = ByteBufferDataInputStream(bytes)
- dis.setDecoder(cs.newDecoder)
+ val finfo = FormatInfoForUnitTest()
+ finfo.reset(cs)
dis.setBitLimit0b(MaybeULong(12))
val cb = CharBuffer.allocate(8)
- dis.skip(2)
- val ml = dis.fillCharBuffer(cb)
+ dis.skip(2, finfo)
+ val ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEquals(3, ml.get)
@@ -195,11 +202,12 @@ class TestByteBufferDataInputStream3Bit {
val cs = OctalLSBF3BitCharset
val bytes = BitsBitte.toBytes(BitsBitte.rtl(BitsBitte.rtl("1"), BitsBitte.encode3(dat)))
val dis = ByteBufferDataInputStream(bytes)
- dis.setDecoder(cs.newDecoder)
+ val finfo = FormatInfoForUnitTest()
+ finfo.reset(cs)
dis.setBitLimit0b(MaybeULong(8))
val cb = CharBuffer.allocate(8)
- dis.skip(1)
- val ml = dis.fillCharBuffer(cb)
+ dis.skip(1, finfo)
+ val ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEquals(2, ml.get)
@@ -217,10 +225,11 @@ class TestByteBufferDataInputStream3Bit {
val cs = OctalLSBF3BitCharset
val bytes = BitsBitte.toBytes(BitsBitte.rtl(BitsBitte.encode3(dat)))
val dis = ByteBufferDataInputStream(bytes)
- dis.setDecoder(cs.newDecoder)
+ val finfo = FormatInfoForUnitTest()
+ finfo.reset(cs)
dis.setBitLimit0b(MaybeULong(5))
val cb = CharBuffer.allocate(8)
- val ml = dis.fillCharBuffer(cb)
+ val ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEquals(1, ml.get)
@@ -233,10 +242,11 @@ class TestByteBufferDataInputStream3Bit {
val cs = OctalLSBF3BitCharset
val bytes = BitsBitte.toBytes(BitsBitte.rtl(BitsBitte.encode3(dat)))
val dis = ByteBufferDataInputStream(bytes)
- dis.setDecoder(cs.newDecoder)
+ val finfo = FormatInfoForUnitTest()
+ finfo.reset(cs)
dis.setBitLimit0b(MaybeULong(14))
val cb = CharBuffer.allocate(8)
- val ml = dis.fillCharBuffer(cb)
+ val ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEquals(4, ml.get)
@@ -249,11 +259,12 @@ class TestByteBufferDataInputStream3Bit {
val cs = OctalLSBF3BitCharset
val bytes = BitsBitte.toBytes(BitsBitte.rtl(BitsBitte.rtl("1"), BitsBitte.encode3(dat)))
val dis = ByteBufferDataInputStream(bytes)
- dis.setDecoder(cs.newDecoder)
+ val finfo = FormatInfoForUnitTest()
+ finfo.reset(cs)
dis.setBitLimit0b(MaybeULong(6))
val cb = CharBuffer.allocate(8)
- dis.skip(1)
- val ml = dis.fillCharBuffer(cb)
+ dis.skip(1, finfo)
+ val ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEquals(1, ml.get)
@@ -275,25 +286,28 @@ class TestByteBufferDataInputStream3Bit {
@Test def testCharIteratorWithInterruptingBitSkips1 {
val dis = ByteBufferDataInputStream(BitsBitte.enc("01234567"))
- dis.setDecoder(OctalLSBF3BitCharset.newDecoder)
+ val cs = OctalLSBF3BitCharset
+ val finfo = FormatInfoForUnitTest()
+ finfo.reset(cs)
dis.setBitLimit0b(MaybeULong(24))
val iter = dis.asIteratorChar
- dis.skip(3)
+ iter.setFormatInfo(finfo)
+ dis.skip(3, finfo)
assertTrue(iter.hasNext)
assertEquals(3, dis.bitPos0b)
assertEquals('1', iter.next)
assertEquals(6, dis.bitPos0b)
- dis.skip(3)
+ dis.skip(3, finfo)
assertTrue(iter.hasNext)
assertEquals(9, dis.bitPos0b)
assertEquals('3', iter.next)
assertEquals(12, dis.bitPos0b)
- dis.skip(1)
- dis.skip(2)
+ dis.skip(1, finfo)
+ dis.skip(2, finfo)
assertTrue(iter.hasNext)
assertEquals('5', iter.next)
assertEquals(18, dis.bitPos0b)
- assertTrue(dis.skip(3))
+ assertTrue(dis.skip(3, finfo))
assertEquals('7', iter.next)
assertFalse(iter.hasNext)
}
diff --git a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream4.scala b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream4.scala
index c620921..2550e65 100644
--- a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream4.scala
+++ b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream4.scala
@@ -39,6 +39,9 @@ import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
class TestByteBufferDataInputStream4 {
+ val leFinfo = FormatInfoForUnitTest()
+ leFinfo.byteOrder = ByteOrder.LittleEndian
+
@Test def testLittleEndianFloat1() {
val bb = ByteBuffer.allocate(4)
bb.order(java.nio.ByteOrder.LITTLE_ENDIAN)
@@ -47,9 +50,9 @@ class TestByteBufferDataInputStream4 {
fb.put(1.125.toFloat)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.LittleEndian)
+
val expected = 1.125.toFloat
- val md = dis.getBinaryFloat()
+ val md = dis.getBinaryFloat(leFinfo)
assertEquals(expected, md, 0.01)
assertEquals(32, dis.bitPos0b)
}
@@ -62,9 +65,8 @@ class TestByteBufferDataInputStream4 {
fb.put(1.125)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.LittleEndian)
val expected = 1.125
- val md = dis.getBinaryDouble()
+ val md = dis.getBinaryDouble(leFinfo)
assertEquals(expected, md, 0.01)
assertEquals(64, dis.bitPos0b)
}
@@ -78,8 +80,7 @@ class TestByteBufferDataInputStream4 {
fb.put(expected)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.LittleEndian)
- val md = dis.getSignedLong(64)
+ val md = dis.getSignedLong(64, leFinfo)
assertEquals(expected, md)
assertEquals(64, dis.bitPos0b)
}
@@ -93,8 +94,7 @@ class TestByteBufferDataInputStream4 {
fb.put(expected)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.LittleEndian)
- val md = dis.getSignedLong(32)
+ val md = dis.getSignedLong(32, leFinfo)
assertEquals(expected, md)
assertEquals(32, dis.bitPos0b)
}
@@ -108,8 +108,7 @@ class TestByteBufferDataInputStream4 {
fb.put(expected)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.LittleEndian)
- val md = dis.getSignedLong(64)
+ val md = dis.getSignedLong(64, leFinfo)
assertEquals(expected, md)
assertEquals(64, dis.bitPos0b)
}
@@ -123,8 +122,7 @@ class TestByteBufferDataInputStream4 {
fb.put(data)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.LittleEndian)
- val md = dis.getSignedLong(32)
+ val md = dis.getSignedLong(32, leFinfo)
assertEquals(0x80706050.toInt, md)
assertEquals(32, dis.bitPos0b)
}
@@ -138,8 +136,7 @@ class TestByteBufferDataInputStream4 {
fb.put(data)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.LittleEndian)
- val md = dis.getSignedLong(1)
+ val md = dis.getSignedLong(1, leFinfo)
assertEquals(1, md)
assertEquals(1, dis.bitPos0b)
}
diff --git a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream5.scala b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream5.scala
index 85ffa81..3485aa8 100644
--- a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream5.scala
+++ b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream5.scala
@@ -39,6 +39,9 @@ import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
class TestByteBufferDataInputStream5 {
+ val leFinfo = FormatInfoForUnitTest()
+ leFinfo.byteOrder = ByteOrder.LittleEndian
+ leFinfo.bitOrder = BitOrder.LeastSignificantBitFirst
@Test def testLittleEndianLSBFirstLong1() {
val bb = ByteBuffer.allocate(8)
@@ -49,9 +52,7 @@ class TestByteBufferDataInputStream5 {
fb.put(expected)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.LittleEndian)
- dis.setBitOrder(BitOrder.LeastSignificantBitFirst)
- val md = dis.getSignedLong(64)
+ val md = dis.getSignedLong(64, leFinfo)
assertEquals(expected, md)
assertEquals(64, dis.bitPos0b)
}
@@ -65,9 +66,7 @@ class TestByteBufferDataInputStream5 {
fb.put(expected)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.LittleEndian)
- dis.setBitOrder(BitOrder.LeastSignificantBitFirst)
- val md = dis.getSignedLong(32)
+ val md = dis.getSignedLong(32, leFinfo)
assertEquals(expected, md)
assertEquals(32, dis.bitPos0b)
}
@@ -81,9 +80,7 @@ class TestByteBufferDataInputStream5 {
fb.put(expected)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.LittleEndian)
- dis.setBitOrder(BitOrder.LeastSignificantBitFirst)
- val md = dis.getSignedLong(64)
+ val md = dis.getSignedLong(64, leFinfo)
assertEquals(expected, md)
assertEquals(64, dis.bitPos0b)
}
@@ -97,9 +94,7 @@ class TestByteBufferDataInputStream5 {
fb.put(data)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.LittleEndian)
- dis.setBitOrder(BitOrder.LeastSignificantBitFirst)
- val md = dis.getSignedLong(32)
+ val md = dis.getSignedLong(32, leFinfo)
assertEquals(0x80706050.toInt, md)
assertEquals(32, dis.bitPos0b)
}
@@ -113,9 +108,8 @@ class TestByteBufferDataInputStream5 {
fb.put(data)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.LittleEndian)
- dis.setBitOrder(BitOrder.LeastSignificantBitFirst)
- val md = dis.getSignedLong(1)
+
+ val md = dis.getSignedLong(1, leFinfo)
assertEquals(1, md)
assertEquals(1, dis.bitPos0b)
}
diff --git a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream6.scala b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream6.scala
index e89aea1..b4e9f43 100644
--- a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream6.scala
+++ b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream6.scala
@@ -38,11 +38,19 @@ import java.nio.ByteBuffer
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
import java.nio.CharBuffer
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.EncodingErrorPolicy
import edu.illinois.ncsa.daffodil.util.MaybeULong
class TestByteBufferDataInputStream6 {
+ val beFinfo = FormatInfoForUnitTest()
+
+ val leFinfo = FormatInfoForUnitTest()
+ leFinfo.byteOrder = ByteOrder.LittleEndian
+
+ val lsbfFinfo = FormatInfoForUnitTest()
+ lsbfFinfo.byteOrder = ByteOrder.LittleEndian
+ lsbfFinfo.bitOrder = BitOrder.LeastSignificantBitFirst
+
@Test def testUnalignedByteArrayMSBFirst() {
val bb = ByteBuffer.allocate(8)
bb.order(java.nio.ByteOrder.BIG_ENDIAN)
@@ -52,10 +60,8 @@ class TestByteBufferDataInputStream6 {
fb.put(data)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.BigEndian)
- dis.setBitOrder(BitOrder.MostSignificantBitFirst)
- dis.skip(1) // move over one bit
- val arr = dis.getByteArray((8 * 8) - 1)
+ dis.skip(1, beFinfo) // move over one bit
+ val arr = dis.getByteArray((8 * 8) - 1, beFinfo)
assertEquals(8, arr.size)
assertEquals(0x71.toByte, arr(0))
assertEquals(0x02.toByte, arr(1))
@@ -77,11 +83,9 @@ class TestByteBufferDataInputStream6 {
fb.put(data)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.LittleEndian)
- dis.setBitOrder(BitOrder.MostSignificantBitFirst)
- dis.skip(1)
+ dis.skip(1, leFinfo)
assertEquals(1, dis.bitPos0b)
- val arr = dis.getByteArray((8 * 8) - 1)
+ val arr = dis.getByteArray((8 * 8) - 1, leFinfo)
assertEquals(8, arr.size)
assertEquals(0x71.toByte, arr(0))
assertEquals(0x05.toByte, arr(1))
@@ -103,10 +107,8 @@ class TestByteBufferDataInputStream6 {
fb.put(data)
val bytes = bb.array()
val dis = ByteBufferDataInputStream(bytes)
- dis.setByteOrder(ByteOrder.LittleEndian)
- dis.setBitOrder(BitOrder.LeastSignificantBitFirst)
- dis.skip(1)
- val arr = dis.getByteArray(((8 * 8) - 1))
+ dis.skip(1, lsbfFinfo)
+ val arr = dis.getByteArray(((8 * 8) - 1), lsbfFinfo)
assertEquals(8, arr.size)
assertEquals(0x00.toByte, arr(0))
assertEquals(0x81.toByte, arr(1))
@@ -128,8 +130,8 @@ class TestByteBufferDataInputStream6 {
@Test def testFillCharBuffer1 {
val dis = ByteBufferDataInputStream("01".getBytes())
val cb = CharBuffer.allocate(1)
- dis.getSignedLong(1)
- val ml = dis.fillCharBuffer(cb)
+ dis.getSignedLong(1, beFinfo)
+ val ml = dis.fillCharBuffer(cb, beFinfo)
assertTrue(ml.isDefined)
assertEquals(1, ml.get)
assertEquals(16, dis.bitPos0b)
@@ -139,8 +141,8 @@ class TestByteBufferDataInputStream6 {
@Test def testFillCharBuffer2 {
val dis = ByteBufferDataInputStream("0年月日".getBytes("utf-8"))
val cb = CharBuffer.allocate(3)
- dis.getSignedLong(4)
- val ml = dis.fillCharBuffer(cb)
+ dis.getSignedLong(4, beFinfo)
+ val ml = dis.fillCharBuffer(cb, beFinfo)
assertTrue(ml.isDefined)
assertEquals(3, ml.get)
assertEquals('年', cb.get(0))
@@ -153,7 +155,7 @@ class TestByteBufferDataInputStream6 {
val dis = ByteBufferDataInputStream("年月日".getBytes("utf-8"))
dis.setBitLimit0b(MaybeULong((8 * 6) + 2)) // 2 extra bits after first 2 chars
val cb = CharBuffer.allocate(3)
- val ml = dis.fillCharBuffer(cb)
+ val ml = dis.fillCharBuffer(cb, beFinfo)
assertTrue(ml.isDefined)
assertEquals(2, ml.get)
assertEquals('年', cb.get(0))
@@ -163,33 +165,31 @@ class TestByteBufferDataInputStream6 {
@Test def testFillCharBufferDataEndsMidByte2 {
val dis = ByteBufferDataInputStream("年月日".getBytes("utf-8"))
- dis.setEncodingErrorPolicy(EncodingErrorPolicy.Replace)
dis.setBitLimit0b(MaybeULong((8 * 6) + 2)) // 2 extra bits after first 2 chars
val cb = CharBuffer.allocate(3)
- val ml = dis.fillCharBuffer(cb)
+ val ml = dis.fillCharBuffer(cb, beFinfo)
assertTrue(ml.isDefined)
assertEquals(2, ml.get)
assertEquals('年', cb.get(0))
assertEquals('月', cb.get(1))
assertEquals(8 * 6, dis.bitPos0b)
cb.clear()
- val ml2 = dis.fillCharBuffer(cb) // ask for next character
+ val ml2 = dis.fillCharBuffer(cb, beFinfo) // ask for next character
assertEquals(MaybeULong.Nope, ml2)
}
@Test def testFillCharBufferDataEndsMidByte3 {
val dis = ByteBufferDataInputStream("年月日".getBytes("utf-8"))
- dis.setEncodingErrorPolicy(EncodingErrorPolicy.Replace)
dis.setBitLimit0b(MaybeULong((8 * 6) + 10)) // 1 more byte plus 2 extra bits after first 2 chars
val cb = CharBuffer.allocate(3)
- val ml = dis.fillCharBuffer(cb)
+ val ml = dis.fillCharBuffer(cb, beFinfo)
assertTrue(ml.isDefined)
assertEquals(2, ml.get)
assertEquals('年', cb.get(0))
assertEquals('月', cb.get(1))
assertEquals(8 * 6, dis.bitPos0b)
cb.clear()
- val ml2 = dis.fillCharBuffer(cb) // ask for next character
+ val ml2 = dis.fillCharBuffer(cb, beFinfo) // ask for next character
//
// because it has 1 more byte available, it doesn't stop the attempt to decode
// and that attempt fails and we replace it.
@@ -215,21 +215,22 @@ class TestByteBufferDataInputStream6 {
@Test def testCharIteratorWithInterruptingBitSkips1 {
val dis = ByteBufferDataInputStream("0年1月2日".getBytes("utf-8"))
val iter = dis.asIteratorChar
- dis.skip(1)
+ iter.setFormatInfo(beFinfo)
+ dis.skip(1, beFinfo)
assertTrue(iter.hasNext) // examining a character here requires aligning to mandatory alignment of 8 bit boundary.
assertEquals(1, dis.bitPos0b)
assertEquals('年', iter.next)
assertEquals(32, dis.bitPos0b)
- dis.skip(1)
+ dis.skip(1, beFinfo)
assertTrue(iter.hasNext)
assertEquals(33, dis.bitPos0b)
assertEquals('月', iter.next)
assertEquals(64, dis.bitPos0b)
- dis.skip(1)
+ dis.skip(1, beFinfo)
assertTrue(iter.hasNext)
assertEquals('日', iter.next)
assertEquals(96, dis.bitPos0b)
- assertFalse(dis.skip(1))
+ assertFalse(dis.skip(1, beFinfo))
assertFalse(iter.hasNext)
}
@@ -244,10 +245,11 @@ class TestByteBufferDataInputStream6 {
@Test def testCharIteratorWithInterruptingBitSkipsBetweenHasNextAndNext {
val dis = ByteBufferDataInputStream("0年1月2日".getBytes("utf-8"))
val iter = dis.asIteratorChar
- dis.skip(1)
+ iter.setFormatInfo(beFinfo)
+ dis.skip(1, beFinfo)
assertTrue(iter.hasNext) // examining a character here requires aligning to mandatory alignment of 8 bit boundary.
assertEquals(1, dis.bitPos0b)
- dis.skip(1) // this skip should invalidate the character cached by hasNext.
+ dis.skip(1, beFinfo) // this skip should invalidate the character cached by hasNext.
assertEquals(2, dis.bitPos0b)
assertTrue(iter.hasNext)
assertEquals(2, dis.bitPos0b)
@@ -255,7 +257,7 @@ class TestByteBufferDataInputStream6 {
assertEquals(32, dis.bitPos0b) // has next doesn't cause movement even to align to mandatory.
assertEquals('年', c)
assertTrue(iter.hasNext)
- dis.skip(4)
+ dis.skip(4, beFinfo)
assertEquals(36, dis.bitPos0b)
val d = iter.next
assertEquals(64, dis.bitPos0b)
diff --git a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream7.scala b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream7.scala
index 24d451b..810ec1e 100644
--- a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream7.scala
+++ b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestByteBufferDataInputStream7.scala
@@ -92,6 +92,9 @@ object Bitte {
*/
class TestByteBufferDataInputStream7 {
+ val finfo = FormatInfoForUnitTest()
+ finfo.reset(USASCII7BitPackedCharset)
+
/** Test the test rig */
@Test def testBitteBits = {
@@ -130,9 +133,8 @@ class TestByteBufferDataInputStream7 {
*/
@Test def testGetByteArrayLengthLimit1 {
val dis = ByteBufferDataInputStream(Bitte.enc("abc"))
- dis.setDecoder(USASCII7BitPackedCharset.newDecoder)
dis.setBitLimit0b(MaybeULong(21))
- val arr = dis.getByteArray(21)
+ val arr = dis.getByteArray(21, finfo)
assertEquals(21, dis.bitLimit0b.get)
assertEquals(21, dis.bitPos0b)
assertEquals(3, arr.size)
@@ -146,9 +148,8 @@ class TestByteBufferDataInputStream7 {
*/
@Test def testFillCharBufferOne7BitChar {
val dis = ByteBufferDataInputStream(Bitte.enc("abcdefgh"))
- dis.setDecoder(USASCII7BitPackedCharset.newDecoder)
val cb = CharBuffer.allocate(1)
- val ml = dis.fillCharBuffer(cb)
+ val ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEquals(1, ml.get)
@@ -158,9 +159,8 @@ class TestByteBufferDataInputStream7 {
@Test def testFillCharBuffer7BitString {
val dis = ByteBufferDataInputStream(Bitte.enc("abcdefgh"))
- dis.setDecoder(USASCII7BitPackedCharset.newDecoder)
val cb = CharBuffer.allocate(8)
- val ml = dis.fillCharBuffer(cb)
+ val ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEquals(8, ml.get)
@@ -170,10 +170,9 @@ class TestByteBufferDataInputStream7 {
@Test def testFillCharBuffer7BitStringOffBy3 {
val bytes = Bitte.toBytes(Bitte.rtl(Bitte.rtl("101"), Bitte.encode7("abcdefgh")))
val dis = ByteBufferDataInputStream(bytes)
- dis.setDecoder(USASCII7BitPackedCharset.newDecoder)
val cb = CharBuffer.allocate(8)
- dis.skip(3)
- val ml = dis.fillCharBuffer(cb)
+ dis.skip(3, finfo)
+ val ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEquals(8, ml.get)
@@ -184,11 +183,10 @@ class TestByteBufferDataInputStream7 {
@Test def testFillCharBufferDataEndsMidByte {
val bytes = Bitte.toBytes(Bitte.rtl(Bitte.rtl("101"), Bitte.encode7("abcdefgh")))
val dis = ByteBufferDataInputStream(bytes)
- dis.setDecoder(USASCII7BitPackedCharset.newDecoder)
dis.setBitLimit0b(MaybeULong(25))
val cb = CharBuffer.allocate(8)
- dis.skip(3)
- val ml = dis.fillCharBuffer(cb)
+ dis.skip(3, finfo)
+ val ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEquals(3, ml.get)
@@ -208,11 +206,10 @@ class TestByteBufferDataInputStream7 {
@Test def testFillCharBufferDataEndsMidByte2 {
val bytes = Bitte.toBytes(Bitte.rtl(Bitte.rtl("101"), Bitte.encode7("abcdefgh")))
val dis = ByteBufferDataInputStream(bytes)
- dis.setDecoder(USASCII7BitPackedCharset.newDecoder)
dis.setBitLimit0b(MaybeULong(20))
val cb = CharBuffer.allocate(8)
- dis.skip(3)
- val ml = dis.fillCharBuffer(cb)
+ dis.skip(3, finfo)
+ val ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEquals(2, ml.get)
@@ -227,11 +224,10 @@ class TestByteBufferDataInputStream7 {
@Test def testFillCharBufferDataEndsMidByte3 {
val bytes = Bitte.toBytes(Bitte.rtl(Bitte.rtl("101"), Bitte.encode7("abcdefgh")))
val dis = ByteBufferDataInputStream(bytes)
- dis.setDecoder(USASCII7BitPackedCharset.newDecoder)
dis.setBitLimit0b(MaybeULong(16))
val cb = CharBuffer.allocate(8)
- dis.skip(3)
- val ml = dis.fillCharBuffer(cb)
+ dis.skip(3, finfo)
+ val ml = dis.fillCharBuffer(cb, finfo)
cb.flip
assertTrue(ml.isDefined)
assertEquals(1, ml.get)
@@ -249,20 +245,20 @@ class TestByteBufferDataInputStream7 {
@Test def testCharIteratorWithInterruptingBitSkips1 {
val dis = ByteBufferDataInputStream(Bitte.enc("0a1b2c"))
- dis.setDecoder(USASCII7BitPackedCharset.newDecoder)
dis.setBitLimit0b(MaybeULong(42))
val iter = dis.asIteratorChar
- dis.skip(7)
+ iter.setFormatInfo(finfo)
+ dis.skip(7, finfo)
assertTrue(iter.hasNext)
assertEquals(7, dis.bitPos0b)
assertEquals('a', iter.next)
assertEquals(14, dis.bitPos0b)
- dis.skip(7)
+ dis.skip(7, finfo)
assertTrue(iter.hasNext)
assertEquals(21, dis.bitPos0b)
assertEquals('b', iter.next)
assertEquals(28, dis.bitPos0b)
- dis.skip(7)
+ dis.skip(7, finfo)
assertTrue(iter.hasNext)
// Problem. We have 42 bits, so it *should* be letting us get the last character.
// However, it is not, because we're not being allowed to get the next byte (the whole next byte isn't available)
@@ -273,7 +269,7 @@ class TestByteBufferDataInputStream7 {
//
assertEquals('c', iter.next)
assertEquals(42, dis.bitPos0b)
- assertFalse(dis.skip(1))
+ assertFalse(dis.skip(1, finfo))
assertFalse(iter.hasNext)
}
@@ -289,19 +285,19 @@ class TestByteBufferDataInputStream7 {
*/
@Test def testCharIteratorWithInterruptingBitSkipsBetweenHasNextAndNext {
val dis = ByteBufferDataInputStream(Bitte.enc("0a1b2c"))
- dis.setDecoder(USASCII7BitPackedCharset.newDecoder)
dis.setBitLimit0b(MaybeULong(42))
val iter = dis.asIteratorChar
+ iter.setFormatInfo(finfo)
assertTrue(iter.hasNext)
- dis.skip(7)
+ dis.skip(7, finfo)
assertEquals(7, dis.bitPos0b)
assertEquals('a', iter.next)
assertEquals(14, dis.bitPos0b)
- dis.skip(7)
+ dis.skip(7, finfo)
assertEquals(21, dis.bitPos0b)
assertEquals('b', iter.next)
assertEquals(28, dis.bitPos0b)
- dis.skip(7)
+ dis.skip(7, finfo)
assertEquals(35, dis.bitPos0b)
assertTrue(iter.hasNext)
assertEquals(35, dis.bitPos0b)
diff --git a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream.scala b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream.scala
index 744caaa..3bae966 100644
--- a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream.scala
+++ b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream.scala
@@ -33,21 +33,26 @@
package edu.illinois.ncsa.daffodil.io
import junit.framework.Assert._
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
import org.junit.Test
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
class TestDataOutputStream {
+ val beFinfo = FormatInfoForUnitTest()
+
+ def newDirectOrBufferedDataOutputStream(jos: java.io.OutputStream, creator: DirectOrBufferedDataOutputStream,
+ bo: BitOrder = BitOrder.MostSignificantBitFirst) = {
+ val os = DirectOrBufferedDataOutputStream(jos, creator)
+ os.setPriorBitOrder(bo)
+ os
+ }
+
@Test def testPutLongDirect1_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
-
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.putLong(-1L, 32)
+ out.putLong(-1L, 32, beFinfo)
val buf = baos.getBuf()
@@ -63,14 +68,11 @@ class TestDataOutputStream {
@Test def testPutLongDirect1Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
-
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.putLong(1, 1)
+ out.putLong(1, 1, beFinfo)
- out.setFinished()
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -82,14 +84,11 @@ class TestDataOutputStream {
@Test def testPutLongDirect2Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
+ out.putLong(3, 2, beFinfo)
- out.putLong(3, 2)
-
- out.setFinished()
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -101,14 +100,11 @@ class TestDataOutputStream {
@Test def testPutLongDirect7Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
-
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.putLong(0xA5, 7)
+ out.putLong(0xA5, 7, beFinfo)
- out.setFinished()
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -120,14 +116,11 @@ class TestDataOutputStream {
@Test def testPutLongDirect8Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
+ out.putLong(0xA5, 8, beFinfo)
- out.putLong(0xA5, 8)
-
- out.setFinished()
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -139,14 +132,11 @@ class TestDataOutputStream {
@Test def testPutLongDirect9Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
-
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.putLong(0xA5A5, 9)
+ out.putLong(0xA5A5, 9, beFinfo)
- out.setFinished()
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -159,14 +149,11 @@ class TestDataOutputStream {
@Test def testPutLongDirect63Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
+ out.putLong(0xA5A5A5A5A5A5A5A5L, 63, beFinfo)
- out.putLong(0xA5A5A5A5A5A5A5A5L, 63)
-
- out.setFinished()
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -185,14 +172,11 @@ class TestDataOutputStream {
@Test def testPutLongDirect64Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
-
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.putLong(0xA5A5A5A5A5A5A5A5L, 64)
+ out.putLong(0xA5A5A5A5A5A5A5A5L, 64, beFinfo)
- out.setFinished()
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -216,16 +200,14 @@ class TestDataOutputStream {
@Test def testPutLongBuffered1_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(-1L, 32)
+ out.putLong(-1L, 32, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -241,16 +223,14 @@ class TestDataOutputStream {
@Test def testPutLongBuffered1Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(1, 1)
+ out.putLong(1, 1, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -262,16 +242,14 @@ class TestDataOutputStream {
@Test def testPutLongBuffered2Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(3, 2)
+ out.putLong(3, 2, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -283,16 +261,14 @@ class TestDataOutputStream {
@Test def testPutLongBuffered7Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5, 7)
+ out.putLong(0xA5, 7, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -304,16 +280,15 @@ class TestDataOutputStream {
@Test def testPutLongBuffered8Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
+ direct.setPriorBitOrder(BitOrder.MostSignificantBitFirst)
val out = direct.addBuffered
- out.putLong(0xA5, 8)
+ out.putLong(0xA5, 8, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -325,16 +300,14 @@ class TestDataOutputStream {
@Test def testPutLongBuffered9Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5A5, 9)
+ out.putLong(0xA5A5, 9, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -347,16 +320,14 @@ class TestDataOutputStream {
@Test def testPutLongBuffered63Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5A5A5A5A5A5A5A5L, 63)
+ out.putLong(0xA5A5A5A5A5A5A5A5L, 63, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -375,16 +346,14 @@ class TestDataOutputStream {
@Test def testPutLongBuffered64Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5A5A5A5A5A5A5A5L, 64)
+ out.putLong(0xA5A5A5A5A5A5A5A5L, 64, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -408,17 +377,15 @@ class TestDataOutputStream {
@Test def testPutLongDirectAndBuffered1_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(-1L, 32)
- direct.putLong(-1L, 32)
+ out.putLong(-1L, 32, beFinfo)
+ direct.putLong(-1L, 32, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -438,17 +405,15 @@ class TestDataOutputStream {
@Test def testPutLongDirectAndBuffered1Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(1, 1)
- direct.putLong(1, 1)
+ out.putLong(1, 1, beFinfo)
+ direct.putLong(1, 1, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -460,17 +425,15 @@ class TestDataOutputStream {
@Test def testPutLongDirectAndBuffered2Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(3, 2)
- direct.putLong(2, 2)
+ out.putLong(3, 2, beFinfo)
+ direct.putLong(2, 2, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -482,17 +445,15 @@ class TestDataOutputStream {
@Test def testPutLongDirectAndBuffered7Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5, 7)
- direct.putLong(0xA5, 7)
+ out.putLong(0xA5, 7, beFinfo)
+ direct.putLong(0xA5, 7, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -505,17 +466,15 @@ class TestDataOutputStream {
@Test def testPutLongDirectAndBuffered8Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5, 8)
- direct.putLong(0xA5, 8)
+ out.putLong(0xA5, 8, beFinfo)
+ direct.putLong(0xA5, 8, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -528,17 +487,15 @@ class TestDataOutputStream {
@Test def testPutLongDirectAndBuffered9Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xDEAD, 9)
- direct.putLong(0xBEEF, 9)
+ out.putLong(0xDEAD, 9, beFinfo)
+ direct.putLong(0xBEEF, 9, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -552,17 +509,15 @@ class TestDataOutputStream {
@Test def testPutLongDirectAndBuffered63BitPlus1Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5A5A5A5A5A5A5A5L, 63)
- direct.putLong(1, 1)
+ out.putLong(0xA5A5A5A5A5A5A5A5L, 63, beFinfo)
+ direct.putLong(1, 1, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -581,17 +536,15 @@ class TestDataOutputStream {
@Test def testPutLongDirectAndBuffered63BitPlus63Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5A5A5A5A5A5A5A5L, 63)
- direct.putLong(0xA5A5A5A5A5A5A5A5L, 63)
+ out.putLong(0xA5A5A5A5A5A5A5A5L, 63, beFinfo)
+ direct.putLong(0xA5A5A5A5A5A5A5A5L, 63, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -619,17 +572,15 @@ class TestDataOutputStream {
@Test def testPutLongDirectAndBuffered64BitPlus64Bit_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5A5A5A5A5A5A5A5L, 64)
- direct.putLong(0xA5A5A5A5A5A5A5A5L, 64)
+ out.putLong(0xA5A5A5A5A5A5A5A5L, 64, beFinfo)
+ direct.putLong(0xA5A5A5A5A5A5A5A5L, 64, beFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(beFinfo)
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -658,14 +609,11 @@ class TestDataOutputStream {
@Test def testPutLong5_4Bits_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
-
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.putLong(5L, 4)
+ out.putLong(5L, 4, beFinfo)
- out.setFinished()
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
diff --git a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream2.scala b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream2.scala
index 0199fc1..d3af058 100644
--- a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream2.scala
+++ b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream2.scala
@@ -33,24 +33,21 @@
package edu.illinois.ncsa.daffodil.io
import junit.framework.Assert._
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
import org.junit.Test
import java.nio.ByteBuffer
class TestDataOutputStream2 {
+ val beFinfo = FormatInfoForUnitTest()
+
@Test def testPutBitsDirect0_BE_MSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
val out = DirectOrBufferedDataOutputStream(baos, null)
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
-
- out.putBits(List(0xA5.toByte, 0xBE.toByte, 0xEF.toByte).toArray, 1, 16)
+ out.putBits(List(0xA5.toByte, 0xBE.toByte, 0xEF.toByte).toArray, 1, 16, beFinfo)
- out.setFinished()
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -66,12 +63,9 @@ class TestDataOutputStream2 {
val baos = new ByteArrayOutputStreamWithGetBuf()
val out = DirectOrBufferedDataOutputStream(baos, null)
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
+ out.putBits(List(0xA5.toByte, 0xBE.toByte, 0xEF.toByte).toArray, 1, 9, beFinfo)
- out.putBits(List(0xA5.toByte, 0xBE.toByte, 0xEF.toByte).toArray, 1, 9)
-
- out.setFinished()
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -87,12 +81,9 @@ class TestDataOutputStream2 {
val baos = new ByteArrayOutputStreamWithGetBuf()
val out = DirectOrBufferedDataOutputStream(baos, null)
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
-
- out.putBits(List(0xA5.toByte, 0xBE.toByte, 0xEF.toByte).toArray, 1, 15)
+ out.putBits(List(0xA5.toByte, 0xBE.toByte, 0xEF.toByte).toArray, 1, 15, beFinfo)
- out.setFinished()
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -112,14 +103,11 @@ class TestDataOutputStream2 {
val baos = new ByteArrayOutputStreamWithGetBuf()
val out = DirectOrBufferedDataOutputStream(baos, null)
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
-
val bb = ByteBuffer.wrap(List(0xA5.toByte, 0xBE.toByte, 0xEF.toByte).toArray)
bb.position(1)
- out.putBitBuffer(bb, 16)
+ out.putBitBuffer(bb, 16, beFinfo)
- out.setFinished()
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -135,14 +123,11 @@ class TestDataOutputStream2 {
val baos = new ByteArrayOutputStreamWithGetBuf()
val out = DirectOrBufferedDataOutputStream(baos, null)
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
-
val bb = ByteBuffer.wrap(List(0xA5.toByte, 0xBE.toByte, 0xEF.toByte).toArray)
bb.position(1)
- out.putBitBuffer(bb, 9)
+ out.putBitBuffer(bb, 9, beFinfo)
- out.setFinished()
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
@@ -158,14 +143,11 @@ class TestDataOutputStream2 {
val baos = new ByteArrayOutputStreamWithGetBuf()
val out = DirectOrBufferedDataOutputStream(baos, null)
- out.setBitOrder(BitOrder.MostSignificantBitFirst)
- out.setByteOrder(ByteOrder.BigEndian)
-
val bb = ByteBuffer.wrap(List(0xA5.toByte, 0xBE.toByte, 0xEF.toByte).toArray)
bb.position(1)
- out.putBitBuffer(bb, 15)
+ out.putBitBuffer(bb, 15, beFinfo)
- out.setFinished()
+ out.setFinished(beFinfo)
val buf = baos.getBuf()
diff --git a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream3.scala b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream3.scala
index 347141c..b8b3505 100644
--- a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream3.scala
+++ b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream3.scala
@@ -39,15 +39,23 @@ import org.junit.Test
class TestDataOutputStream3 {
+ val lsbfFinfo = FormatInfoForUnitTest()
+ lsbfFinfo.byteOrder = ByteOrder.LittleEndian
+ lsbfFinfo.bitOrder = BitOrder.LeastSignificantBitFirst
+
+ def newDirectOrBufferedDataOutputStream(jos: java.io.OutputStream, creator: DirectOrBufferedDataOutputStream,
+ bo: BitOrder = BitOrder.MostSignificantBitFirst) = {
+ val os = DirectOrBufferedDataOutputStream(jos, creator)
+ os.setPriorBitOrder(bo)
+ os
+ }
+
@Test def testPutLongDirect1_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.setBitOrder(BitOrder.LeastSignificantBitFirst)
- out.setByteOrder(ByteOrder.LittleEndian)
-
- out.putLong(-1L, 32)
+ out.putLong(-1L, 32, lsbfFinfo)
val buf = baos.getBuf()
@@ -63,14 +71,11 @@ class TestDataOutputStream3 {
@Test def testPutLongDirect1Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
-
- out.setBitOrder(BitOrder.LeastSignificantBitFirst)
- out.setByteOrder(ByteOrder.LittleEndian)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.putLong(1, 1)
+ out.putLong(1, 1, lsbfFinfo)
- out.setFinished()
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -82,14 +87,11 @@ class TestDataOutputStream3 {
@Test def testPutLongDirect2Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.setBitOrder(BitOrder.LeastSignificantBitFirst)
- out.setByteOrder(ByteOrder.LittleEndian)
+ out.putLong(3, 2, lsbfFinfo)
- out.putLong(3, 2)
-
- out.setFinished()
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -101,14 +103,11 @@ class TestDataOutputStream3 {
@Test def testPutLongDirect7Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
-
- out.setBitOrder(BitOrder.LeastSignificantBitFirst)
- out.setByteOrder(ByteOrder.LittleEndian)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.putLong(0xA5, 7)
+ out.putLong(0xA5, 7, lsbfFinfo)
- out.setFinished()
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -120,14 +119,11 @@ class TestDataOutputStream3 {
@Test def testPutLongDirect8Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
-
- out.setBitOrder(BitOrder.LeastSignificantBitFirst)
- out.setByteOrder(ByteOrder.LittleEndian)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.putLong(0xA5, 8)
+ out.putLong(0xA5, 8, lsbfFinfo)
- out.setFinished()
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -139,14 +135,11 @@ class TestDataOutputStream3 {
@Test def testPutLongDirect9Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.setBitOrder(BitOrder.LeastSignificantBitFirst)
- out.setByteOrder(ByteOrder.LittleEndian)
+ out.putLong(0xA5A5, 9, lsbfFinfo)
- out.putLong(0xA5A5, 9)
-
- out.setFinished()
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -159,14 +152,11 @@ class TestDataOutputStream3 {
@Test def testPutLongDirect63Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
-
- out.setBitOrder(BitOrder.LeastSignificantBitFirst)
- out.setByteOrder(ByteOrder.LittleEndian)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.putLong(0xA5A5A5A5A5A5A5A5L, 63)
+ out.putLong(0xA5A5A5A5A5A5A5A5L, 63, lsbfFinfo)
- out.setFinished()
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -185,14 +175,11 @@ class TestDataOutputStream3 {
@Test def testPutLongDirect64Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.setBitOrder(BitOrder.LeastSignificantBitFirst)
- out.setByteOrder(ByteOrder.LittleEndian)
+ out.putLong(0xA5A5A5A5A5A5A5A5L, 64, lsbfFinfo)
- out.putLong(0xA5A5A5A5A5A5A5A5L, 64)
-
- out.setFinished()
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -216,16 +203,13 @@ class TestDataOutputStream3 {
@Test def testPutLongBuffered1_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(-1L, 32)
+ out.putLong(-1L, 32, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -241,16 +225,13 @@ class TestDataOutputStream3 {
@Test def testPutLongBuffered1Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(1, 1)
+ out.putLong(1, 1, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -262,16 +243,13 @@ class TestDataOutputStream3 {
@Test def testPutLongBuffered2Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(3, 2)
+ out.putLong(3, 2, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -283,16 +261,13 @@ class TestDataOutputStream3 {
@Test def testPutLongBuffered7Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5, 7)
+ out.putLong(0xA5, 7, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -304,16 +279,13 @@ class TestDataOutputStream3 {
@Test def testPutLongBuffered8Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5, 8)
+ out.putLong(0xA5, 8, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -325,16 +297,13 @@ class TestDataOutputStream3 {
@Test def testPutLongBuffered9Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5A5, 9)
+ out.putLong(0xA5A5, 9, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -347,16 +316,13 @@ class TestDataOutputStream3 {
@Test def testPutLongBuffered63Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5A5A5A5A5A5A5A5L, 63)
+ out.putLong(0xA5A5A5A5A5A5A5A5L, 63, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -375,16 +341,13 @@ class TestDataOutputStream3 {
@Test def testPutLongBuffered64Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5A5A5A5A5A5A5A5L, 64)
+ out.putLong(0xA5A5A5A5A5A5A5A5L, 64, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -408,17 +371,14 @@ class TestDataOutputStream3 {
@Test def testPutLongDirectAndBuffered1_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(-1L, 32)
- direct.putLong(-1L, 32)
+ out.putLong(-1L, 32, lsbfFinfo)
+ direct.putLong(-1L, 32, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -438,17 +398,14 @@ class TestDataOutputStream3 {
@Test def testPutLongDirectAndBuffered1Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null, BitOrder.LeastSignificantBitFirst)
val out = direct.addBuffered
- out.putLong(1, 1)
- direct.putLong(1, 1)
+ out.putLong(1, 1, lsbfFinfo)
+ direct.putLong(1, 1, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -460,17 +417,14 @@ class TestDataOutputStream3 {
@Test def testPutLongDirectAndBuffered2Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null, BitOrder.LeastSignificantBitFirst)
val out = direct.addBuffered
- out.putLong(3, 2)
- direct.putLong(2, 2)
+ out.putLong(3, 2, lsbfFinfo)
+ direct.putLong(2, 2, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -482,17 +436,15 @@ class TestDataOutputStream3 {
@Test def testPutLongDirectAndBuffered7Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
+ direct.setPriorBitOrder(BitOrder.LeastSignificantBitFirst)
val out = direct.addBuffered
- out.putLong(0xA5, 7)
- direct.putLong(0xA5, 7)
+ out.putLong(0xA5, 7, lsbfFinfo)
+ direct.putLong(0xA5, 7, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -505,17 +457,14 @@ class TestDataOutputStream3 {
@Test def testPutLongDirectAndBuffered8Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5, 8)
- direct.putLong(0xA5, 8)
+ out.putLong(0xA5, 8, lsbfFinfo)
+ direct.putLong(0xA5, 8, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -528,17 +477,14 @@ class TestDataOutputStream3 {
@Test def testPutLongDirectAndBuffered9Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null, BitOrder.LeastSignificantBitFirst)
val out = direct.addBuffered
- out.putLong(0xDEAD, 9)
- direct.putLong(0xBEEF, 9)
+ out.putLong(0xDEAD, 9, lsbfFinfo)
+ direct.putLong(0xBEEF, 9, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -552,17 +498,14 @@ class TestDataOutputStream3 {
@Test def testPutLongDirectAndBuffered63BitPlus1Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null, BitOrder.LeastSignificantBitFirst)
val out = direct.addBuffered
- out.putLong(0xA5A5A5A5A5A5A5A5L, 63)
- direct.putLong(1, 1)
+ out.putLong(0xA5A5A5A5A5A5A5A5L, 63, lsbfFinfo)
+ direct.putLong(1, 1, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -581,17 +524,14 @@ class TestDataOutputStream3 {
@Test def testPutLongDirectAndBuffered63BitPlus63Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null, BitOrder.LeastSignificantBitFirst)
val out = direct.addBuffered
- out.putLong(0xA5A5A5A5A5A5A5A5L, 63)
- direct.putLong(0xA5A5A5A5A5A5A5A5L, 63)
+ out.putLong(0xA5A5A5A5A5A5A5A5L, 63, lsbfFinfo)
+ direct.putLong(0xA5A5A5A5A5A5A5A5L, 63, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -619,17 +559,14 @@ class TestDataOutputStream3 {
@Test def testPutLongDirectAndBuffered64BitPlus64Bit_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.LeastSignificantBitFirst)
- direct.setByteOrder(ByteOrder.LittleEndian)
-
+ val direct = newDirectOrBufferedDataOutputStream(baos, null)
val out = direct.addBuffered
- out.putLong(0xA5A5A5A5A5A5A5A5L, 64)
- direct.putLong(0xA5A5A5A5A5A5A5A5L, 64)
+ out.putLong(0xA5A5A5A5A5A5A5A5L, 64, lsbfFinfo)
+ direct.putLong(0xA5A5A5A5A5A5A5A5L, 64, lsbfFinfo)
- direct.setFinished()
- out.setFinished()
+ direct.setFinished(lsbfFinfo)
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
@@ -658,14 +595,11 @@ class TestDataOutputStream3 {
@Test def testPutLong5_4Bits_LE_LSBF {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val out = DirectOrBufferedDataOutputStream(baos, null)
-
- out.setBitOrder(BitOrder.LeastSignificantBitFirst)
- out.setByteOrder(ByteOrder.LittleEndian)
+ val out = newDirectOrBufferedDataOutputStream(baos, null)
- out.putLong(5L, 4)
+ out.putLong(5L, 4, lsbfFinfo)
- out.setFinished()
+ out.setFinished(lsbfFinfo)
val buf = baos.getBuf()
diff --git a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream4.scala b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream4.scala
index bfd8ad8..414c4a3 100644
--- a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream4.scala
+++ b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDataOutputStream4.scala
@@ -33,21 +33,21 @@
package edu.illinois.ncsa.daffodil.io
import junit.framework.Assert._
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
import org.junit.Test
import passera.unsigned.ULong
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
class TestDataOutputStream4 {
- def setup(setAbs: Boolean = true) = {
+ val finfo = FormatInfoForUnitTest()
+ finfo.fillByte = 0.toByte
+
+ def setup(setAbs: Boolean = true, bitOrd: BitOrder = BitOrder.MostSignificantBitFirst) = {
val baos = new ByteArrayOutputStreamWithGetBuf()
val direct = DirectOrBufferedDataOutputStream(baos, null)
- direct.setBitOrder(BitOrder.MostSignificantBitFirst)
- direct.setByteOrder(ByteOrder.BigEndian)
- direct.setFillByte(0)
+ direct.setPriorBitOrder(bitOrd)
- direct.putLong(0x5a5a5, 19)
+ direct.putLong(0x5a5a5, 19, finfo)
// X101 1010 0101 1010 0101
// 1011 0100 1011 0100 101X big endian, MSBF
@@ -59,11 +59,11 @@ class TestDataOutputStream4 {
if (setAbs)
out2.setAbsStartingBitPos0b(ULong(39))
- out.putLong(0x5a5a5, 19)
+ out.putLong(0x5a5a5, 19, finfo)
// 101 1010 0101 1010 0101
// 1 0110 1001 0110 1001 01XX
- out2.putLong(0x5a5a5, 19)
+ out2.putLong(0x5a5a5, 19, finfo)
// 101 1010 0101 1010 0101
// 10 1101 0010 1101 0010 1XXX
@@ -92,9 +92,9 @@ class TestDataOutputStream4 {
@Test def testPutLong19FinishInOrderAbs {
val (baos, direct, out, out2) = setup()
- direct.setFinished()
- out.setFinished()
- out2.setFinished()
+ direct.setFinished(finfo)
+ out.setFinished(finfo)
+ out2.setFinished(finfo)
checkResults(baos)
@@ -103,9 +103,9 @@ class TestDataOutputStream4 {
@Test def testPutLong19FinishOutOfOrder1Abs {
val (baos, direct, out, out2) = setup()
- out.setFinished()
- direct.setFinished()
- out2.setFinished()
+ out.setFinished(finfo)
+ direct.setFinished(finfo)
+ out2.setFinished(finfo)
checkResults(baos)
@@ -114,9 +114,9 @@ class TestDataOutputStream4 {
@Test def testPutLong19FinishOutOfOrder2Abs {
val (baos, direct, out, out2) = setup()
- out2.setFinished()
- out.setFinished()
- direct.setFinished()
+ out2.setFinished(finfo)
+ out.setFinished(finfo)
+ direct.setFinished(finfo)
checkResults(baos)
@@ -125,9 +125,9 @@ class TestDataOutputStream4 {
@Test def testPutLong19FinishOutOfOrder3Abs {
val (baos, direct, out, out2) = setup()
- out2.setFinished()
- direct.setFinished()
- out.setFinished()
+ out2.setFinished(finfo)
+ direct.setFinished(finfo)
+ out.setFinished(finfo)
checkResults(baos)
@@ -136,9 +136,9 @@ class TestDataOutputStream4 {
@Test def testPutLong19FinishInOrder {
val (baos, direct, out, out2) = setup(false)
- direct.setFinished()
- out.setFinished()
- out2.setFinished()
+ direct.setFinished(finfo)
+ out.setFinished(finfo)
+ out2.setFinished(finfo)
checkResults(baos)
@@ -147,9 +147,9 @@ class TestDataOutputStream4 {
@Test def testPutLong19FinishOutOfOrder1 {
val (baos, direct, out, out2) = setup(false)
- out.setFinished()
- direct.setFinished()
- out2.setFinished()
+ out.setFinished(finfo)
+ direct.setFinished(finfo)
+ out2.setFinished(finfo)
checkResults(baos)
@@ -158,9 +158,9 @@ class TestDataOutputStream4 {
@Test def testPutLong19FinishOutOfOrder2 {
val (baos, direct, out, out2) = setup(false)
- out2.setFinished()
- out.setFinished()
- direct.setFinished()
+ out2.setFinished(finfo)
+ out.setFinished(finfo)
+ direct.setFinished(finfo)
checkResults(baos)
@@ -169,9 +169,9 @@ class TestDataOutputStream4 {
@Test def testPutLong19FinishOutOfOrder3 {
val (baos, direct, out, out2) = setup(false)
- out2.setFinished()
- direct.setFinished()
- out.setFinished()
+ out2.setFinished(finfo)
+ direct.setFinished(finfo)
+ out.setFinished(finfo)
checkResults(baos)
diff --git a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDirectOrBufferedDataOutputStream.scala b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDirectOrBufferedDataOutputStream.scala
index e63bfd9..2956fcc 100644
--- a/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDirectOrBufferedDataOutputStream.scala
+++ b/daffodil-io/src/test/scala/edu/illinois/ncsa/daffodil/io/TestDirectOrBufferedDataOutputStream.scala
@@ -38,6 +38,7 @@ import junit.framework.Assert._
import java.io.ByteArrayInputStream
import java.io.InputStreamReader
import org.apache.commons.io.IOUtils
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
class TestDirectOrBufferedDataOutputStream {
@@ -49,33 +50,41 @@ class TestDirectOrBufferedDataOutputStream {
res
}
+ def newDirectOrBufferedDataOutputStream(jos: java.io.OutputStream, creator: DirectOrBufferedDataOutputStream,
+ bo: BitOrder = BitOrder.MostSignificantBitFirst) = {
+ val os = DirectOrBufferedDataOutputStream(jos, creator)
+ os.setPriorBitOrder(bo)
+ os
+ }
/**
* Tests that the toString method doesn't throw. Can't even use a debugger
* if that happens.
*/
@Test def testToStringDoesNotThrow {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val layered = DirectOrBufferedDataOutputStream(baos, null)
+ val layered = newDirectOrBufferedDataOutputStream(baos, null)
assertFalse(layered.toString().isEmpty())
}
+ val finfo = FormatInfoForUnitTest()
+
@Test def testCollapsingBufferIntoDirect1 {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val layered = DirectOrBufferedDataOutputStream(baos, null)
+ val layered = newDirectOrBufferedDataOutputStream(baos, null)
val hw = "Hello World!"
val hwBytes = hw.getBytes("ascii")
- layered.putBytes(hwBytes)
+ layered.putBytes(hwBytes, finfo)
assertEquals(hw, getString(baos))
val buf1 = layered.addBuffered
- buf1.putBytes("buf1".getBytes("ascii"))
+ buf1.putBytes("buf1".getBytes("ascii"), finfo)
- layered.setFinished() // collapses layered into buf1.
+ layered.setFinished(finfo) // collapses layered into buf1.
assertTrue(layered.isDead)
@@ -89,24 +98,24 @@ class TestDirectOrBufferedDataOutputStream {
@Test def testCollapsingFinishedBufferIntoLayered {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val layered = DirectOrBufferedDataOutputStream(baos, null)
+ val layered = newDirectOrBufferedDataOutputStream(baos, null)
val hw = "Hello World!"
val hwBytes = hw.getBytes("ascii")
- layered.putBytes(hwBytes)
+ layered.putBytes(hwBytes, finfo)
assertEquals(hw, getString(baos))
val buf1 = layered.addBuffered
- buf1.putBytes("buf1".getBytes("ascii"))
+ buf1.putBytes("buf1".getBytes("ascii"), finfo)
- buf1.setFinished()
+ buf1.setFinished(finfo)
assertTrue(buf1.isFinished)
- layered.setFinished() // collapses layered into buf1.
+ layered.setFinished(finfo) // collapses layered into buf1.
assertTrue(buf1.isDead) // because it was finished when layered was subsequently finished
assertTrue(layered.isDead)
@@ -118,28 +127,28 @@ class TestDirectOrBufferedDataOutputStream {
@Test def testCollapsingTwoBuffersIntoDirect {
val baos = new ByteArrayOutputStreamWithGetBuf()
- val layered = DirectOrBufferedDataOutputStream(baos, null)
+ val layered = newDirectOrBufferedDataOutputStream(baos, null)
val hw = "Hello World!"
val hwBytes = hw.getBytes("ascii")
- layered.putBytes(hwBytes)
+ layered.putBytes(hwBytes, finfo)
assertEquals(hw, getString(baos))
val buf1 = layered.addBuffered
val buf2 = buf1.addBuffered
- buf1.putBytes("buf1".getBytes("ascii"))
- buf2.putBytes("buf2".getBytes("ascii"))
+ buf1.putBytes("buf1".getBytes("ascii"), finfo)
+ buf2.putBytes("buf2".getBytes("ascii"), finfo)
assertTrue(buf2.isBuffering)
- buf1.setFinished() // buf1 finished while layered (before it) is still unfinished.
+ buf1.setFinished(finfo) // buf1 finished while layered (before it) is still unfinished.
assertTrue(buf1.isFinished)
- layered.setFinished() // collapses layered into buf1. Since buf1 is finished already, this melds them, outputs everything
+ layered.setFinished(finfo) // collapses layered into buf1. Since buf1 is finished already, this melds them, outputs everything
// and leaves the whole thing finished.
// leaves layered dead/unusable.
diff --git a/daffodil-lib/src/main/scala/edu/illinois/ncsa/daffodil/util/Maybe.scala b/daffodil-lib/src/main/scala/edu/illinois/ncsa/daffodil/util/Maybe.scala
index bf5036e..243e9ba 100644
--- a/daffodil-lib/src/main/scala/edu/illinois/ncsa/daffodil/util/Maybe.scala
+++ b/daffodil-lib/src/main/scala/edu/illinois/ncsa/daffodil/util/Maybe.scala
@@ -50,7 +50,8 @@ import edu.illinois.ncsa.daffodil.exceptions.Assert
final class Maybe[+T <: AnyRef](val v: AnyRef) extends AnyVal with Serializable {
@inline final def get: T = if (isDefined) value else noneGet
@inline final def value: T = v.asInstanceOf[T]
- final def noneGet = throw new NoSuchElementException("Nope.get")
+ final def noneGet =
+ throw new NoSuchElementException("Nope.get") // good place for a breakpoint
@inline final def isEmpty: Boolean = NopeValue eq v
@inline final def isDefined: Boolean = !isEmpty
diff --git a/daffodil-lib/src/main/scala/passera/unsigned/ULong.scala b/daffodil-lib/src/main/scala/passera/unsigned/ULong.scala
index 922d159..a9081a0 100644
--- a/daffodil-lib/src/main/scala/passera/unsigned/ULong.scala
+++ b/daffodil-lib/src/main/scala/passera/unsigned/ULong.scala
@@ -216,6 +216,7 @@ case class ULong(override val longValue: Long) extends AnyVal with Unsigned[ULon
object ULong {
val MinValue = ULong(0L)
+ val Zero = MinValue
val MaxValue = ULong(~0L)
val MaxValueAsBigInt = BigInt(Long.MinValue).abs
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/BinaryBooleanUnparsers.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/BinaryBooleanUnparsers.scala
index 5eba5f7..edd43f9 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/BinaryBooleanUnparsers.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/BinaryBooleanUnparsers.scala
@@ -44,6 +44,7 @@ import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.LengthKind
import edu.illinois.ncsa.daffodil.util.Numbers
import edu.illinois.ncsa.daffodil.util.MaybeULong
import passera.unsigned.ULong
+import edu.illinois.ncsa.daffodil.io.FormatInfo
class BinaryBooleanUnparser(e: ElementRuntimeData,
binaryBooleanTrueRep: MaybeULong,
@@ -73,8 +74,8 @@ class BinaryBooleanUnparser(e: ElementRuntimeData,
override lazy val runtimeDependencies = List(lengthEv)
- protected def putNumber(dos: DataOutputStream, value: ULong, nBits: Int): Boolean = {
- dos.putULong(value, nBits)
+ protected def putNumber(dos: DataOutputStream, value: ULong, nBits: Int, finfo: FormatInfo): Boolean = {
+ dos.putULong(value, nBits, finfo)
}
def unparse(state: UState): Unit = {
@@ -93,9 +94,9 @@ class BinaryBooleanUnparser(e: ElementRuntimeData,
val res =
if (value) {
val trueRep = if (binaryBooleanTrueRep.isDefined) binaryBooleanTrueRep.getULong else ~binaryBooleanFalseRep
- putNumber(dos, trueRep, nBits)
+ putNumber(dos, trueRep, nBits, state)
} else {
- putNumber(dos, binaryBooleanFalseRep, nBits)
+ putNumber(dos, binaryBooleanFalseRep, nBits, state)
}
if (!res) {
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/BinaryNumberUnparsers.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/BinaryNumberUnparsers.scala
index ae1539b..e3c6335 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/BinaryNumberUnparsers.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/BinaryNumberUnparsers.scala
@@ -44,13 +44,14 @@ import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.LengthUnits
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.YesNo
import edu.illinois.ncsa.daffodil.io.DataOutputStream
import edu.illinois.ncsa.daffodil.util.Numbers._
+import edu.illinois.ncsa.daffodil.io.FormatInfo
abstract class BinaryNumberBaseUnparser(e: ElementRuntimeData)
extends PrimUnparserObject(e) {
protected def getBitLength(s: ParseOrUnparseState): Int
- protected def putNumber(dos: DataOutputStream, number: JNumber, nBits: Int): Boolean
+ protected def putNumber(dos: DataOutputStream, number: JNumber, nBits: Int, finfo: FormatInfo): Boolean
def unparse(state: UState): Unit = {
val nBits = getBitLength(state)
@@ -60,7 +61,7 @@ abstract class BinaryNumberBaseUnparser(e: ElementRuntimeData)
val res =
if (nBits > 0) {
- putNumber(dos, value, nBits)
+ putNumber(dos, value, nBits, state)
} else {
true
}
@@ -77,11 +78,11 @@ abstract class BinaryNumberBaseUnparser(e: ElementRuntimeData)
abstract class BinaryIntegerBaseUnparser(e: ElementRuntimeData, signed: Boolean)
extends BinaryNumberBaseUnparser(e) {
- override def putNumber(dos: DataOutputStream, value: JNumber, nBits: Int): Boolean = {
+ override def putNumber(dos: DataOutputStream, value: JNumber, nBits: Int, finfo: FormatInfo): Boolean = {
if (nBits > 64) {
- dos.putBigInt(asBigInt(value), nBits, signed)
+ dos.putBigInt(asBigInt(value), nBits, signed, finfo)
} else {
- dos.putLong(asLong(value), nBits)
+ dos.putLong(asLong(value), nBits, finfo)
}
}
}
@@ -103,8 +104,8 @@ class BinaryFloatUnparser(e: ElementRuntimeData)
override def getBitLength(s: ParseOrUnparseState) = 32
- override def putNumber(dos: DataOutputStream, value: JNumber, nBits: Int): Boolean = {
- dos.putBinaryFloat(asFloat(value))
+ override def putNumber(dos: DataOutputStream, value: JNumber, nBits: Int, finfo:FormatInfo): Boolean = {
+ dos.putBinaryFloat(asFloat(value), finfo)
}
}
@@ -114,8 +115,8 @@ class BinaryDoubleUnparser(e: ElementRuntimeData)
override def getBitLength(s: ParseOrUnparseState) = 64
- override def putNumber(dos: DataOutputStream, value: JNumber, nBits: Int): Boolean = {
- dos.putBinaryDouble(asDouble(value))
+ override def putNumber(dos: DataOutputStream, value: JNumber, nBits: Int, finfo: FormatInfo): Boolean = {
+ dos.putBinaryDouble(asDouble(value), finfo)
}
}
@@ -134,7 +135,7 @@ class BinaryDecimalRuntimeLengthUnparser(val e: ElementRuntimeData, signed: YesN
abstract class BinaryDecimalUnparserBase(e: ElementRuntimeData, signed: YesNo, binaryDecimalVirtualPoint: Int)
extends BinaryNumberBaseUnparser(e) {
- override def putNumber(dos: DataOutputStream, value: JNumber, nBits: Int): Boolean = {
+ override def putNumber(dos: DataOutputStream, value: JNumber, nBits: Int, finfo: FormatInfo): Boolean = {
// We want to scale the bigInt by binaryDecimalVirtualPoint so that it is a BigInt
val bigDec = asBigDecimal(value)
@@ -148,7 +149,7 @@ abstract class BinaryDecimalUnparserBase(e: ElementRuntimeData, signed: YesNo, b
if (binaryDecimalVirtualPoint != 0) bigDec.scaleByPowerOfTen(binaryDecimalVirtualPoint).toBigInteger()
else bigDec.toBigInteger()
- dos.putBigInt(bigInt, nBits, signed == YesNo.Yes)
+ dos.putBigInt(bigInt, nBits, signed == YesNo.Yes, finfo)
}
}
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/BitOrderChangeUnparser.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/BitOrderChangeUnparser.scala
deleted file mode 100644
index 29535d8..0000000
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/BitOrderChangeUnparser.scala
+++ /dev/null
@@ -1,76 +0,0 @@
-/* Copyright (c) 2016 Tresys Technology, LLC. All rights reserved.
- *
- * Developed by: Tresys Technology, LLC
- * http://www.tresys.com
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy of
- * this software and associated documentation files (the "Software"), to deal with
- * the Software without restriction, including without limitation the rights to
- * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
- * of the Software, and to permit persons to whom the Software is furnished to do
- * so, subject to the following conditions:
- *
- * 1. Redistributions of source code must retain the above copyright notice,
- * this list of conditions and the following disclaimers.
- *
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimers in the
- * documentation and/or other materials provided with the distribution.
- *
- * 3. Neither the names of Tresys Technology, nor the names of its contributors
- * may be used to endorse or promote products derived from this Software
- * without specific prior written permission.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE
- * SOFTWARE.
- */
-
-package edu.illinois.ncsa.daffodil.processors.unparsers
-
-import edu.illinois.ncsa.daffodil.processors.CheckByteAndBitOrderEv
-import edu.illinois.ncsa.daffodil.processors.CheckBitOrderAndCharsetEv
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
-import edu.illinois.ncsa.daffodil.processors.TermRuntimeData
-import edu.illinois.ncsa.daffodil.processors.SuspendableOperation
-import edu.illinois.ncsa.daffodil.processors.SuspendableUnparser
-
-class BitOrderChangeUnparserSuspendableOperation(
- override val rd: TermRuntimeData,
- val bitOrder: BitOrder,
- val checkByteAndBitOrder: CheckByteAndBitOrderEv,
- val checkBitOrderAndCharset: CheckBitOrderAndCharsetEv)
- extends SuspendableOperation {
-
- override def test(ustate: UState) = {
- val dos = ustate.dataOutputStream
- val res = dos.maybeAbsBitPos0b.isDefined
- res
- }
-
- override def continuation(state: UState) {
- checkByteAndBitOrder(state)
- checkBitOrderAndCharset(state)
- rd.schemaDefinitionUnless(state.bitPos1b % 8 == 1, "Can only change dfdl:bitOrder on a byte boundary")
- state.dataOutputStream.setBitOrder(bitOrder)
- }
-}
-
-class BitOrderChangeUnparser(
- val rd: TermRuntimeData,
- val bitOrder: BitOrder,
- val checkByteAndBitOrder: CheckByteAndBitOrderEv,
- val checkBitOrderAndCharset: CheckBitOrderAndCharsetEv)
- extends PrimUnparserObject(rd)
- with SuspendableUnparser {
-
- override lazy val runtimeDependencies = Seq(checkByteAndBitOrder, checkBitOrderAndCharset)
-
- override def suspendableOperation = new BitOrderChangeUnparserSuspendableOperation(
- rd, bitOrder, checkByteAndBitOrder, checkBitOrderAndCharset)
-
-}
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ByteOrderChangeUnparser.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ByteOrderChangeUnparser.scala
deleted file mode 100644
index 0ac7634..0000000
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ByteOrderChangeUnparser.scala
+++ /dev/null
@@ -1,49 +0,0 @@
-/* Copyright (c) 2016 Tresys Technology, LLC. All rights reserved.
- *
- * Developed by: Tresys Technology, LLC
- * http://www.tresys.com
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy of
- * this software and associated documentation files (the "Software"), to deal with
- * the Software without restriction, including without limitation the rights to
- * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
- * of the Software, and to permit persons to whom the Software is furnished to do
- * so, subject to the following conditions:
- *
- * 1. Redistributions of source code must retain the above copyright notice,
- * this list of conditions and the following disclaimers.
- *
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimers in the
- * documentation and/or other materials provided with the distribution.
- *
- * 3. Neither the names of Tresys Technology, nor the names of its contributors
- * may be used to endorse or promote products derived from this Software
- * without specific prior written permission.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE
- * SOFTWARE.
- */
-
-package edu.illinois.ncsa.daffodil.processors.unparsers
-
-import edu.illinois.ncsa.daffodil.processors._
-
-class ByteOrderChangeUnparser(val elementRuntimeData: ElementRuntimeData, byteOrdEv: ByteOrderEv, check: CheckByteAndBitOrderEv)
- extends PrimUnparser with BinaryParserUnparserRuntimeMixin {
-
- override def context = elementRuntimeData
-
- override lazy val runtimeDependencies = Seq(byteOrdEv, check)
-
- def unparse(state: UState): Unit = {
- check(state)
- setupByteOrder(state, elementRuntimeData, byteOrdEv)
- }
-
-}
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ConvertTextBooleanUnparser.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ConvertTextBooleanUnparser.scala
index cdf034e..16e5711 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ConvertTextBooleanUnparser.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ConvertTextBooleanUnparser.scala
@@ -39,7 +39,7 @@ import java.lang.Boolean.{ TRUE => JTrue, FALSE => JFalse }
case class ConvertTextBooleanUnparser(erd: ElementRuntimeData,
textBooleanTrueRepEv: TextBooleanTrueRepEv,
textBooleanFalseRepEv: TextBooleanFalseRepEv)
- extends PrimUnparser {
+ extends TextPrimUnparser {
override def context = erd
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ConvertTextCalendarUnparser.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ConvertTextCalendarUnparser.scala
index 1bfef4b..095e0f8 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ConvertTextCalendarUnparser.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ConvertTextCalendarUnparser.scala
@@ -60,7 +60,7 @@ case class ConvertTextCalendarUnparser(erd: ElementRuntimeData,
extends ConvertTextCalendarProcessorBase(erd,
xsdType, prettyType, pattern, hasTZ, localeEv, calendarEv, infosetPattern, firstDay, calendarDaysInFirstWeek,
calendarCheckPolicy, calendarTz, tz)
- with PrimUnparser {
+ with TextPrimUnparser {
/**
* Primitive unparsers must override runtimeDependencies
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ConvertTextNumberUnparser.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ConvertTextNumberUnparser.scala
index dcaa84e..f7f99fd 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ConvertTextNumberUnparser.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ConvertTextNumberUnparser.scala
@@ -41,7 +41,7 @@ import edu.illinois.ncsa.daffodil.processors.parsers.NumberFormatFactoryBase
import edu.illinois.ncsa.daffodil.processors.parsers.ConvertTextNumberParserUnparserHelperBase
case class ConvertTextCombinatorUnparser(
- rd: RuntimeData,
+ rd: TermRuntimeData,
valueUnparser: Unparser,
converterUnparser: Unparser)
extends UnparserObject(rd) {
@@ -49,11 +49,11 @@ case class ConvertTextCombinatorUnparser(
override lazy val childProcessors = Seq(converterUnparser, valueUnparser)
override def unparse(state: UState): Unit = {
- converterUnparser.unparse1(state, rd)
+ converterUnparser.unparse1(state)
if (state.processorStatus ne Success) return
- valueUnparser.unparse1(state, rd)
+ valueUnparser.unparse1(state)
}
}
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/DelimitedUnparsers.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/DelimitedUnparsers.scala
index f3b2fd6..449adb9 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/DelimitedUnparsers.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/DelimitedUnparsers.scala
@@ -51,7 +51,7 @@ import edu.illinois.ncsa.daffodil.util.Maybe.One
sealed class StringDelimitedUnparser(erd: ElementRuntimeData,
escapeScheme: Maybe[EscapeSchemeUnparseEv],
isDelimRequired: Boolean)
- extends PrimUnparserObject(erd) with TextUnparserRuntimeMixin {
+ extends TextPrimUnparserObject(erd) {
val fieldDFA = CreateFieldDFA()
val textUnparser = new TextDelimitedUnparser(erd)
@@ -61,8 +61,6 @@ sealed class StringDelimitedUnparser(erd: ElementRuntimeData,
def unparse(state: UState): Unit = {
- setupEncoding(state, erd)
-
val schemeOpt = if (escapeScheme.isDefined) One(escapeScheme.get.evaluate(state)) else Nope
try {
@@ -73,7 +71,7 @@ sealed class StringDelimitedUnparser(erd: ElementRuntimeData,
state.withUnparserDataInputStream { dis =>
val terminatingMarkup = state.allTerminatingMarkup
- dis.reset(valueString)
+ dis.reset(valueString, state)
val scheme = schemeOpt.get
@@ -112,7 +110,7 @@ sealed class StringDelimitedUnparser(erd: ElementRuntimeData,
} else valueString // No EscapeScheme
val outStream = state.dataOutputStream
- val nCharsWritten = outStream.putString(escapedValue)
+ val nCharsWritten = outStream.putString(escapedValue, state)
if (nCharsWritten != escapedValue.length) UE(state, "%s - Too many bits in field: IndexOutOfBounds. Insufficient space to write %s characters.",
nom, escapedValue.length)
log(LogLevel.Debug, "Ended at bit position " + outStream.relBitPos0b)
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/DelimiterUnparsers.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/DelimiterUnparsers.scala
index 48c77ee..a957ec4 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/DelimiterUnparsers.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/DelimiterUnparsers.scala
@@ -42,7 +42,7 @@ import edu.illinois.ncsa.daffodil.util.Misc
import edu.illinois.ncsa.daffodil.exceptions.Assert
class DelimiterTextUnparser(erd: TermRuntimeData, delimiterType: DelimiterTextType.Type)
- extends PrimUnparserObject(erd) with TextUnparserRuntimeMixin {
+ extends TextPrimUnparserObject(erd) {
override lazy val nom = {
if (delimiterType == DelimiterTextType.Initiator) "InitiatorUnparser"
@@ -56,7 +56,6 @@ class DelimiterTextUnparser(erd: TermRuntimeData, delimiterType: DelimiterTextTy
}
def unparse(state: UState): Unit = {
- setupEncoding(state, erd)
log(LogLevel.Debug, "Unparsing starting at bit position: %s", state.dataOutputStream.maybeAbsBitPos0b)
@@ -76,7 +75,7 @@ class DelimiterTextUnparser(erd: TermRuntimeData, delimiterType: DelimiterTextTy
val valueString = delimDFA.unparseValue
val outStream = state.dataOutputStream
- val nCharsWritten = outStream.putString(valueString)
+ val nCharsWritten = outStream.putString(valueString, state)
if (nCharsWritten != valueString.length)
UE(state, "%s - Too many bits in delimiter: IndexOutOfBounds. Insufficient space to write delimiter '%s'.",
nom, Misc.remapStringToVisibleGlyphs(valueString))
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ElementKindUnparsers.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ElementKindUnparsers.scala
index 2caec71..39db1b9 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ElementKindUnparsers.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ElementKindUnparsers.scala
@@ -31,7 +31,8 @@
*/
package edu.illinois.ncsa.daffodil.processors.unparsers
-import edu.illinois.ncsa.daffodil.processors._ ; import edu.illinois.ncsa.daffodil.infoset._
+import edu.illinois.ncsa.daffodil.processors._
+import edu.illinois.ncsa.daffodil.infoset._
import edu.illinois.ncsa.daffodil.processors.RuntimeData
import edu.illinois.ncsa.daffodil.util.Maybe._
import edu.illinois.ncsa.daffodil.api.ValidationMode
@@ -52,7 +53,7 @@ class ComplexTypeUnparser(rd: RuntimeData, bodyUnparser: Unparser)
def unparse(start: UState): Unit = {
start.childIndexStack.push(1L) // one-based indexing
- bodyUnparser.unparse1(start, rd)
+ bodyUnparser.unparse1(start)
start.childIndexStack.pop()
}
}
@@ -120,7 +121,7 @@ class SequenceCombinatorUnparser(rdArg: ModelGroupRuntimeData, childUnparsers: V
}
}
if (doUnparser) {
- childUnparser.unparse1(start, childRD)
+ childUnparser.unparse1(start)
}
index += 1
//
@@ -163,7 +164,7 @@ class ChoiceCombinatorUnparser(mgrd: ModelGroupRuntimeData, eventUnparserMap: Ma
UnparseError(One(mgrd.schemaFileLocation), One(state.currentLocation), "Encountered event %s. Expected one of %s.",
key, eventUnparserMap.keys.mkString(", "))
}
- childUnparser.unparse1(state, mgrd)
+ childUnparser.unparse1(state)
}
}
@@ -181,14 +182,14 @@ class HiddenChoiceCombinatorUnparser(mgrd: ModelGroupRuntimeData, branchUnparser
override lazy val childProcessors: Seq[Processor] = Seq(branchUnparser)
def unparse(state: UState): Unit = {
- branchUnparser.unparse1(state, mgrd)
+ branchUnparser.unparse1(state)
}
}
class DelimiterStackUnparser(initiatorOpt: Maybe[InitiatorUnparseEv],
separatorOpt: Maybe[SeparatorUnparseEv],
terminatorOpt: Maybe[TerminatorUnparseEv],
- override val context: RuntimeData,
+ override val context: TermRuntimeData,
bodyUnparser: Unparser)
extends Unparser {
override def nom = "DelimiterStack"
@@ -216,13 +217,13 @@ class DelimiterStackUnparser(initiatorOpt: Maybe[InitiatorUnparseEv],
state.pushDelimiters(node)
- bodyUnparser.unparse1(state, context)
+ bodyUnparser.unparse1(state)
state.popDelimiters
}
}
-class DynamicEscapeSchemeUnparser(escapeScheme: EscapeSchemeUnparseEv, override val context: RuntimeData, bodyUnparser: Unparser)
+class DynamicEscapeSchemeUnparser(escapeScheme: EscapeSchemeUnparseEv, override val context: TermRuntimeData, bodyUnparser: Unparser)
extends Unparser {
override def nom = "EscapeSchemeStack"
@@ -239,7 +240,7 @@ class DynamicEscapeSchemeUnparser(escapeScheme: EscapeSchemeUnparseEv, override
escapeScheme.evaluate(state)
// Unparse
- bodyUnparser.unparse1(state, context)
+ bodyUnparser.unparse1(state)
// invalidate the escape scheme cache
escapeScheme.invalidateCache(state)
@@ -258,7 +259,7 @@ class ArrayCombinatorUnparser(erd: ElementRuntimeData, bodyUnparser: Unparser)
var event = state.advanceOrError
Assert.invariant(event.isStart && event.node.isInstanceOf[DIArray])
- bodyUnparser.unparse1(state, erd)
+ bodyUnparser.unparse1(state)
event = state.advanceOrError
if (!(event.isEnd && event.node.isInstanceOf[DIArray])) {
@@ -303,7 +304,7 @@ class OptionalCombinatorUnparser(erd: ElementRuntimeData, bodyUnparser: Unparser
val event = state.inspectOrError
Assert.invariant(event.isStart && !event.node.isInstanceOf[DIArray])
- bodyUnparser.unparse1(state, erd)
+ bodyUnparser.unparse1(state)
state.arrayIndexStack.pop()
state.occursBoundsStack.pop()
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ElementUnparser.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ElementUnparser.scala
index fa3e624..feddb26 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ElementUnparser.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ElementUnparser.scala
@@ -160,7 +160,7 @@ sealed abstract class ElementUnparserBase(
{
var i = 0;
while (i < setVarUnparsers.length) {
- setVarUnparsers(i).unparse1(state, context)
+ setVarUnparsers(i).unparse1(state)
i += 1
}
}
@@ -168,17 +168,17 @@ sealed abstract class ElementUnparserBase(
protected def doBeforeContentUnparser(state: UState) {
if (eBeforeUnparser.isDefined)
- eBeforeUnparser.get.unparse1(state, context)
+ eBeforeUnparser.get.unparse1(state)
}
protected def doAfterContentUnparser(state: UState) {
if (eAfterUnparser.isDefined)
- eAfterUnparser.get.unparse1(state, context)
+ eAfterUnparser.get.unparse1(state)
}
protected def runContentUnparser(state: UState) {
if (eUnparser.isDefined)
- eUnparser.get.unparse1(state, context)
+ eUnparser.get.unparse1(state)
}
override def unparse(state: UState): Unit = {
@@ -186,6 +186,7 @@ sealed abstract class ElementUnparserBase(
if (state.dataProc.isDefined) state.dataProc.value.startElement(state, this)
unparseBegin(state)
+ UnparserBitOrderChecks.checkUnparseBitOrder(state)
captureRuntimeValuedExpressionValues(state)
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/EncodingChangeUnparser.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/EncodingChangeUnparser.scala
deleted file mode 100644
index 4ca1318..0000000
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/EncodingChangeUnparser.scala
+++ /dev/null
@@ -1,50 +0,0 @@
-/* Copyright (c) 2016 Tresys Technology, LLC. All rights reserved.
- *
- * Developed by: Tresys Technology, LLC
- * http://www.tresys.com
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy of
- * this software and associated documentation files (the "Software"), to deal with
- * the Software without restriction, including without limitation the rights to
- * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
- * of the Software, and to permit persons to whom the Software is furnished to do
- * so, subject to the following conditions:
- *
- * 1. Redistributions of source code must retain the above copyright notice,
- * this list of conditions and the following disclaimers.
- *
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimers in the
- * documentation and/or other materials provided with the distribution.
- *
- * 3. Neither the names of Tresys Technology, nor the names of its contributors
- * may be used to endorse or promote products derived from this Software
- * without specific prior written permission.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE
- * SOFTWARE.
- */
-
-package edu.illinois.ncsa.daffodil.processors.unparsers
-
-import edu.illinois.ncsa.daffodil.processors.TermRuntimeData
-import edu.illinois.ncsa.daffodil.processors.CheckBitOrderAndCharsetEv
-import edu.illinois.ncsa.daffodil.processors.CheckEncodingEv
-
-class EncodingChangeUnparser(val context: TermRuntimeData, checkBitOrderAndCharset: CheckBitOrderAndCharsetEv, checkEncoding: CheckEncodingEv)
- extends PrimUnparser with TextUnparserRuntimeMixin {
-
- lazy val runtimeDependencies = Seq(context.encodingInfo.charsetEv, checkBitOrderAndCharset, checkEncoding)
-
- def unparse(state: UState): Unit = {
- checkBitOrderAndCharset(state)
- checkEncoding(state)
- setupEncoder(state, context)
- }
-
-}
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ExpressionEvaluatingUnparsers.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ExpressionEvaluatingUnparsers.scala
index 5088cff..f13332b 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ExpressionEvaluatingUnparsers.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/ExpressionEvaluatingUnparsers.scala
@@ -55,6 +55,16 @@ final class SetVariableSuspendableExpression(
override protected def maybeKnownLengthInBits(ustate: UState) = MaybeULong(0)
}
+/**
+ * Used when unparsing to evaluate dfdl:setVariable statements.
+ *
+ * TODO: Possible bug. This will allow expressions to forward reference, even
+ * when the variables are being referenced from expressions that are NOT
+ * allowed to forward reference - e.g., property value expressions such
+ * as delimiters and byte order.
+ *
+ * This forward suspension is only supposed to be allowed for dfdl:outputValueCalc.
+ */
final class SetVariableUnparser(
val expr: CompiledExpression[AnyRef],
val rd: VariableRuntimeData,
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/FramingUnparsers.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/FramingUnparsers.scala
index d136ff8..3602901 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/FramingUnparsers.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/FramingUnparsers.scala
@@ -32,30 +32,28 @@
package edu.illinois.ncsa.daffodil.processors.unparsers
-import edu.illinois.ncsa.daffodil.processors.RuntimeData
-import edu.illinois.ncsa.daffodil.processors.FillByteEv
import edu.illinois.ncsa.daffodil.processors.SuspendableOperation
import edu.illinois.ncsa.daffodil.processors.SuspendableUnparser
import edu.illinois.ncsa.daffodil.util.LogLevel
+import edu.illinois.ncsa.daffodil.processors.TextProcessor
+import edu.illinois.ncsa.daffodil.processors.TermRuntimeData
class SkipRegionUnparser(
skipInBits: Int,
- e: RuntimeData,
- fillByteEv: FillByteEv)
+ e: TermRuntimeData)
extends PrimUnparserObject(e) {
- override def runtimeDependencies = List(fillByteEv)
+ override def runtimeDependencies = Nil
override def unparse(state: UState) = {
val dos = state.dataOutputStream
- if (!dos.skip(skipInBits)) UE(state, "Unable to skip %s(bits).", skipInBits)
+ if (!dos.skip(skipInBits, state)) UE(state, "Unable to skip %s(bits).", skipInBits)
}
}
class AlignmentFillUnparserSuspendableOperation(
alignmentInBits: Int,
- override val rd: RuntimeData,
- fillByteEv: FillByteEv)
+ override val rd: TermRuntimeData)
extends SuspendableOperation {
override def test(ustate: UState) = {
@@ -68,10 +66,8 @@ class AlignmentFillUnparserSuspendableOperation(
override def continuation(state: UState) {
val dos = state.dataOutputStream
- val fb = fillByteEv.evaluate(state)
- dos.setFillByte(fb)
val b4 = dos.relBitPos0b
- if (!dos.align(alignmentInBits))
+ if (!dos.align(alignmentInBits, state))
UE(state, "Unable to align to %s(bits).", alignmentInBits)
val aft = dos.relBitPos0b
val delta = aft - b4
@@ -84,20 +80,19 @@ class AlignmentFillUnparserSuspendableOperation(
class AlignmentFillUnparser(
alignmentInBits: Int,
- val rd: RuntimeData,
- fillByteEv: FillByteEv)
+ val rd: TermRuntimeData)
extends PrimUnparserObject(rd)
with SuspendableUnparser {
- override def runtimeDependencies = List(fillByteEv)
+ override def runtimeDependencies = Nil
override def suspendableOperation =
new AlignmentFillUnparserSuspendableOperation(
- alignmentInBits, rd, fillByteEv)
+ alignmentInBits, rd)
}
class MandatoryTextAlignmentUnparser(
alignmentInBits: Int,
- e: RuntimeData,
- fillByteEv: FillByteEv)
- extends AlignmentFillUnparser(alignmentInBits, e, fillByteEv)
+ e: TermRuntimeData)
+ extends AlignmentFillUnparser(alignmentInBits, e)
+ with TextProcessor
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/HexBinaryLengthUnparser.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/HexBinaryLengthUnparser.scala
index 42d5dcb..cd3bb47 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/HexBinaryLengthUnparser.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/HexBinaryLengthUnparser.scala
@@ -34,12 +34,11 @@ package edu.illinois.ncsa.daffodil.processors.unparsers
import edu.illinois.ncsa.daffodil.processors.ElementRuntimeData
import edu.illinois.ncsa.daffodil.util.Maybe._
-import edu.illinois.ncsa.daffodil.processors.FillByteEv
import edu.illinois.ncsa.daffodil.infoset.RetryableException
import edu.illinois.ncsa.daffodil.processors.UnparseTargetLengthInBitsEv
import edu.illinois.ncsa.daffodil.exceptions.Assert
-abstract class HexBinaryUnparserBase(erd: ElementRuntimeData, fillByteEv: FillByteEv)
+abstract class HexBinaryUnparserBase(erd: ElementRuntimeData)
extends PrimUnparserObject(erd) {
protected def getLengthInBits(state: UState): Long
@@ -71,7 +70,7 @@ abstract class HexBinaryUnparserBase(erd: ElementRuntimeData, fillByteEv: FillBy
// put the hex binary array
if (bitsFromValueToPut > 0) {
- val ret = dos.putByteArray(value, bitsFromValueToPut.toInt)
+ val ret = dos.putByteArray(value, bitsFromValueToPut.toInt, state)
if (!ret) {
UnparseError(One(erd.schemaFileLocation), One(state.currentLocation), "Failed to write %d hexBinary bits", bitsFromValueToPut)
}
@@ -80,9 +79,7 @@ abstract class HexBinaryUnparserBase(erd: ElementRuntimeData, fillByteEv: FillBy
// calculate the skip bits
val nFillBits = lengthInBits - bitsFromValueToPut
if (nFillBits > 0) {
- val fillByte = fillByteEv.evaluate(state)
- dos.setFillByte(fillByte) // TODO: PEFORMANCE: this and many other settings should be set via a changeFillByte processor, so that it is always pre-set???
- val ret = dos.skip(nFillBits)
+ val ret = dos.skip(nFillBits, state)
if (!ret) {
UnparseError(Nope, One(state.currentLocation), "Failed to skip %d bits.", nFillBits)
}
@@ -90,18 +87,18 @@ abstract class HexBinaryUnparserBase(erd: ElementRuntimeData, fillByteEv: FillBy
}
}
-final class HexBinaryMinLengthInBytesUnparser(minLengthInBytes: Long, erd: ElementRuntimeData, fillByteEv: FillByteEv)
- extends HexBinaryUnparserBase(erd, fillByteEv) {
+final class HexBinaryMinLengthInBytesUnparser(minLengthInBytes: Long, erd: ElementRuntimeData)
+ extends HexBinaryUnparserBase(erd) {
override def getLengthInBits(state: UState): Long = {
val len = state.currentNode.get.asSimple.dataValue.asInstanceOf[Array[Byte]].length * 8
- val min = minLengthInBytes * 8
+ val min = minLengthInBytes * 8
scala.math.max(len, min)
}
}
-final class HexBinarySpecifiedLengthUnparser(erd: ElementRuntimeData, val lengthEv: UnparseTargetLengthInBitsEv, fillByteEv: FillByteEv)
- extends HexBinaryUnparserBase(erd, fillByteEv) {
+final class HexBinarySpecifiedLengthUnparser(erd: ElementRuntimeData, val lengthEv: UnparseTargetLengthInBitsEv)
+ extends HexBinaryUnparserBase(erd) {
override def getLengthInBits(state: UState): Long = {
val l: Long = try {
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/NilEmptyCombinatorUnparsers.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/NilEmptyCombinatorUnparsers.scala
index 3325b0b..c278dab 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/NilEmptyCombinatorUnparsers.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/NilEmptyCombinatorUnparsers.scala
@@ -51,9 +51,9 @@ case class SimpleNilOrEmptyOrValueUnparser(ctxt: ElementRuntimeData,
// suspends, this call to isNilled will throw a InfosetNoDataException
// because _isNilled has not been set yet. Rather than having to deal with
// suspending, only check isNilled for non-OVC elements.
- if (ctxt.outputValueCalcExpr.isEmpty && inode.isNilled) nilUnparser.unparse(state)
- else if (inode.isEmpty) emptyUnparser.unparse(state)
- else valueUnparser.unparse(state)
+ if (ctxt.outputValueCalcExpr.isEmpty && inode.isNilled) nilUnparser.unparse1(state)
+ else if (inode.isEmpty) emptyUnparser.unparse1(state)
+ else valueUnparser.unparse1(state)
}
}
@@ -66,8 +66,8 @@ case class SimpleNilOrValueUnparser(ctxt: ElementRuntimeData,
Assert.invariant(Maybe.WithNulls.isDefined(state.currentInfosetNode))
val inode = state.currentInfosetNode.asSimple
// see comment above for why this OVC check is necessary
- if (ctxt.outputValueCalcExpr.isEmpty && inode.isNilled) nilUnparser.unparse(state)
- else valueUnparser.unparse(state)
+ if (ctxt.outputValueCalcExpr.isEmpty && inode.isNilled) nilUnparser.unparse1(state)
+ else valueUnparser.unparse1(state)
}
}
@@ -79,8 +79,8 @@ case class SimpleEmptyOrValueUnparser(ctxt: ElementRuntimeData,
def unparse(state: UState): Unit = {
Assert.invariant(Maybe.WithNulls.isDefined(state.currentInfosetNode))
val inode = state.currentInfosetNode.asSimple
- if (inode.isEmpty) emptyUnparser.unparse(state)
- else valueUnparser.unparse(state)
+ if (inode.isEmpty) emptyUnparser.unparse1(state)
+ else valueUnparser.unparse1(state)
}
}
@@ -92,7 +92,7 @@ case class ComplexNilOrContentUnparser(ctxt: ElementRuntimeData,
def unparse(state: UState): Unit = {
Assert.invariant(Maybe.WithNulls.isDefined(state.currentInfosetNode))
val inode = state.currentInfosetNode.asComplex
- if (inode.isNilled) nilUnparser.unparse(state)
- else contentUnparser.unparse(state)
+ if (inode.isNilled) nilUnparser.unparse1(state)
+ else contentUnparser.unparse1(state)
}
}
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/OptionalInfixSepUnparser.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/OptionalInfixSepUnparser.scala
index 0129fdd..1492f66 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/OptionalInfixSepUnparser.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/OptionalInfixSepUnparser.scala
@@ -32,15 +32,15 @@
package edu.illinois.ncsa.daffodil.processors.unparsers
-import edu.illinois.ncsa.daffodil.processors.RuntimeData
+import edu.illinois.ncsa.daffodil.processors.TermRuntimeData
-class OptionalInfixSepUnparser(contextArg: RuntimeData,
+class OptionalInfixSepUnparser(contextArg: TermRuntimeData,
sepUnparser: Unparser)
extends UnparserObject(contextArg) {
override lazy val childProcessors = List(sepUnparser)
- def unparse(state: UState): Unit = {
+ override def unparse(state: UState): Unit = {
if (state.arrayPos > 1) sepUnparser.unparse1(state, contextArg)
else if (state.groupPos > 1) sepUnparser.unparse1(state, contextArg)
}
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/SpecifiedLength2.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/SpecifiedLength2.scala
index e148871..efdd9db 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/SpecifiedLength2.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/SpecifiedLength2.scala
@@ -37,7 +37,6 @@ import edu.illinois.ncsa.daffodil.infoset.DIComplex
import edu.illinois.ncsa.daffodil.infoset.DIElement
import edu.illinois.ncsa.daffodil.infoset.DISimple
import edu.illinois.ncsa.daffodil.processors.ElementRuntimeData
-import edu.illinois.ncsa.daffodil.processors.FillByteEv
import edu.illinois.ncsa.daffodil.processors.SuspendableOperation
import edu.illinois.ncsa.daffodil.processors.UnparseTargetLengthInBitsEv
import edu.illinois.ncsa.daffodil.processors.charset.DFDLCharset
@@ -190,6 +189,15 @@ class OVCRetryUnparserSuspendableOperation(override val rd: ElementRuntimeData,
// knownLengthInBits. This is because even if an OVC element has fixed
// length, the result of the OVC might not actually write that many bits,
// relying on padding and/or right fill to fill in the remaining bits
+ //
+ // TODO: The above is too pessimistic. Many formats have no notion of
+ // padding nor filling, so the length could be computed from the targetLengthInBitsEv
+ // just fine
+ //
+ // We could make it the responsibility of the caller of this to supply or not
+ // the maybeUnparserTargetLengthInBitsEv depending on whether it can be
+ // depended upon or not.
+ //
MaybeULong.Nope
}
@@ -198,7 +206,7 @@ class OVCRetryUnparserSuspendableOperation(override val rd: ElementRuntimeData,
}
protected def continuation(state: UState) {
- vUnparser.unparse1(state, rd)
+ vUnparser.unparse1(state)
}
}
@@ -287,6 +295,12 @@ class CaptureEndOfValueLengthUnparser(override val context: ElementRuntimeData)
/**
* Carries out computation of the target length for a specified-length element.
+ *
+ * This is not a SuspendableExpression because the dfdl:length property cannot
+ * be forward referencing. However, it can refer backward to elements that have
+ * dfdl:outputValueCalc or variables that have not yet been computed. So we have
+ * to retry this in order to get the target length used to compute the amount of
+ * padding or the amount of unused space.
*/
class TargetLengthOperation(override val rd: ElementRuntimeData,
targetLengthEv: UnparseTargetLengthInBitsEv)
@@ -303,6 +317,9 @@ class TargetLengthOperation(override val rd: ElementRuntimeData,
override protected def maybeKnownLengthInBits(ustate: UState): MaybeULong = MaybeULong(0L)
override def test(ustate: UState): Boolean = {
+ //
+ // regular evaluation - can only look backwards
+ //
targetLengthEv.evaluate(ustate) // can we successfully evaluate without blocking (blocking would throw)
true
}
@@ -397,8 +414,7 @@ class ElementUnusedUnparserSuspendableOperation(
override val targetLengthEv: UnparseTargetLengthInBitsEv,
override val maybeLengthEv: Maybe[LengthEv],
override val maybeCharsetEv: Maybe[CharsetEv],
- override val maybeLiteralNilEv: Maybe[NilStringLiteralForUnparserEv],
- fillByteEv: FillByteEv)
+ override val maybeLiteralNilEv: Maybe[NilStringLiteralForUnparserEv])
extends SuspendableOperation
with NeedValueAndTargetLengthMixin {
@@ -417,9 +433,7 @@ class ElementUnusedUnparserSuspendableOperation(
protected final def skipTheBits(ustate: UState, skipInBits: Long) {
if (skipInBits > 0) {
val dos = ustate.dataOutputStream
- val fb = fillByteEv.evaluate(ustate)
- dos.setFillByte(fb)
- if (!dos.skip(skipInBits))
+ if (!dos.skip(skipInBits, ustate))
UE(ustate, "Unable to skip %s(bits).", skipInBits)
}
if (skipInBits == 0) {
@@ -438,16 +452,15 @@ class ElementUnusedUnparser(
targetLengthEv: UnparseTargetLengthInBitsEv,
maybeLengthEv: Maybe[LengthEv],
maybeCharsetEv: Maybe[CharsetEv],
- maybeLiteralNilEv: Maybe[NilStringLiteralForUnparserEv],
- fillByteEv: FillByteEv)
+ maybeLiteralNilEv: Maybe[NilStringLiteralForUnparserEv])
extends PrimUnparserObject(rd)
with SuspendableUnparser {
- override lazy val runtimeDependencies = List(targetLengthEv, fillByteEv)
+ override lazy val runtimeDependencies = List(targetLengthEv)
override def suspendableOperation =
new ElementUnusedUnparserSuspendableOperation(
- rd, targetLengthEv, maybeLengthEv, maybeCharsetEv, maybeLiteralNilEv, fillByteEv)
+ rd, targetLengthEv, maybeLengthEv, maybeCharsetEv, maybeLiteralNilEv)
}
@@ -486,15 +499,12 @@ trait PaddingUnparserMixin
val nChars = numPadChars(skipInBits, cs.padCharWidthInBits)
if (nChars > 0) {
val dos = state.dataOutputStream
- // TODO: Performance - do we need to do this?
- dos.setEncoder(state.getEncoder(cs.charset))
- // TODO: Performance - do this better. No reason to be doing this one character at a time
var i = 0
val padChar = maybePadChar.get
val padString = padChar.toString
while (i < nChars) {
try {
- if (dos.putString(padString) != 1)
+ if (dos.putString(padString, state) != 1)
UE(state, "Unable to output %s %s characters.", nChars, charsKind)
} catch {
case m: MalformedInputException => { UnparseError(One(self.rd.schemaFileLocation), One(state.currentLocation), "MalformedInputException: \n%s", m.getMessage()) }
@@ -525,7 +535,7 @@ class OnlyPaddingUnparser(
maybeCharsetEv: Maybe[CharsetEv],
maybeLiteralNilEv: Maybe[NilStringLiteralForUnparserEv],
maybePadChar: MaybeChar)
- extends PrimUnparserObject(rd)
+ extends TextPrimUnparserObject(rd)
with SuspendableUnparser {
override lazy val runtimeDependencies = List(targetLengthEv)
@@ -575,7 +585,7 @@ class NilLiteralCharacterUnparser(
val maybeLengthEv: Maybe[LengthEv],
val maybeCharsetEv: Maybe[CharsetEv],
literalNilChar: Char)
- extends PrimUnparserObject(rd)
+ extends TextPrimUnparserObject(rd)
with SuspendableUnparser {
override lazy val runtimeDependencies = List(targetLengthEv)
@@ -648,9 +658,8 @@ class RightFillUnparserSuspendableOperation(
maybeLengthEv: Maybe[LengthEv],
maybeCharsetEv: Maybe[CharsetEv],
maybeLiteralNilEv: Maybe[NilStringLiteralForUnparserEv],
- fillByteEv: FillByteEv,
override val maybePadChar: MaybeChar)
- extends ElementUnusedUnparserSuspendableOperation(rd, targetLengthEv, maybeLengthEv, maybeCharsetEv, maybeLiteralNilEv, fillByteEv)
+ extends ElementUnusedUnparserSuspendableOperation(rd, targetLengthEv, maybeLengthEv, maybeCharsetEv, maybeLiteralNilEv)
with PaddingUnparserMixin {
override def continuation(state: UState) {
@@ -679,13 +688,12 @@ class RightFillUnparser(
maybeLengthEv: Maybe[LengthEv],
maybeCharsetEv: Maybe[CharsetEv],
maybeLiteralNilEv: Maybe[NilStringLiteralForUnparserEv],
- fillByteEv: FillByteEv,
val maybePadChar: MaybeChar)
- extends ElementUnusedUnparser(rd, targetLengthEv, maybeLengthEv, maybeCharsetEv, maybeLiteralNilEv, fillByteEv) {
+ extends ElementUnusedUnparser(rd, targetLengthEv, maybeLengthEv, maybeCharsetEv, maybeLiteralNilEv) {
override def suspendableOperation =
new RightFillUnparserSuspendableOperation(
- rd, targetLengthEv, maybeLengthEv, maybeCharsetEv, maybeLiteralNilEv, fillByteEv, maybePadChar)
+ rd, targetLengthEv, maybeLengthEv, maybeCharsetEv, maybeLiteralNilEv, maybePadChar)
}
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/SpecifiedLengthUnparsers.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/SpecifiedLengthUnparsers.scala
index 64e0342..2496616 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/SpecifiedLengthUnparsers.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/SpecifiedLengthUnparsers.scala
@@ -65,7 +65,7 @@ final class SpecifiedLengthExplicitImplicitUnparser(
erd: ElementRuntimeData,
targetLengthInBitsEv: UnparseTargetLengthInBitsEv,
maybeTargetLengthInCharactersEv: Maybe[UnparseTargetLengthInCharactersEv])
- extends UnparserObject(erd) with TextUnparserRuntimeMixin {
+ extends UnparserObject(erd) {
override lazy val childProcessors = Seq(eUnparser)
@@ -133,7 +133,7 @@ final class SpecifiedLengthExplicitImplicitUnparser(
//
diSimple.overwriteDataValue(newV)
}
- eUnparser.unparse1(state, erd)
+ eUnparser.unparse1(state)
} else {
// target length unknown
// ignore constraining the output length. Just unparse it.
@@ -143,7 +143,7 @@ final class SpecifiedLengthExplicitImplicitUnparser(
// on this element.
// This breaks the chicken-egg cycle.
//
- eUnparser.unparse1(state, erd)
+ eUnparser.unparse1(state)
}
}
diff --git a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/StringLengthUnparsers.scala b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/StringLengthUnparsers.scala
index bdf83d2..9da3816 100644
--- a/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/StringLengthUnparsers.scala
+++ b/daffodil-runtime1-unparser/src/main/scala/edu/illinois/ncsa/daffodil/processors/unparsers/StringLengthUnparsers.scala
@@ -46,7 +46,7 @@ import java.nio.charset.UnmappableCharacterException
sealed abstract class StringSpecifiedLengthUnparserBase(
val erd: ElementRuntimeData)
- extends PrimUnparser {
+ extends TextPrimUnparser {
override def context = erd
@@ -68,7 +68,7 @@ class StringNoTruncateUnparser(
val dos = state.dataOutputStream
val valueToWrite = contentString(state)
val nCharsWritten = try {
- dos.putString(valueToWrite)
+ dos.putString(valueToWrite, state)
} catch {
case m: MalformedInputException => { UE(state, "%s - MalformedInputException: \n%s", nom, m.getMessage()) }
case u: UnmappableCharacterException => { UE(state, "%s - UnmappableCharacterException: \n%s", nom, u.getMessage()) }
@@ -162,8 +162,7 @@ class StringMaybeTruncateBitsUnparser(
//
state.withByteArrayOutputStream {
case (_, dos) =>
- dos.setEncoder(state.dataOutputStream.encoder)
- val nChars = dos.putString(str)
+ val nChars = dos.putString(str, state)
val nBits = dos.relBitPos0b.toLong
(nBits, nChars)
}
@@ -221,7 +220,7 @@ class StringMaybeTruncateBitsUnparser(
truncatedValue
}
- val nCharsWritten = dos.putString(valueToWrite)
+ val nCharsWritten = dos.putString(valueToWrite, state)
Assert.invariant(nCharsWritten == valueToWrite.length) // assertion because we figured this out above based on available space.
//
// Filling of unused bits is done elsewhere now
@@ -275,7 +274,7 @@ class StringMaybeTruncateCharactersUnparser(
}
}
//
- val nCharsWritten = dos.putString(valueToWrite)
+ val nCharsWritten = dos.putString(valueToWrite, state)
Assert.invariant(nCharsWritten == valueToWrite.length)
}
}
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/debugger/InteractiveDebugger.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/debugger/InteractiveDebugger.scala
index b3b5eab..b75f396 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/debugger/InteractiveDebugger.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/debugger/InteractiveDebugger.scala
@@ -34,7 +34,8 @@ package edu.illinois.ncsa.daffodil.debugger
import edu.illinois.ncsa.daffodil.exceptions.Assert
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.Representation
-import edu.illinois.ncsa.daffodil.processors._ ; import edu.illinois.ncsa.daffodil.infoset._
+import edu.illinois.ncsa.daffodil.processors._
+import edu.illinois.ncsa.daffodil.infoset._
import edu.illinois.ncsa.daffodil.processors.parsers._
import edu.illinois.ncsa.daffodil.xml.XMLUtils
import edu.illinois.ncsa.daffodil.xml.GlobalQName
@@ -61,7 +62,6 @@ import edu.illinois.ncsa.daffodil.dsom.RuntimeSchemaDefinitionError
import edu.illinois.ncsa.daffodil.util.Misc
import edu.illinois.ncsa.daffodil.infoset.InfosetItem
import edu.illinois.ncsa.daffodil.infoset.InfosetElement
-import edu.illinois.ncsa.daffodil.infoset.InfosetDocument
import edu.illinois.ncsa.daffodil.infoset.XMLTextInfosetOutputter
import edu.illinois.ncsa.daffodil.processors.parsers.ConvertTextCombinatorParser
@@ -1033,10 +1033,7 @@ class InteractiveDebugger(runner: InteractiveDebuggerRunner, eCompilers: Express
debugPrintln("eval: There is no infoset currently.")
return DebugState.Pause
}
- val element = state.infoset match {
- case e: InfosetElement => e
- case d: InfosetDocument => d.getRootElement()
- }
+ val element = state.infoset
// this adjustment is so that automatic display of ".." doesn't fail
// for the root element.
val adjustedExpression =
@@ -1078,9 +1075,24 @@ class InteractiveDebugger(runner: InteractiveDebuggerRunner, eCompilers: Express
}
}
case s: scala.util.control.ControlThrowable => throw s
- // case e: ExpressionEvaluationException => println(e)
- // case e: InfosetException => println(e)
- // case e: VariableException => println(e)
+ //
+ // If we eval(.) on a node that has no value, we get a RSDE thrown.
+ //
+ // Users (such as tests in daffodil's cli module) can set up a 'display eval (.)' and then
+ // single steps until they start parsing an element which has no value.
+ // That will throw this RSDE. If we recognize this situation, we
+ // display the empty element.
+ //
+ case r: RuntimeSchemaDefinitionError if r.getCause() ne null => r.getCause() match {
+ case nd: InfosetNoDataException => {
+ //
+ // Displays the empty element since it has no value.
+ //
+ debugPrettyPrintXML(nd.diElement)
+ state.suppressDiagnosticAndSucceed(r)
+ }
+ case _ => throw r
+ }
case e: Throwable => {
val ex = e // just so we can see it in the debugger.
throw new DebugException("expression evaluation failed: %s".format(Misc.getSomeMessage(ex).get))
@@ -1314,7 +1326,7 @@ class InteractiveDebugger(runner: InteractiveDebuggerRunner, eCompilers: Express
} else {
if (state.hasInfoset) {
state.infoset match {
- case e: DIElement => Some(e.runtimeData.impliedRepresentation)
+ case e: DIElement => Some(e.erd.impliedRepresentation)
}
} else {
None
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/dpath/SuspendableExpression.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/dpath/SuspendableExpression.scala
index 72b2d85..67b7781 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/dpath/SuspendableExpression.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/dpath/SuspendableExpression.scala
@@ -40,9 +40,17 @@ import edu.illinois.ncsa.daffodil.util.Maybe._
import edu.illinois.ncsa.daffodil.processors.Suspension
import edu.illinois.ncsa.daffodil.util.LogLevel
+/**
+ * Base for unparse-time expression evaluation that can have forward reference.
+ * There are only two such cases, which is dfdl:outputValueCalc, and
+ * dfdl:setVariable expressions (which variables are in-turn used by
+ * dfdl:outputValueCalc.
+ */
trait SuspendableExpression
extends Suspension {
+ override val isReadOnly = true
+
protected def expr: CompiledExpression[AnyRef]
override def toString = "SuspendableExpression(" + rd.diagnosticDebugName + ", expr=" + expr.prettyExpr + ")"
@@ -71,29 +79,5 @@ trait SuspendableExpression
}
}
- // def run(ustate: UState) {
- // val tst =
- // try {
- // Assert.invariant(ustate.dState.mode eq UnparserBlocking)
- // ustate.dState.setMode(UnparserNonBlocking) // temporarily set to just test for blocking
- // val result = Maybe(expr.evaluate(ustate))
- // if (result.isDefined) {
- // processExpressionResult(ustate, result.get)
- // true
- // } else false
- // } catch {
- // case _: RetryableException =>
- // false
- // } finally {
- // ustate.dState.setMode(UnparserBlocking) // restore invariant.
- // }
- // if (tst) {
- // // nothing. We're done. Don't need the task object.
- // } else {
- // val mkl = maybeKnownLengthInBits(ustate)
- // setup(ustate, mkl)
- // }
- // }
-
}
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/infoset/InfosetImpl.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/infoset/InfosetImpl.scala
index 19b7f7b..1458114 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/infoset/InfosetImpl.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/infoset/InfosetImpl.scala
@@ -248,6 +248,7 @@ sealed trait DITerm {
}
def trd: TermRuntimeData
+ final def termRuntimeData = trd
protected final def dafPrefix = {
val ee = trd.dpathCompileInfo.immediateEnclosingCompileInfo.getOrElse {
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/infoset/InfosetInputter.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/infoset/InfosetInputter.scala
index 32ec52b..40e278e 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/infoset/InfosetInputter.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/infoset/InfosetInputter.scala
@@ -434,7 +434,7 @@ class InfosetAccessor private (var kind: InfosetEventKind, var node: DINode) ext
def namedQName = node.namedQName
def erd: ElementRuntimeData = node match {
case a: DIArray => a.parent.runtimeData
- case e: DIElement => e.runtimeData
+ case e: DIElement => e.erd
}
override def toString = {
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/EncodingRuntimeData.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/EncodingRuntimeData.scala
index 50c690a..35bf367 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/EncodingRuntimeData.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/EncodingRuntimeData.scala
@@ -32,9 +32,6 @@
package edu.illinois.ncsa.daffodil.processors
-import java.nio.charset.CharsetDecoder
-import java.nio.charset.CharsetEncoder
-
import edu.illinois.ncsa.daffodil.dsom._
import edu.illinois.ncsa.daffodil.exceptions.Assert
import edu.illinois.ncsa.daffodil.exceptions.SchemaFileLocation
@@ -46,6 +43,7 @@ import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.UTF16Width
import edu.illinois.ncsa.daffodil.util.PreSerialization
import edu.illinois.ncsa.daffodil.util.TransientParam
import java.nio.charset.StandardCharsets
+import edu.illinois.ncsa.daffodil.util.Maybe
/**
* To eliminate circularities between RuntimeData objects and the
@@ -141,8 +139,9 @@ trait KnownEncodingMixin { self: ThrowsSDE =>
final class EncodingRuntimeData(
@TransientParam termRuntimeDataArg: => TermRuntimeData,
@TransientParam charsetEvArg: => CharsetEv,
+ @TransientParam checkEncodingEvArg: => CheckEncodingEv,
override val schemaFileLocation: SchemaFileLocation,
- val optionUTF16Width: Option[UTF16Width],
+ optionUTF16WidthArg: Option[UTF16Width],
val defaultEncodingErrorPolicy: EncodingErrorPolicy,
val summaryEncoding: EncodingLattice,
val isKnownEncoding: Boolean,
@@ -150,29 +149,41 @@ final class EncodingRuntimeData(
override val knownEncodingAlignmentInBits: Int)
extends KnownEncodingMixin with ImplementsThrowsSDE with PreSerialization {
+ private val optionUTF16Width_ = optionUTF16WidthArg
+ private val maybeUTF16Width_ = Maybe.toMaybe[UTF16Width](optionUTF16WidthArg)
+
+ @deprecated("20170530", "Use maybeUTF16Width instead.")
+ def optionUTF16Width = optionUTF16Width_
+ def maybeUTF16Width = maybeUTF16Width_
+
lazy val termRuntimeData = termRuntimeDataArg
lazy val charsetEv = charsetEvArg
+ lazy val checkEncodingEv = checkEncodingEvArg
- lazy val runtimeDependencies = List(charsetEv)
+ lazy val runtimeDependencies = List(charsetEv, checkEncodingEv)
- def getDecoder(state: ParseOrUnparseState): CharsetDecoder = {
+ def getDecoderInfo(state: ParseOrUnparseState) = {
+ checkEncodingEv.evaluate(state)
val cs = charsetEv.evaluate(state)
- val dec = state.getDecoder(cs.charset)
+ val dec = state.getDecoderInfo(cs.charset)
dec
}
- def getEncoder(state: ParseOrUnparseState): CharsetEncoder = {
+ def getEncoderInfo(state: ParseOrUnparseState) = {
+ checkEncodingEv.evaluate(state)
val cs = charsetEv.evaluate(state)
- val enc = state.getEncoder(cs.charset)
+ val enc = state.getEncoderInfo(cs.charset)
enc
}
- def getEncoder(state: ParseOrUnparseState, dcs: DFDLCharset): CharsetEncoder = {
+ def getEncoder(state: ParseOrUnparseState, dcs: DFDLCharset) = {
+ checkEncodingEv.evaluate(state)
val enc = state.getEncoder(dcs.charset)
enc
}
def getDFDLCharset(state: ParseOrUnparseState): DFDLCharset = {
+ checkEncodingEv.evaluate(state)
val cs = charsetEv.evaluate(state)
cs
}
@@ -181,6 +192,7 @@ final class EncodingRuntimeData(
super.preSerialization
termRuntimeData
charsetEv
+ checkEncodingEv
}
@throws(classOf[java.io.IOException])
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/EvByteOrder.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/EvByteOrder.scala
index 36994ec..cf4ccaf 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/EvByteOrder.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/EvByteOrder.scala
@@ -34,14 +34,13 @@ package edu.illinois.ncsa.daffodil.processors
import edu.illinois.ncsa.daffodil.schema.annotation.props.gen._
import edu.illinois.ncsa.daffodil.dsom._
-import edu.illinois.ncsa.daffodil.util.Maybe
import edu.illinois.ncsa.daffodil.equality._
import edu.illinois.ncsa.daffodil.io.NonByteSizeCharset
/**
* Runtime valued properties that are enums would all work like ByteOrder here.
*/
-class ByteOrderEv(expr: CompiledExpression[String], erd: ElementRuntimeData)
+class ByteOrderEv(override val expr: CompiledExpression[String], erd: ElementRuntimeData)
extends EvaluatableConvertedExpression[String, ByteOrder](
expr,
ByteOrder,
@@ -60,24 +59,29 @@ class Ok private () extends Serializable {
}
object Ok extends Ok()
-class CheckByteAndBitOrderEv(t: TermRuntimeData, bitOrder: BitOrder, maybeByteOrder: Maybe[ByteOrderEv])
+class CheckByteAndBitOrderEv(t: TermRuntimeData, bitOrder: BitOrder)
extends Evaluatable[Ok](t)
with InfosetCachedEvaluatable[Ok] { // can't use unit here, not <: AnyRef
- override lazy val runtimeDependencies = maybeByteOrder.toList
+ override lazy val runtimeDependencies = Nil
- final protected def compute(state: ParseOrUnparseState): Ok = {
- if (maybeByteOrder.isEmpty) return Ok
- val byteOrderEv = maybeByteOrder.get
- val byteOrder = byteOrderEv.evaluate(state)
- bitOrder match {
- case BitOrder.MostSignificantBitFirst => // ok
- case BitOrder.LeastSignificantBitFirst =>
- if (byteOrder =:= ByteOrder.BigEndian) {
- t.schemaDefinitionError("Bit order 'leastSignificantBitFirst' requires byte order 'littleEndian', but byte order was '%s'.", byteOrder)
+ override final protected def compute(state: ParseOrUnparseState): Ok = {
+ t match {
+ case erd: ElementRuntimeData => {
+ if (erd.maybeByteOrderEv.isDefined) {
+ val byteOrderEv = erd.maybeByteOrderEv.get
+ val byteOrder = byteOrderEv.evaluate(state)
+ bitOrder match {
+ case BitOrder.MostSignificantBitFirst => // ok
+ case BitOrder.LeastSignificantBitFirst =>
+ if (byteOrder =:= ByteOrder.BigEndian) {
+ t.schemaDefinitionError("Bit order 'leastSignificantBitFirst' requires byte order 'littleEndian', but byte order was '%s'.", byteOrder)
+ }
+ }
}
+ }
+ case _ => // ok. No checks required.
}
-
Ok
}
}
@@ -88,7 +92,7 @@ class CheckBitOrderAndCharsetEv(t: TermRuntimeData, bitOrder: BitOrder, charsetE
override lazy val runtimeDependencies = List(charsetEv)
- final protected def compute(state: ParseOrUnparseState): Ok = {
+ override final protected def compute(state: ParseOrUnparseState): Ok = {
val dfdlCS = charsetEv.evaluate(state)
dfdlCS.charset match {
case nbsc: NonByteSizeCharset =>
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/EvEncoding.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/EvEncoding.scala
index e5f717b..0be70e7 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/EvEncoding.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/EvEncoding.scala
@@ -110,10 +110,12 @@ class FillByteEv(fillByteRaw: String, charsetEv: CharsetEv, val trd: TermRuntime
private val maybeSingleRawByteValue: MaybeInt = {
val RawByte = """\%\#r([0-9a-fA-F]{2})\;""".r
- fillByteRaw match {
- case RawByte(hex) => MaybeInt(Integer.parseInt(hex, 16))
+ val mfb = fillByteRaw match {
+ case RawByte(hex) =>
+ MaybeInt(Integer.parseInt(hex, 16))
case _ => MaybeInt.Nope
}
+ mfb
}
override protected def compute(state: ParseOrUnparseState): Integer = {
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/ProcessorBases.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/ProcessorBases.scala
index d17d8a8..a9d393b 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/ProcessorBases.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/ProcessorBases.scala
@@ -64,7 +64,7 @@ trait Processor
with Serializable {
// things common to both unparser and parser go here.
def context: RuntimeData
- def childProcessors: Seq[Processor]
+ override def childProcessors: Seq[Processor]
def runtimeDependencies: Seq[Evaluatable[AnyRef]]
var isInitialized: Boolean = false
@@ -74,16 +74,8 @@ trait PrimProcessor extends Processor {
override def childProcessors: Seq[Processor] = Nil
}
-trait BinaryParserUnparserRuntimeMixin {
-
- final protected def setupByteOrder(state: ParseOrUnparseState, erd: ElementRuntimeData, byteOrdEv: ByteOrderEv) {
- if (state.dataStream.isEmpty) return
- val dis = state.dataStream.get
- val byteOrd = byteOrdEv.evaluate(state)
- dis.setByteOrder(byteOrd)
- dis.setBitOrder(erd.defaultBitOrder)
- }
-}
+/** must mixin to all processors that deal with text */
+trait TextProcessor
/**
* BriefXML is XML-style output, but intended for specific purposes. It is NOT
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/ProcessorStateBases.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/ProcessorStateBases.scala
index a4dbc4e..85e9fee 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/ProcessorStateBases.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/ProcessorStateBases.scala
@@ -58,6 +58,18 @@ import edu.illinois.ncsa.daffodil.util.Maybe
import edu.illinois.ncsa.daffodil.exceptions.ThrowsSDE
import edu.illinois.ncsa.daffodil.exceptions.SavesErrorsAndWarnings
import edu.illinois.ncsa.daffodil.infoset._
+import edu.illinois.ncsa.daffodil.processors.charset.EncoderInfo
+import edu.illinois.ncsa.daffodil.processors.charset.DecoderInfo
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
+import edu.illinois.ncsa.daffodil.io.FormatInfo
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BinaryFloatRep
+import edu.illinois.ncsa.daffodil.util.MaybeInt
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.UTF16Width
+import edu.illinois.ncsa.daffodil.processors.charset.CoderInfo
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.EncodingErrorPolicy
+import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.ByteOrder
+import java.nio.charset.CharsetDecoder
+import edu.illinois.ncsa.daffodil.io.NonByteSizeCharset
/**
* Trait mixed into the PState.Mark object class and the ParseOrUnparseState
@@ -83,6 +95,30 @@ case class TupleForDebugger(
val bitLimit0b: MaybeULong,
override val discriminator: Boolean) extends StateForDebugger
+trait SetProcessorMixin {
+ private var maybeProcessor_ : Maybe[Processor] = Nope
+
+ final def maybeProcessor = maybeProcessor_
+
+ final def processor = {
+ Assert.usage(maybeProcessor_.isDefined) // failure means setProcessor wasn't called.
+ maybeProcessor_.value
+ }
+
+ /**
+ * Must be called on the state before any
+ * call to the I/O layer, as that will call back to get property-related
+ * information and that has to be obtained via the runtime data for the
+ * term - whether element or model group. The
+ */
+ final def setProcessor(p: Processor) {
+ maybeProcessor_ = One(p)
+ }
+
+ final def setMaybeProcessor(mp: Maybe[Processor]) {
+ maybeProcessor_ = mp
+ }
+}
/**
* A parser takes a state, and returns an updated state
*
@@ -104,11 +140,118 @@ abstract class ParseOrUnparseState protected (
with SavesErrorsAndWarnings
with LocalBufferMixin
with EncoderDecoderMixin
- with Logging {
+ with Logging
+ with FormatInfo
+ with SetProcessorMixin {
def this(vmap: VariableMap, diags: List[Diagnostic], dataProc: Maybe[DataProcessor], tunable: DaffodilTunables) =
this(new VariableBox(vmap), diags, dataProc, tunable)
+ def infoset: DIElement
+
+ private def simpleElement = infoset.asInstanceOf[DISimple]
+
+ /*
+ * Implement the FormatInfo trait needed by the I/O layer.
+ */
+ final def replacingDecoder: CharsetDecoder = decoderEntry.replacingCoder
+ final def reportingDecoder: CharsetDecoder = decoderEntry.reportingCoder
+ final def binaryFloatRep: BinaryFloatRep = simpleElement.erd.maybeBinaryFloatRepEv.get.evaluate(this)
+
+ private def runtimeData = processor.context
+ private def termRuntimeData = runtimeData.asInstanceOf[TermRuntimeData]
+
+ /**
+ * Returns bit order. If text, this is the bit order for the character set
+ * encoding. If binary, this is the bitOrder property value.
+ */
+ final def bitOrder: BitOrder = {
+ val res = processor match {
+ case txtProc: TextProcessor =>
+ encoder.charset() match {
+ case nbs: NonByteSizeCharset => nbs.requiredBitOrder
+ case _ => BitOrder.MostSignificantBitFirst
+ }
+ case _ => processor.context match {
+ case trd: TermRuntimeData => trd.defaultBitOrder
+ case ntrd: NonTermRuntimeData =>
+ Assert.usageError("Cannot ask for bitOrder for non-terms - NonTermRuntimeData: " + ntrd)
+ }
+ }
+ res
+ }
+
+ final def byteOrder: ByteOrder = {
+ runtimeData match {
+ case erd: ElementRuntimeData => erd.maybeByteOrderEv.get.evaluate(this)
+ case mgrd: ModelGroupRuntimeData => {
+ //
+ // Model Groups can't have byte order.
+ // However, I/O layer still requests it because alignment regions
+ // use skip, which ultimately uses getLong/putLong, which asks for
+ // byteOrder.
+ //
+ // A model group DOES care about bit order for its alignment regions,
+ // and for the charset encoding of say, initiators or prefix separators.
+ // A bitOrder change requires that we check the new bitOrder against the
+ // byte order to insure compatibility. (byteOrder can be an expression),
+ // so of necessity, we also need byte order. However, if byte order isn't defined
+ // we can assume littleEndian since that works with all bit orders.
+ // (Big endian only allows MSBF bit order)
+ //
+ ByteOrder.LittleEndian
+ }
+ case _ => Assert.usageError("byte order of non term: " + runtimeData)
+ }
+ }
+
+ final def maybeCharWidthInBits: MaybeInt = { coderCacheEntry_.maybeCharWidthInBits }
+ final def encodingMandatoryAlignmentInBits: Int = { decoder; coderCacheEntry_.encodingMandatoryAlignmentInBits }
+ final def maybeUTF16Width: Maybe[UTF16Width] = termRuntimeData.encodingInfo.maybeUTF16Width
+ final def fillByte: Byte = termRuntimeData.maybeFillByteEv.get.evaluate(this).toByte
+
+ final def decoder = {
+ val de = decoderEntry
+ if (encodingErrorPolicy eq EncodingErrorPolicy.Error)
+ de.reportingCoder
+ else
+ de.replacingCoder
+ }
+
+ final def encoder = {
+ val ee = encoderEntry
+ if (encodingErrorPolicy eq EncodingErrorPolicy.Error)
+ ee.reportingCoder
+ else
+ ee.replacingCoder
+ }
+
+ final def encodingErrorPolicy: EncodingErrorPolicy = {
+ val eep = termRuntimeData.encodingInfo.defaultEncodingErrorPolicy
+ eep
+ }
+
+ private def decoderEntry = {
+ val nextEntry = termRuntimeData.encodingInfo.getDecoderInfo(this)
+ if (coderCacheEntry_ == null || coderCacheEntry_ != nextEntry) {
+ coderCacheEntry_ = nextEntry
+ }
+ coderCacheEntry_.asInstanceOf[DecoderInfo]
+ }
+
+ private def encoderEntry = {
+ val nextEntry = termRuntimeData.encodingInfo.getEncoderInfo(this)
+ if (coderCacheEntry_ == null || coderCacheEntry_ != nextEntry) {
+ coderCacheEntry_ = nextEntry
+ }
+ coderCacheEntry_.asInstanceOf[EncoderInfo]
+ }
+
+ private var coderCacheEntry_ : CoderInfo = _
+
+ /**
+ * Variable map provides access to variable bindings.
+ */
def variableMap = variableBox.vmap
def setVariableMap(newMap: VariableMap) {
variableBox.setVMap(newMap)
@@ -152,6 +295,16 @@ abstract class ParseOrUnparseState protected (
def setSuccess() {
_processorStatus = Success
}
+
+ /**
+ * Used when errors are caught by interactive debugger expression evaluation.
+ * We don't want to accumulate the diagnostics that we're suppressing.
+ */
+ final def suppressDiagnosticAndSucceed(d: Diagnostic) {
+ Assert.usage(diagnostics.contains(d))
+ diagnostics = diagnostics.filterNot{ _ eq d}
+ setSuccess()
+ }
def currentNode: Maybe[DINode]
@@ -208,7 +361,6 @@ abstract class ParseOrUnparseState protected (
def occursBoundsStack: MStackOfLong
def hasInfoset: Boolean
- def infoset: InfosetItem
def maybeERD = {
if (hasInfoset)
@@ -270,19 +422,15 @@ abstract class ParseOrUnparseState protected (
* So they don't require a "real" state.
*
* This serves two purposes. First it lets us obey the regular API for evaluation, so we don't need
- * one way to evaluate and another very similar thing for analyzing expressions to see if they are constnat.
+ * one way to evaluate and another very similar thing for analyzing expressions to see if they are constant.
*
* Second, it serves as a detector of when an expression is non-constant by blowing up when things
* inconsistent with constant-value are attempted to be extracted from the state. By "blow up" it throws
* a structured set of exceptions, typically children of InfosetException or VariableException.
*/
-class CompileState(trd: RuntimeData, maybeDataProc: Maybe[DataProcessor])
+final class CompileState(trd: RuntimeData, maybeDataProc: Maybe[DataProcessor])
extends ParseOrUnparseState(trd.variableMap, Nil, maybeDataProc, tunable = trd.tunable) {
- /**
- * As seen from class CompileState, the missing signatures are as follows.
- * * For convenience, these are usable as stub implementations.
- */
- // Members declared in edu.illinois.ncsa.daffodil.processors.ParseOrUnparseState
+
def arrayPos: Long = 1L
def bitLimit0b: MaybeULong = MaybeULong.Nope
def bitPos0b: Long = 0L
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/Runtime.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/Runtime.scala
index e7e9625..b74e649 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/Runtime.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/Runtime.scala
@@ -108,7 +108,7 @@ class DataProcessor(val ssrd: SchemaSetRuntimeData)
extends DFDL.DataProcessor with Logging
with HasSetDebugger with Serializable
with MultipleEventHandler {
-
+
protected var tunablesObj = ssrd.tunable // Compiler-set tunables
def setValidationMode(mode: ValidationMode.Type): Unit = { ssrd.validationMode = mode }
@@ -161,7 +161,7 @@ class DataProcessor(val ssrd: SchemaSetRuntimeData)
def setTunable(tunable: String, value: String): Unit = tunablesObj = tunablesObj.setTunable(tunable, value)
def setTunables(tunables: Map[String, String]): Unit = tunablesObj = tunablesObj.setTunables(tunables)
def resetTunables(): Unit = tunablesObj = ssrd.tunable // Compiler-set values
-
+
override def isError = false // really there is no compiling at all currently, so there can be no errors.
override def getDiagnostics = ssrd.diagnostics
@@ -241,6 +241,16 @@ class DataProcessor(val ssrd: SchemaSetRuntimeData)
this.startElement(state, p)
p.parse1(state)
this.endElement(state, p)
+ //
+ // After the end of all processing, we still call things that ask for the
+ // ERD, and expect to find it on the processor.context. If we don't set
+ // this, then the processor.context is Nope (because it is set on the
+ // way into a parser, and unset back when a parser unwinds). We
+ // want it to do this wind/unwind, but here at the ultimate top
+ // level we want to defeat that final unwind
+ // so that subsequent use of the state can generally work and have a context.
+ //
+ state.setMaybeProcessor(Maybe(p))
/* Verify that all stacks are empty */
Assert.invariant(state.mpstate.arrayIndexStack.length == 1)
@@ -278,7 +288,8 @@ class DataProcessor(val ssrd: SchemaSetRuntimeData)
state.setFailed(e)
}
case us: UnsuppressableException => throw us
- case x: Throwable => Assert.invariantFailed("Runtime.scala - Leaked exception: " + x)
+ case x: Throwable =>
+ Assert.invariantFailed("Runtime.scala - Leaked exception: " + x)
}
state.dataInputStream.validateFinalStreamState
@@ -288,7 +299,6 @@ class DataProcessor(val ssrd: SchemaSetRuntimeData)
Assert.usage(!this.isError)
val outStream = java.nio.channels.Channels.newOutputStream(output)
val out = DirectOrBufferedDataOutputStream(outStream, null) // null means no other stream created this one.
-
val unparserState =
UState.createInitialUState(
out,
@@ -302,6 +312,7 @@ class DataProcessor(val ssrd: SchemaSetRuntimeData)
}
inputter.initialize(ssrd.elementRuntimeData, unparserState.tunable)
unparserState.dataProc.get.init(ssrd.unparser)
+ out.setPriorBitOrder(ssrd.elementRuntimeData.defaultBitOrder)
doUnparse(unparserState)
unparserState.evalSuspensions(unparserState) // handles outputValueCalc that were suspended due to forward references.
unparserState.unparseResult
@@ -350,7 +361,7 @@ class DataProcessor(val ssrd: SchemaSetRuntimeData)
private def doUnparse(state: UState): Unit = {
val rootUnparser = ssrd.unparser
// LoggingDefaults.setLoggingLevel(LogLevel.Debug)
- rootUnparser.unparse(state)
+ rootUnparser.unparse1(state)
/* Verify that all stacks are empty */
Assert.invariant(state.arrayIndexStack.length == 1)
@@ -374,7 +385,7 @@ class DataProcessor(val ssrd: SchemaSetRuntimeData)
//
Assert.invariant(!state.dataOutputStream.isFinished)
try {
- state.dataOutputStream.setFinished()
+ state.dataOutputStream.setFinished(state)
} catch {
case boc: BitOrderChangeException =>
state.SDE(boc)
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/RuntimeData.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/RuntimeData.scala
index f882d8b..713a5b0 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/RuntimeData.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/RuntimeData.scala
@@ -127,7 +127,10 @@ sealed abstract class TermRuntimeData(
@TransientParam alignmentValueInBitsArg: => Int,
@TransientParam hasNoSkipRegionsArg: => Boolean,
@TransientParam defaultBitOrderArg: => BitOrder,
- @TransientParam optIgnoreCaseArg: => Option[YesNo])
+ @TransientParam optIgnoreCaseArg: => Option[YesNo],
+ @TransientParam maybeFillByteEvArg: => Maybe[FillByteEv],
+ @TransientParam maybeCheckByteAndBitOrderEvArg: => Maybe[CheckByteAndBitOrderEv],
+ @TransientParam maybeCheckBitOrderAndCharsetEvArg: => Maybe[CheckBitOrderAndCharsetEv])
extends RuntimeData
with Serializable
with PreSerialization {
@@ -166,6 +169,9 @@ sealed abstract class TermRuntimeData(
lazy val hasNoSkipRegions = hasNoSkipRegionsArg
lazy val defaultBitOrder = defaultBitOrderArg
lazy val optIgnoreCase = optIgnoreCaseArg
+ lazy val maybeFillByteEv = maybeFillByteEvArg
+ lazy val maybeCheckByteAndBitOrderEv = maybeCheckByteAndBitOrderEvArg
+ lazy val maybeCheckBitOrderAndCharsetEv = maybeCheckBitOrderAndCharsetEvArg
override def preSerialization: Unit = {
super.preSerialization
@@ -179,10 +185,14 @@ sealed abstract class TermRuntimeData(
hasNoSkipRegions
defaultBitOrder
optIgnoreCase
+ maybeFillByteEv
tunable
+ maybeCheckByteAndBitOrderEv
+ maybeCheckBitOrderAndCharsetEv
}
@throws(classOf[java.io.IOException])
final private def writeObject(out: java.io.ObjectOutputStream): Unit = serializeObject(out)
+
}
sealed class NonTermRuntimeData(
@@ -444,7 +454,7 @@ final class SimpleTypeRuntimeData(
}
private def checkMinLength(diNode: DISimple, minValue: java.math.BigDecimal,
- e: ThrowsSDE, primType: PrimType): java.lang.Boolean = {
+ e: ThrowsSDE, primType: PrimType): java.lang.Boolean = {
val minAsLong = minValue.longValueExact()
primType match {
case PrimType.String => {
@@ -467,7 +477,7 @@ final class SimpleTypeRuntimeData(
}
private def checkMaxLength(diNode: DISimple, maxValue: java.math.BigDecimal,
- e: ThrowsSDE, primType: PrimType): java.lang.Boolean = {
+ e: ThrowsSDE, primType: PrimType): java.lang.Boolean = {
val maxAsLong = maxValue.longValueExact()
primType match {
case PrimType.String => {
@@ -634,13 +644,13 @@ final class ElementRuntimeData(
@TransientParam isOptionalArg: => Boolean, // can have only 0 or 1 occurrence
@TransientParam isRequiredArg: => Boolean, // must have at least 1 occurrence
/**
- * This is the properly qualified name for recognizing this
- * element.
- *
- * This takes into account xs:schema's elementFormDefault attribute.
- * If 'qualified' then there will be a namespace component.
- * If 'unqualified' the the namespace component will be No_Namespace.
- */
+ * This is the properly qualified name for recognizing this
+ * element.
+ *
+ * This takes into account xs:schema's elementFormDefault attribute.
+ * If 'qualified' then there will be a namespace component.
+ * If 'unqualified' the the namespace component will be No_Namespace.
+ */
@TransientParam namedQNameArg: => NamedQName,
@TransientParam isRepresentedArg: => Boolean,
@TransientParam couldHaveTextArg: => Boolean,
@@ -653,15 +663,22 @@ final class ElementRuntimeData(
// Unparser-specific arguments
//
/**
- * pass true for this if the corresponding infoset element is never
- * accessed by way of expressions. Enables the element to be dropped
- * from the infoset immediately after unparsing is complete.
- */
+ * pass true for this if the corresponding infoset element is never
+ * accessed by way of expressions. Enables the element to be dropped
+ * from the infoset immediately after unparsing is complete.
+ */
@TransientParam notReferencedByExpressionsArg: => Boolean,
@TransientParam optTruncateSpecifiedLengthStringArg: => Option[Boolean],
- @TransientParam outputValueCalcExprArg: => Option[CompiledExpression[AnyRef]])
+ @TransientParam outputValueCalcExprArg: => Option[CompiledExpression[AnyRef]],
+ @TransientParam maybeBinaryFloatRepEvArg: => Maybe[BinaryFloatRepEv],
+ @TransientParam maybeByteOrderEvArg: => Maybe[ByteOrderEv],
+ @TransientParam maybeFillByteEvArg: => Maybe[FillByteEv],
+ @TransientParam maybeCheckByteAndBitOrderEvArg: => Maybe[CheckByteAndBitOrderEv],
+ @TransientParam maybeCheckBitOrderAndCharsetEvArg: => Maybe[CheckBitOrderAndCharsetEv])
extends TermRuntimeData(parentArg, parentTermArg, encInfoArg, dpathElementCompileInfoArg, isRepresentedArg, couldHaveTextArg, alignmentValueInBitsArg, hasNoSkipRegionsArg,
- defaultBitOrderArg, optIgnoreCaseArg){
+ defaultBitOrderArg, optIgnoreCaseArg, maybeFillByteEvArg,
+ maybeCheckByteAndBitOrderEvArg,
+ maybeCheckBitOrderAndCharsetEvArg) {
lazy val parent = parentArg
lazy val parentTerm = parentTermArg
@@ -696,6 +713,8 @@ final class ElementRuntimeData(
lazy val notReferencedByExpressions = notReferencedByExpressionsArg
lazy val optTruncateSpecifiedLengthString = optTruncateSpecifiedLengthStringArg
lazy val outputValueCalcExpr = outputValueCalcExprArg
+ lazy val maybeBinaryFloatRepEv = maybeBinaryFloatRepEvArg
+ lazy val maybeByteOrderEv = maybeByteOrderEvArg
def isReferencedByExpressions = !notReferencedByExpressions
@@ -734,6 +753,8 @@ final class ElementRuntimeData(
notReferencedByExpressions
optTruncateSpecifiedLengthString
outputValueCalcExpr
+ maybeBinaryFloatRepEv
+ maybeByteOrderEv
}
@throws(classOf[java.io.IOException])
@@ -782,11 +803,16 @@ sealed abstract class ModelGroupRuntimeData(
@TransientParam couldHaveTextArg: => Boolean,
@TransientParam alignmentValueInBitsArg: => Int,
@TransientParam hasNoSkipRegionsArg: => Boolean,
- @TransientParam optIgnoreCaseArg: => Option[YesNo])
+ @TransientParam optIgnoreCaseArg: => Option[YesNo],
+ @TransientParam maybeFillByteEvArg: => Maybe[FillByteEv],
+ @TransientParam maybeCheckByteAndBitOrderEvArg: => Maybe[CheckByteAndBitOrderEv],
+ @TransientParam maybeCheckBitOrderAndCharsetEvArg: => Maybe[CheckBitOrderAndCharsetEv])
extends TermRuntimeData(Some(erdArg),
Maybe(trdArg),
encInfoArg, ciArg, isRepresentedArg, couldHaveTextArg, alignmentValueInBitsArg, hasNoSkipRegionsArg,
- defaultBitOrderArg, optIgnoreCaseArg) {
+ defaultBitOrderArg, optIgnoreCaseArg, maybeFillByteEvArg,
+ maybeCheckByteAndBitOrderEvArg,
+ maybeCheckBitOrderAndCharsetEvArg) {
lazy val variableMap = variableMapArg
lazy val encInfo = encInfoArg
@@ -837,9 +863,15 @@ final class SequenceRuntimeData(
@TransientParam couldHaveTextArg: => Boolean,
@TransientParam alignmentValueInBitsArg: => Int,
@TransientParam hasNoSkipRegionsArg: => Boolean,
- @TransientParam optIgnoreCaseArg: => Option[YesNo])
+ @TransientParam optIgnoreCaseArg: => Option[YesNo],
+ @TransientParam maybeFillByteEvArg: => Maybe[FillByteEv],
+ @TransientParam maybeCheckByteAndBitOrderEvArg: => Maybe[CheckByteAndBitOrderEv],
+ @TransientParam maybeCheckBitOrderAndCharsetEvArg: => Maybe[CheckBitOrderAndCharsetEv])
extends ModelGroupRuntimeData(variableMapArg, encInfoArg, schemaFileLocationArg, ciArg, diagnosticDebugNameArg, pathArg, namespacesArg, defaultBitOrderArg, groupMembersArg,
- erdArg, trdArg, isRepresentedArg, couldHaveTextArg, alignmentValueInBitsArg, hasNoSkipRegionsArg, optIgnoreCaseArg)
+ erdArg, trdArg, isRepresentedArg, couldHaveTextArg, alignmentValueInBitsArg, hasNoSkipRegionsArg, optIgnoreCaseArg,
+ maybeFillByteEvArg,
+ maybeCheckByteAndBitOrderEvArg,
+ maybeCheckBitOrderAndCharsetEvArg)
final class ChoiceRuntimeData(
/**
@@ -862,9 +894,14 @@ final class ChoiceRuntimeData(
@TransientParam couldHaveTextArg: => Boolean,
@TransientParam alignmentValueInBitsArg: => Int,
@TransientParam hasNoSkipRegionsArg: => Boolean,
- @TransientParam optIgnoreCaseArg: => Option[YesNo])
+ @TransientParam optIgnoreCaseArg: => Option[YesNo],
+ @TransientParam maybeFillByteEvArg: => Maybe[FillByteEv],
+ @TransientParam maybeCheckByteAndBitOrderEvArg: => Maybe[CheckByteAndBitOrderEv],
+ @TransientParam maybeCheckBitOrderAndCharsetEvArg: => Maybe[CheckBitOrderAndCharsetEv])
extends ModelGroupRuntimeData(variableMapArg, encInfoArg, schemaFileLocationArg, ciArg, diagnosticDebugNameArg, pathArg, namespacesArg, defaultBitOrderArg, groupMembersArg,
- erdArg, trdArg, isRepresentedArg, couldHaveTextArg, alignmentValueInBitsArg, hasNoSkipRegionsArg, optIgnoreCaseArg)
+ erdArg, trdArg, isRepresentedArg, couldHaveTextArg, alignmentValueInBitsArg, hasNoSkipRegionsArg, optIgnoreCaseArg, maybeFillByteEvArg,
+ maybeCheckByteAndBitOrderEvArg,
+ maybeCheckBitOrderAndCharsetEvArg)
final class VariableRuntimeData(
@TransientParam schemaFileLocationArg: => SchemaFileLocation,
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/SuspendableOperation.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/SuspendableOperation.scala
index 4353f7b..32cc35a 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/SuspendableOperation.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/SuspendableOperation.scala
@@ -40,6 +40,7 @@ import edu.illinois.ncsa.daffodil.api.Diagnostic
import edu.illinois.ncsa.daffodil.util.Maybe
import edu.illinois.ncsa.daffodil.util.Maybe._
import edu.illinois.ncsa.daffodil.infoset.RetryableException
+import edu.illinois.ncsa.daffodil.processors.unparsers.Unparser
/**
* SuspendableOperation is used for suspending and retrying things that aren't
@@ -52,7 +53,7 @@ import edu.illinois.ncsa.daffodil.infoset.RetryableException
trait SuspendableOperation
extends Suspension {
- override def rd: RuntimeData
+ override def rd: TermRuntimeData
override def toString = "%s for %s".format(Misc.getNameFromClass(this), rd.diagnosticDebugName)
@@ -105,7 +106,8 @@ trait SuspendableOperation
}
}
-trait SuspendableUnparser {
+trait SuspendableUnparser
+ extends Unparser {
protected def suspendableOperation: SuspendableOperation
@@ -116,6 +118,6 @@ trait SuspendableUnparser {
class SuspendableOperationException(m: String)
extends Diagnostic(Nope, Nope, Nope, Maybe(m)) {
- def isError = true
- def modeName= "Unparse"
+ override def isError = true
+ override def modeName = "Unparse"
}
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/Suspension.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/Suspension.scala
index 00e468f..a839de0 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/Suspension.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/Suspension.scala
@@ -42,7 +42,6 @@ import edu.illinois.ncsa.daffodil.util.LogLevel
import edu.illinois.ncsa.daffodil.util.Maybe
import edu.illinois.ncsa.daffodil.util.Maybe._
import edu.illinois.ncsa.daffodil.util.MaybeInt
-import passera.unsigned.ULong
import edu.illinois.ncsa.daffodil.processors.unparsers.UnparseError
import edu.illinois.ncsa.daffodil.io.BitOrderChangeException
@@ -58,6 +57,15 @@ import edu.illinois.ncsa.daffodil.io.BitOrderChangeException
trait Suspension
extends Serializable with Logging {
+ /**
+ * Specifies that this suspension does not write to the data output stream.
+ *
+ * Override in TargetLengthOperation,and in SuspendableExpression as they
+ * don't write to the DOS hence, if a DOS is created it can be setFinished
+ * immediately.
+ *
+ * TODO: Redundant with implementing maybeKnownLengthInBits as MaybeULong(0L)
+ */
val isReadOnly = false
def UE(ustate: UState, s: String, args: Any*) = {
@@ -89,7 +97,11 @@ trait Suspension
doTask(savedUstate)
if (isDone && !isReadOnly) {
try {
- savedUstate.dataOutputStream.setFinished()
+ //
+ // We are done, and we're not readOnly, so the
+ // DOS needs to be set finished now.
+ //
+ savedUstate.dataOutputStream.setFinished(savedUstate)
} catch {
case boc: BitOrderChangeException =>
savedUstate.SDE(boc)
@@ -98,12 +110,103 @@ trait Suspension
}
}
+ /**
+ * Run the first time.
+ *
+ */
final def run(ustate: UState) {
doTask(ustate)
if (!isDone) {
- val mkl = maybeKnownLengthInBits(ustate)
- setup(ustate, mkl)
+ prepareToSuspend(ustate)
+ }
+ }
+
+ private def prepareToSuspend(ustate: UState) {
+ val mkl = maybeKnownLengthInBits(ustate)
+ //
+ // It seems like we have too many splits going on.
+ //
+ // As written, we have a bunch of suspensions that occur, but have
+ // specifically known length of zero bits. So nothing being written out.
+ // In that case, why do we need to split at all?
+ //
+ val original = ustate.dataOutputStream.asInstanceOf[DirectOrBufferedDataOutputStream]
+ if (mkl.isEmpty || (mkl.isDefined && mkl.get > 0)) {
+ //
+ // only split if the length is either unknown
+ // or known and greater than 0.
+ //
+ // If length known 0, then no need for another DOS
+ //
+ splitDOS(ustate, mkl, original)
+ }
+ suspend(ustate, original)
+ }
+
+ private def splitDOS(ustate: UState,
+ maybeKnownLengthInBits: MaybeULong,
+ original: DirectOrBufferedDataOutputStream) {
+ Assert.usage(ustate.currentInfosetNodeMaybe.isDefined)
+
+ val buffered = original.addBuffered
+
+ if (maybeKnownLengthInBits.isDefined) {
+ // since we know the length of the unparsed representation that we're skipping for now,
+ // that means we know the absolute position of the bits in the buffer we're creating
+ // and that means alignment operations don't have to suspend waiting for this knowledge
+ if (original.maybeAbsBitPos0b.isDefined) {
+ // direct streams always know this, but buffered streams may not.
+
+ val originalAbsBitPos0b = original.maybeAbsBitPos0b.getULong
+
+ // we are passed this length (in bits)
+ // and can use it to initialize the absolute bit pos of the buffered output stream.
+ //
+ // This allows us to deal with alignment regions, that is, we can determine
+ // their size since we know the absolute bit position.
+
+ val mkl = maybeKnownLengthInBits.getULong
+ buffered.setAbsStartingBitPos0b(originalAbsBitPos0b + mkl)
+
+ }
+ } else {
+ log(LogLevel.Debug, "Buffered DOS created for %s without knowning absolute start bit pos: %s\n",
+ ustate.currentInfosetNode.erd.diagnosticDebugName, buffered)
}
+
+ // the main-thread will carry on using the original ustate but unparsing
+ // into this buffered stream.
+ ustate.dataOutputStream = buffered
+ }
+
+ private def suspend(ustate: UState, original: DirectOrBufferedDataOutputStream) {
+ //
+ // clone the ustate for use when evaluating the expression
+ //
+ // TODO: Performance - copying this whole state, just for OVC is painful.
+ // Some sort of copy-on-write scheme would be better.
+ //
+ val didSplit = (ustate.dataOutputStream ne original)
+ val cloneUState = ustate.asInstanceOf[UStateMain].cloneForSuspension(original)
+ if (isReadOnly && didSplit) {
+ Assert.invariantFailed("Shouldn't have split. read-only case")
+ // try {
+ // // We did a DOS split, but we know we'll not be writing to it
+ // //
+ // // So we set finished immediately
+ // //
+ // // TODO: Begs the question of why we needed the split to begin with in that
+ // // case. Figure out why and document it!
+ // //
+ // original.setFinished(cloneUState)
+ // } catch {
+ // case boc: BitOrderChangeException => ustate.SDE(boc)
+ // }
+ }
+
+ savedUstate_ = cloneUState
+
+ ustate.asInstanceOf[UStateMain].addSuspension(this)
}
final def explain() {
@@ -195,70 +298,5 @@ trait Suspension
res
}
- final protected def setup(ustate: UState, maybeKnownLengthInBits: MaybeULong) {
- Assert.usage(ustate.currentInfosetNodeMaybe.isDefined)
-
- val original = ustate.dataOutputStream.asInstanceOf[DirectOrBufferedDataOutputStream]
-
- // // if we know the length will be 0, no need to allocate a buffered DOS
- // // but if we don't know the length or we know the length is non-zero
- // // then we do need a buffered DOS
- // //
- // if (maybeKnownLengthInBits.isEmpty ||
- // (maybeKnownLengthInBits.get != 0)) {
-
- val buffered = original.addBuffered
-
- if (maybeKnownLengthInBits.isDefined) {
- // since we know the length of the unparsed representation that we're skipping for now,
- // that means we know the absolute position of the bits in the buffer we're creating
- // and that means alignment operations don't have to suspend waiting for this knowledge
- if (original.maybeAbsBitPos0b.isDefined) {
- // direct streams always know this, but buffered streams may not.
-
- val originalAbsBitPos0b = original.maybeAbsBitPos0b.getULong
-
- // we are passed this length (in bits)
- // and can use it to initialize the absolute bit pos of the buffered output stream.
- //
- // This allows us to deal with alignment regions, that is, we can determine
- // their size since we know the absolute bit position.
-
- val mkl = maybeKnownLengthInBits.getULong
- Assert.invariant(mkl >= ULong(0))
- buffered.setAbsStartingBitPos0b(originalAbsBitPos0b + mkl)
-
- }
- } else {
- // log(LogLevel.Debug,
- log(LogLevel.Debug, "Buffered DOS created for %s without knowning absolute start bit pos: %s\n",
- ustate.currentInfosetNode.erd.diagnosticDebugName, buffered)
- }
-
- // the main-thread will carry on using the original ustate but unparsing
- // into this buffered stream.
- ustate.dataOutputStream = buffered
-
- // }
-
- //
- // clone the ustate for use when evaluating the expression
- //
- // TODO: Performance - copying this whole state, just for OVC is painful.
- // Some sort of copy-on-write scheme would be better.
- //
- val cloneUState = ustate.asInstanceOf[UStateMain].cloneForSuspension(original)
- if (isReadOnly) {
- try {
- original.setFinished()
- } catch {
- case boc: BitOrderChangeException => ustate.SDE(boc)
- }
- }
-
- savedUstate_ = cloneUState
-
- ustate.asInstanceOf[UStateMain].addSuspension(this)
- }
}
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/CreatePaddingDFA.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/CreatePaddingDFA.scala
index e30409a..9b49e19 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/CreatePaddingDFA.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/CreatePaddingDFA.scala
@@ -34,8 +34,8 @@ package edu.illinois.ncsa.daffodil.processors.dfa
import scala.collection.mutable.ArrayBuffer
import edu.illinois.ncsa.daffodil.processors.Delimiter
-import edu.illinois.ncsa.daffodil.processors.RuntimeData
import edu.illinois.ncsa.daffodil.processors.parsers.DelimiterTextType
+import edu.illinois.ncsa.daffodil.processors.TermRuntimeData
object CreatePaddingDFA {
@@ -43,7 +43,7 @@ object CreatePaddingDFA {
* Constructs a DFADelimiter object that specifically
* looks for padChar.
*/
- def apply(padChar: Char, rd: RuntimeData): DFADelimiter = {
+ def apply(padChar: Char, rd: TermRuntimeData): DFADelimiter = {
// TODO: In the future we will need to change this because the padChar isn't necessarily a char.
// One can use it to specify a numeric byte to be used to pad as well.
@@ -60,7 +60,7 @@ object CreatePaddingDFA {
* Constructs a DFADelimiter object that specifically
* looks for padChar.
*/
- def apply(padChar: Char, outputNewLine: String, rd: RuntimeData): DFADelimiter = {
+ def apply(padChar: Char, outputNewLine: String, rd: TermRuntimeData): DFADelimiter = {
// TODO: In the future we will need to change this because the padChar isn't necessarily a char.
// One can use it to specify a numeric byte to be used to pad as well.
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/Registers.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/Registers.scala
index 2fdca0e..9e0206f 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/Registers.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/Registers.scala
@@ -32,11 +32,12 @@
package edu.illinois.ncsa.daffodil.processors.dfa
+import edu.illinois.ncsa.daffodil.equality.ViewEqual
import edu.illinois.ncsa.daffodil.io.DataInputStream
-import edu.illinois.ncsa.daffodil.equality._
-import edu.illinois.ncsa.daffodil.util.Pool
import edu.illinois.ncsa.daffodil.processors.DelimiterIterator
+import edu.illinois.ncsa.daffodil.util.Pool
import edu.illinois.ncsa.daffodil.util.Poolable
+import edu.illinois.ncsa.daffodil.io.FormatInfo
private[dfa] object TLRegistersPool extends ThreadLocal[RegistersPool] {
override def initialValue = new RegistersPool()
@@ -83,10 +84,11 @@ class Registers() extends Poolable with Serializable {
* and then reset before first use. I.e.,
* reset() is also init().
*/
- def reset(input: DataInputStream, delimIter: DelimiterIterator, m: DataInputStream.MarkPos = DataInputStream.MarkPos.NoMarkPos) {
+ def reset(finfo: FormatInfo, input: DataInputStream, delimIter: DelimiterIterator, m: DataInputStream.MarkPos = DataInputStream.MarkPos.NoMarkPos) {
dataInputStream = input
- if (m !=#= DataInputStream.MarkPos.NoMarkPos) dataInputStream.resetPos(m)
- resetChars
+ if (m !=#= DataInputStream.MarkPos.NoMarkPos)
+ dataInputStream.resetPos(m)
+ resetChars(finfo)
resultString.clear()
delimString.clear()
numCharsRead = 0
@@ -94,7 +96,6 @@ class Registers() extends Poolable with Serializable {
numCharsDropped = 0
Registers.this.matchStartPos = matchStartPos
matchedAtLeastOnce = false
- charIterator = dataInputStream.asIteratorChar
actionNum = 0
state = 0
nextState = 0
@@ -102,8 +103,9 @@ class Registers() extends Poolable with Serializable {
delimitersIter = delimIter
}
- def resetChars {
+ def resetChars(finfo: FormatInfo) {
charIterator = dataInputStream.asIteratorChar
+ charIterator.setFormatInfo(finfo)
data0 = charIterator.peek()
data1 = charIterator.peek2()
}
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextDelimitedParser.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextDelimitedParser.scala
index d199534..931b0f3 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextDelimitedParser.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextDelimitedParser.scala
@@ -43,6 +43,7 @@ import edu.illinois.ncsa.daffodil.io.DataInputStream
import edu.illinois.ncsa.daffodil.equality._
import edu.illinois.ncsa.daffodil.util.MaybeChar
import edu.illinois.ncsa.daffodil.processors.DelimiterIterator
+import edu.illinois.ncsa.daffodil.io.FormatInfo
abstract class TextDelimitedParserBase(
override val justificationTrim: TextJustificationType.Type,
@@ -53,14 +54,14 @@ abstract class TextDelimitedParserBase(
private lazy val padCharInfo = if (parsingPadChar.isDefined) parsingPadChar.toString else "NONE"
lazy val info: String = "justification='" + justificationTrim + "', padChar='" + padCharInfo + "'"
- final def parse(input: DataInputStream, field: DFAField, delimIter: DelimiterIterator, isDelimRequired: Boolean): Maybe[ParseResult] = {
+ final def parse(finfo: FormatInfo, input: DataInputStream, field: DFAField, delimIter: DelimiterIterator, isDelimRequired: Boolean): Maybe[ParseResult] = {
Assert.invariant(field != null)
val lmt = new LongestMatchTracker()
val fieldReg: Registers = TLRegistersPool.getFromPool("TextDelimitedParserBase1")
- fieldReg.reset(input, delimIter) // Initialization
+ fieldReg.reset(finfo, input, delimIter) // Initialization
var stillSearching: Boolean = true
var beforeDelimiter: DataInputStream.MarkPos = DataInputStream.MarkPos.NoMarkPos
@@ -81,7 +82,7 @@ abstract class TextDelimitedParserBase(
input.resetPos(beforeDelimiter)
beforeDelimiter = input.markPos
val delimReg: Registers = TLRegistersPool.getFromPool("TextDelimitedParserBase2")
- delimReg.reset(input, delimIter)
+ delimReg.reset(finfo, input, delimIter)
d.run(delimReg)
if (delimReg.status == StateKind.Succeeded) {
lmt.successfulMatch(delimReg.matchStartPos, delimReg.delimString, d, delimIter.currentIndex)
@@ -199,11 +200,11 @@ class TextDelimitedParserWithEscapeBlock(
}
}
- protected def removeLeftPadding(input: DataInputStream, delimIter: DelimiterIterator): Unit = {
+ protected def removeLeftPadding(finfo: FormatInfo, input: DataInputStream, delimIter: DelimiterIterator): Unit = {
justificationTrim match {
case TextJustificationType.Center | TextJustificationType.Right if parsingPadChar.isDefined => {
val leftPaddingRegister = TLRegistersPool.getFromPool("removeLeftPadding")
- leftPaddingRegister.reset(input, delimIter)
+ leftPaddingRegister.reset(finfo, input, delimIter)
leftPadding.run(leftPaddingRegister)
TLRegistersPool.returnToPool(leftPaddingRegister)
}
@@ -211,11 +212,11 @@ class TextDelimitedParserWithEscapeBlock(
}
}
- protected def removeRightPadding(input: DataInputStream, delimIter: DelimiterIterator): Unit = {
+ protected def removeRightPadding(finfo: FormatInfo, input: DataInputStream, delimIter: DelimiterIterator): Unit = {
justificationTrim match {
case TextJustificationType.Center | TextJustificationType.Left if parsingPadChar.isDefined => {
val rightPaddingRegister = TLRegistersPool.getFromPool("removeRightPadding")
- rightPaddingRegister.reset(input, delimIter)
+ rightPaddingRegister.reset(finfo, input, delimIter)
rightPadding.run(rightPaddingRegister)
TLRegistersPool.returnToPool(rightPaddingRegister)
}
@@ -223,9 +224,9 @@ class TextDelimitedParserWithEscapeBlock(
}
}
- protected def parseStartBlock(input: DataInputStream, startBlock: DFADelimiter, delimIter: DelimiterIterator): Boolean = {
+ protected def parseStartBlock(finfo: FormatInfo, input: DataInputStream, startBlock: DFADelimiter, delimIter: DelimiterIterator): Boolean = {
val startBlockRegister = TLRegistersPool.getFromPool("parseStartBlock")
- startBlockRegister.reset(input, delimIter)
+ startBlockRegister.reset(finfo, input, delimIter)
startBlock.run(startBlockRegister) // find the block start, fail otherwise
val startStatus = startBlockRegister.status
@@ -240,7 +241,7 @@ class TextDelimitedParserWithEscapeBlock(
* Called to parse the rest of the field until we reach a block end, but
* beyond that, after we reach a block-end out until we reach the delimiter.
*/
- protected def parseRemainder(input: DataInputStream,
+ protected def parseRemainder(finfo: FormatInfo, input: DataInputStream,
fieldEsc: DFAField,
startBlock: DFADelimiter, endBlock: DFADelimiter,
delimIter: DelimiterIterator, isDelimRequired: Boolean): Maybe[ParseResult] = {
@@ -248,7 +249,7 @@ class TextDelimitedParserWithEscapeBlock(
val lmt = new LongestMatchTracker()
val fieldRegister = TLRegistersPool.getFromPool("parseRemainder")
- fieldRegister.reset(input, delimIter)
+ fieldRegister.reset(finfo, input, delimIter)
var stillSearching: Boolean = true
var foundBlockEnd: Boolean = false
@@ -265,11 +266,9 @@ class TextDelimitedParserWithEscapeBlock(
case StateKind.EndOfData => stillSearching = false
case StateKind.Failed => stillSearching = false
case StateKind.Paused => {
- // Pick up where field left off, we are looking for
- // the blockEnd.
+ // Pick up where field left off, we are looking for the blockEnd.
val endBlockRegister = TLRegistersPool.getFromPool("parseRemainder2")
- endBlockRegister.reset(input, delimIter) // copy(fieldRegister) // TODO: This should just be a reset of the registers. No need to copy.
-
+ endBlockRegister.reset(finfo, input, delimIter)
endBlock.run(endBlockRegister)
val endBlockStatus = endBlockRegister.status
TLRegistersPool.returnToPool(endBlockRegister)
@@ -278,7 +277,7 @@ class TextDelimitedParserWithEscapeBlock(
case StateKind.Succeeded => {
// Found the unescaped block end, now we need to
// find any padding.
- this.removeRightPadding(input, delimIter)
+ this.removeRightPadding(finfo, input, delimIter)
beforeDelimiter = input.markPos
delimIter.reset()
@@ -288,7 +287,7 @@ class TextDelimitedParserWithEscapeBlock(
val delimRegister = TLRegistersPool.getFromPool("parseRemainder3")
input.resetPos(beforeDelimiter)
beforeDelimiter = input.markPos
- delimRegister.reset(input, delimIter)
+ delimRegister.reset(finfo, input, delimIter)
d.run(delimRegister)
if (delimRegister.status == StateKind.Succeeded) {
@@ -312,7 +311,7 @@ class TextDelimitedParserWithEscapeBlock(
//
input.resetPos(beforeDelimiter)
beforeDelimiter = DataInputStream.MarkPos.NoMarkPos
- fieldRegister.resetChars
+ fieldRegister.resetChars(finfo)
// resume field parse
//
@@ -364,7 +363,7 @@ class TextDelimitedParserWithEscapeBlock(
result
}
- def parse(input: DataInputStream, field: DFAField, fieldEsc: DFAField,
+ def parse(finfo: FormatInfo, input: DataInputStream, field: DFAField, fieldEsc: DFAField,
startBlock: DFADelimiter, endBlock: DFADelimiter,
delimIter: DelimiterIterator, isDelimRequired: Boolean): Maybe[ParseResult] = {
Assert.invariant(fieldEsc != null)
@@ -372,12 +371,12 @@ class TextDelimitedParserWithEscapeBlock(
Assert.invariant(startBlock != null)
Assert.invariant(endBlock != null)
- removeLeftPadding(input, delimIter)
- val foundStartBlock = parseStartBlock(input, startBlock, delimIter)
+ removeLeftPadding(finfo, input, delimIter)
+ val foundStartBlock = parseStartBlock(finfo, input, startBlock, delimIter)
val res = if (!foundStartBlock) {
- super.parse(input, field, delimIter, isDelimRequired)
+ super.parse(finfo, input, field, delimIter, isDelimRequired)
} else {
- parseRemainder(input, fieldEsc, startBlock, endBlock, delimIter, isDelimRequired)
+ parseRemainder(finfo, input, fieldEsc, startBlock, endBlock, delimIter, isDelimRequired)
}
TLRegistersPool.pool.finalCheck
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextDelimitedUnparser.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextDelimitedUnparser.scala
index 80877d7..351f3ca 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextDelimitedUnparser.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextDelimitedUnparser.scala
@@ -109,9 +109,7 @@ class TextDelimitedUnparser(override val context: TermRuntimeData)
}
val delimIter = new AllDelimiterIterator(ArrayBuffer(delims: _*))
- fieldReg.reset(input, fieldEscapesIter)
-
- // val initialCharPos = 0
+ fieldReg.reset(state, input, fieldEscapesIter)
var stillSearching: Boolean = true
var numCharsInserted: Int = 0
@@ -146,7 +144,7 @@ class TextDelimitedUnparser(override val context: TermRuntimeData)
// We check for a blockEnd first, if it exists then we MUST
// generate an escape block
val blockEndReg: Registers = TLRegistersPool.getFromPool("escapeBlock2")
- blockEndReg.reset(input, blockEndDelimIter)
+ blockEndReg.reset(state, input, blockEndDelimIter)
blockEnd.run(blockEndReg)
val blockEndStatus = blockEndReg.status
blockEndStatus match {
@@ -191,7 +189,7 @@ class TextDelimitedUnparser(override val context: TermRuntimeData)
val delimReg: Registers = TLRegistersPool.getFromPool("escapeBlock3")
input.resetPos(beforeDelimiter)
beforeDelimiter = input.markPos
- delimReg.reset(input, delimIter)
+ delimReg.reset(state, input, delimIter)
d.run(delimReg)
val delimStatus = delimReg.status
delimStatus match {
@@ -210,7 +208,7 @@ class TextDelimitedUnparser(override val context: TermRuntimeData)
}
input.resetPos(beforeDelimiter)
beforeDelimiter = DataInputStream.MarkPos.NoMarkPos
- fieldReg.resetChars
+ fieldReg.resetChars(state)
if (successes.isEmpty) {
// did not match any delimiters, go to the next rule in the
// field DFA, effectively resuming the field parse. This is possible
@@ -270,7 +268,7 @@ class TextDelimitedUnparser(override val context: TermRuntimeData)
val delimIter = new AllDelimiterIterator(ArrayBuffer(delims: _*))
- fieldReg.reset(input, delimIter)
+ fieldReg.reset(state, input, delimIter)
var stillSearching: Boolean = true
fieldReg.state = 0 // initial state is 0
@@ -305,7 +303,7 @@ class TextDelimitedUnparser(override val context: TermRuntimeData)
while (delimIter.hasNext()) {
val d = delimIter.next()
val delimReg: Registers = TLRegistersPool.getFromPool("escapeCharacter2")
- delimReg.reset(input, delimIter)
+ delimReg.reset(state, input, delimIter)
input.resetPos(beforeDelimiter)
beforeDelimiter = input.markPos
d.run(delimReg)
@@ -350,8 +348,8 @@ class TextDelimitedUnparser(override val context: TermRuntimeData)
// delimiter
//
input.resetPos(beforeDelimiter)
- Assert.invariant(input.skipChars(delim.length))
- fieldReg.resetChars
+ Assert.invariant(input.skipChars(delim.length, state))
+ fieldReg.resetChars(state)
successes.foreach { case (d, r) => TLRegistersPool.returnToPool(r) }
successes.clear
stillSearching = true
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextPaddingParser.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextPaddingParser.scala
index 16b8318..224b1c5 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextPaddingParser.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextPaddingParser.scala
@@ -38,6 +38,7 @@ import edu.illinois.ncsa.daffodil.processors.TermRuntimeData
import edu.illinois.ncsa.daffodil.processors.DelimiterIterator
import edu.illinois.ncsa.daffodil.io.DataInputStream
import scala.collection.mutable.ArrayBuffer
+import edu.illinois.ncsa.daffodil.io.FormatInfo
class TextPaddingParser(val padChar: Char,
override val context: TermRuntimeData)
@@ -48,11 +49,11 @@ class TextPaddingParser(val padChar: Char,
val paddingDFA = CreatePaddingDFA(padChar, context)
- def parse(input: DataInputStream, delimIter: DelimiterIterator): Maybe[ParseResult] = {
+ def parse(finfo: FormatInfo, input: DataInputStream, delimIter: DelimiterIterator): Maybe[ParseResult] = {
val paddingReg: Registers = TLRegistersPool.getFromPool("TextPaddingParser1")
- paddingReg.reset(input, delimIter)
+ paddingReg.reset(finfo, input, delimIter)
paddingDFA.run(paddingReg) // Will always succeed.
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextParser.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextParser.scala
index f5a7261..413350a 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextParser.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/dfa/TextParser.scala
@@ -37,15 +37,16 @@ import edu.illinois.ncsa.daffodil.util.Maybe._
import edu.illinois.ncsa.daffodil.processors.TermRuntimeData
import edu.illinois.ncsa.daffodil.processors.DelimiterIterator
import edu.illinois.ncsa.daffodil.io.DataInputStream
+import edu.illinois.ncsa.daffodil.io.FormatInfo
class TextParser(
override val context: TermRuntimeData)
extends Parser {
- lazy val name: String = "TextParser"
- lazy val info: String = "" // Nothing additional to add here
+ override lazy val name: String = "TextParser"
+ override lazy val info: String = "" // Nothing additional to add here
- def parse(input: DataInputStream, delimIter: DelimiterIterator, isDelimRequired: Boolean): Maybe[ParseResult] = {
+ def parse(finfo: FormatInfo, input: DataInputStream, delimIter: DelimiterIterator, isDelimRequired: Boolean): Maybe[ParseResult] = {
val lmt = new LongestMatchTracker()
@@ -54,7 +55,7 @@ class TextParser(
while (delimIter.hasNext()) {
val d = delimIter.next()
val reg = TLRegistersPool.getFromPool("TextParser1")
- reg.reset(input, delimIter, m)
+ reg.reset(finfo, input, delimIter, m)
m = input.markPos
d.run(reg)
if (reg.status == StateKind.Succeeded) {
@@ -69,7 +70,7 @@ class TextParser(
if (isDelimRequired) Nope
else {
val totalNumCharsRead = 0
- input.getString(totalNumCharsRead)
+ input.getString(totalNumCharsRead, finfo)
One(new ParseResult(Nope, Nope, lmt.longestMatches))
}
} else {
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/AssertPatternParsers.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/AssertPatternParsers.scala
index 58ffc4c..0318d70 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/AssertPatternParsers.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/AssertPatternParsers.scala
@@ -64,7 +64,7 @@ abstract class AssertPatternParserBase(
val dis = start.dataInputStream
val mark = dis.markPos
withMatcher { m =>
- val isMatch = dis.lookingAt(m)
+ val isMatch = dis.lookingAt(m, start)
afterParse(start, isMatch, m)
}
dis.resetPos(mark)
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/BinaryBooleanParsers.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/BinaryBooleanParsers.scala
index 3edfc25..fe05d58 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/BinaryBooleanParsers.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/BinaryBooleanParsers.scala
@@ -86,7 +86,7 @@ class BinaryBooleanParser(val e: ElementRuntimeData,
return
}
- val sl = dis.getUnsignedLong(nBits)
+ val sl = dis.getUnsignedLong(nBits, start)
val bool: JBoolean =
if (!binaryBooleanTrueRep.isDefined) {
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/BinaryNumberParsers.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/BinaryNumberParsers.scala
index 9beb639..b8f1e12 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/BinaryNumberParsers.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/BinaryNumberParsers.scala
@@ -50,7 +50,7 @@ class BinaryFloatParser(e: ElementRuntimeData)
return
}
- val f: JFloat = dis.getBinaryFloat()
+ val f: JFloat = dis.getBinaryFloat(start)
start.simpleElement.overwriteDataValue(f)
}
}
@@ -66,7 +66,7 @@ class BinaryDoubleParser(e: ElementRuntimeData)
return
}
- val d: JDouble = dis.getBinaryDouble()
+ val d: JDouble = dis.getBinaryDouble(start)
start.simpleElement.overwriteDataValue(d)
}
}
@@ -89,15 +89,14 @@ abstract class BinaryDecimalParserBase(e: ElementRuntimeData, signed: YesNo, bin
def parse(start: PState): Unit = {
val nBits = getBitLength(start)
val dis = start.dataInputStream
-
if (!dis.isDefinedForLength(nBits)) {
PE(start, "Insufficient bits in data. Needed %d bit(s) but found only %d available.", nBits, dis.remainingBits.get)
return
}
val bigInt: BigInt =
- if (signed == YesNo.Yes) { dis.getSignedBigInt(nBits) }
- else { dis.getUnsignedBigInt(nBits) }
+ if (signed == YesNo.Yes) { dis.getSignedBigInt(nBits, start) }
+ else { dis.getUnsignedBigInt(nBits, start) }
val bigDec = BigDecimal(bigInt, binaryDecimalVirtualPoint)
start.simpleElement.overwriteDataValue(bigDec)
@@ -123,7 +122,6 @@ abstract class BinaryIntegerBaseParser(e: ElementRuntimeData, signed: Boolean)
val nBits = getBitLength(start)
if (nBits == 0) return // zero length is used for outputValueCalc often.
val dis = start.dataInputStream
-
if (!dis.isDefinedForLength(nBits)) {
PE(start, "Insufficient bits in data. Needed %d bit(s) but found only %d available.", nBits, dis.remainingBits.get)
return
@@ -131,11 +129,11 @@ abstract class BinaryIntegerBaseParser(e: ElementRuntimeData, signed: Boolean)
val int: JNumber =
if (signed) {
- if (nBits > 64) { dis.getSignedBigInt(nBits) }
- else { dis.getSignedLong(nBits) }
+ if (nBits > 64) { dis.getSignedBigInt(nBits, start) }
+ else { dis.getSignedLong(nBits, start) }
} else {
- if (nBits >= 64) { dis.getUnsignedBigInt(nBits) }
- else { dis.getUnsignedLong(nBits).toLong }
+ if (nBits >= 64) { dis.getUnsignedBigInt(nBits, start) }
+ else { dis.getUnsignedLong(nBits, start).toLong }
}
start.simpleElement.overwriteDataValue(int)
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/BitOrderChange1.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/BitOrderChange1.scala
deleted file mode 100644
index 7be736f..0000000
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/BitOrderChange1.scala
+++ /dev/null
@@ -1,59 +0,0 @@
-/* Copyright (c) 2012-2014 Tresys Technology, LLC. All rights reserved.
- *
- * Developed by: Tresys Technology, LLC
- * http://www.tresys.com
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy of
- * this software and associated documentation files (the "Software"), to deal with
- * the Software without restriction, including without limitation the rights to
- * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
- * of the Software, and to permit persons to whom the Software is furnished to do
- * so, subject to the following conditions:
- *
- * 1. Redistributions of source code must retain the above copyright notice,
- * this list of conditions and the following disclaimers.
- *
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimers in the
- * documentation and/or other materials provided with the distribution.
- *
- * 3. Neither the names of Tresys Technology, nor the names of its contributors
- * may be used to endorse or promote products derived from this Software
- * without specific prior written permission.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE
- * SOFTWARE.
- */
-
-package edu.illinois.ncsa.daffodil.processors.parsers
-
-import edu.illinois.ncsa.daffodil.schema.annotation.props.gen.BitOrder
-import edu.illinois.ncsa.daffodil.processors.CheckBitOrderAndCharsetEv
-import edu.illinois.ncsa.daffodil.processors.CheckByteAndBitOrderEv
-import edu.illinois.ncsa.daffodil.processors.TermRuntimeData
-
-class BitOrderChangeParser(
- val termRuntimeData: TermRuntimeData,
- val bitOrder: BitOrder,
- val checkByteAndBitOrder: CheckByteAndBitOrderEv,
- val checkBitOrderAndCharset: CheckBitOrderAndCharsetEv)
- extends PrimParser {
-
- override def context = termRuntimeData
-
- override lazy val runtimeDependencies = List(checkByteAndBitOrder, checkBitOrderAndCharset)
-
- def parse(pstate: PState): Unit = {
- checkByteAndBitOrder(pstate)
- checkBitOrderAndCharset(pstate)
-
- termRuntimeData.schemaDefinitionUnless(pstate.bitPos1b % 8 == 1, "Can only change dfdl:bitOrder on a byte boundary")
- pstate.dataInputStream.setBitOrder(bitOrder)
- }
-
-}
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/ByteOrderChange1.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/ByteOrderChange1.scala
deleted file mode 100644
index f037df7..0000000
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/ByteOrderChange1.scala
+++ /dev/null
@@ -1,54 +0,0 @@
-/* Copyright (c) 2012-2014 Tresys Technology, LLC. All rights reserved.
- *
- * Developed by: Tresys Technology, LLC
- * http://www.tresys.com
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy of
- * this software and associated documentation files (the "Software"), to deal with
- * the Software without restriction, including without limitation the rights to
- * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
- * of the Software, and to permit persons to whom the Software is furnished to do
- * so, subject to the following conditions:
- *
- * 1. Redistributions of source code must retain the above copyright notice,
- * this list of conditions and the following disclaimers.
- *
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimers in the
- * documentation and/or other materials provided with the distribution.
- *
- * 3. Neither the names of Tresys Technology, nor the names of its contributors
- * may be used to endorse or promote products derived from this Software
- * without specific prior written permission.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE
- * SOFTWARE.
- */
-
-package edu.illinois.ncsa.daffodil.processors.parsers
-
-import edu.illinois.ncsa.daffodil.processors.BinaryParserUnparserRuntimeMixin
-import edu.illinois.ncsa.daffodil.processors.ByteOrderEv
-import edu.illinois.ncsa.daffodil.processors.CheckByteAndBitOrderEv
-import edu.illinois.ncsa.daffodil.processors.ElementRuntimeData
-
-class ByteOrderChangeParser(val elementRuntimeData: ElementRuntimeData, byteOrdEv: ByteOrderEv, check: CheckByteAndBitOrderEv)
- extends PrimParser with BinaryParserUnparserRuntimeMixin {
-
- override val context = elementRuntimeData
-
- override lazy val runtimeDependencies = List(check)
-
- override def briefXMLAttributes = "byteOrder='%s'".format(byteOrdEv.toBriefXML())
-
- def parse(pstate: PState): Unit = {
- check(pstate)
- setupByteOrder(pstate, elementRuntimeData, byteOrdEv)
- }
-
-}
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/DelimitedParsers.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/DelimitedParsers.scala
index 2ff373b..9b8108f 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/DelimitedParsers.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/DelimitedParsers.scala
@@ -54,7 +54,7 @@ class StringDelimitedParser(
textParser: TextDelimitedParserBase,
fieldDFAEv: FieldDFAParseEv,
isDelimRequired: Boolean)
- extends PrimParserObject(erd)
+ extends TextPrimParserObject(erd)
with CaptureParsingValueLength {
override val runtimeDependencies = List(fieldDFAEv, erd.encInfo.charsetEv)
@@ -90,9 +90,9 @@ class StringDelimitedParser(
val result = {
if (textParser.isInstanceOf[TextDelimitedParserWithEscapeBlock]) {
val s = fieldDFAEv.escapeSchemeEv.get.evaluate(start).asInstanceOf[EscapeSchemeBlockParserHelper]
- textParser.asInstanceOf[TextDelimitedParserWithEscapeBlock].parse(start.dataInputStream, fieldDFA, s.fieldEscDFA, s.blockStartDFA, s.blockEndDFA, delimIter, isDelimRequired)
+ textParser.asInstanceOf[TextDelimitedParserWithEscapeBlock].parse(start, start.dataInputStream, fieldDFA, s.fieldEscDFA, s.blockStartDFA, s.blockEndDFA, delimIter, isDelimRequired)
} else {
- textParser.parse(start.dataInputStream, fieldDFA, delimIter, isDelimRequired)
+ textParser.parse(start, start.dataInputStream, fieldDFA, delimIter, isDelimRequired)
}
}
processResult(result, start)
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/DelimiterParsers.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/DelimiterParsers.scala
index bbde02f..5f08601 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/DelimiterParsers.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/DelimiterParsers.scala
@@ -59,8 +59,7 @@ class DelimiterTextParser(
textParser: TextParser,
positionalInfo: String,
delimiterType: DelimiterTextType.Type)
- extends PrimParser
- with TextParserRuntimeMixin {
+ extends TextPrimParser {
override lazy val runtimeDependencies = rd.encodingInfo.runtimeDependencies
override def context = rd
@@ -112,9 +111,6 @@ class DelimiterTextParser(
}
override def parse(start: PState): Unit = {
-
- setupEncoding(start, rd)
-
val foundDelimiter =
if (delimiterType == DelimiterTextType.Initiator || !start.delimitedParseResult.isDefined) {
//
@@ -142,7 +138,7 @@ class DelimiterTextParser(
// group, which might be difficult to do.
val delimIter = new RemoteTerminatingMarkupAndLocalTypedDelimiterIterator(delimiterType, start.mpstate.delimiters, start.mpstate.delimitersLocalIndexStack)
- val result = textParser.parse(start.dataInputStream, delimIter, true)
+ val result = textParser.parse(start, start.dataInputStream, delimIter, true)
result
} else {
start.delimitedParseResult
@@ -157,7 +153,7 @@ class DelimiterTextParser(
// Consume the found local delimiter
val nChars = foundDelimiter.get.matchedDelimiterValue.get.length
- val wasDelimiterTextSkipped = start.dataInputStream.skipChars(nChars)
+ val wasDelimiterTextSkipped = start.dataInputStream.skipChars(nChars, start)
Assert.invariant(wasDelimiterTextSkipped)
start.clearDelimitedParseResult()
} else {
diff --git a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/ElementCombinator1.scala b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/ElementCombinator1.scala
index 53e43ed..767cfb7 100644
--- a/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/ElementCombinator1.scala
+++ b/daffodil-runtime1/src/main/scala/edu/illinois/ncsa/daffodil/processors/parsers/ElementCombinator1.scala
@@ -42,11 +42,11 @@ import edu.illinois.ncsa.daffodil.api.Diagnostic
import edu.illinois.ncsa.daffodil.exceptions.Assert
import edu.illinois.ncsa.daffodil.processors.ElementRuntimeData
import edu.illinois.ncsa.daffodil.processors.Processor
-import edu.illinois.ncsa.daffodil.processors.RuntimeData
... 2213 lines suppressed ...
--
To stop receiving notification emails like this one, please contact
"commits@daffodil.apache.org" <co...@daffodil.apache.org>.