Data Exchange: LSByte first, LSBit first

["Costello, Roger L." <co...@mitre.org>]

Hello DFDL community,

The data specification for my input file says this:
Data Exchange
Data sending order - Little Endian (Intel standard). LSByte first, LSBit first.

I think that means: for a 16-bit unsigned integer the number one ( 1 ) is represented this way:

1000 0000 0000 0000

The "1" is the LSBit in the LSByte ... do you agree?

How to specify the DFDL properties for this file format?

I discovered that this isn't correct:

<xs:element name="input">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="two-byte-integer" type="unsignedint16"
                dfdl:byteOrder="littleEndian"
                dfdl:bitOrder="leastSignificantBitFirst"/>
        </xs:sequence>
    </xs:complexType>
</xs:element>
As it generates the wrong value:

<input>
  <two-byte-integer>128</two-byte-integer>
</input>

Eek!

It seems that I will have to create a hidden group which places each bit into a different element. Then, create an element which uses inputValueCalc to reverse the bytes and then reverse the bits in each byte and then concatenate the bits and then cast to unsignedInt ... is that correct? Eek! That's a lot of work. Is there a simpler solution?

/Roger

Re: Data Exchange: LSByte first, LSBit first

["Beckerle, Mike" <mb...@tresys.com>]

I think the data should be a 16 bit unsigned integer for the number 1 is:


(byte 1)     (byte 2)

0000 0001 0000 0000


When dealing with data which has the bits numbered as bitOrder 'leastSignificantBitFirst', and littleEndian byte order it is best to lay out the bits and bytes from *right to left*.


Then your number looks like:


(byte 2)     (byte 1)

0000 0000 0000 0001


Which looks exactly as it should for a 16 bit integer with numeric value 1.


The LSBit first just means we call that bit that has the 1 in it, bit position 1, not bit position 7. It's just about the numbering scheme for how we choose to talk about the bits.


If your data has converted the original numeric value 1 into the value 128, that's reversing the bits of the byte, which is an entirely different concept of bit ordering. To date, I have never seen this, and there's a reason for that: It indicates that bytes of data were transmitted one way, but received asymmetrically. The transmission did not preserve the data, it corrupted it. Real systems don't do this (at least that I've seen).





________________________________
From: Costello, Roger L. <co...@mitre.org>
Sent: Tuesday, May 28, 2019 1:44:04 PM
To: users@daffodil.apache.org
Subject: Data Exchange: LSByte first, LSBit first


Hello DFDL community,



The data specification for my input file says this:

Data Exchange

Data sending order – Little Endian (Intel standard). LSByte first, LSBit first.



I think that means: for a 16-bit unsigned integer the number one ( 1 ) is represented this way:



1000 0000 0000 0000



The “1” is the LSBit in the LSByte … do you agree?



How to specify the DFDL properties for this file format?



I discovered that this isn’t correct:



<xs:element name="input">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="two-byte-integer" type="unsignedint16"
                dfdl:byteOrder="littleEndian"
                dfdl:bitOrder="leastSignificantBitFirst"/>
        </xs:sequence>
    </xs:complexType>
</xs:element>

As it generates the wrong value:



<input>
  <two-byte-integer>128</two-byte-integer>
</input>



Eek!



It seems that I will have to create a hidden group which places each bit into a different element. Then, create an element which uses inputValueCalc to reverse the bytes and then reverse the bits in each byte and then concatenate the bits and then cast to unsignedInt … is that correct? Eek! That’s a lot of work. Is there a simpler solution?



/Roger