You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@daffodil.apache.org by "Costello, Roger L." <co...@mitre.org> on 2019/02/12 21:27:26 UTC
Can type="hexBinary" be used on a 31-bit field?
Hello DFDL community,
My input contains a field that is 32 bits. The last bit indicates the type of the field. If the last bit is 1, then the prior 31 bits represents an address. I could declare that 31-bit field like this:
<xs:element name="Hint_Name_Table_RVA" type="unsignedint31" />
The will produce XML such as this:
<Hint_Name_Table_RVA>58222</Hint_Name_Table_RVA>
Ugh!
I don't like showing an address as an unsigned integer. I prefer showing addresses as hex. Is there anyway to show a 31-bit field as a hex value?
/Roger
Re: Can type="hexBinary" be used on a 31-bit field?
Posted by Steve Lawrence <sl...@apache.org>.
Actually, we do have a function that will work: dfdl:hexBinary(). This
function takes in an numeric value and converts it to hexBinary. So
first parse your 31 bit field as an int (which will take into account
byteOrder) then use inputValueCalc to convert that int value to a hex
binary value for display. So something like:
<xs:element name="Int" type="xs:int"
dfdl:lengthKind="explicit"
dfdl:length="31"
dfdl:lengthUnits="bits"
dfdl:byteOrder="littleEndian"
dfdl:bitOrder="leastSignificantBitFirst" />
<xs:element name="HB" type="xs:hexBinary"
dfdl:inputValueCalc="{ dfdl:hexBinary(../Int) }" />
A new function that can convert a number to a string of any base might
still potentially be useful, but it shouldn't be needed to support what
you're tying to do.
- Steve
On 2/12/19 8:22 PM, Steve Lawrence wrote:
> Sortof. Daffodil does support non-byte size hexBinary. The resulting
> hexBinary in the infoset is padded with zeros to reach a full byte.
>
> However, I don't think it would behave the way you would like it to work
> since you're working in byteOrder="littleEndian" and
> bitOrder="leastSignificantBitFirst".
>
> As an example, let's say we have the following data and 0-based bit
> positions:
>
> Pos: 7 0 15 8 23 16 31 24
> Bin: 11011010 11111111 00001101 11000001
> Hex: D A F F 0 D C 1
>
> And assume we have the following schema to parse the first 31 bits of
> that data.
>
> <xs:element name="HB" type="xs:hexBinary"
> dfdl:lengthKind="explicit"
> dfdl:length="31"
> dfdl:lengthUnits="bits"
> dfdl:byteOrder="littleEndian"
> dfdl:bitOrder="leastSignificantBitFirst" />
>
> Since this is littleEndian/leastSignificantBitFirst, one might expect
> the following infoset, which is probably the memory address you would want:
>
> <HB>410DFFDA</HB>
>
> So the order of the bytes are flipped and only the 7 least significant
> bits of the 4th byte are included. The most significant bit of the last
> byte becomes a 0 resulting in 0x41.
>
> However, that isn't the case (at least in the next version of Daffodil
> where a bug is fixed). What actually happens is that since the type is
> hexBinary, byteOrder is completely ignored and the resulting infoset
> becomes:
>
> <HB>DAFF0D41</HB>
>
> So the most significant bit of last byte is still not included due to
> the bitOrder, but the bytes aren't flipped because byteOrder is ignored.
> So this doesn't really represent the memory address you want.
>
> However, it sounds to me like maybe you don't actually want hexBinary in
> the infoset. The data isn't an array of bytes, which is what hexBinary
> is usually used to represent. The data is really just an integer (which
> does obey byteOrder) that you just so happen to want to represent in
> base-16 in the infoset, because that integer represents a memory address
> and that's how memory addresses are most often displayed.
>
> And I can imagine cases (and I think I've come across some) where one
> would prefer on octal, trinary, or binary representation too. So even if
> we did support a mode where hexBinary does obey byteOrder, it wouldn't
> solve those other cases.
>
> So maybe the right way to support this is via dfdl:inputValueCalc and an
> XPath function that could convert between bases. Unfortunately, no
> functions in the DFDL spec allow for such a conversion. One might be
> able to come up with an ugly DFDL expression that could convert a fixed
> length number to base-16, but something like the dp:radix-convert() [1]
> seems like something that might be worth adding to DFDL/Daffodil.
>
> - Steve
>
> [1]
> https://www.ibm.com/support/knowledgecenter/en/SS9H2Y_7.7.0/com.ibm.dp.doc/radix-convert_cryptographicfunction.html
>
>
>
> On 2/12/19 4:27 PM, Costello, Roger L. wrote:
>> Hello DFDL community,
>>
>> My input contains a field that is 32 bits. The last bit indicates the type of the field. If the last bit is 1, then the prior 31 bits represents an address. I could declare that 31-bit field like this:
>>
>> <xs:element name="Hint_Name_Table_RVA" type="unsignedint31" />
>>
>> The will produce XML such as this:
>>
>> <Hint_Name_Table_RVA>58222</Hint_Name_Table_RVA>
>>
>> Ugh!
>>
>> I don't like showing an address as an unsigned integer. I prefer showing addresses as hex. Is there anyway to show a 31-bit field as a hex value?
>>
>> /Roger
>>
>
Re: Can type="hexBinary" be used on a 31-bit field?
Posted by Steve Lawrence <sl...@apache.org>.
Sortof. Daffodil does support non-byte size hexBinary. The resulting
hexBinary in the infoset is padded with zeros to reach a full byte.
However, I don't think it would behave the way you would like it to work
since you're working in byteOrder="littleEndian" and
bitOrder="leastSignificantBitFirst".
As an example, let's say we have the following data and 0-based bit
positions:
Pos: 7 0 15 8 23 16 31 24
Bin: 11011010 11111111 00001101 11000001
Hex: D A F F 0 D C 1
And assume we have the following schema to parse the first 31 bits of
that data.
<xs:element name="HB" type="xs:hexBinary"
dfdl:lengthKind="explicit"
dfdl:length="31"
dfdl:lengthUnits="bits"
dfdl:byteOrder="littleEndian"
dfdl:bitOrder="leastSignificantBitFirst" />
Since this is littleEndian/leastSignificantBitFirst, one might expect
the following infoset, which is probably the memory address you would want:
<HB>410DFFDA</HB>
So the order of the bytes are flipped and only the 7 least significant
bits of the 4th byte are included. The most significant bit of the last
byte becomes a 0 resulting in 0x41.
However, that isn't the case (at least in the next version of Daffodil
where a bug is fixed). What actually happens is that since the type is
hexBinary, byteOrder is completely ignored and the resulting infoset
becomes:
<HB>DAFF0D41</HB>
So the most significant bit of last byte is still not included due to
the bitOrder, but the bytes aren't flipped because byteOrder is ignored.
So this doesn't really represent the memory address you want.
However, it sounds to me like maybe you don't actually want hexBinary in
the infoset. The data isn't an array of bytes, which is what hexBinary
is usually used to represent. The data is really just an integer (which
does obey byteOrder) that you just so happen to want to represent in
base-16 in the infoset, because that integer represents a memory address
and that's how memory addresses are most often displayed.
And I can imagine cases (and I think I've come across some) where one
would prefer on octal, trinary, or binary representation too. So even if
we did support a mode where hexBinary does obey byteOrder, it wouldn't
solve those other cases.
So maybe the right way to support this is via dfdl:inputValueCalc and an
XPath function that could convert between bases. Unfortunately, no
functions in the DFDL spec allow for such a conversion. One might be
able to come up with an ugly DFDL expression that could convert a fixed
length number to base-16, but something like the dp:radix-convert() [1]
seems like something that might be worth adding to DFDL/Daffodil.
- Steve
[1]
https://www.ibm.com/support/knowledgecenter/en/SS9H2Y_7.7.0/com.ibm.dp.doc/radix-convert_cryptographicfunction.html
On 2/12/19 4:27 PM, Costello, Roger L. wrote:
> Hello DFDL community,
>
> My input contains a field that is 32 bits. The last bit indicates the type of the field. If the last bit is 1, then the prior 31 bits represents an address. I could declare that 31-bit field like this:
>
> <xs:element name="Hint_Name_Table_RVA" type="unsignedint31" />
>
> The will produce XML such as this:
>
> <Hint_Name_Table_RVA>58222</Hint_Name_Table_RVA>
>
> Ugh!
>
> I don't like showing an address as an unsigned integer. I prefer showing addresses as hex. Is there anyway to show a 31-bit field as a hex value?
>
> /Roger
>