ASN.1 encoding experiment

[Emmanuel Lécharny <el...@gmail.com>]

Hi guys,

I had some time last evening and during the following insomnia to do
some tests and bechnmark using the reverse order encoding of ASN.1 PDU
(the PDU is encoded starting from the end, which allows the encoding of
Length when this length is known, which saves the computeLength() call
and the storage of intermediary values, leading to less GC)

Last week-end, I played with a Kerberos PDU (Ticket), and today I used
the AddRequest message.

It's more than 2 times faster.

I confirmed the result with JMH, which shown a 2x faster encoding
(20,529,987 encoding/s vs 9,393,737 encoding/s, on a smallest PDU, the
VLV request control)

Actually, modifying the code to adapt the various messages is not that
painful, it's more or less a matter of 1h per message, tests included.

I'll keep going for the moment, and I'll see if we can go any further
with the Decorator removal later on.

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org

Re: ASN.1 encoding experiment

[Emmanuel Lécharny <el...@gmail.com>]

A bit more about it.


Here is the code we are using for the AddRequest encoding :

public final class AddRequestDecorator extends
AbstractSingleReplyRequestDecorator<AddRequest> implements
    AddRequest
{
    /** The add request length */
    private int addRequestLength;

    /** The Entry length */
    private int entryLength;

    /** The list of all attributes length */
    private List<Integer> attributesLength;

    /** The list of all attributes Id bytes */
    private List<byte[]> attributeIds;

    /** The list of all vals length */
    private List<Integer> valuesLength;

    /** The bytes containing the Dn */
    private byte[] dnBytes;
...

    public int computeLength()
    {
        AddRequest addRequest = getDecorated();
        Entry entry = addRequest.getEntry();

        if ( entry == null )
        {
            throw new IllegalArgumentException( I18n.err(
I18n.ERR_05002_ENTRY_NULL_VALUE ) );
        }

        dnBytes = Strings.getBytesUtf8( entry.getDn().getName() );
        int dnLen = dnBytes.length;

        // The entry Dn
        addRequestLength = 1 + TLV.getNbBytes( dnLen ) + dnLen;

        // The attributes sequence
        entryLength = 0;

        if ( entry.size() != 0 )
        {
            attributesLength = new LinkedList<>();
            attributeIds = new LinkedList<>();
            valuesLength = new LinkedList<>();

            // Compute the attributes length
            for ( Attribute attribute : entry )
            {
                int localAttributeLength;
                int localValuesLength;

                // Get the type length
                byte[] attributeIdBytes = Strings.getBytesUtf8(
attribute.getUpId() );
                attributeIds.add( attributeIdBytes );

                int idLength = attributeIdBytes.length;
                localAttributeLength = 1 + TLV.getNbBytes( idLength ) +
idLength;

                // The values
                if ( attribute.size() != 0 )
                {
                    localValuesLength = 0;

                    for ( Value value : attribute )
                    {
                        if ( value.getBytes() == null )
                        {
                            localValuesLength += 1 + 1;
                        }
                        else
                        {
                            int valueLength = value.getBytes().length;
                            localValuesLength += 1 + TLV.getNbBytes(
valueLength ) + valueLength;
                        }
                    }

                    localAttributeLength += 1 + TLV.getNbBytes(
localValuesLength ) + localValuesLength;
                }
                else
                {
                    // No value : we still have to store the
encapsulating Sequence
                    localValuesLength = 1 + 1;
                    localAttributeLength += 1 + 1 + localValuesLength;
                }

                // add the attribute length to the attributes length
                entryLength += 1 + TLV.getNbBytes( localAttributeLength
) + localAttributeLength;

                attributesLength.add( localAttributeLength );
                valuesLength.add( localValuesLength );
            }
        }

        addRequestLength += 1 + TLV.getNbBytes( entryLength ) + entryLength;

        // Return the result.
        return 1 + TLV.getNbBytes( addRequestLength ) + addRequestLength;
    }

...
    public ByteBuffer encode( ByteBuffer buffer ) throws EncoderException
    {
        try
        {
            // The AddRequest Tag
            buffer.put( LdapCodecConstants.ADD_REQUEST_TAG );
            buffer.put( TLV.getBytes( addRequestLength ) );

            // The entry
            BerValue.encode( buffer, dnBytes );

            // The attributes sequence
            buffer.put( UniversalTag.SEQUENCE.getValue() );
            buffer.put( TLV.getBytes( entryLength ) );

            // The partial attribute list
            Entry entry = getEntry();

            if ( entry.size() != 0 )
            {
                int attributeNumber = 0;

                // Compute the attributes length
                for ( Attribute attribute : entry )
                {
                    // The attributes list sequence
                    buffer.put( UniversalTag.SEQUENCE.getValue() );
                    int localAttributeLength = attributesLength.get(
attributeNumber );
                    buffer.put( TLV.getBytes( localAttributeLength ) );

                    // The attribute type
                    BerValue.encode( buffer, attributeIds.get(
attributeNumber ) );

                    // The values
                    buffer.put( UniversalTag.SET.getValue() );
                    int localValuesLength = valuesLength.get(
attributeNumber );
                    buffer.put( TLV.getBytes( localValuesLength ) );

                    if ( attribute.size() != 0 )
                    {
                        for ( Value value : attribute )
                        {
                            BerValue.encode( buffer, value.getBytes() );
                        }
                    }
                    else
                    {
                        BerValue.encode( buffer, Strings.EMPTY_BYTES );
                    }

                    // Go to the next attribute number
                    attributeNumber++;
                }
            }

            return buffer;
        }
        catch ( BufferOverflowException boe )
        {
            throw new EncoderException( I18n.err(
I18n.ERR_08212_PDU_BUFFER_TOO_SMALL, boe.getMessage() ) );
        }
    }



And here the new encoder code :

    public void encode( Asn1Buffer buffer )
    {
        int addRequestPos = buffer.getPos();

        // The partial attribute list
        Entry entry = getEntry();

        if ( entry.size() != 0 )
        {
            // Compute the attributes length
            for ( Attribute attribute : entry )
            {
                // The values
                int attributePos = buffer.getPos();

                if ( attribute.size() != 0 )
                {
                    for ( Value value : attribute )
                    {
                        BerValue.encodeOctetString( buffer,
value.getBytes() );
                    }
                }
                else
                {
                    BerValue.encodeOctetString( buffer,
Strings.EMPTY_BYTES );
                }

                TLV.encodeLength( buffer, attributePos - buffer.getPos() );
                buffer.put( UniversalTag.SET.getValue() );

                // The attribute ID
                BerValue.encodeOctetString( buffer,
Strings.getBytesUtf8( attribute.getUpId() ) );

                // The attributes list sequence
                TLV.encodeLength( buffer, attributePos - buffer.getPos() );
                buffer.put( UniversalTag.SEQUENCE.getValue() );
            }
        }

        // The attributes sequence
        TLV.encodeLength( buffer, addRequestPos - buffer.getPos() );
        buffer.put( UniversalTag.SEQUENCE.getValue() );

        // The entry DN
        BerValue.encodeOctetString( buffer, dnBytes );

        // The AddRequest Tag
        TLV.encodeLength( buffer, addRequestPos - buffer.getPos() );
        buffer.put( LdapCodecConstants.ADD_REQUEST_TAG );
    }



I think it says it all...

-- 
Emmanuel Lecharny

Symas.com
directory.apache.org