[asn.1] Decoder service design ( was: Paper on original SNACC)

[Alex Karasulu <ao...@bellsouth.net>]

Wes,

> From: "Wes McKean" <wm...@bellsouth.net>
> I don't have permission to access the snacc.pdf.  What do I do?

Sorry dude I just set the perms to allow access.  Give it another 
try.

> I'm going to go ahead and get started on the BER encoder/decoder.  For now,
> I will keep the interface the same as the current one.  Namely, the
> encoder/decoder will read from and write to a stream.  The primary
> difference is I will detect non-blocking streams.  If anyone has any idea on
> this, let me know.  It might truly be difficult to implement the decoder as
> non-blocking, if that means decoding TLVs in chunks.  

TLVs? Forgive me I'm bad with acronyms.  Oh and another thing take a look at
the new front end stuff in eve/frontend/common/api where there are events 
defined.  Namely take a look at the InputEvent which carries with it a 
direct nio buffer.  Look towards defining your interfaces around this.

> That would require the decoder to almost be like a state machine.  
> If we decide that is the route to go, then fine with me.  For getting 
> started, I will use just a regular old stream.

I've given limited time to thinking about this one.  Basically we have
chunks of data that will be delivered over to the decoder in the form
of input events associated with a client.  The InputEvent (caught a bug)
should extend the ClientEvent (which it currently does not - my bad!) 
which has the ClientKey associated with it.  The InputEvent is a SEDA
event.

So what you do with the buffer content is upto you but here are some
thoughts.  I'm thinking out loud here.  

First of let me state a couple things about the existing state of 
the frontend.  Don't worry about manipulating the byte buffer with 
a concern for effecting other peices of code that work with the same 
input event.  When you access the buffer you really have a read only 
copy of the buffer which is backed by the direct buffer allocated 
from the buffer pool.  Meaning you can advance and read and do a 
bunch o stuff to change where you are in the buffer but you cannot 
affect the content to mess with other code that may also have the 
event delivered to it.  So advancing in the buffer will not throw of
other listeners that have the event delivered to them.

You got the buffer.  What you do with it is upto you.  

Now the decoder service may eventually be used to decode more than
just input comming in from a client stream down the road.  Just wanted
to mention this but I don't think you should factor it into what you're
dealing with right now.  This is for decoding stuff through the LdapContext
JNDI interface which still use encoded ASN.1 for Controls.  I'll deal
with this later but just thought I'd mention it.

Ok we want to basically not block or rather dedicate a thread while 
processing an entire message.  That would mean two threads used per
request.  One thread to deliver the event, and another to read and 
decode it.  So this is the main problem.

Perhaps we can setup a small framework here inside the decoder with
a callback.  When a new message is detected, a non-direct buffer can 
be assigned to span the time required to decode the request.  This
buffer collects the data coming in from the direct buffer in the
InputEvent.  The input buffers are copied into the collector.  Now
if the decoder was a state machine then we would not have to do a
copy right?  The state machine based decoder could read from the 
direct buffer and between chunks freeze its state until the next
chunk appears.  That would be the best approach as you mentioned but
I don't know how complex that would be.  And when processing a chunk
the state machine should return even when it has not reached a 
terminal state.

Now if we continue on the less efficient collector buffer approach 
then you need to have the decoder detect the start of a new PDU and
the end of a PDU (Protocol Data Unit an ASN.1 message envelope).  It 
must detect the start of the PDU to create and assign a collector 
buffer to the current request.  It must detect the end of the PDU
to basically trigger the true decoding machinery to slurp up the
content in the collector buffer and decode it which can be done in
one step and can be blocking.  You can use 1.4 regex functionality
to detect the PDU start sequence and the end sequence of bytes 
comming in as you transfer bytes from the direct buffer into the
collector buffer.  Again this is sounding convoluted too.

What are your thoughts?  There has got to be a better way perhaps
we just need to sleep on it.

Alex