You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@directory.apache.org by ak...@apache.org on 2004/03/14 03:44:16 UTC
svn commit: rev 9450 - incubator/directory/snickers/trunk/ber/xdocs
Author: akarasulu
Date: Sat Mar 13 18:44:15 2004
New Revision: 9450
Removed:
incubator/directory/snickers/trunk/ber/xdocs/news.xml
incubator/directory/snickers/trunk/ber/xdocs/rt.xml
Modified:
incubator/directory/snickers/trunk/ber/xdocs/eureka.xml
Log:
cleaning up docs
Modified: incubator/directory/snickers/trunk/ber/xdocs/eureka.xml
==============================================================================
--- incubator/directory/snickers/trunk/ber/xdocs/eureka.xml (original)
+++ incubator/directory/snickers/trunk/ber/xdocs/eureka.xml Sat Mar 13 18:44:15 2004
@@ -16,22 +16,22 @@
<source>
[SNIP]
-Wes at Work says:
+Wes says:
I've been thinking about the decoding process a bit over the weekend.
Alex Karasulu says:
k I'm listening
-Wes at Work says:
+Wes says:
and encoding.
-Wes at Work says:
+Wes says:
I'm not sure at the initial stage there will be *one* decoder.
-Wes at Work says:
+Wes says:
We will need some place to hold our TLV tree.
-Wes at Work says:
+Wes says:
and also, I was thinking about really long messages.
Alex Karasulu says:
@@ -42,22 +42,22 @@
[SNIP]
-Wes at Work says:
+Wes says:
We got one part that builds the tree
-Wes at Work says:
+Wes says:
part two should be the translation.
[SNIP]
-Wes at Work says:
+Wes says:
I think the only issue we have is how to handle chunking, and blocking versus
non-blocking code.
-Wes at Work says:
+Wes says:
And also, dealing with really huge messages.
-Wes at Work says:
+Wes says:
It obviously won't make sense to build a TLV tree in its entirety for a huge
search result.
@@ -68,7 +68,7 @@
for encoding there is a mechanism for breaking down large TLVs of simple types
down
-Wes at Work says:
+Wes says:
encoding is a non issue as far as chunking goes.
Alex Karasulu says:
@@ -89,14 +89,14 @@
Alex Karasulu says:
follow me out
-Wes at Work says:
+Wes says:
You give the encoder an output interface, and every time it fills up the byte
bufffer, it spits it out.
Alex Karasulu says:
Strictly talking about decoding and chunk sizes for now.
-Wes at Work says:
+Wes says:
K. decoding then.
Alex Karasulu says:
@@ -111,22 +111,22 @@
Basically there is some threashold u use to judge whether or not the blob is too
big and needs to be chopped up.
-Wes at Work says:
+Wes says:
I did read the section the length.
Alex Karasulu says:
cool
-Wes at Work says:
+Wes says:
I actually printed the whole appendix out and read it.
-Wes at Work says:
+Wes says:
on BER.
Alex Karasulu says:
cool that's what I was reffering to
-Wes at Work says:
+Wes says:
An encoder can choose any one he wants.
Alex Karasulu says:
@@ -138,20 +138,20 @@
The key here is not to keep all the tlvs in memory or the entire encoded buffer
in memory
-Wes at Work says:
+Wes says:
For decoding, there are messages where keeping the intermediate form in memory
is not an issue, and with others, there are.
-Wes at Work says:
+Wes says:
issues.
Alex Karasulu says:
Right depends on the message size
-Wes at Work says:
+Wes says:
The client will want to process most of the messages as a complete object.
-Wes at Work says:
+Wes says:
By definition, it will be in memory.
Alex Karasulu says:
@@ -160,7 +160,7 @@
user to determine how the data is dealt with. Eventually we can take messures
to stream data if we want instead of having it all in memory.
-Wes at Work says:
+Wes says:
Back up just a second.
Alex Karasulu says:
@@ -170,47 +170,47 @@
Alex Karasulu says:
sure talk to me
-Wes at Work says:
+Wes says:
I used this technique in a Btrieve interface I wrote for U. S. South...
-Wes at Work says:
+Wes says:
which I stole from OpenTDS.
Alex Karasulu says:
Btrieve?
-Wes at Work says:
+Wes says:
Yea, an ISAM database.
Alex Karasulu says:
oh ok
-Wes at Work says:
+Wes says:
It used byte buffers to send and retrieve records.
-Wes at Work says:
+Wes says:
I wrote a java class that basically treated the byte array as primitives.
Alex Karasulu says:
cool so you're already of the mindset to keeping the decoding and encoding
memory footprints small
-Wes at Work says:
+Wes says:
That might not work with us though.
-Wes at Work says:
+Wes says:
It might.
-Wes at Work says:
+Wes says:
All we need to know
-Wes at Work says:
+Wes says:
is that this field goes with this TLV.
-Wes at Work says:
+Wes says:
and convert it on the fly.
-Wes at Work says:
+Wes says:
Also, we an simply dump the TLVs when we are done.
Alex Karasulu says:
@@ -244,13 +244,13 @@
Alex Karasulu says:
does that make sense I know its a lil nebulous
-Wes at Work says:
+Wes says:
Keep it simple
Alex Karasulu says:
ok in decoding bytes go in and TLVs come out
-Wes at Work says:
+Wes says:
Right.
Alex Karasulu says:
@@ -259,13 +259,13 @@
Alex Karasulu says:
wit me?
-Wes at Work says:
+Wes says:
Yup.
Alex Karasulu says:
now the TLVs comming out are a peice of the TLV tree
-Wes at Work says:
+Wes says:
You got to be able to handle partial Ts, Ls, and Vs.
Alex Karasulu says:
@@ -278,7 +278,7 @@
Alex Karasulu says:
wit me?
-Wes at Work says:
+Wes says:
right.
Alex Karasulu says:
@@ -293,23 +293,23 @@
Alex Karasulu says:
This is one of those requirements you agree?
-Wes at Work says:
+Wes says:
I don't see that being an issue.
-Wes at Work says:
+Wes says:
The parent needs to know about the children, but not vis a versa.
Alex Karasulu says:
right
-Wes at Work says:
+Wes says:
and I don't see how you are going to be able to assemble an ASN.1 message in a
state driven fashion without making it very complicated.
Alex Karasulu says:
that's our primary issue here
-Wes at Work says:
+Wes says:
and have two decoders hooked together as well.
Alex Karasulu says:
@@ -329,34 +329,34 @@
Alex Karasulu says:
you didn't think this was gonna be a cake walk did ya
-Wes at Work says:
+Wes says:
Hmmmm.
Alex Karasulu says:
you do understand where I was coming from wit the sax and dom stuff right?
-Wes at Work says:
+Wes says:
yea.
-Wes at Work says:
+Wes says:
That I understand.
Alex Karasulu says:
do you think its possible?
-Wes at Work says:
+Wes says:
So you have an event driven ASN.1 parser.
-Wes at Work says:
+Wes says:
I think that's still easy.
-Wes at Work says:
+Wes says:
However, assembling them into the messages is still complicated.
-Wes at Work says:
+Wes says:
every ASN.1 message type would have to be derived from our parser.
-Wes at Work says:
+Wes says:
Then a factory could create the message type based on the application type.
Alex Karasulu says:
@@ -366,7 +366,7 @@
what do you mean by: "every ASN.1 message type would have to be derived from
our parser.
-Wes at Work says:
+Wes says:
You want the ASN.1 messages to be able to assemble themselves? or no.
Alex Karasulu says:
@@ -385,28 +385,28 @@
Alex Karasulu says:
question is do we need a compiler now?
-Wes at Work says:
+Wes says:
Right.
-Wes at Work says:
+Wes says:
Factory returns the ASN.1 message on the application tag.
Alex Karasulu says:
right I see where your going with the design
-Wes at Work says:
+Wes says:
the parser then passes everything to the ASN.reader interface,
-Wes at Work says:
+Wes says:
SAX like.
Alex Karasulu says:
Hmm sounds like it should be very possible
-Wes at Work says:
+Wes says:
of the application object.
-Wes at Work says:
+Wes says:
who knows how to assemble himself.
Alex Karasulu says:
@@ -418,10 +418,10 @@
Alex Karasulu says:
I wonder if other ASN.1 tools have this sax like mechanism already in place.
-Wes at Work says:
+Wes says:
But how do we handle ASN.1 messages which need to be streamed.
-Wes at Work says:
+Wes says:
like a huge search result.
Alex Karasulu says:
@@ -433,15 +433,15 @@
Alex Karasulu says:
sorry n+1
-Wes at Work says:
+Wes says:
You have a search result tight.
-Wes at Work says:
+Wes says:
Tag = Applicationz
Length = 00
Value = Search Results
-Wes at Work says:
+Wes says:
Now V is made up of thousands of result messages.
Alex Karasulu says:
@@ -454,16 +454,16 @@
Alex Karasulu says:
n+1 messages
-Wes at Work says:
+Wes says:
Ah.
-Wes at Work says:
+Wes says:
But are they wrapped in an application TLV?
Alex Karasulu says:
but think of a large blob of data
-Wes at Work says:
+Wes says:
or is it just one stream of TLVs.
Alex Karasulu says:
@@ -475,14 +475,14 @@
response types have you know some enumeration values to determine which
response type the top level envelope or application TLV represents
-Wes at Work says:
+Wes says:
Right.
Alex Karasulu says:
but your question is valid for say a single SearchEntryResponse where one of
the attributes is a huge binary chunk
-Wes at Work says:
+Wes says:
So the event firing for the top level envelope will be different than the TLVs
which are part of the envelope.
@@ -496,10 +496,10 @@
Alex Karasulu says:
same one every time
-Wes at Work says:
+Wes says:
Right, but not after the entire TLV is read into memry.
-Wes at Work says:
+Wes says:
that would defeat our SAX based parser.
Alex Karasulu says:
@@ -511,19 +511,19 @@
Alex Karasulu says:
exactly
-Wes at Work says:
+Wes says:
I'm with you.
Alex Karasulu says:
you would get a start_ldap_message event
-Wes at Work says:
+Wes says:
Actually,
-Wes at Work says:
+Wes says:
for the envelope, you would need to hit the factory.
-Wes at Work says:
+Wes says:
to get the appropriate LDAP message.
Alex Karasulu says:
@@ -535,10 +535,10 @@
fire its arrival. Like sax where you say start tag for this element then the
contained elemenets then close tags etc.
-Wes at Work says:
+Wes says:
Got ya.
-Wes at Work says:
+Wes says:
I think that's pretty cool.
Alex Karasulu says:
@@ -563,10 +563,10 @@
Alex Karasulu says:
y.
-Wes at Work says:
+Wes says:
That's fine for us. We have control over the encoding.
-Wes at Work says:
+Wes says:
We won't be so lucky on the inbound side.
Alex Karasulu says:
@@ -600,16 +600,16 @@
TLVs into the indeterminate form and spits those out in peices rather than the
one large primitive TLV.
-Wes at Work says:
+Wes says:
What does that buy us?
Alex Karasulu says:
streaming
-Wes at Work says:
+Wes says:
Is not the ASN message gonna re-assemble it anyways.
-Wes at Work says:
+Wes says:
Do you still end up with 200K picture in the ASN.1 message.
Alex Karasulu says:
@@ -619,7 +619,7 @@
Alex Karasulu says:
the other codec is Type to TLV
-Wes at Work says:
+Wes says:
If we are using a SAX based parser, then the Type will be assembling itself as
the TLVs are decoded and fired.
@@ -630,7 +630,7 @@
Alex Karasulu says:
right
-Wes at Work says:
+Wes says:
At some point, you are going to have to put your faith in the garbage collector.
Alex Karasulu says:
@@ -639,14 +639,14 @@
Alex Karasulu says:
keep that lean and mean - why you ask
-Wes at Work says:
+Wes says:
Also, if you want a truly small memory footprint, then you could put stuff like
that in a small embedded database.
Alex Karasulu says:
well the TLV to Type code can be made lean and mean too
-Wes at Work says:
+Wes says:
I just don't think at this stage that we need to be all that worried about huge
blocks of binary data.
@@ -654,7 +654,7 @@
right we use referrals to data on disk to manage large peices of data that
needsto be streamed but this we can do later.
-Wes at Work says:
+Wes says:
Exactly.
Alex Karasulu says:
@@ -669,16 +669,16 @@
Alex Karasulu says:
I don't care if the first implementation is a hog
-Wes at Work says:
+Wes says:
The BER stuff today doesn't deal with this.
-Wes at Work says:
+Wes says:
It doesn't care.
Alex Karasulu says:
for large peices of data
-Wes at Work says:
+Wes says:
It's an application issue.
Alex Karasulu says:
@@ -692,7 +692,7 @@
Alex Karasulu says:
wit me?
-Wes at Work says:
+Wes says:
K.
Alex Karasulu says:
@@ -707,22 +707,22 @@
Alex Karasulu says:
that's all you and Jeff with the C based version of this thang
-Wes at Work says:
+Wes says:
Right.
-Wes at Work says:
+Wes says:
You won't find much other ASN.1 stuff out there.
-Wes at Work says:
+Wes says:
I'm comfortable that no one is doing it this way, either.
-Wes at Work says:
+Wes says:
It will make it unqiuely, Apache.
Alex Karasulu says:
Ok. Let's touch base in a day or two to regroup
-Wes at Work says:
+Wes says:
Do you think ASN.1 is going to die?
Alex Karasulu says:
@@ -734,7 +734,7 @@
Alex Karasulu says:
ASN.1 is awesome stuff
-Wes at Work says:
+Wes says:
We'll see.
Alex Karasulu says:
@@ -743,7 +743,7 @@
Alex Karasulu says:
what's the alternative?
-Wes at Work says:
+Wes says:
XML is what everyone is using now.
Alex Karasulu says:
@@ -755,20 +755,20 @@
Alex Karasulu says:
ASN.1 can go to BER, PER, XER, and DER
-Wes at Work says:
+Wes says:
Yes.
Alex Karasulu says:
the encoding does not effect the ASN.1 specification and that is what makes
ASN.1 a winner always.
-Wes at Work says:
+Wes says:
Slapping XML on ASN.1 ain't the same.
Alex Karasulu says:
the XML format is just for the encoding of the data types
-Wes at Work says:
+Wes says:
I agree that ASN.1 is a good protocol.
Alex Karasulu says:
@@ -777,10 +777,10 @@
Alex Karasulu says:
it kicks ass I think and is here to stay.
-Wes at Work says:
+Wes says:
If we do this, we are going to go backwards right?
-Wes at Work says:
+Wes says:
Do the compiler last.
Alex Karasulu says:
@@ -789,13 +789,13 @@
Alex Karasulu says:
yeah that might be the case or we can work it together.
-Wes at Work says:
+Wes says:
You need to let me work this.
Alex Karasulu says:
I can do the compiler with you and you can handle the runtime
-Wes at Work says:
+Wes says:
You got other things to do.
Alex Karasulu says:
@@ -804,16 +804,16 @@
Alex Karasulu says:
I'm just a follower
-Wes at Work says:
+Wes says:
I won't mind help with the compiler.
-Wes at Work says:
+Wes says:
Just don't get going on it any time soon
Alex Karasulu says:
sure I have extensive javacc and antlr experience
-Wes at Work says:
+Wes says:
Deal.
Alex Karasulu says:
@@ -825,16 +825,16 @@
Alex Karasulu says:
I'll catch ya later I need to hit the head
-Wes at Work says:
+Wes says:
Talk about the decoder's stream.
-Wes at Work says:
+Wes says:
K
Alex Karasulu says:
ttyl
-Wes at Work says:
+Wes says:
Talk later then.
Alex Karasulu says:
@@ -846,7 +846,7 @@
Alex Karasulu says:
what about the decoder's stream.
-Wes at Work says:
+Wes says:
So, how do we feed the decoder then.
Alex Karasulu says:
@@ -864,7 +864,7 @@
Alex Karasulu says:
wit me?
-Wes at Work says:
+Wes says:
right.
Alex Karasulu says:
@@ -879,13 +879,13 @@
Alex Karasulu says:
its down again damn
-Wes at Work says:
+Wes says:
I got my update
Alex Karasulu says:
cool
-Wes at Work says:
+Wes says:
Must of brought it down.
Alex Karasulu says:
@@ -909,17 +909,17 @@
Alex Karasulu says:
Then we use those interfaces to implement the ASN.1 stuff.
-Wes at Work says:
+Wes says:
Right.
Alex Karasulu says:
We do this in the snickers area but put back as much into the commons codec as
we can. You game with this strategy?
-Wes at Work says:
+Wes says:
Yea, that's fine.
-Wes at Work says:
+Wes says:
I'll check out commons code as soon as it comes up.
</source>