You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@directory.apache.org by bu...@apache.org on 2018/01/13 13:03:27 UTC
svn commit: r1023789 - in /websites/staging/directory/trunk/content: ./
api/dev-guide/ api/dev-guide/images/
Author: buildbot
Date: Sat Jan 13 13:03:26 2018
New Revision: 1023789
Log:
Staging update by buildbot for directory
Added:
websites/staging/directory/trunk/content/api/dev-guide/4.1-asn1-tlv.html
websites/staging/directory/trunk/content/api/dev-guide/images/TLVs.png (with props)
websites/staging/directory/trunk/content/api/dev-guide/images/grammar-action.graphml (with props)
websites/staging/directory/trunk/content/api/dev-guide/images/grammar-action.png (with props)
websites/staging/directory/trunk/content/api/dev-guide/images/tlv-bervalue.graphml (with props)
websites/staging/directory/trunk/content/api/dev-guide/images/tlv-bervalue.png (with props)
Modified:
websites/staging/directory/trunk/content/ (props changed)
websites/staging/directory/trunk/content/api/dev-guide/1-introduction.html
websites/staging/directory/trunk/content/api/dev-guide/4-asn1.html
Propchange: websites/staging/directory/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Sat Jan 13 13:03:26 2018
@@ -1 +1 @@
-1820988
+1821059
Modified: websites/staging/directory/trunk/content/api/dev-guide/1-introduction.html
==============================================================================
--- websites/staging/directory/trunk/content/api/dev-guide/1-introduction.html (original)
+++ websites/staging/directory/trunk/content/api/dev-guide/1-introduction.html Sat Jan 13 13:03:26 2018
@@ -189,7 +189,8 @@ h2:hover > .headerlink, h3:hover > .head
<ul>
<li><a href="2-general-structure.html">2 - General structure</a></li>
<li><a href="3-building.html">3 - Building</a></li>
-<li><a href="4-asn1.html">4 - ASN/1</a></li>
+<li><a href="4-asn1.html">4 - ASN/1</a>
+** <a href="4.1-asn1-tlv.html">4.1 - ASN/1 TLV</a></li>
<li><a href="5-network.html">5 - Network</a></li>
<li><a href="6-codec.html">6 - Encoding/Decoding</a></li>
<li><a href="7-ldap-messages.html">7 - LDAP Messages</a></li>
Modified: websites/staging/directory/trunk/content/api/dev-guide/4-asn1.html
==============================================================================
--- websites/staging/directory/trunk/content/api/dev-guide/4-asn1.html (original)
+++ websites/staging/directory/trunk/content/api/dev-guide/4-asn1.html Sat Jan 13 13:03:26 2018
@@ -165,7 +165,7 @@
</div>
<div class="nav_next">
- <a href="5-network.html">5 - Network</a>
+ <a href="4.1-asn1-tlv.html">4.1 - ASN/1 TLV</a>
</div>
<div class="clearfix"></div>
@@ -237,6 +237,7 @@ h2:hover > .headerlink, h3:hover > .head
<h3 id="the-state-machine">The state machine<a class="headerlink" href="#the-state-machine" title="Permanent link">¶</a></h3>
<p>So we decode a message using a state machine, which basically transit from one state to another, and optionally execute an action in between :</p>
<p><img alt="State Machine transition" src="images/sm-transition.png" /></p>
+<p>Now, let's see a real example =</p>
<div class="nav">
@@ -252,7 +253,7 @@ h2:hover > .headerlink, h3:hover > .head
</div>
<div class="nav_next">
- <a href="5-network.html">5 - Network</a>
+ <a href="4.1-asn1-tlv.html">4.1 - ASN/1 TLV</a>
</div>
<div class="clearfix"></div>
Added: websites/staging/directory/trunk/content/api/dev-guide/4.1-asn1-tlv.html
==============================================================================
--- websites/staging/directory/trunk/content/api/dev-guide/4.1-asn1-tlv.html (added)
+++ websites/staging/directory/trunk/content/api/dev-guide/4.1-asn1-tlv.html Sat Jan 13 13:03:26 2018
@@ -0,0 +1,296 @@
+<!DOCTYPE html>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<html>
+ <head>
+ <title>4-1 - ASN/1 TLV — Apache Directory</title>
+
+ <link href="./../../css/common.css" rel="stylesheet" type="text/css">
+ <link href="./../../css/brown.css" rel="stylesheet" type="text/css">
+
+
+ <link rel="shortcut icon" href="./../../images/api-icon_16x16.png">
+
+ <!-- Google Analytics -->
+ <script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
+ <script type="text/javascript">
+ _uacct = "UA-1358462-1";
+ urchinTracker();
+ </script>
+ </head>
+ <body>
+ <div id="container">
+ <div id="header">
+ <div id="subProjectsNavBar">
+ <a href="./../../">
+
+ Main
+
+ </a>
+ |
+ <a href="./../../apacheds">
+
+ ApacheDS
+
+ </a>
+ |
+ <a href="./../../studio">
+
+ Studio
+
+ </a>
+ |
+ <a href="./../../api">
+
+ <STRONG>LDAP API</STRONG>
+
+ </a>
+ |
+ <a href="./../../mavibot">
+
+ Mavibot
+
+ </a>
+ |
+ <a href="./../../escimo">
+
+ eSCIMo
+
+ </a>
+ |
+ <a href="./../../fortress">
+
+ Fortress
+
+ </a>
+ |
+ <a href="./../../kerby">
+
+ Kerby
+
+ </a>
+ </div><!-- subProjectsNavBar -->
+ </div><!-- header -->
+ <div id="content">
+ <div id="leftColumn">
+
+<div id="navigation">
+
+ <!--ul>
+ <li>
+ <a href="http://bit.ly/1n9YlQT" target="_blank">
+ <img src="./../../images/ApacheConBudapest.png" width="125" height="125" alt="I'm Speaking at ApacheCon Europe 2014! Join me!" title="I'm Speaking at ApacheCon Europe 2014! Join me!" border="0" style="margin-bottom:-3px;"/>
+ </a>
+ </li>
+ </ul-->
+ <h5>LDAP API 1.0</h5>
+ <ul>
+ <li><a href="./../../api/">Home</a></li>
+ <li><a href="./../../api/news.html">News</a></li>
+ </ul>
+ <h5>Downloads</h5>
+ <ul>
+ <li><a href="./../../api/downloads.html">Version 1.0.0</a> <IMG src="./../../images/new_badge.gif" alt="" style="margin-bottom:-3px;" border="0"></li>
+ <li><a href="./../../api/download-old-versions.html">Older versions</a></li>
+ </ul>
+ <h5>Getting Started</h5>
+ <ul>
+ <li><a href="./../../api/vision.html">Vision</a></li>
+ <li><a href="./../../api/java-api.html">Java API</a></li>
+ </ul>
+ <h5>Documentation</h5>
+ <ul>
+ <li><a href="./../../api/five-minutes-tutorial.html">Five minutes tutorial</a></li>
+ <li><a href="./../../api/user-guide.html">User Guide</a></li>
+ <li><a href="./../../api/gen-docs/latest/apidocs/">JavaDocs</a></li>
+ <li><a href="./../../api/gen-docs/latest/xref/">Cross-Reference</a></li>
+ <!--li><a href="./../../api/gen-docs/latest/">Generated Reports</a></li-->
+ <li><a href="./../../api/developer-guide.html">Developer Guide</a></li>
+ </ul>
+
+
+ <h5>Support</h5>
+ <ul>
+ <li><a href="./../../mailing-lists-and-irc.html">Mailing Lists & IRC</a></li>
+ <li><a href="./../../sources.html">Sources</a></li>
+ <li><a href="./../../issue-tracking.html">Issue Tracking</a></li>
+ <li><a href="./../../commercial-support.html">Commercial Support</a></li>
+ </ul>
+ <h5>Community</h5>
+ <ul>
+ <li><a href="./../../contribute.html">How to Contribute</a></li>
+ <li><a href="./../../team.html">Team</a></li>
+ <li><a href="./../../original-project-proposal.html">Original Project Proposal</a></li>
+ <li><a href="./../../special-thanks.html" class="external-link" rel="nofollow">Special Thanks</a></li>
+ </ul>
+ <h5>About Apache</h5>
+ <ul>
+ <li><a href="http://www.apache.org/">Apache</a></li>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+
+</div><!-- navigation -->
+
+ </div><!-- leftColumn -->
+ <div id="rightColumn">
+
+
+ <div class="nav">
+ <div class="nav_prev">
+
+ <a href="4-asn1.html">4 - ASN/1</a>
+
+ </div>
+ <div class="nav_up">
+
+ <a href="../dev-guide.html">Developer Guide</a>
+
+ </div>
+ <div class="nav_next">
+
+ <a href="5-network.html">5 - Network</a>
+
+ </div>
+ <div class="clearfix"></div>
+ </div>
+
+
+<style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<h1 id="4-asn1-tlv">4 - ASN/1 TLV<a class="headerlink" href="#4-asn1-tlv" title="Permanent link">¶</a></h1>
+<h2 id="what-are-tlvs">What are TLVs ?<a class="headerlink" href="#what-are-tlvs" title="Permanent link">¶</a></h2>
+<p>The acronym <strong>TLV</strong> stands for <strong>T</strong>ag, <strong>L</strong>ength and <strong>V</strong>alue. It's a way to encode a piece of information with a type, a length followed by the information itself. Three points must be known:</p>
+<ul>
+<li>The <strong>Value</strong> part may contents other <strong>TLV</strong>s. One can see <strong>TLV</strong>s as C structures, that can contain sub-structures.</li>
+<li>The <strong>Value</strong> may not exist, and in this case the <strong>Length</strong> will be 0.</li>
+<li>The <strong>Length</strong> part may not give the <strong>Value</strong> length: it is called an <em>indefinite Length</em>. In this - not so frequent - case, the <strong>Value</strong> must end with a specific terminator.</li>
+</ul>
+<h3 id="a-quick-sample">A quick sample<a class="headerlink" href="#a-quick-sample" title="Permanent link">¶</a></h3>
+<p>Let's begin with a simple example, without too many explanations. This is the <strong>PDU</strong> (<strong>P</strong>acket <strong>D</strong>ata <strong>U</strong>nit) of a LDAP <em>BindRequest</em>:</p>
+<p><img alt="TLV" src="../images/TLVs.png" /></p>
+<p>We can see in this picture that you have what is called a first level <strong>TLV</strong>. It encapsulates other <strong>TLV</strong>s. It's basically a stream of bytes.</p>
+<h3 id="type">Type<a class="headerlink" href="#type" title="Permanent link">¶</a></h3>
+<p>Each <strong>Type</strong> contains information about the <strong>Value</strong> part of the <strong>TLV</strong>. It tells if the <strong>Value</strong> is a primitive or a constructed one, which type of primitive is the value, gives some contextual information. A <strong>Type</strong> can be coded on more than one byte. The first 3 bits give some contextual information about the <strong>Type</strong>, and the 5 following bits are either a label or the beginning of a multi-bytes label.</p>
+<ul>
+<li>Labels are numbers in [0..30], and they represent a specific <strong>ASN/1</strong> element in the protocol description, like _CompareRequest ::= [APPLICATION 14] _ (here, the label is 14). </li>
+<li>If the label is 31, then more than one byte is used to encode the <strong>Type</strong>. In this case, we use the following bytes to compute the label, where each byte which high bit is one will be followed by another byte. </li>
+</ul>
+<p>We limit the label to 2,097,151 as we encode the <strong>Type</strong> in one Java int : </p>
+<div class="codehilite"><pre><span class="n">b1</span> <span class="n">xxx</span><span class="p">[</span>1<span class="o">-</span>1111<span class="p">],</span> <span class="n">b2</span> 1<span class="p">[</span>111<span class="o">-</span>1111<span class="p">],</span> <span class="n">b3</span> 1<span class="p">[</span>111<span class="o">-</span>1111<span class="p">],</span> <span class="n">b4</span> 0<span class="p">[</span>111<span class="o">-</span>1111<span class="p">]</span>
+<span class="o">-></span>
+<span class="p">[</span>111<span class="o">-</span>1111<span class="p">][</span>111<span class="o">-</span>1111<span class="p">][</span>111<span class="o">-</span>1111<span class="p">]</span>
+<span class="o">-></span>
+0001<span class="o">-</span>1111 1111<span class="o">-</span>1111 1111<span class="o">-</span>1111
+<span class="o">-></span>
+0<span class="n">x1FFFFF</span>
+<span class="o">-></span>
+2<span class="p">,</span>097<span class="p">,</span>151
+</pre></div>
+
+
+<p>In <strong>LDAP</strong> or <strong>Kerberos</strong>, no label is higher than 30, so we always use 1 byte <strong>Type</strong>s.</p>
+<p>Other interesting information that we need to grab from a <em>Type</em> are stored in the two first bits (bit 7 and 6), and in the third bit (bit 5). The first two bits describe the class, the third tells if the <strong>TLV</strong> is a <strong>primitive</strong> (b5 = 0) or a <strong>constructed</strong> <strong>TLV</strong> (b5 = 1).</p>
+<h3 id="length">Length<a class="headerlink" href="#length" title="Permanent link">¶</a></h3>
+<p><strong>Length</strong> gives the number of bytes of the <strong>Value</strong>, and nothing else. So the total length of a <strong>TLV</strong> will be:</p>
+<div class="codehilite"><pre><span class="n">TLV</span> <span class="nb">length</span> <span class="p">=</span> <span class="n">Tag</span> <span class="nb">length</span> <span class="o">+</span> <span class="n">Length</span> <span class="nb">length</span> <span class="o">+</span> <span class="n">Value</span> <span class="nb">length</span><span class="p">,</span>
+</pre></div>
+
+
+<p>where the <strong>Value</strong> length is stored in the <strong>Length</strong> element.</p>
+<p>The <strong>Length</strong> may be 0, which means that there is no value following.</p>
+<p>How is <strong>Length</strong> encoded? A <strong>Value</strong> may be from 0 to N bytes long, with N < (256 ^ 126) - 1. This limit is purely hypothetic, of course. If we have to deal with huge objects like pictures or movies, their length will not exceed a few MBytes or a few GBytes</p>
+<p>Typically, we will find five kind of <strong>Length</strong>s :</p>
+<ul>
+<li>zero length values;</li>
+<li>values with a length less than 128 bytes</li>
+<li>values with a length between 128 and 256 ^ 4 bytes long (an int will be able to hold 4 bytes);</li>
+<li>values above 256 ^ 4 bytes long</li>
+<li>values which length is not defined by the <strong>Length</strong> element.</li>
+</ul>
+<p>The last type of Length could occurs if the sender does not know the length of the value while it is sending it. <strong>LDAP</strong> protocol does not allow those kind of values, which are dangerous because you need to read the full <strong>Value</strong> to know its length.</p>
+<p>The fourth type could also be ignored (4 GBytes is quite a huge size for an LDAP element ...), so we can decide that we won't accept those <strong>Values</strong>. It seems reasonable.</p>
+<p>As the <strong>Length</strong> can be stored in more than one byte, we have to take care of fragmented <strong>PDU</strong> : we may receive only the first bytes, and have to wait for the rest to be received. The idea is to freeze and start again when we receive some more data.</p>
+<p>In any case, if the first byte is > 0x7F, that means it's a multi-bytes length, and we have to process the following bytesto get the real length. In this case, the first byte contains teh number of expected following bytes. Typically :</p>
+<div class="codehilite"><pre>0<span class="n">x74</span> <span class="o">-></span> 116
+0<span class="n">x82</span> 0<span class="n">x02</span> 0<span class="n">x84</span> <span class="p">:</span> 2 <span class="n">bytes</span><span class="p">,</span> <span class="nb">length</span> <span class="p">=</span> 2 <span class="o">*</span> 16 <span class="o">+</span> 132 <span class="o">-></span> 164
+0<span class="n">x81</span> 0<span class="n">x84</span> <span class="p">:</span> 1 <span class="n">byte</span><span class="p">,</span> <span class="nb">length</span> <span class="p">=</span> 132
+</pre></div>
+
+
+<h3 id="value">Value<a class="headerlink" href="#value" title="Permanent link">¶</a></h3>
+<p><strong>Value</strong> carries the 'meat' of a <strong>TLV</strong>. Depending on the <strong>Type</strong>, it's either a primitive value, or a constructed one (which means it contains a <strong>TLV</strong> or a set of <strong>TLV</strong>s). Remember that bit 5 of the <strong>Type</strong> tells if the **Value is <em>primitive</em> ( b5 == 0) or _constructed (b5 == 1).</p>
+<ul>
+<li>If we have a <em>primitive</em> <strong>Value</strong>, we have to read <strong>Length</strong> byte and we are done
+<strong> If we have a <em>constructed</em> </strong>Value<strong>, we have to process it as one or more </strong>TLV**.</li>
+</ul>
+<h2 id="tlv-processing">TLV processing<a class="headerlink" href="#tlv-processing" title="Permanent link">¶</a></h2>
+<p>In the <strong>API</strong>, the <strong>TLV</strong> processing is done in the <em>Asn1Decoder</em> class, and more specifically by the <em>decode( ByteBuffer stream, Asn1Container container )</em> method, which takes a ByteBuffer as input, and feed a container as a result. </p>
+<p>The important thing to understand is that this method can be called repetively, until a message is fully decoded, as soon as we feed it with some new <em>ByteBuffer</em>. The <em>Container</em> instance will contain the result, as soon as its state has switched to <strong>PDU_DECODED</strong>.</p>
+<p>While processing a <strong>TLV</strong>, when we are done with the <strong>Value</strong> part, the decoder will check if any action is to be executed. The action is associated to the grammar in use, which is stored in the container. Here is the part that call the action, in the <em>Asn1Decoder.treatTLVDoneState()</em> method :</p>
+<div class="codehilite"><pre><span class="kd">private</span> <span class="kt">boolean</span> <span class="nf">treatTLVDoneState</span><span class="o">(</span> <span class="n">ByteBuffer</span> <span class="n">stream</span><span class="o">,</span> <span class="n">Asn1Container</span> <span class="n">container</span> <span class="o">)</span> <span class="kd">throws</span> <span class="n">DecoderException</span>
+<span class="o">{</span>
+ <span class="c1">// First, we have to execute the associated action</span>
+ <span class="n">container</span><span class="o">.</span><span class="na">getGrammar</span><span class="o">().</span><span class="na">executeAction</span><span class="o">(</span> <span class="n">container</span> <span class="o">);</span>
+ <span class="o">...</span>
+</pre></div>
+
+
+<p>So the <em>Container</em> must contain the grammar, and the current state. We may not have any action to execute, either because none is associated with the current transition or because we are at the end of the message.</p>
+
+
+ <div class="nav">
+ <div class="nav_prev">
+
+ <a href="4-asn1.html">4 - ASN/1</a>
+
+ </div>
+ <div class="nav_up">
+
+ <a href="../dev-guide.html">Developer Guide</a>
+
+ </div>
+ <div class="nav_next">
+
+ <a href="5-network.html">5 - Network</a>
+
+ </div>
+ <div class="clearfix"></div>
+ </div>
+
+
+ </div><!-- rightColumn -->
+ <div id="endContent"></div>
+ </div><!-- content -->
+ <div id="footer">© 2003-2015, <a href="http://www.apache.org">The Apache Software Foundation</a> - <a href="./../../privacy-policy.html">Privacy Policy</a><br />
+ Apache Directory, ApacheDS, Apache Directory Server, Apache Directory Studio, Apache LDAP API, Apache Triplesec, Triplesec, Apache Mavibot, Mavibot, Apache eSCIMo, eSCIMo, Fortress, Apache Fortress, EnMasse, Apache EnMasse, Apache Kerby, Kerby
+ Apache, the Apache feather logo, and the Apache Directory project logos are trademarks of The Apache Software Foundation.
+ </div>
+ </div><!-- container -->
+ </body>
+</html>
\ No newline at end of file
Added: websites/staging/directory/trunk/content/api/dev-guide/images/TLVs.png
==============================================================================
Binary file - no diff available.
Propchange: websites/staging/directory/trunk/content/api/dev-guide/images/TLVs.png
------------------------------------------------------------------------------
svn:mime-type = image/png
Added: websites/staging/directory/trunk/content/api/dev-guide/images/grammar-action.graphml
==============================================================================
Binary file - no diff available.
Propchange: websites/staging/directory/trunk/content/api/dev-guide/images/grammar-action.graphml
------------------------------------------------------------------------------
svn:mime-type = application/xml
Added: websites/staging/directory/trunk/content/api/dev-guide/images/grammar-action.png
==============================================================================
Binary file - no diff available.
Propchange: websites/staging/directory/trunk/content/api/dev-guide/images/grammar-action.png
------------------------------------------------------------------------------
svn:mime-type = image/png
Added: websites/staging/directory/trunk/content/api/dev-guide/images/tlv-bervalue.graphml
==============================================================================
Binary file - no diff available.
Propchange: websites/staging/directory/trunk/content/api/dev-guide/images/tlv-bervalue.graphml
------------------------------------------------------------------------------
svn:mime-type = application/xml
Added: websites/staging/directory/trunk/content/api/dev-guide/images/tlv-bervalue.png
==============================================================================
Binary file - no diff available.
Propchange: websites/staging/directory/trunk/content/api/dev-guide/images/tlv-bervalue.png
------------------------------------------------------------------------------
svn:mime-type = image/png