You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficserver.apache.org by am...@apache.org on 2013/10/09 04:01:04 UTC
git commit: DOC: Added HostDB architecture proposal. Added proposal
section to architecture area.
Updated Branches:
refs/heads/master 3c0c835c1 -> 049571519
DOC: Added HostDB architecture proposal. Added proposal section to architecture area.
Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/04957151
Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/04957151
Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/04957151
Branch: refs/heads/master
Commit: 04957151907c4d12d89972e3aad5a7b24b718428
Parents: 3c0c835
Author: Alan M. Carroll <am...@network-geographics.com>
Authored: Tue Oct 8 21:00:36 2013 -0500
Committer: Alan M. Carroll <am...@network-geographics.com>
Committed: Tue Oct 8 21:00:36 2013 -0500
----------------------------------------------------------------------
doc/arch/index.en.rst | 19 +++-
doc/arch/proposals/hostdb.en.rst | 168 ++++++++++++++++++++++++++++++++++
2 files changed, 184 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/04957151/doc/arch/index.en.rst
----------------------------------------------------------------------
diff --git a/doc/arch/index.en.rst b/doc/arch/index.en.rst
index ea73adc..e5958bf 100644
--- a/doc/arch/index.en.rst
+++ b/doc/arch/index.en.rst
@@ -8,9 +8,9 @@ Architecture
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -21,7 +21,13 @@ Architecture
Introduction
--------------
-The original architectural documents for Traffic Server were lost in the transation to an open source project. The documents in this section are provisional and were written based on the existing code. The purpose is to have a high level description of aspects of Traffic Server to better inform ongoing work.
+The original architectural documents for Traffic Server were lost in the transation to an open source project. The
+documents in this section are provisional and were written based on the existing code. The purpose is to have a high
+level description of aspects of Traffic Server to better inform ongoing work.
+
+In addition, future proposed architectural changes are included in the *Proposals* section. These are not current, and
+may not be implemented in the future, and if implemented may be signficantly changed. They are included here to provide
+direction for those interested in working on the Traffic Server architecture.
Contents:
@@ -29,3 +35,10 @@ Contents:
:maxdepth: 2
cache/cache.en
+
+Proposals
+
+.. toctree::
+ :maxdepth: 1
+
+ proposals/hostdb.en
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/04957151/doc/arch/proposals/hostdb.en.rst
----------------------------------------------------------------------
diff --git a/doc/arch/proposals/hostdb.en.rst b/doc/arch/proposals/hostdb.en.rst
new file mode 100644
index 0000000..16a6b4e
--- /dev/null
+++ b/doc/arch/proposals/hostdb.en.rst
@@ -0,0 +1,168 @@
+Host Resolution
+******************
+
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Introduction
+--------------
+
+The current mechanism for resolving host names to IP addresses for Traffic Server is contained the HostDB and DNS
+libraries. These take hostnames and provide IP addresses for them.
+
+The current implementation is generally considered inadequate, both from a functionality point of view and difficulty in
+working with it in other parts of Traffic Server. As Traffic Server is used in more complex situtations this inadequacy
+presents increasing problems.
+
+Goals
+-----
+
+Updating the host name resolution (currently referred to as "HostDB") has several functions goals
+
+* Enable additional processing layers to be easily added.
+* Enable plugins to directly access the name resolution logic
+* Enable plugins to provide name resolution
+* Asynchronous (immediate resolve or callback on block)
+* Minimize allocations -- in particular no allocations for cached resolutions
+* Simplify interactions with the resolution, particularly with regard to nameservers, origin server failover, and
+ address family handling.
+
+It is also necessary to support a number of specific features that are either currently available or strongly desired.
+
+* SplitDNS or its equivalent
+* Use of a hosts file (e.g. ``/etc/hosts``)
+* Simultaneous IPv4 and IPv6 queries
+* IP family control
+* Negative caching
+ * Server connection failures
+ * Query failures
+ * Nameserver failures.
+* Address validity time out control
+* Address round robin support
+* SRV record support (weighted records)
+* Nameserver round robin
+* Plugin access to nameserver data (add, remove, enumerate)
+* Plugin provision of resolvers.
+* Hooks for plugin detection / recovery from resolution events.
+
+One issue is persistence of the cached resolutions. This creates problems for the current implementation (because of
+size limits it imposes on the cached data) but also allows for quicker restarts in a busy environment.
+
+Basics
+------
+
+The basic design is to separate the functionality into chainable layers so that a resolver with the desired attributes
+can be assembled from those layers. The core interface is that of a lazy iterator. This object returns one of four
+results when asked for an address
+
+* An IP address
+* Done(no more addresses are available)
+* Wait(an address may be available in the future)
+* Fail (no address is available and none will be so in the future)
+
+Each layer (except the bottom) uses this API and also provides it. This enables higher level logic such as the state
+machine to simply use the resolver as a list without having to backtrack states in the case of failures, or have special
+cases for different resolution sources.
+
+To perform a resolution, a client creates a query object (potentially on the stack), initializes it with the required
+data (at least the hostname) and then starts the resolution. Methods on the query object allow its state and IP address
+data to be accessed.
+
+Required Resolvers
+------------------------
+
+Nameserver
+ A bottom level resolver that directly queries a nameserver for DNS data. This contains much of the functionality
+ currently in the ``iocore/dns`` directory.
+
+SplitDNS
+ A resolver that directs requests to one of several resolvers. To emulate current behavior these would be Nameserver
+ instances.
+
+NameserverGroup
+ A grouping mechanism for Nameserver instances that provides failover, round robin, and ordering capabilities. It may be
+ reasonable to merge this with the SplitDNS resolver.
+
+HostFile
+ A resolver that uses a local file to resolve names.
+
+AddressCache
+ A resolver that also has a cache for resolution results. It requires another resolver instance to perform the actual
+ resolution.
+
+Preloaded
+ A resolver that can contain one or more explicitly set IP addresses which are returned. When those are exhausted it
+ falls back to another resolver.
+
+Configuration
+-------------
+
+To configuration the resolution, each resolver would be assigned a tag. It is not, however, sufficient to simply provide
+the list of resolver tags because some resolvers require additional configuration. Unfortunately this will likely
+require a separate configuration file outside of :file:`records.config`, although we would be able to remove
+:file:`splitdns.config`. In this case we would need chain start / end markers around a list of resolver tags. Each tag
+would the be able to take additional resolver configuration data. For instance, for a SplitDNS resolver the nameservers.
+
+Examples
+--------
+
+Transparent operations would benefit from the *Preloaded* resolver. This would be loaded with the origin host address
+provided by the client connection. This could be done early in processing and then no more logic would be required to
+skip DNS processing as it would happen without additional action by the state machine. It would handle the problem of de
+facto denial of service if an origin server becomes unavailable in that configuration, as *Preloaded* would switch to
+alternate addresses automatically.
+
+Adding host file access would be easier as well, as it could be done in a much more modular fashion and then added to
+the stack at configuration time. Whether such addresses were cached would be controlled by chain arrangement rather yet
+more configuration knobs.
+
+The default configuration would be *Preloaded* : *AddressCache* : *Nameserver*.
+
+In all cases the state machine makes requests against the request object to get IP addresses as needed.
+
+Issues
+------
+
+Request object allocation
+=========================
+
+The biggest hurdle is being able to unwind a resolver chain when a block is encountered. There are some ways to deal with this.
+
+1) Set a maximum resolver chain length and declare the request instance so that there is storage for state for that many
+resolvers. If needed and additional value of maximum storage per chain could be set as well. The expected number of
+elements in a chain is expected to be limited, 10 would likely be a reaosnable limit. If settable at source
+configuration time this should be sufficient.
+
+2) Embed class allocators in resolver chains and mark the top / outermost / first resolver. The maximum state size for a
+resolution can be calculated when the chain is created and then the top level resolver can use an allocation pool to
+efficiently allocate request objects. This has an advantage that with a wrapper class the request object can be passed
+along cheaply. Whether that's an advantage in practice is unclear.
+
+Plugin resolvers
+================
+
+If plugins can provide resolvers, how can these can integrated in to existing resolver chains for use by the HTTP SM for
+instance?
+
+Feedback
+========
+
+It should be possible for a client to provide feedback about addresses (e.g., the origin server at this address is not
+available). Not all resolvers will handle feedback but some will and that must be possible.
+
+Related to this is that caching resolvers (such as *AddressCache*) must be able to iterator over all resolved addresses
+even if their client does not ask for them. In effect they must background fill the address data.