You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficserver.apache.org by am...@apache.org on 2014/08/17 03:57:43 UTC
git commit: TS-3000: Change option name to 'id', update docs,
improve var naming.
Repository: trafficserver
Updated Branches:
refs/heads/master ba342b905 -> 93f283ef3
TS-3000: Change option name to 'id', update docs, improve var naming.
Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/93f283ef
Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/93f283ef
Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/93f283ef
Branch: refs/heads/master
Commit: 93f283ef387047d4d8a2325ec1ad9376600b56ed
Parents: ba342b9
Author: Alan M. Carroll <am...@network-geographics.com>
Authored: Sat Aug 16 20:57:11 2014 -0500
Committer: Alan M. Carroll <am...@network-geographics.com>
Committed: Sat Aug 16 20:57:11 2014 -0500
----------------------------------------------------------------------
.../configuration/storage.config.en.rst | 49 +++++++++++---------
iocore/cache/Cache.cc | 6 +--
iocore/cache/I_Store.h | 8 ++--
iocore/cache/P_CacheDisk.h | 2 +-
iocore/cache/Store.cc | 14 +++---
5 files changed, 41 insertions(+), 38 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/93f283ef/doc/reference/configuration/storage.config.en.rst
----------------------------------------------------------------------
diff --git a/doc/reference/configuration/storage.config.en.rst b/doc/reference/configuration/storage.config.en.rst
index af019b1..e1c1ea5 100644
--- a/doc/reference/configuration/storage.config.en.rst
+++ b/doc/reference/configuration/storage.config.en.rst
@@ -31,12 +31,13 @@ Format
The format of the :file:`storage.config` file is a series of lines of the form
- *pathname* *size* [ ``volume=``\ *number* ] [ ``seed=``\ *string* ]
+ *pathname* *size* [ ``volume=``\ *number* ] [ ``id=``\ *string* ]
-where :arg:`pathname` is the name of a partition, directory or file, :arg:`size` is the size of the named partition,
-directory or file (in bytes), and :arg:`volume` is the volume number used in the files :file:`volume.config` and
-:file:`hosting.config`. :arg:`seed` is used for seeding the :ref:`assignment-table`. You must specify a size for
-directories or files; size is optional for raw partitions. :arg:`volume` is optional and :arg:`seed` are optional.
+where :arg:`pathname` is the name of a partition, directory or file, :arg:`size` is the size of the
+named partition, directory or file (in bytes), and :arg:`volume` is the volume number used in the
+files :file:`volume.config` and :file:`hosting.config`. :arg:`id` is used for seeding the
+:ref:`assignment-table`. You must specify a size for directories or files; size is optional for raw
+partitions. :arg:`volume` and arg:`seed` are optional.
.. note::
@@ -45,7 +46,7 @@ directories or files; size is optional for raw partitions. :arg:`volume` is opti
.. note::
- If the :arg:`seed` option is used every use must have a unique value for :arg:`string`.
+ If the :arg:`id` option is used every use must have a unique value for :arg:`string`.
You can use any partition of any size. For best performance:
@@ -80,17 +81,18 @@ supported. They include
Assignment Table
----------------
-Each storage element defined in :file:`storage.config` is divided in to :term:`stripes`. The assignment table maps from
-an object URL to a specific stripe. The table is initialized based on a pseudo-random process which is seeded by hashing
-a string for each stripe. This string is composed of a seed string, an offset (the start of the stripe on the storage
-element) and the length of the stripe. By default the path for the storage is used as the seed string. This ensures that
-each stripe has a unique string for the assignment hash. This does make the assignment table very sensitive to the path
-for the storage elements and changing even one can have a cascading effect which will effectively clear most of the cache.
-This can be problem when drives fail and a system reboot causes the path names to change.
+Each storage element defined in :file:`storage.config` is divided in to :term:`stripes`. The
+assignment table maps from an object URL to a specific stripe. The table is initialized based on a
+pseudo-random process which is seeded by hashing a string for each stripe. This string is composed
+of a base string, an offset (the start of the stripe on the storage element), and the length of the
+stripe. By default the path for the storage is used as the base string. This ensures that each
+stripe has a unique string for the assignment hash. This does make the assignment table very
+sensitive to the path for the storage elements and changing even one can have a cascading effect
+which will effectively clear most of the cache. This can be problem when drives fail and a system
+reboot causes the path names to change.
-The :arg:`seed` option can be used to create a fixed string that an administrator can use to keep the assignment table
-consistent even if a device has a changed path. This value of the option is used instead of the path as the seed string
-for the assignment table hash.
+The :arg:`id` option can be used to create a fixed string that an administrator can use to keep the
+assignment table consistent by maintaing the mapping from physical device to base string even in the presence of hardware changes and failures.
Examples
========
@@ -149,16 +151,17 @@ In order to apply these settings, trigger a reload with :manpage:`udevadm(8)`:::
udevadm trigger --subsystem-match=block
As an implementation note, modern Linux supports `alternative symlinked names for disk devices
-<https://wiki.archlinux.org/index.php/persistent_block_device_naming>`_ in the ``/dev/disk`` directory structure. As
-noted for the :ref:`assignment-table` the path used for the disk can effect the cache if it changes. This can be
-ameloriated in some cases by using one of the alternate paths in via ``/dev/disk``. Note that if the ``by-id`` style is
-used, replacing a failed drive will cause that path to change because the new drive will have a different physical ID.
+<https://wiki.archlinux.org/index.php/persistent_block_device_naming>`_ in the ``/dev/disk``
+directory structure. As noted for the :ref:`assignment-table` the path used for the disk can effect
+the cache if it changes. This can be ameloriated in some cases by using one of the alternate paths
+in via ``/dev/disk``. Note that if the ``by-id`` style is used, replacing a failed drive will cause
+that path to change because the new drive will have a different physical ID. The original hash string can be kept by adding :arg:`id` with the original path to the storage line.
-If this is not sufficient then the :arg:`seed` argument should be used to create a more permanent assignment table. An
+If this is not sufficient then the :arg:`id` argument should be used to create a more permanent assignment table. An
example would be::
- /dev/sde seed=cache.disk.0
- /dev/sdg seed=cache.disk.1
+ /dev/sde id=cache.disk.0
+ /dev/sdg id=cache.disk.1
FreeBSD Example
---------------
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/93f283ef/iocore/cache/Cache.cc
----------------------------------------------------------------------
diff --git a/iocore/cache/Cache.cc b/iocore/cache/Cache.cc
index 3913b71..36c3012 100644
--- a/iocore/cache/Cache.cc
+++ b/iocore/cache/Cache.cc
@@ -743,8 +743,8 @@ CacheProcessor::start_internal(int flags)
gdisks[gndisks] = new CacheDisk();
gdisks[gndisks]->forced_volume_num = sd->forced_volume_num;
- if (sd->hash_seed_string)
- gdisks[gndisks]->hash_seed_string = ats_strdup(sd->hash_seed_string);
+ if (sd->hash_base_string)
+ gdisks[gndisks]->hash_base_string = ats_strdup(sd->hash_base_string);
Debug("cache_hosting", "Disk: %d, blocks: %d", gndisks, blocks);
@@ -1263,7 +1263,7 @@ Vol::clear_dir()
int
Vol::init(char *s, off_t blocks, off_t dir_skip, bool clear)
{
- char* seed_str = disk->hash_seed_string ? disk->hash_seed_string : s;
+ char* seed_str = disk->hash_base_string ? disk->hash_base_string : s;
const size_t hash_seed_size = strlen(seed_str);
const size_t hash_text_size = hash_seed_size + 32;
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/93f283ef/iocore/cache/I_Store.h
----------------------------------------------------------------------
diff --git a/iocore/cache/I_Store.h b/iocore/cache/I_Store.h
index 303f880..f951e6b 100644
--- a/iocore/cache/I_Store.h
+++ b/iocore/cache/I_Store.h
@@ -57,7 +57,7 @@ public:
// v- used as a magic location for copy constructor.
// we memcpy everything before this member and do explicit assignment for the rest.
ats_scoped_str pathname;
- ats_scoped_str hash_seed_string; ///< Used to seed the stripe assignment hash.
+ ats_scoped_str hash_base_string; ///< Used to seed the stripe assignment hash.
SLINK(Span, link);
bool is_mmapable() { return is_mmapable_internal; }
@@ -99,7 +99,7 @@ public:
char *buf, int buflen); // where to store the path
/// Set the hash seed string.
- void hash_seed_string_set(char const* s);
+ void hash_base_string_set(char const* s);
/// Set the volume number.
void volume_number_set(int n);
@@ -121,7 +121,7 @@ public:
Span(Span const& that) {
memcpy(this, &that, reinterpret_cast<intptr_t>(&(static_cast<Span*>(0)->pathname)));
if (that.pathname) pathname = ats_strdup(that.pathname);
- if (that.hash_seed_string) hash_seed_string = ats_strdup(that.hash_seed_string);
+ if (that.hash_base_string) hash_base_string = ats_strdup(that.hash_base_string);
link.next = NULL;
}
@@ -210,7 +210,7 @@ struct Store
/// Additional configuration key values.
static char const VOLUME_KEY[];
- static char const HASH_SEED_KEY[];
+ static char const HASH_BASE_STRING_KEY[];
};
extern Store theStore;
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/93f283ef/iocore/cache/P_CacheDisk.h
----------------------------------------------------------------------
diff --git a/iocore/cache/P_CacheDisk.h b/iocore/cache/P_CacheDisk.h
index 76c11ef..2734f04 100644
--- a/iocore/cache/P_CacheDisk.h
+++ b/iocore/cache/P_CacheDisk.h
@@ -106,7 +106,7 @@ struct CacheDisk: public Continuation
// Extra configuration values
int forced_volume_num; ///< Volume number for this disk.
- ats_scoped_str hash_seed_string; ///< Base string for hash seed.
+ ats_scoped_str hash_base_string; ///< Base string for hash seed.
CacheDisk()
: Continuation(new_ProxyMutex()), header(NULL),
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/93f283ef/iocore/cache/Store.cc
----------------------------------------------------------------------
diff --git a/iocore/cache/Store.cc b/iocore/cache/Store.cc
index be4ffc9..9df8f9a 100644
--- a/iocore/cache/Store.cc
+++ b/iocore/cache/Store.cc
@@ -45,7 +45,7 @@ Store theStore;
//
char const Store::VOLUME_KEY[] = "volume";
-char const Store::HASH_SEED_KEY[] = "seed";
+char const Store::HASH_BASE_STRING_KEY[] = "id";
Ptr<ProxyMutex> tmp_p;
Store::Store():n_disks(0), disk(NULL)
@@ -200,9 +200,9 @@ Span::path(char *filename, int64_t * aoffset, char *buf, int buflen)
}
void
-Span::hash_seed_string_set(char const* s)
+Span::hash_base_string_set(char const* s)
{
- hash_seed_string = s ? ats_strdup(s) : NULL;
+ hash_base_string = s ? ats_strdup(s) : NULL;
}
void
@@ -329,8 +329,8 @@ Store::read_config(int fd)
err = "error parsing size";
goto Lfail;
}
- } else if (0 == strncasecmp(HASH_SEED_KEY, e, sizeof(HASH_SEED_KEY)-1)) {
- e += sizeof(HASH_SEED_KEY) - 1;
+ } else if (0 == strncasecmp(HASH_BASE_STRING_KEY, e, sizeof(HASH_BASE_STRING_KEY)-1)) {
+ e += sizeof(HASH_BASE_STRING_KEY) - 1;
if ('=' == *e) ++e;
if (*e && !ParseRules::is_space(*e))
seed = e;
@@ -347,7 +347,7 @@ Store::read_config(int fd)
char *pp = Layout::get()->relative(path);
ns = new Span;
Debug("cache_init", "Store::read_config - ns = new Span; ns->init(\"%s\",%" PRId64 "), forced volume=%d%s%s",
- pp, size, volume_num, seed ? " seed=" : "", seed ? seed : "");
+ pp, size, volume_num, seed ? " id=" : "", seed ? seed : "");
if ((err = ns->init(pp, size))) {
RecSignalWarning(REC_SIGNAL_SYSTEM_ERROR, "could not initialize storage \"%s\" [%s]", pp, err);
Debug("cache_init", "Store::read_config - could not initialize storage \"%s\" [%s]", pp, err);
@@ -359,7 +359,7 @@ Store::read_config(int fd)
n_dsstore++;
// Set side values if present.
- if (seed) ns->hash_seed_string_set(seed);
+ if (seed) ns->hash_base_string_set(seed);
if (volume_num > 0) ns->volume_number_set(volume_num);
// new Span