You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficserver.apache.org by zw...@apache.org on 2018/08/13 18:29:15 UTC
[trafficserver] branch 8.0.x updated (c942236 -> ea89d09)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a change to branch 8.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git.
from c942236 cppapi: InterceptPlugin fix
new d669199 Runroot: add an option to specify copy style
new 7899e8e Runroot: add new option to specify layout during creating
new ea89d09 Runroot: Add feature to use runroot by providing yaml file
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
doc/appendices/command-line/traffic_layout.en.rst | 98 ++++++++++++----------
lib/records/RecConfigParse.cc | 17 ++++
lib/ts/runroot.cc | 85 ++++++++++---------
lib/ts/runroot.h | 7 +-
src/traffic_layout/engine.cc | 99 ++++++++++++++++++++---
src/traffic_layout/engine.h | 10 ++-
src/traffic_layout/file_system.cc | 26 +++---
src/traffic_layout/file_system.h | 5 +-
src/traffic_layout/traffic_layout.cc | 2 -
tests/gold_tests/runroot/runroot_error.test.py | 4 +-
tests/gold_tests/runroot/runroot_init.test.py | 9 ++-
11 files changed, 249 insertions(+), 113 deletions(-)
[trafficserver] 01/03: Runroot: add an option to specify copy style
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit d669199869519c356b72cbdd0d946e83429b50ae
Author: Xavier Chi <ch...@gmail.com>
AuthorDate: Fri Jun 29 10:30:19 2018 -0500
Runroot: add an option to specify copy style
(cherry picked from commit 58beea4fec0b84205d502e08fadd8b3d19eb2ec7)
---
doc/appendices/command-line/traffic_layout.en.rst | 94 +++++++++++++----------
src/traffic_layout/engine.cc | 16 +++-
src/traffic_layout/engine.h | 5 ++
src/traffic_layout/file_system.cc | 24 ++++--
src/traffic_layout/file_system.h | 5 +-
5 files changed, 92 insertions(+), 52 deletions(-)
diff --git a/doc/appendices/command-line/traffic_layout.en.rst b/doc/appendices/command-line/traffic_layout.en.rst
index e9eca56..c236816 100644
--- a/doc/appendices/command-line/traffic_layout.en.rst
+++ b/doc/appendices/command-line/traffic_layout.en.rst
@@ -20,22 +20,12 @@
traffic_layout
**************
+========
Synopsis
========
:program:`traffic_layout` SUBCOMMAND [OPTIONS]
-Options
-=============
-.. program:: traffic_layout
-
-.. option:: --run-root=[<path>]
-
- Use the run root file at :arg:`path`.
-
-.. option:: -V, --version
-
- Print version information and exit.
-
+===========
Environment
===========
@@ -43,8 +33,9 @@ Environment
The path to the run root file. It has the same effect as the command line option :option:`--run-root`.
+===========
Description
-=============
+===========
Document for the special functionality of ``runroot`` inside :program:`traffic_layout`. This feature
is for the setup of traffic server runroot. It will create a runtime sandbox for any program of
traffic server to run under.
@@ -60,59 +51,78 @@ How it works
#. Emit a yaml file that defines layout structure for other programs to use (relative path).
#. Users are able to remove runroot and verify permission of the runroot.
+===========
Subcommands
-=============
+===========
-- Initialize the runroot: ::
+``init``
+ Use the current working directory or the specific path to create runroot.
+ The path can be relative or set up in :envvar:`TS_RUNROOT`.
- traffic_layout init (--path /path/to/sandbox/)
+``remove``
+ Find the sandbox to remove in following order:
+ #. specified in --path as absolute or relative.
+ #. ENV variable: :envvar:`TS_RUNROOT`
+ #. current working directory
+ #. installed directory.
+
+``verify``
+ Verify the permission of the sandbox.
+
+=======
+Options
+=======
+.. program:: traffic_layout
- Use the current working directory or the specific path to create runroot.
- The path can be relative or set up in :envvar:`TS_RUNROOT`.
+.. option:: --run-root=[<path>]
+ Use the run root file at :arg:`path`.
+
+.. option:: -V, --version
-- Remove the runroot: ::
+ Print version information and exit.
- traffic_layout remove (--path /path/to/sandbox/)
+.. option:: -h, --help
- Find the sandbox to remove in following order:
+ Print usage information and exit.
- 1. specified in --path as absolute or relative.
- 2. :envvar:`TS_RUNROOT`
- 3. current working directory
- 4. installed directory.
+.. option:: --force
-- Verify the runroot: ::
+ Force init will create sandbox even if the directory is not empty.
+ Force remove will remove a directory even if directory has no yaml file.
- traffic_layout verify (--path /path/to/sandbox/)
+.. option:: --absolute
- Verify the permission of the sandbox.
+ Put directories in the yaml file in the form of absolute path when creating.
+
+.. option:: --fix
-Subcommands options
--------------------
+ Fix the permission issues verify found. ``--fix`` requires root privilege (sudo).
-- Force option: ::
+.. option:: --copy-style=[HARD/SOFT/FULL]
- traffic_layout init --force (--path /path/to/sandbox)
- traffic_layout remove --force (--path /path/to/sandbox)
+ Specify the way of copying executables when creating runroot
- Force init will create sandbox even if the directory is not empty.
- Force remove will remove a directory even if directory has no yaml file.
+========
+Examples
+========
-- Absolute option: ::
+Initialize the runroot. ::
- traffic_layout init --absolute (--path /path/to/sandbox)
+ traffic_layout init (--path /path/to/sandbox/) (--force) (--absolute) (--copy-style=[HARD/SOFT/FULL])
- create the sandbox and put directories in the yaml file in the form of absolute path.
+Remove the runroot. ::
-- Fix option: ::
+ traffic_layout remove (--path /path/to/sandbox/) (--force)
- traffic_layout verify --fix (--path /path/to/sandbox)
+Verify the runroot. ::
+
+ traffic_layout verify (--path /path/to/sandbox/) (--fix)
- Fix the permission issues verify found. ``--fix`` requires root privilege (sudo).
+=========================
Usage for other programs:
-==============================================
+=========================
All programs can find the runroot to use in the following order
diff --git a/src/traffic_layout/engine.cc b/src/traffic_layout/engine.cc
index 0e9a142..b6af4f6 100644
--- a/src/traffic_layout/engine.cc
+++ b/src/traffic_layout/engine.cc
@@ -36,6 +36,7 @@
#include "file_system.h"
#include "ts/runroot.h"
+#include <cctype>
#include <fstream>
#include <iostream>
#include <ftw.h>
@@ -147,6 +148,7 @@ RunrootEngine::runroot_help_message(const bool runflag, const bool cleanflag, co
"--force Force to create ts_runroot even directory already exists\n"
"--absolute Produce absolute path in the yaml file\n"
"--run-root(=/path) Using specified TS_RUNROOT as sandbox\n"
+ "--copy-style=[STYLE] Specify style (FULL, HARD, SOFT) when copying executable\n"
<< std::endl;
}
if (cleanflag) {
@@ -216,6 +218,18 @@ RunrootEngine::runroot_parse()
fix_flag = true;
continue;
}
+ if (argument.substr(0, COPYSTYLE_WORD_LENGTH) == "--copy-style") {
+ std::string style = argument.substr(COPYSTYLE_WORD_LENGTH + 1);
+ transform(style.begin(), style.end(), style.begin(), ::tolower);
+ if (style == "full") {
+ copy_style = FULL;
+ } else if (style == "soft") {
+ copy_style = SOFT;
+ } else {
+ copy_style = HARD;
+ }
+ continue;
+ }
if (argument == "--path") {
if (i + 1 >= _argv.size() || _argv[i + 1][0] == '-') {
// invalid path
@@ -463,7 +477,7 @@ RunrootEngine::copy_runroot(const std::string &original_root, const std::string
// don't copy the prefix, mandir, localstatedir and infodir
if (it.first != LAYOUT_EXEC_PREFIX && it.first != LAYOUT_LOCALSTATEDIR && it.first != LAYOUT_MANDIR &&
it.first != LAYOUT_INFODIR) {
- if (!copy_directory(old_path, new_path, it.first)) {
+ if (!copy_directory(old_path, new_path, it.first, copy_style)) {
ink_warning("Unable to copy '%s' - %s", it.first.c_str(), strerror(errno));
ink_notice("Creating '%s': %s", it.first.c_str(), new_path.c_str());
// if copy failed for certain directory, we create it in runroot
diff --git a/src/traffic_layout/engine.h b/src/traffic_layout/engine.h
index 82c7581..f83339b 100644
--- a/src/traffic_layout/engine.h
+++ b/src/traffic_layout/engine.h
@@ -23,11 +23,14 @@
#pragma once
+#include "file_system.h"
+
#include <vector>
#include <string>
#include <unordered_map>
#define RUNROOT_WORD_LENGTH 10
+#define COPYSTYLE_WORD_LENGTH 12
typedef std::unordered_map<std::string, std::string> RunrootMapType;
@@ -71,6 +74,8 @@ struct RunrootEngine {
// for parsing
int command_num = 0;
+ CopyStyle copy_style = HARD;
+
// the path for create & remove
std::string path;
diff --git a/src/traffic_layout/file_system.cc b/src/traffic_layout/file_system.cc
index e7bca3a..681038e 100644
--- a/src/traffic_layout/file_system.cc
+++ b/src/traffic_layout/file_system.cc
@@ -43,6 +43,7 @@ static std::string dst_root;
static std::string src_root;
static std::string copy_dir; // the current dir we are copying. e.x. sysconfdir, bindir...
static std::string remove_path;
+CopyStyle copy_style;
// list of all executables of traffic server
std::set<std::string> const executables = {"traffic_crashlog", "traffic_ctl", "traffic_layout", "traffic_logcat",
@@ -255,12 +256,18 @@ ts_copy_function(const char *src_path, const struct stat *sb, int flag)
}
// hardlink bin executable
if (sb->st_mode & S_IEXEC) {
- if (link(src_path, dst_path.c_str()) != 0) {
- if (errno != EEXIST) {
+ if (copy_style == SOFT) {
+ if (symlink(src_path, dst_path.c_str()) != 0 && errno != EEXIST) {
+ ink_warning("failed to create symlink - %s", strerror(errno));
+ } else {
+ return 0;
+ }
+ } else if (copy_style == HARD) {
+ if (link(src_path, dst_path.c_str()) != 0 && errno != EEXIST) {
ink_warning("failed to create hard link - %s", strerror(errno));
+ } else {
+ return 0;
}
- } else {
- return 0;
}
}
// for normal other files
@@ -276,11 +283,12 @@ ts_copy_function(const char *src_path, const struct stat *sb, int flag)
// copy directory recursively using ftw to iterate
bool
-copy_directory(const std::string &src, const std::string &dst, const std::string &dir)
+copy_directory(const std::string &src, const std::string &dst, const std::string &dir, CopyStyle style)
{
- src_root = src;
- dst_root = dst;
- copy_dir = dir;
+ src_root = src;
+ dst_root = dst;
+ copy_dir = dir;
+ copy_style = style;
remove_slash(src_root);
append_slash(dst_root);
diff --git a/src/traffic_layout/file_system.h b/src/traffic_layout/file_system.h
index 54a93bb..b6d450e 100644
--- a/src/traffic_layout/file_system.h
+++ b/src/traffic_layout/file_system.h
@@ -29,6 +29,9 @@
// size can be changed accordingly
#define OPEN_MAX_FILE 256
+// full copy, hardlink, softlink
+enum CopyStyle { FULL, HARD, SOFT };
+
// append slash & remove slash of path for convinient use
void append_slash(std::string &path);
@@ -47,4 +50,4 @@ bool remove_directory(const std::string &dir);
// remove everything inside this directory
bool remove_inside_directory(const std::string &dir);
-bool copy_directory(const std::string &src, const std::string &dst, const std::string &dir = "");
+bool copy_directory(const std::string &src, const std::string &dst, const std::string &dir = "", CopyStyle style = HARD);
[trafficserver] 02/03: Runroot: add new option to specify layout
during creating
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 7899e8e34ede26adb77aa65d1872254d70ddbda0
Author: Xavier Chi <ch...@gmail.com>
AuthorDate: Tue Jul 10 16:04:18 2018 -0500
Runroot: add new option to specify layout during creating
(cherry picked from commit 5ca1665f6928f347af16b1e193e791d7a6a2ce7f)
---
doc/appendices/command-line/traffic_layout.en.rst | 4 ++
lib/records/RecConfigParse.cc | 17 ++++++++
lib/ts/runroot.cc | 6 +++
lib/ts/runroot.h | 3 ++
src/traffic_layout/engine.cc | 48 +++++++++++++++++++----
src/traffic_layout/engine.h | 7 +++-
src/traffic_layout/file_system.cc | 2 +-
src/traffic_layout/traffic_layout.cc | 2 -
tests/gold_tests/runroot/runroot_init.test.py | 9 +++--
9 files changed, 81 insertions(+), 17 deletions(-)
diff --git a/doc/appendices/command-line/traffic_layout.en.rst b/doc/appendices/command-line/traffic_layout.en.rst
index c236816..a56c7a0 100644
--- a/doc/appendices/command-line/traffic_layout.en.rst
+++ b/doc/appendices/command-line/traffic_layout.en.rst
@@ -103,6 +103,10 @@ Options
Specify the way of copying executables when creating runroot
+.. option:: --layout=[<YAML file>]
+
+ Use specific layout (providing YAML file) to create runroot
+
========
Examples
========
diff --git a/lib/records/RecConfigParse.cc b/lib/records/RecConfigParse.cc
index 3042b0e..81fecad 100644
--- a/lib/records/RecConfigParse.cc
+++ b/lib/records/RecConfigParse.cc
@@ -28,6 +28,7 @@
#include "ts/Tokenizer.h"
#include "ts/ink_defs.h"
#include "ts/ink_string.h"
+#include "ts/runroot.h"
#include "P_RecFile.h"
#include "P_RecUtils.h"
@@ -83,6 +84,20 @@ RecFileImport_Xmalloc(const char *file, char **file_buf, int *file_size)
}
//-------------------------------------------------------------------------
+// RecConfigOverrideFromRunroot
+//-------------------------------------------------------------------------
+bool
+RecConfigOverrideFromRunroot(const char *name)
+{
+ if (use_runroot()) {
+ if (!strcmp(name, "proxy.config.bin_path") || !strcmp(name, "proxy.config.local_state_dir") ||
+ !strcmp(name, "proxy.config.log.logfile_dir") || !strcmp(name, "proxy.config.plugin.plugin_dir"))
+ return true;
+ }
+ return false;
+}
+
+//-------------------------------------------------------------------------
// RecConfigOverrideFromEnvironment
//-------------------------------------------------------------------------
const char *
@@ -106,6 +121,8 @@ RecConfigOverrideFromEnvironment(const char *name, const char *value)
envval = getenv((const char *)envname);
if (envval) {
return envval;
+ } else if (RecConfigOverrideFromRunroot(name)) {
+ return nullptr;
}
return value;
diff --git a/lib/ts/runroot.cc b/lib/ts/runroot.cc
index 119cd56..38edaa4 100644
--- a/lib/ts/runroot.cc
+++ b/lib/ts/runroot.cc
@@ -246,3 +246,9 @@ check_runroot()
}
return runroot_map(using_runroot);
}
+
+bool
+use_runroot()
+{
+ return !using_runroot.empty();
+}
diff --git a/lib/ts/runroot.h b/lib/ts/runroot.h
index 4256cfc..a06636f 100644
--- a/lib/ts/runroot.h
+++ b/lib/ts/runroot.h
@@ -61,3 +61,6 @@ RunrootMapType runroot_map(const std::string &prefix);
// help check runroot for layout
RunrootMapType check_runroot();
+
+// helper method for records config to check using runroot or not
+bool use_runroot();
diff --git a/src/traffic_layout/engine.cc b/src/traffic_layout/engine.cc
index b6af4f6..764514d 100644
--- a/src/traffic_layout/engine.cc
+++ b/src/traffic_layout/engine.cc
@@ -44,7 +44,7 @@
#include <yaml-cpp/yaml.h>
// for nftw check_directory
-std::string directory_check = "";
+std::string directory_check;
// check if we can create the runroot using path
// return true if the path is good to use
@@ -149,6 +149,7 @@ RunrootEngine::runroot_help_message(const bool runflag, const bool cleanflag, co
"--absolute Produce absolute path in the yaml file\n"
"--run-root(=/path) Using specified TS_RUNROOT as sandbox\n"
"--copy-style=[STYLE] Specify style (FULL, HARD, SOFT) when copying executable\n"
+ "--layout=[/path] Use specific layout (providing yaml file) to create runroot\n"
<< std::endl;
}
if (cleanflag) {
@@ -192,7 +193,7 @@ RunrootEngine::runroot_parse()
abs_flag = true;
continue;
}
- if (argument.substr(0, RUNROOT_WORD_LENGTH) == "--run-root") {
+ if (argument.substr(0, RUNROOT_WORD.size()) == RUNROOT_WORD) {
continue;
}
// set init flag
@@ -218,8 +219,8 @@ RunrootEngine::runroot_parse()
fix_flag = true;
continue;
}
- if (argument.substr(0, COPYSTYLE_WORD_LENGTH) == "--copy-style") {
- std::string style = argument.substr(COPYSTYLE_WORD_LENGTH + 1);
+ if (argument.substr(0, COPYSTYLE_WORD.size()) == COPYSTYLE_WORD) {
+ std::string style = argument.substr(COPYSTYLE_WORD.size() + 1);
transform(style.begin(), style.end(), style.begin(), ::tolower);
if (style == "full") {
copy_style = FULL;
@@ -230,6 +231,14 @@ RunrootEngine::runroot_parse()
}
continue;
}
+ if (argument.substr(0, LAYOUT_WORD.size()) == LAYOUT_WORD) {
+ if (argument.size() <= LAYOUT_WORD.size() + 1) {
+ layout_file = "";
+ } else {
+ layout_file = argument.substr(LAYOUT_WORD.size() + 1);
+ }
+ continue;
+ }
if (argument == "--path") {
if (i + 1 >= _argv.size() || _argv[i + 1][0] == '-') {
// invalid path
@@ -454,6 +463,29 @@ RunrootEngine::copy_runroot(const std::string &original_root, const std::string
original_map[LAYOUT_INFODIR] = TS_BUILD_INFODIR;
original_map[LAYOUT_CACHEDIR] = TS_BUILD_CACHEDIR;
+ RunrootMapType new_map = original_map;
+ // use the user provided layout: layout_file
+ if (layout_file.size() != 0) {
+ try {
+ YAML::Node yamlfile = YAML::LoadFile(layout_file);
+ for (auto it : yamlfile) {
+ std::string key = it.first.as<std::string>();
+ std::string value = it.second.as<std::string>();
+ auto iter = new_map.find(key);
+ if (iter != new_map.end()) {
+ iter->second = value;
+ } else {
+ if (key != "prefix") {
+ ink_warning("Unknown item from %s: '%s'", layout_file.c_str(), key.c_str());
+ }
+ }
+ }
+ } catch (YAML::Exception &e) {
+ ink_warning("Unable to read provided YAML file '%s': %s", layout_file.c_str(), e.what());
+ ink_notice("Continuing with default value");
+ }
+ }
+
// copy each directory to the runroot path
// symlink the executables
// set up path_map for yaml to emit key-value pairs
@@ -465,15 +497,15 @@ RunrootEngine::copy_runroot(const std::string &original_root, const std::string
} else {
join_path = it.second;
}
+ std::string new_join_path = new_map[it.first];
std::string old_path = Layout::relative_to(original_root, join_path);
- std::string new_path = Layout::relative_to(ts_runroot, join_path);
+ std::string new_path = Layout::relative_to(ts_runroot, new_join_path);
if (abs_flag) {
- path_map[it.first] = Layout::relative_to(ts_runroot, join_path);
+ path_map[it.first] = Layout::relative_to(ts_runroot, new_join_path);
} else {
- path_map[it.first] = Layout::relative_to(".", join_path);
+ path_map[it.first] = Layout::relative_to(".", new_join_path);
}
-
// don't copy the prefix, mandir, localstatedir and infodir
if (it.first != LAYOUT_EXEC_PREFIX && it.first != LAYOUT_LOCALSTATEDIR && it.first != LAYOUT_MANDIR &&
it.first != LAYOUT_INFODIR) {
diff --git a/src/traffic_layout/engine.h b/src/traffic_layout/engine.h
index f83339b..857877d 100644
--- a/src/traffic_layout/engine.h
+++ b/src/traffic_layout/engine.h
@@ -29,8 +29,9 @@
#include <string>
#include <unordered_map>
-#define RUNROOT_WORD_LENGTH 10
-#define COPYSTYLE_WORD_LENGTH 12
+static const std::string_view RUNROOT_WORD{"--run-root"};
+static const std::string_view COPYSTYLE_WORD{"--copy-style"};
+static const std::string_view LAYOUT_WORD{"--layout"};
typedef std::unordered_map<std::string, std::string> RunrootMapType;
@@ -74,6 +75,8 @@ struct RunrootEngine {
// for parsing
int command_num = 0;
+ std::string layout_file;
+
CopyStyle copy_style = HARD;
// the path for create & remove
diff --git a/src/traffic_layout/file_system.cc b/src/traffic_layout/file_system.cc
index 681038e..02c6c5d 100644
--- a/src/traffic_layout/file_system.cc
+++ b/src/traffic_layout/file_system.cc
@@ -191,7 +191,7 @@ ts_copy_function(const char *src_path, const struct stat *sb, int flag)
std::string full_src_path = src_path;
if (full_src_path == src_root) {
if (!create_directory(dst_root)) {
- ink_fatal("create directory failed during copy");
+ ink_fatal("create directory '%s' failed during copy", dst_root.c_str());
}
return 0;
}
diff --git a/src/traffic_layout/traffic_layout.cc b/src/traffic_layout/traffic_layout.cc
index c77e40c..51b0762 100644
--- a/src/traffic_layout/traffic_layout.cc
+++ b/src/traffic_layout/traffic_layout.cc
@@ -46,7 +46,6 @@ struct subcommand {
// Command line arguments (parsing)
struct CommandLineArgs {
- int layout;
int features;
int json;
};
@@ -54,7 +53,6 @@ struct CommandLineArgs {
static CommandLineArgs cl;
const ArgumentDescription argument_descriptions[] = {
- {"layout", 'l', "Show the layout (this is the default with no options given)", "T", &cl.layout, nullptr, nullptr},
{"features", 'f', "Show the compiled features", "T", &cl.features, nullptr, nullptr},
{"json", 'j', "Produce output in JSON format (when supported)", "T", &cl.json, nullptr, nullptr},
VERSION_ARGUMENT_DESCRIPTION(),
diff --git a/tests/gold_tests/runroot/runroot_init.test.py b/tests/gold_tests/runroot/runroot_init.test.py
index fbb5117..7d5e105 100644
--- a/tests/gold_tests/runroot/runroot_init.test.py
+++ b/tests/gold_tests/runroot/runroot_init.test.py
@@ -66,7 +66,8 @@ tr.Processes.Default.Streams.All = Testers.ContainsExpression("Forcing creating
path5 = os.path.join(ts_root, "runroot5")
junk1 = os.path.join(path1, "bin/junk1")
junk2 = os.path.join(path1, "lib/junk2")
-junk3 = os.path.join(path1, "include/junk3")
+junk3 = os.path.join(path1, "var/junk3")
+
tr = Test.AddTestRun("Test traffic_layout init #5")
# create the junk files in runroot1 and create(init and copy) runroot5 from runroot 1
tr.Processes.Default.Command = "touch " + junk1 + ";" + "touch " + junk2 + ";" + \
@@ -77,13 +78,13 @@ f.Exists = True
# check if the junk file is created and not copied to the new runroot
f = tr.Disk.File(junk1)
f.Exists = True
-f = tr.Disk.File(os.path.join(path5, "bin/junk"))
+f = tr.Disk.File(os.path.join(path5, "bin/junk1"))
f.Exists = False
f = tr.Disk.File(junk2)
f.Exists = True
-f = tr.Disk.File(os.path.join(path5, "lib/junk"))
+f = tr.Disk.File(os.path.join(path5, "lib/junk2"))
f.Exists = False
f = tr.Disk.File(junk3)
f.Exists = True
-f = tr.Disk.File(os.path.join(path5, "include/junk"))
+f = tr.Disk.File(os.path.join(path5, "var/junk3"))
f.Exists = False
[trafficserver] 03/03: Runroot: Add feature to use runroot by
providing yaml file
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 8.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit ea89d097086392e684034d95e2e74cad9b3c1f50
Author: Xavier Chi <ch...@gmail.com>
AuthorDate: Fri Jul 20 15:14:18 2018 -0500
Runroot: Add feature to use runroot by providing yaml file
(cherry picked from commit 37aa9fefc729ab320b2acffca480ce37ea3f21c6)
---
lib/ts/runroot.cc | 81 +++++++++++++-------------
lib/ts/runroot.h | 4 --
src/traffic_layout/engine.cc | 39 ++++++++++++-
tests/gold_tests/runroot/runroot_error.test.py | 4 +-
4 files changed, 80 insertions(+), 48 deletions(-)
diff --git a/lib/ts/runroot.cc b/lib/ts/runroot.cc
index 38edaa4..8e5f0f4 100644
--- a/lib/ts/runroot.cc
+++ b/lib/ts/runroot.cc
@@ -38,6 +38,7 @@ datadir, libexecdir, libdir, runtimedir, cachedir.
*/
#include "ts/ink_error.h"
+#include "ts/ink_file.h"
#include "ts/I_Layout.h"
#include "runroot.h"
@@ -47,27 +48,33 @@ datadir, libexecdir, libdir, runtimedir, cachedir.
#include <unistd.h>
#include <yaml-cpp/yaml.h>
-static std::string using_runroot = {};
+static std::string runroot_file = {};
// the function for the checking of the yaml file in the passed in path
-// if found return the path, if not return empty string
-std::string
-check_path(const std::string &path)
+// if found return the path to the yaml file, if not return empty string.
+static std::string
+get_yaml_path(const std::string &path)
{
- std::string whole_path = path;
- std::string yaml_path = Layout::relative_to(whole_path, "runroot_path.yml");
std::ifstream check_file;
- check_file.open(yaml_path);
- if (check_file.good()) {
- return whole_path;
+ if (ink_file_is_directory(path.c_str())) {
+ std::string yaml_path = Layout::relative_to(path, "runroot_path.yml");
+ check_file.open(yaml_path);
+ if (check_file.good()) {
+ return yaml_path;
+ }
+ } else {
+ check_file.open(path);
+ if (check_file.good() && path.substr(path.find_last_of("/") + 1) == "runroot_path.yml") {
+ return path;
+ }
}
return {};
}
// the function for the checking of the yaml file in passed in directory or parent directory
-// if found return the parent path containing the yaml file
-std::string
-check_parent_path(const std::string &path)
+// if found return the parent path to the yaml file
+static std::string
+get_parent_yaml_path(const std::string &path)
{
std::string whole_path = path;
if (whole_path.back() == '/') {
@@ -79,25 +86,17 @@ check_parent_path(const std::string &path)
if (whole_path.empty()) {
return {};
}
- if (!check_path(whole_path).empty()) {
- return whole_path;
+ std::string yaml_file = get_yaml_path(whole_path);
+ if (!yaml_file.empty()) {
+ return yaml_file;
}
whole_path = whole_path.substr(0, whole_path.find_last_of("/"));
}
return {};
}
-// until I get a <filesystem> impl in
-bool
-is_directory(const char *directory)
-{
- struct stat buffer;
- int result = stat(directory, &buffer);
- return (!result && (S_IFDIR & buffer.st_mode)) ? true : false;
-}
-
// handler for ts runroot
-// this function set up using_runroot
+// this function set up runroot_file
void
runroot_handler(const char **argv, bool json)
{
@@ -121,12 +120,12 @@ runroot_handler(const char **argv, bool json)
if (!arg.empty() && arg != prefix) {
// 1. pass in path
prefix += "=";
- path = check_path(arg.substr(prefix.size(), arg.size() - 1));
+ path = get_yaml_path(arg.substr(prefix.size(), arg.size() - 1));
if (!path.empty()) {
if (!json) {
ink_notice("using command line path as RUNROOT");
}
- using_runroot = path;
+ runroot_file = path;
return;
} else {
if (!json) {
@@ -137,10 +136,10 @@ runroot_handler(const char **argv, bool json)
// 2. check Environment variable
char *env_val = getenv("TS_RUNROOT");
- if ((env_val != nullptr) && is_directory(env_val)) {
- path = check_path(env_val);
+ if (env_val) {
+ path = get_yaml_path(env_val);
if (!path.empty()) {
- using_runroot = env_val;
+ runroot_file = path;
if (!json) {
ink_notice("using the environment variable TS_RUNROOT");
}
@@ -155,9 +154,9 @@ runroot_handler(const char **argv, bool json)
// 3. find cwd or parent path of cwd to check
char cwd[PATH_MAX] = {0};
if (getcwd(cwd, sizeof(cwd)) != nullptr) {
- path = check_parent_path(cwd);
+ path = get_parent_yaml_path(cwd);
if (!path.empty()) {
- using_runroot = path;
+ runroot_file = path;
if (!json) {
ink_notice("using cwd as TS_RUNROOT");
}
@@ -170,9 +169,9 @@ runroot_handler(const char **argv, bool json)
if ((argv[0] != nullptr) && realpath(argv[0], RealBinPath) != nullptr) {
std::string bindir = RealBinPath;
bindir = bindir.substr(0, bindir.find_last_of("/")); // getting the bin dir not executable path
- path = check_parent_path(bindir);
+ path = get_parent_yaml_path(bindir);
if (!path.empty()) {
- using_runroot = path;
+ runroot_file = path;
if (!json) {
ink_notice("using the installed dir as TS_RUNROOT");
}
@@ -209,11 +208,13 @@ runroot_map_default()
// return a map of all path in runroot_path.yml
RunrootMapType
-runroot_map(const std::string &prefix)
+runroot_map(const std::string &file)
{
RunrootMapType map;
try {
- YAML::Node yamlfile = YAML::LoadFile(Layout::relative_to(prefix, "runroot_path.yml"));
+ YAML::Node yamlfile = YAML::LoadFile(file);
+ std::string prefix = file.substr(0, file.find_last_of("/"));
+
for (auto it : yamlfile) {
// key value pairs of dirs
std::string value = it.second.as<std::string>();
@@ -223,7 +224,7 @@ runroot_map(const std::string &prefix)
map[it.first.as<std::string>()] = value;
}
} catch (YAML::Exception &e) {
- ink_warning("Unable to read runroot_path.yml from '%s': %s", prefix.c_str(), e.what());
+ ink_warning("Unable to read '%s': %s", file.c_str(), e.what());
ink_notice("Continuing with default value");
return RunrootMapType{};
}
@@ -236,19 +237,19 @@ runroot_map(const std::string &prefix)
RunrootMapType
check_runroot()
{
- if (using_runroot.empty()) {
+ if (runroot_file.empty()) {
return RunrootMapType{};
}
- int len = using_runroot.size();
+ int len = runroot_file.size();
if ((len + 1) > PATH_NAME_MAX) {
ink_fatal("runroot path is too big: %d, max %d\n", len, PATH_NAME_MAX - 1);
}
- return runroot_map(using_runroot);
+ return runroot_map(runroot_file);
}
bool
use_runroot()
{
- return !using_runroot.empty();
+ return !runroot_file.empty();
}
diff --git a/lib/ts/runroot.h b/lib/ts/runroot.h
index a06636f..fcb44e4 100644
--- a/lib/ts/runroot.h
+++ b/lib/ts/runroot.h
@@ -48,10 +48,6 @@ const std::string LAYOUT_CACHEDIR = "cachedir";
typedef std::unordered_map<std::string, std::string> RunrootMapType;
-std::string check_path(const std::string &path);
-
-std::string check_parent_path(const std::string &path);
-
void runroot_handler(const char **argv, bool json = false);
// get a map from default layout
diff --git a/src/traffic_layout/engine.cc b/src/traffic_layout/engine.cc
index 764514d..36c21e9 100644
--- a/src/traffic_layout/engine.cc
+++ b/src/traffic_layout/engine.cc
@@ -46,6 +46,41 @@
// for nftw check_directory
std::string directory_check;
+// the function for the checking of the yaml file in the passed in path
+// if found return the path, if not return empty string
+std::string
+check_path(const std::string &path)
+{
+ std::ifstream check_file;
+ check_file.open(Layout::relative_to(path, "runroot_path.yml"));
+ if (check_file.good()) {
+ return path;
+ }
+ return {};
+}
+
+// the function for the checking of the yaml file in passed in directory or parent directory
+// if found return the parent path containing the yaml file
+std::string
+check_parent_path(const std::string &path)
+{
+ std::string yaml_path = path;
+ if (yaml_path.back() == '/') {
+ yaml_path.pop_back();
+ }
+ // go up to 4 level of parent directories
+ for (int i = 0; i < 4; i++) {
+ if (yaml_path.empty()) {
+ return {};
+ }
+ if (!check_path(yaml_path).empty()) {
+ return yaml_path;
+ }
+ yaml_path = yaml_path.substr(0, yaml_path.find_last_of("/"));
+ }
+ return {};
+}
+
// check if we can create the runroot using path
// return true if the path is good to use
static bool
@@ -340,7 +375,7 @@ RunrootEngine::clean_runroot()
}
} else {
// handle the map and deleting of each directories specified in the yml file
- RunrootMapType map = runroot_map(clean_root);
+ RunrootMapType map = runroot_map(Layout::relative_to(clean_root, "runroot_path.yml"));
map.erase(LAYOUT_PREFIX);
map.erase(LAYOUT_EXEC_PREFIX);
for (auto it : map) {
@@ -728,7 +763,7 @@ RunrootEngine::verify_runroot()
path_map = runroot_map_default();
std::cout << "Verifying default build ..." << std::endl;
} else {
- path_map = runroot_map(path);
+ path_map = runroot_map(Layout::relative_to(path, "runroot_path.yml"));
}
RunrootMapType permission_map; // contains rwx bits
diff --git a/tests/gold_tests/runroot/runroot_error.test.py b/tests/gold_tests/runroot/runroot_error.test.py
index 2776243..e32df98 100644
--- a/tests/gold_tests/runroot/runroot_error.test.py
+++ b/tests/gold_tests/runroot/runroot_error.test.py
@@ -62,9 +62,9 @@ f.Exists = False
path_invalid = os.path.join(ts_root, "tmp")
tr = Test.AddTestRun("remove invalid runroot")
tr.Processes.Default.Command = "$ATS_BIN/traffic_layout remove --path " + path_invalid
-tr.Processes.Default.Streams.All = Testers.ContainsExpression("Unable to read runroot_path.yml", "remove incorrect usage")
+tr.Processes.Default.Streams.All = Testers.ContainsExpression("Unable to read", "remove incorrect usage")
# verify invalid runroot
tr = Test.AddTestRun("verify invalid runroot")
tr.Processes.Default.Command = "$ATS_BIN/traffic_layout verify --path " + path_invalid
-tr.Processes.Default.Streams.All = Testers.ContainsExpression("Unable to read runroot_path.yml", "verify incorrect usage")
+tr.Processes.Default.Streams.All = Testers.ContainsExpression("Unable to read", "verify incorrect usage")