You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficcontrol.apache.org by zr...@apache.org on 2021/04/28 21:54:11 UTC
[trafficcontrol] branch master updated: Provide initial groundwork
implementation for PostgreSQL Traffic Vault (#5792)
This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
The following commit(s) were added to refs/heads/master by this push:
new f64365c Provide initial groundwork implementation for PostgreSQL Traffic Vault (#5792)
f64365c is described below
commit f64365c3f1bbf0e9bc35a9502279ca1e362a7ab8
Author: Rawlin Peters <ra...@apache.org>
AuthorDate: Wed Apr 28 15:53:03 2021 -0600
Provide initial groundwork implementation for PostgreSQL Traffic Vault (#5792)
* Provide initial groundwork implementation for PostgreSQL Traffic Vault
Add a Postgres Traffic Vault backend and lay the groundwork for its
implementation, only actually implementing the `Ping` method and leaving
the rest unimplemented for now.
* Update example cdn.conf to match trafficvault/dbconf.yml
* Fix TO API test
* Address review comments
---
CHANGELOG.md | 2 +
docs/source/admin/index.rst | 4 +-
docs/source/admin/traffic_vault.rst | 55 ++++-
docs/source/api/migrating-from-v1.rst | 2 +-
docs/source/development/traffic_ops.rst | 20 +-
docs/source/overview/traffic_vault.rst | 4 +-
docs/source/tools/traffic_vault_util.rst | 10 +-
lib/go-tc/deliveryservice_ssl_keys.go | 13 +-
lib/go-tc/tovalidate/rules.go | 4 +
traffic_ops/app/db/admin.go | 98 +++++----
traffic_ops/app/db/trafficvault/create_tables.sql | 231 ++++++++++++++++++++
traffic_ops/app/db/trafficvault/dbconf.yml | 34 +++
.../migrations/2021042200000000_init.sql | 22 ++
.../testing/api/v4/traffic_vault_ping_test.go | 31 +++
traffic_ops/traffic_ops_golang/cdn/dnssec.go | 14 +-
.../traffic_ops_golang/cdn/dnssecrefresh.go | 5 +-
traffic_ops/traffic_ops_golang/cdn/genksk.go | 4 +-
traffic_ops/traffic_ops_golang/cdn/sslkeys.go | 2 +-
.../traffic_ops_golang/deliveryservice/acme.go | 2 +-
.../deliveryservice/acme_renew.go | 8 +-
.../deliveryservice/autorenewcerts.go | 6 +-
.../deliveryservice/deleteoldcerts.go | 2 +-
.../deliveryservice/deliveryservices.go | 2 +-
.../traffic_ops_golang/deliveryservice/dnssec.go | 7 +-
.../traffic_ops_golang/deliveryservice/keys.go | 8 +-
.../traffic_ops_golang/deliveryservice/sslkeys.go | 7 +-
.../traffic_ops_golang/deliveryservice/urlkey.go | 10 +-
traffic_ops/traffic_ops_golang/ping/keys.go | 2 +-
traffic_ops/traffic_ops_golang/ping/riak.go | 4 +-
traffic_ops/traffic_ops_golang/ping/vault.go | 4 +-
.../traffic_ops_golang/traffic_ops_golang.go | 1 +
.../traffic_ops_golang/trafficvault/README.md | 13 +-
.../vault.go => trafficvault/backends/backends.go} | 24 +--
.../trafficvault/backends/disabled/disabled.go | 31 +--
.../trafficvault/backends/postgres/postgres.go | 236 +++++++++++++++++++++
.../trafficvault/backends/riaksvc/riak.go | 31 +--
.../trafficvault/trafficvault.go | 37 ++--
.../traffic_ops_golang/urisigning/urisigning.go | 8 +-
traffic_ops/v4-client/vault.go | 32 +++
39 files changed, 851 insertions(+), 179 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 104ff4d..7837f52 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
## [unreleased]
### Added
+- Added support for PostgreSQL as a Traffic Vault backend
- [#5449](https://github.com/apache/trafficcontrol/issues/5449) The `todb-tests` GitHub action now runs the Traffic Ops DB tests
- Python client: [#5611](https://github.com/apache/trafficcontrol/pull/5611) Added server_detail endpoint
- Ported the Postinstall script to Python. The Perl version has been moved to `install/bin/_postinstall.pl` and has been deprecated, pending removal in a future release.
@@ -75,6 +76,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
- The format of the `/servers/{{host name}}/update_status` Traffic Ops API endpoint has been changed to use a top-level `response` property, in keeping with (most of) the rest of the API.
### Deprecated
+- The Riak Traffic Vault backend is now deprecated and its support may be removed in a future release. It is highly recommended to use the new PostgreSQL backend instead.
- The `riak.conf` config file and its corresponding `--riakcfg` option in `traffic_ops_golang` have been deprecated. Please use `"traffic_vault_backend": "riak"` and `"traffic_vault_config"` (with the existing contents of riak.conf) instead.
- The Traffic Ops API route `GET /api/{version}/vault/bucket/{bucket}/key/{key}/values` has been deprecated and will no longer be available as of Traffic Ops API v4
- The `riak_port` option in cdn.conf is now deprecated. Please use the `"port"` field in `traffic_vault_config` instead.
diff --git a/docs/source/admin/index.rst b/docs/source/admin/index.rst
index 3d3224a..3c4087d 100644
--- a/docs/source/admin/index.rst
+++ b/docs/source/admin/index.rst
@@ -21,9 +21,9 @@ Traffic Control is distributed in source form for the developer, but also as a b
When installing a complete CDN from scratch, a sample recommended order is:
-#. Traffic Ops DB (Postgresql)
+#. Traffic Ops DB (PostgreSQL)
#. `InfluxDB [Optional] <https://github.com/influxdata/influxdb>`_
-#. Traffic Vault (Riak) [Optional]
+#. Traffic Vault (PostgreSQL or Riak) [Optional]
#. Fake Origin [Optional]
#. Traffic Ops
#. Traffic Portal
diff --git a/docs/source/admin/traffic_vault.rst b/docs/source/admin/traffic_vault.rst
index cf275bf..29a64f8 100644
--- a/docs/source/admin/traffic_vault.rst
+++ b/docs/source/admin/traffic_vault.rst
@@ -19,12 +19,61 @@
Traffic Vault Administration
****************************
-Currently, the only supported backend for Traffic Vault is Riak, but more backends may be supported in the future.
+Currently, the supported backends for Traffic Vault are PostgreSQL and Riak, but Riak support is deprecated and may be removed in a future release. More backends may be supported in the future.
+
+.. _traffic_vault_postgresql_backend:
+
+PostgreSQL
+==========
+
+In order to use the PostgreSQL backend for Traffic Vault, you will need to set the ``traffic_vault_backend`` option to ``"postgres"`` and include the necessary configuration in the ``traffic_vault_config`` section in :file:`cdn.conf`. The ``traffic_vault_config`` options for the PostgreSQL backend are as follows:
+
+:dbname: The name of the database to use
+:hostname: The hostname of the database server to connect to
+:password: The password to use when connecting to the database
+:port: The port number that the database listens for new connections on (NOTE: the PostgreSQL default is 5432)
+:user: The username to use when connecting to the database
+:conn_max_lifetime_seconds: Optional. The maximum amount of time (in seconds) a connection may be reused. If negative, connections are not closed due to a connection's age. If 0 or unset, the default of 60 is used.
+:max_connections: Optional. The maximum number of open connections to the database. Default: 0 (unlimited)
+:max_idle_connections: Optional. The maximum number of connections in the idle connection pool. If negative, no idle connections are retained. If 0 or unset, the default of 30 is used.
+:query_timeout_seconds: Optional. The duration (in seconds) after which database queries will time out and be cancelled. Default: 30
+:ssl: Optional. Whether or not to use SSL to connect to the database. Default: false
+
+Example cdn.conf snippet:
+-------------------------
+
+.. code-block:: json
+
+ {
+ "traffic_ops_golang": {
+ "traffic_vault_backend": "postgres",
+ "traffic_vault_config": {
+ "dbname": "tv_development",
+ "hostname": "localhost",
+ "user": "traffic_vault",
+ "password": "twelve",
+ "port": 5432,
+ "ssl": false,
+ "conn_max_lifetime_seconds": 60,
+ "max_connections": 500,
+ "max_idle_connections": 30,
+ "query_timeout_seconds": 10
+ }
+ }
+ }
+
+Administration of the PostgreSQL database for Traffic Vault
+-----------------------------------------------------------
+
+Similar to administering the Traffic Ops database, the :ref:`admin <database-management>` tool should be used for administering the PostgreSQL Traffic Vault backend.
.. _traffic_vault_riak_backend:
-Riak
-====
+Riak (deprecated)
+=================
+
+.. deprecated:: ATCv6
+ The Riak Traffic Vault backend is deprecated and support may be removed in a future release. It is highly recommended to use the PostgreSQL Traffic Vault backend instead.
In order to use the Riak backend for Traffic Vault, you will need to set the ``traffic_vault_backend`` option to ``"riak"`` and include the necessary configuration in the ``traffic_vault_config`` section in :file:`cdn.conf`. The ``traffic_vault_config`` options for the Riak backend are as follows:
diff --git a/docs/source/api/migrating-from-v1.rst b/docs/source/api/migrating-from-v1.rst
index 0325fb9..58cc7d4 100644
--- a/docs/source/api/migrating-from-v1.rst
+++ b/docs/source/api/migrating-from-v1.rst
@@ -498,7 +498,7 @@ If your code performs HTTP requests to a Traffic Ops API directly, it may be nec
+-------------+-----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------+
| ``POST`` | :ref:`to-api-v1-regions` | |same| |
+-------------+-----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------+
-| ``GET`` | :ref:`to-api-riak-bucket-bucket-key-key-values` | :ref:`to-api-vault-bucket-bucket-key-key-values` |
+| ``GET`` | :ref:`to-api-riak-bucket-bucket-key-key-values` | |none| |
+-------------+-----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------+
| ``GET`` | :ref:`to-api-v1-riak-ping` | :ref:`to-api-vault-ping` |
+-------------+-----------------------------------------------------------------------------+------------------------------------------------------------------------------------------------+
diff --git a/docs/source/development/traffic_ops.rst b/docs/source/development/traffic_ops.rst
index 93cc710..a384a97 100644
--- a/docs/source/development/traffic_ops.rst
+++ b/docs/source/development/traffic_ops.rst
@@ -180,7 +180,7 @@ Traffic Ops Project Tree Overview
app/db/admin
============
-The :program:`app/db/admin` binary is for use in managing the Traffic Ops database tables. This essentially serves as a front-end for `Goose <https://github.com/kevinburke/goose>`_.
+The :program:`app/db/admin` binary is for use in managing the Traffic Ops (and Traffic Vault PostgreSQL backend) database tables. This essentially serves as a front-end for `Goose <https://github.com/kevinburke/goose>`_.
.. note:: For proper resolution of configuration and SOL statement files, it's recommended that this binary be run from the ``app`` directory
@@ -201,6 +201,10 @@ Options and Arguments
:program:`admin` sets :envvar:`MOJO_MODE` to the value of the environment as specified by this option. (Default: ``development``)
+.. option:: --trafficvault
+
+ When used, commands will be run against the Traffic Vault PostgreSQL backend database as specified in the :file:`app/db/trafficvault/dbconf.yml` configuration file.
+
.. option:: command
The :option:`command` specifies the operation to be performed on the database. It must be one of:
@@ -218,19 +222,17 @@ Options and Arguments
drop_user
Drops the user defined for the current environment
load_schema
- Sets up the database for the current environment according to the SQL statements in ``app/db/create_tables.sql``
+ Sets up the database for the current environment according to the SQL statements in ``app/db/create_tables.sql`` or ``app/db/trafficvault/create_tables.sql`` if the ``--trafficvault`` option is used
migrate
Runs a migration on the database for the current environment
patch
- Patches the database for the current environment using the SQL statements from the ``app/db/patches.sql``
+ Patches the database for the current environment using the SQL statements from the ``app/db/patches.sql``. This command is not supported when using the ``--trafficvault`` option
redo
Rolls back the most recently applied migration, then run it again
reset
Creates the user defined for the current environment, drops the database for the current environment, creates a new one, loads the schema into it, and runs a single migration on it
- reverse_schema
- Reverse engineers the ``app/lib/Schema/Result/*`` files from the environment database
seed
- Executes the SQL statements from the ``app/db/seeds.sql`` file for loading static data
+ Executes the SQL statements from the ``app/db/seeds.sql`` file for loading static data. This command is not supported when using the ``--trafficvault`` option
show_users
Displays a list of all users registered with the PostgreSQL server
status
@@ -243,7 +245,7 @@ Options and Arguments
db/admin --env=test reset
-The environments are defined in the :atc-file:`traffic_ops/app/db/dbconf.yml` file, and the name of the database generated will be the name of the environment for which it was created.
+The environments are defined in the :atc-file:`traffic_ops/app/db/dbconf.yml` file, and the name of the database generated will be the name of the environment for which it was created. If the ``--trafficvault`` option is used, the :file:`app/db/trafficvault/dbconf.yml` file defines this information.
Installing The Developer Environment
====================================
@@ -264,7 +266,7 @@ To install the Traffic Ops Developer environment:
.. seealso:: `PostgreSQL instructions on setting up a database <https://wiki.postgresql.org/wiki/First_steps>`_.
-#. Use the ``reset`` and ``upgrade`` :option:`command`\ s of :program:`admin` (see :ref:`database-management` for usage) to set up the ``traffic_ops`` database(s).
+#. Use the ``reset`` and ``upgrade`` :option:`command`\ s of :program:`admin` (see :ref:`database-management` for usage) to set up the ``traffic_ops`` database(s) and optionally with the ``--trafficvault`` option to set up the ``traffic_vault`` database(s).
#. Run the :atc-file:`traffic_ops/install/bin/postinstall` script, it will prompt for information like the default user login credentials.
#. To run Traffic Ops, follow the instructions in :ref:`to-running`.
@@ -317,7 +319,7 @@ The integration tests are run using :manpage:`go-test(1)`, with two configuratio
.. option:: --includeSystemTests ``no``/``yes``
- Specify whether to run tests that depend on additional components like an SMTP server or a Riak server. Default: ``no``
+ Specify whether to run tests that depend on additional components like an SMTP server or a Traffic Vault server. Default: ``no``
Configuring the Integration Tests
"""""""""""""""""""""""""""""""""
diff --git a/docs/source/overview/traffic_vault.rst b/docs/source/overview/traffic_vault.rst
index 0e7a63c..b438681 100644
--- a/docs/source/overview/traffic_vault.rst
+++ b/docs/source/overview/traffic_vault.rst
@@ -40,6 +40,4 @@ Traffic Vault is a data store used for storing the following types of sensitive
* URL Signing Keys
-As the name suggests, Traffic Vault is meant to be a "vault" of private keys that only certain users are allowed to access. In order to create, add, and retrieve keys a user must have administrative privileges. Keys can be created via the :ref:`tp-overview` UI, but they can only be retrieved via the :ref:`to-api`. Currently, the only supported data store used by Traffic Vault is `Riak <http://basho.com/products/riak-kv/>`_. :ref:`to-overview` accesses Riak via HTTPS on port 8088. :ref:`t [...]
-
-.. seealso:: Information on the Riak API can be found in `their documentation <http://docs.riak.com/riak/latest/dev/references/http/>`_.
+As the name suggests, Traffic Vault is meant to be a "vault" of private keys that only certain users are allowed to access. In order to create, add, and retrieve keys a user must have administrative privileges. Keys can be created via the :ref:`tp-overview` UI, but they can only be retrieved via the :ref:`to-api`. Currently, the supported data stores used by Traffic Vault are `PostgreSQL <https://www.postgresql.org/>`_ and `Riak (deprecated) <https://basho.com/products/riak-kv/>`_. Supp [...]
diff --git a/docs/source/tools/traffic_vault_util.rst b/docs/source/tools/traffic_vault_util.rst
index b8eb611..0ee3cc6 100644
--- a/docs/source/tools/traffic_vault_util.rst
+++ b/docs/source/tools/traffic_vault_util.rst
@@ -15,10 +15,12 @@
.. _traffic_vault_util:
-******************
-Traffic Vault Util
-******************
-The ``traffic_vault_util`` tool - located at :file:`tools/traffic_vault_util.go` in the `Apache Traffic Control repository <https://github.com/apache/trafficcontrol>`_ - is used to view and modify the contents of a Traffic Vault (i.e. Riak) cluster. The tool contains basic operations to display the buckets, keys and values stored within Traffic Vault.
+*************************
+Traffic Vault Util (Riak)
+*************************
+The ``traffic_vault_util`` tool - located at :file:`tools/traffic_vault_util.go` in the `Apache Traffic Control repository <https://github.com/apache/trafficcontrol>`_ - is used to view and modify the contents of a Traffic Vault Riak cluster. The tool contains basic operations to display the buckets, keys and values stored within Riak.
+
+.. note:: This tool does not apply to the PostgreSQL Traffic Vault backend.
``traffic_vault_util`` also has a small converter utility to perform a one-off conversion of key formats within the SSL bucket. This conversion is useful when moving from an older version of Traffic Ops to the current version. In the older version, SSL records were indexed by :term:`Delivery Service` database ID. Currently, SSL records are indexed by :term:`Delivery Service` ``xml_id``.
diff --git a/lib/go-tc/deliveryservice_ssl_keys.go b/lib/go-tc/deliveryservice_ssl_keys.go
index b759eda..ed41374 100644
--- a/lib/go-tc/deliveryservice_ssl_keys.go
+++ b/lib/go-tc/deliveryservice_ssl_keys.go
@@ -221,14 +221,21 @@ type URISignerKeyset struct {
Keys []jwk.EssentialHeader `json:"keys"`
}
-// Deprecated: use TrafficVaultPingResponse instead.
-type RiakPingResp TrafficVaultPingResponse
+// Deprecated: use TrafficVaultPing instead.
+type RiakPingResp TrafficVaultPing
-type TrafficVaultPingResponse struct {
+// TrafficVaultPing represents the status of a given Traffic Vault server.
+type TrafficVaultPing struct {
Status string `json:"status"`
Server string `json:"server"`
}
+// TrafficVaultPingResponse represents the JSON HTTP response returned by the /vault/ping route.
+type TrafficVaultPingResponse struct {
+ Response TrafficVaultPing `json:"response"`
+ Alerts
+}
+
// DNSSECKeys is the DNSSEC keys as stored in Riak, plus the DS record text.
type DNSSECKeys map[string]DNSSECKeySet
diff --git a/lib/go-tc/tovalidate/rules.go b/lib/go-tc/tovalidate/rules.go
index cbe5818..00010b3 100644
--- a/lib/go-tc/tovalidate/rules.go
+++ b/lib/go-tc/tovalidate/rules.go
@@ -140,6 +140,10 @@ func IsGreaterThanZero(value interface{}) error {
func IsValidPortNumber(value interface{}) error {
switch v := value.(type) {
+ case int:
+ if v > 0 && v <= 65535 {
+ return nil
+ }
case *int:
if v == nil || *v > 0 && *v <= 65535 {
return nil
diff --git a/traffic_ops/app/db/admin.go b/traffic_ops/app/db/admin.go
index 035dbf2..a8b11bc 100644
--- a/traffic_ops/app/db/admin.go
+++ b/traffic_ops/app/db/admin.go
@@ -74,22 +74,21 @@ const (
DBNameKey = "dbname"
// available commands
- CmdCreateDB = "createdb"
- CmdDropDB = "dropdb"
- CmdCreateUser = "create_user"
- CmdDropUser = "drop_user"
- CmdShowUsers = "show_users"
- CmdReset = "reset"
- CmdUpgrade = "upgrade"
- CmdMigrate = "migrate"
- CmdDown = "down"
- CmdRedo = "redo"
- CmdStatus = "status"
- CmdDBVersion = "dbversion"
- CmdSeed = "seed"
- CmdLoadSchema = "load_schema"
- CmdReverseSchema = "reverse_schema"
- CmdPatch = "patch"
+ CmdCreateDB = "createdb"
+ CmdDropDB = "dropdb"
+ CmdCreateUser = "create_user"
+ CmdDropUser = "drop_user"
+ CmdShowUsers = "show_users"
+ CmdReset = "reset"
+ CmdUpgrade = "upgrade"
+ CmdMigrate = "migrate"
+ CmdDown = "down"
+ CmdRedo = "redo"
+ CmdStatus = "status"
+ CmdDBVersion = "dbversion"
+ CmdSeed = "seed"
+ CmdLoadSchema = "load_schema"
+ CmdPatch = "patch"
// goose commands that don't match the commands for this tool
GooseUp = "up"
@@ -100,11 +99,16 @@ const (
DBPatchesPath = "db/patches.sql"
DefaultEnvironment = EnvDevelopment
DefaultDBSuperUser = "postgres"
+
+ TrafficVaultDBConfigPath = "db/trafficvault/dbconf.yml"
+ TrafficVaultDir = "db/trafficvault"
+ TrafficVaultSchemaPath = "db/trafficvault/create_tables.sql"
)
var (
// globals that are passed in via CLI flags and used in commands
- Environment string
+ Environment string
+ TrafficVault bool
// globals that are parsed out of DBConfigFile and used in commands
DBName string
@@ -116,9 +120,13 @@ var (
)
func parseDBConfig() error {
- confBytes, err := ioutil.ReadFile(DBConfigPath)
+ dbConfigPath := DBConfigPath
+ if TrafficVault {
+ dbConfigPath = TrafficVaultDBConfigPath
+ }
+ confBytes, err := ioutil.ReadFile(dbConfigPath)
if err != nil {
- return errors.New("reading DB conf '" + DBConfigPath + "': " + err.Error())
+ return errors.New("reading DB conf '" + dbConfigPath + "': " + err.Error())
}
dbConfig := DBConfig{}
@@ -252,8 +260,10 @@ func reset() {
func upgrade() {
goose(GooseUp)
- seed()
- patch()
+ if !TrafficVault {
+ seed()
+ patch()
+ }
}
func migrate() {
@@ -277,6 +287,9 @@ func dbVersion() {
}
func seed() {
+ if TrafficVault {
+ die("seed not supported for trafficvault environment")
+ }
fmt.Println("Seeding database w/ required data.")
seedsBytes, err := ioutil.ReadFile(DBSeedsPath)
if err != nil {
@@ -294,7 +307,11 @@ func seed() {
func loadSchema() {
fmt.Println("Creating database tables.")
- schemaBytes, err := ioutil.ReadFile(DBSchemaPath)
+ schemaPath := DBSchemaPath
+ if TrafficVault {
+ schemaPath = TrafficVaultSchemaPath
+ }
+ schemaBytes, err := ioutil.ReadFile(schemaPath)
if err != nil {
die("unable to read '" + DBSchemaPath + "': " + err.Error())
}
@@ -308,18 +325,10 @@ func loadSchema() {
}
}
-func reverseSchema() {
- fmt.Fprintf(os.Stderr, "WARNING: the '%s' command will be removed with Traffic Ops Perl because it will no longer be necessary\n", CmdReverseSchema)
- cmd := exec.Command("db/reverse_schema.pl")
- cmd.Env = append(os.Environ(), "MOJO_MODE="+Environment)
- out, err := cmd.CombinedOutput()
- fmt.Printf("%s", out)
- if err != nil {
- die("Can't run `db/reverse_schema.pl`: " + err.Error())
- }
-}
-
func patch() {
+ if TrafficVault {
+ die("patch not supported for trafficvault environment")
+ }
fmt.Println("Patching database with required data fixes.")
patchesBytes, err := ioutil.ReadFile(DBPatchesPath)
if err != nil {
@@ -337,7 +346,12 @@ func patch() {
func goose(arg string) {
fmt.Println("Running goose " + arg + "...")
- cmd := exec.Command("goose", "--env="+Environment, arg)
+ args := []string{"--env=" + Environment}
+ if TrafficVault {
+ args = append(args, "--path="+TrafficVaultDir)
+ }
+ args = append(args, arg)
+ cmd := exec.Command("goose", args...)
out, err := cmd.CombinedOutput()
fmt.Printf("%s", out)
if err != nil {
@@ -355,12 +369,14 @@ func usage() string {
home := "$HOME"
home = os.Getenv("HOME")
return `
-Usage: ` + programName + ` [--env (development|test|production|integration)] [arguments]
+Usage: ` + programName + ` [--trafficvault] [--env (development|test|production|integration)] [arguments]
Example: ` + programName + ` --env=test reset
-Purpose: This script is used to manage database. The environments are
- defined in the dbconf.yml, as well as the database names.
+Purpose: This script is used to manage the Traffic Ops database and Traffic Vault PostgreSQL backend database.
+ The Traffic Ops environments and database names are defined in the dbconf.yml, and for Traffic Vault
+ they are defined in trafficvault/dbconf.yml. In order to execute commands against the Traffic Vault
+ database, the the --trafficvault option.
NOTE:
Postgres Superuser: The 'postgres' superuser needs to be created to run ` + programName + ` and setup databases.
@@ -381,6 +397,7 @@ without prompts.
----------------------
*:*:*:postgres:your-postgres-password
*:*:*:traffic_ops:the-password-in-dbconf.yml
+ *:*:*:traffic_vault:the-password-in-trafficvault-dbconf.yml
----------------------
Save the following example into this file ` + home + `/.pgpass with the permissions of this file
@@ -396,11 +413,10 @@ create_user - Execute 'create_user' the user for the current environment (traff
dropdb - Execute db 'dropdb' on the database for the current environment.
down - Roll back a single migration from the current version.
drop_user - Execute 'drop_user' the user for the current environment (traffic_ops).
-patch - Execute sql from db/patches.sql for loading post-migration data patches.
+patch - Execute sql from db/patches.sql for loading post-migration data patches (NOTE: not supported with --trafficvault option).
redo - Roll back the most recently applied migration, then run it again.
reset - Execute db 'dropdb', 'createdb', load_schema, migrate on the database for the current environment.
-reverse_schema - Reverse engineer the lib/Schema/Result files from the environment database.
-seed - Execute sql from db/seeds.sql for loading static data.
+seed - Execute sql from db/seeds.sql for loading static data (NOTE: not supported with --trafficvault option).
show_users - Execute sql to show all of the user for the current environment.
status - Print the status of all migrations.
upgrade - Execute migrate, seed, and patches on the database for the current environment.
@@ -410,6 +426,7 @@ migrate - Execute migrate (without seeds or patches) on the database for the cu
func main() {
flag.StringVar(&Environment, "env", DefaultEnvironment, "The environment to use (defined in "+DBConfigPath+").")
+ flag.BoolVar(&TrafficVault, "trafficvault", false, "Run this for the Traffic Vault database")
flag.Parse()
if len(flag.Args()) != 1 || flag.Arg(0) == "" {
die(usage())
@@ -436,7 +453,6 @@ func main() {
commands[CmdDBVersion] = dbVersion
commands[CmdSeed] = seed
commands[CmdLoadSchema] = loadSchema
- commands[CmdReverseSchema] = reverseSchema
commands[CmdPatch] = patch
userCmd := flag.Arg(0)
diff --git a/traffic_ops/app/db/trafficvault/create_tables.sql b/traffic_ops/app/db/trafficvault/create_tables.sql
new file mode 100644
index 0000000..e82166d
--- /dev/null
+++ b/traffic_ops/app/db/trafficvault/create_tables.sql
@@ -0,0 +1,231 @@
+/*
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+--
+-- PostgreSQL database dump
+--
+
+-- Dumped from database version 13.1
+-- Dumped by pg_dump version 13.1
+
+SET statement_timeout = 0;
+SET lock_timeout = 0;
+SET client_encoding = 'UTF8';
+SET standard_conforming_strings = on;
+SET check_function_bodies = false;
+SET client_min_messages = warning;
+SET row_security = off;
+
+--
+-- Name: plpgsql; Type: EXTENSION; Schema: -; Owner:
+--
+
+CREATE EXTENSION IF NOT EXISTS plpgsql WITH SCHEMA pg_catalog;
+
+
+SET search_path = public, pg_catalog;
+
+--
+-- Name: on_update_current_timestamp_last_updated(); Type: FUNCTION; Schema: public; Owner: traffic_vault
+--
+
+CREATE OR REPLACE FUNCTION on_update_current_timestamp_last_updated() RETURNS trigger
+ LANGUAGE plpgsql
+ AS $$
+BEGIN
+ NEW.last_updated = now();
+ RETURN NEW;
+END;
+$$;
+
+
+ALTER FUNCTION on_update_current_timestamp_last_updated() OWNER TO traffic_vault;
+
+SET default_tablespace = '';
+
+SET default_table_access_method = heap;
+
+--
+-- Name: dnssec; Type: TABLE; Schema: public; Owner: traffic_vault
+--
+
+CREATE TABLE IF NOT EXISTS dnssec (
+ cdn text NOT NULL,
+ data jsonb NOT NULL,
+ last_updated timestamp with time zone DEFAULT now() NOT NULL
+);
+
+
+ALTER TABLE dnssec OWNER TO traffic_vault;
+
+--
+-- Name: sslkey; Type: TABLE; Schema: public; Owner: traffic_vault
+--
+
+CREATE TABLE IF NOT EXISTS sslkey (
+ id bigint NOT NULL,
+ deliveryservice text NOT NULL,
+ cdn text NOT NULL,
+ version text NOT NULL,
+ last_updated timestamp with time zone DEFAULT now() NOT NULL
+);
+
+
+ALTER TABLE sslkey OWNER TO traffic_vault;
+
+--
+-- Name: sslkey_id_seq; Type: SEQUENCE; Schema: public; Owner: traffic_vault
+--
+
+CREATE SEQUENCE IF NOT EXISTS sslkey_id_seq
+ START WITH 1
+ INCREMENT BY 1
+ NO MINVALUE
+ NO MAXVALUE
+ CACHE 1;
+
+
+ALTER TABLE sslkey_id_seq OWNER TO traffic_vault;
+
+--
+-- Name: sslkey_id_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: traffic_vault
+--
+
+ALTER SEQUENCE sslkey_id_seq OWNED BY sslkey.id;
+
+
+--
+-- Name: uri_signing_key; Type: TABLE; Schema: public; Owner: traffic_vault
+--
+
+CREATE TABLE IF NOT EXISTS uri_signing_key (
+ deliveryservice text NOT NULL,
+ data jsonb NOT NULL,
+ last_updated timestamp with time zone DEFAULT now() NOT NULL
+);
+
+
+ALTER TABLE uri_signing_key OWNER TO traffic_vault;
+
+--
+-- Name: url_sig_key; Type: TABLE; Schema: public; Owner: traffic_vault
+--
+
+CREATE TABLE IF NOT EXISTS url_sig_key (
+ deliveryservice text NOT NULL,
+ data jsonb NOT NULL,
+ last_updated timestamp with time zone DEFAULT now() NOT NULL
+);
+
+
+ALTER TABLE url_sig_key OWNER TO traffic_vault;
+
+--
+-- Name: sslkey id; Type: DEFAULT; Schema: public; Owner: traffic_vault
+--
+
+ALTER TABLE ONLY sslkey ALTER COLUMN id SET DEFAULT nextval('sslkey_id_seq'::regclass);
+
+--
+-- Name: sslkey_id_seq; Type: SEQUENCE SET; Schema: public; Owner: traffic_vault
+--
+
+SELECT pg_catalog.setval('sslkey_id_seq', 1, false);
+
+
+--
+-- Name: dnssec dnssec_pkey; Type: CONSTRAINT; Schema: public; Owner: traffic_vault
+--
+
+ALTER TABLE ONLY dnssec
+ ADD CONSTRAINT dnssec_pkey PRIMARY KEY (cdn);
+
+
+--
+-- Name: sslkey sslkey_pkey; Type: CONSTRAINT; Schema: public; Owner: traffic_vault
+--
+
+ALTER TABLE ONLY sslkey
+ ADD CONSTRAINT sslkey_pkey PRIMARY KEY (id);
+
+
+--
+-- Name: uri_signing_key uri_signing_key_pkey; Type: CONSTRAINT; Schema: public; Owner: traffic_vault
+--
+
+ALTER TABLE ONLY uri_signing_key
+ ADD CONSTRAINT uri_signing_key_pkey PRIMARY KEY (deliveryservice);
+
+
+--
+-- Name: url_sig_key url_sig_key_pkey; Type: CONSTRAINT; Schema: public; Owner: traffic_vault
+--
+
+ALTER TABLE ONLY url_sig_key
+ ADD CONSTRAINT url_sig_key_pkey PRIMARY KEY (deliveryservice);
+
+
+--
+-- Name: sslkey_cdn_idx; Type: INDEX; Schema: public; Owner: traffic_vault
+--
+
+CREATE INDEX sslkey_cdn_idx ON sslkey USING btree (cdn);
+
+
+--
+-- Name: sslkey_deliveryservice_idx; Type: INDEX; Schema: public; Owner: traffic_vault
+--
+
+CREATE INDEX sslkey_deliveryservice_idx ON sslkey USING btree (deliveryservice);
+
+
+--
+-- Name: sslkey_version_idx; Type: INDEX; Schema: public; Owner: traffic_vault
+--
+
+CREATE INDEX sslkey_version_idx ON sslkey USING btree (version);
+
+
+--
+-- Name: dnssec dnssec_last_updated; Type: TRIGGER; Schema: public; Owner: traffic_vault
+--
+
+CREATE TRIGGER dnssec_last_updated BEFORE UPDATE ON dnssec FOR EACH ROW EXECUTE FUNCTION on_update_current_timestamp_last_updated();
+
+
+--
+-- Name: sslkey sslkey_last_updated; Type: TRIGGER; Schema: public; Owner: traffic_vault
+--
+
+CREATE TRIGGER sslkey_last_updated BEFORE UPDATE ON sslkey FOR EACH ROW EXECUTE FUNCTION on_update_current_timestamp_last_updated();
+
+
+--
+-- Name: uri_signing_key uri_signing_key_last_updated; Type: TRIGGER; Schema: public; Owner: traffic_vault
+--
+
+CREATE TRIGGER uri_signing_key_last_updated BEFORE UPDATE ON uri_signing_key FOR EACH ROW EXECUTE FUNCTION on_update_current_timestamp_last_updated();
+
+
+--
+-- Name: url_sig_key url_sig_key_last_updated; Type: TRIGGER; Schema: public; Owner: traffic_vault
+--
+
+CREATE TRIGGER url_sig_key_last_updated BEFORE UPDATE ON url_sig_key FOR EACH ROW EXECUTE FUNCTION on_update_current_timestamp_last_updated();
+
+
+--
+-- PostgreSQL database dump complete
+--
+
diff --git a/traffic_ops/app/db/trafficvault/dbconf.yml b/traffic_ops/app/db/trafficvault/dbconf.yml
new file mode 100644
index 0000000..eb71573
--- /dev/null
+++ b/traffic_ops/app/db/trafficvault/dbconf.yml
@@ -0,0 +1,34 @@
+#
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+version: "1.0"
+name: dbconf.yml
+
+development:
+ driver: postgres
+ open: host=127.0.0.1 port=5432 user=traffic_vault password=twelve dbname=tv_development sslmode=disable
+
+test:
+ driver: postgres
+ open: host=127.0.0.1 port=5432 user=traffic_vault password=twelve dbname=tv_test sslmode=disable
+
+integration:
+ driver: postgres
+ open: host=127.0.0.1 port=5432 user=traffic_vault password=twelve dbname=tv_integration sslmode=disable
+
+production:
+ driver: postgres
+ open: host=127.0.0.1 port=5432 user=traffic_vault password=twelve dbname=traffic_vault sslmode=disable
+
diff --git a/traffic_ops/app/db/trafficvault/migrations/2021042200000000_init.sql b/traffic_ops/app/db/trafficvault/migrations/2021042200000000_init.sql
new file mode 100644
index 0000000..29f169f
--- /dev/null
+++ b/traffic_ops/app/db/trafficvault/migrations/2021042200000000_init.sql
@@ -0,0 +1,22 @@
+/*
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+-- +goose Up
+-- SQL in section 'Up' is executed when this migration is applied
+SELECT;
+
+-- +goose Down
+-- SQL section 'Down' is executed when this migration is rolled back
+SELECT;
diff --git a/traffic_ops/testing/api/v4/traffic_vault_ping_test.go b/traffic_ops/testing/api/v4/traffic_vault_ping_test.go
new file mode 100644
index 0000000..76ccf79
--- /dev/null
+++ b/traffic_ops/testing/api/v4/traffic_vault_ping_test.go
@@ -0,0 +1,31 @@
+package v4
+
+/*
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+import (
+ "testing"
+)
+
+func TestTrafficVaultPing(t *testing.T) {
+ if includeSystemTests {
+ WithObjs(t, []TCObj{CDNs, Types, Tenants, Users, Parameters, Profiles, Statuses, Divisions, Regions, PhysLocations, CacheGroups, Servers}, func() {
+ _, _, err := TOSession.TrafficVaultPing()
+ if err != nil {
+ t.Errorf("could not ping Traffic Vault: %v", err)
+ }
+ })
+ }
+}
diff --git a/traffic_ops/traffic_ops_golang/cdn/dnssec.go b/traffic_ops/traffic_ops_golang/cdn/dnssec.go
index d77ab31..06a7c75 100644
--- a/traffic_ops/traffic_ops_golang/cdn/dnssec.go
+++ b/traffic_ops/traffic_ops_golang/cdn/dnssec.go
@@ -20,6 +20,7 @@ package cdn
*/
import (
+ "context"
"database/sql"
"errors"
"net/http"
@@ -81,7 +82,7 @@ func CreateDNSSECKeys(w http.ResponseWriter, r *http.Request) {
return
}
- if err := generateStoreDNSSECKeys(inf.Tx.Tx, cdnName, cdnDomain, uint64(*req.TTL), uint64(*req.KSKExpirationDays), uint64(*req.ZSKExpirationDays), int64(*req.EffectiveDateUnix), inf.Vault); err != nil {
+ if err := generateStoreDNSSECKeys(inf.Tx.Tx, cdnName, cdnDomain, uint64(*req.TTL), uint64(*req.KSKExpirationDays), uint64(*req.ZSKExpirationDays), int64(*req.EffectiveDateUnix), inf.Vault, r.Context()); err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("generating and storing DNSSEC CDN keys: "+err.Error()))
return
}
@@ -113,7 +114,7 @@ func GetDNSSECKeys(w http.ResponseWriter, r *http.Request) {
cdnName := inf.Params["name"]
- tvKeys, keysExist, err := inf.Vault.GetDNSSECKeys(cdnName, inf.Tx.Tx)
+ tvKeys, keysExist, err := inf.Vault.GetDNSSECKeys(cdnName, inf.Tx.Tx, r.Context())
if err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("getting DNSSEC CDN keys: "+err.Error()))
return
@@ -148,7 +149,7 @@ func GetDNSSECKeysV11(w http.ResponseWriter, r *http.Request) {
defer inf.Close()
cdnName := inf.Params["name"]
- riakKeys, keysExist, err := inf.Vault.GetDNSSECKeys(cdnName, inf.Tx.Tx)
+ riakKeys, keysExist, err := inf.Vault.GetDNSSECKeys(cdnName, inf.Tx.Tx, r.Context())
if err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("getting DNSSEC CDN keys: "+err.Error()))
return
@@ -178,13 +179,14 @@ func generateStoreDNSSECKeys(
zExpDays uint64,
effectiveDateUnix int64,
tv trafficvault.TrafficVault,
+ ctx context.Context,
) error {
zExp := time.Duration(zExpDays) * time.Hour * 24
kExp := time.Duration(kExpDays) * time.Hour * 24
ttl := time.Duration(ttlSeconds) * time.Second
- oldKeys, oldKeysExist, err := tv.GetDNSSECKeys(cdnName, tx)
+ oldKeys, oldKeysExist, err := tv.GetDNSSECKeys(cdnName, tx, ctx)
if err != nil {
return errors.New("getting old dnssec keys: " + err.Error())
}
@@ -264,7 +266,7 @@ func generateStoreDNSSECKeys(
}
newKeys[ds.Name] = dsKeys
}
- if err := tv.PutDNSSECKeys(cdnName, newKeys, tx); err != nil {
+ if err := tv.PutDNSSECKeys(cdnName, newKeys, tx, ctx); err != nil {
return errors.New("putting CDN DNSSEC keys in Traffic Vault: " + err.Error())
}
return nil
@@ -348,7 +350,7 @@ func deleteDNSSECKeys(w http.ResponseWriter, r *http.Request, deprecated bool) {
writeError(w, r, inf.Tx.Tx, http.StatusNotFound, nil, nil, deprecated)
return
}
- if err := inf.Vault.DeleteDNSSECKeys(key, inf.Tx.Tx); err != nil {
+ if err := inf.Vault.DeleteDNSSECKeys(key, inf.Tx.Tx, r.Context()); err != nil {
writeError(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("deleting CDN DNSSEC keys: "+err.Error()), deprecated)
return
}
diff --git a/traffic_ops/traffic_ops_golang/cdn/dnssecrefresh.go b/traffic_ops/traffic_ops_golang/cdn/dnssecrefresh.go
index 503dc12..3bac24f 100644
--- a/traffic_ops/traffic_ops_golang/cdn/dnssecrefresh.go
+++ b/traffic_ops/traffic_ops_golang/cdn/dnssecrefresh.go
@@ -20,6 +20,7 @@ package cdn
*/
import (
+ "context"
// "context"
"database/sql"
"errors"
@@ -137,7 +138,7 @@ func doDNSSECKeyRefresh(tx *sql.Tx, tv trafficvault.TrafficVault) {
}
for _, cdnInf := range cdnDNSSECKeyParams {
- keys, ok, err := tv.GetDNSSECKeys(string(cdnInf.CDNName), tx) // TODO get all in a map beforehand
+ keys, ok, err := tv.GetDNSSECKeys(string(cdnInf.CDNName), tx, context.Background()) // TODO get all in a map beforehand
if err != nil {
log.Warnln("refreshing DNSSEC Keys: getting cdn '" + string(cdnInf.CDNName) + "' keys from Traffic Vault, skipping: " + err.Error())
continue
@@ -271,7 +272,7 @@ func doDNSSECKeyRefresh(tx *sql.Tx, tv trafficvault.TrafficVault) {
}
}
if updatedAny {
- if err := tv.PutDNSSECKeys(string(cdnInf.CDNName), keys, tx); err != nil {
+ if err := tv.PutDNSSECKeys(string(cdnInf.CDNName), keys, tx, context.Background()); err != nil {
log.Errorln("refreshing DNSSEC Keys: putting keys into Traffic Vault for cdn '" + string(cdnInf.CDNName) + "': " + err.Error())
}
}
diff --git a/traffic_ops/traffic_ops_golang/cdn/genksk.go b/traffic_ops/traffic_ops_golang/cdn/genksk.go
index 987b869..75abeae 100644
--- a/traffic_ops/traffic_ops_golang/cdn/genksk.go
+++ b/traffic_ops/traffic_ops_golang/cdn/genksk.go
@@ -91,7 +91,7 @@ func GenerateKSK(w http.ResponseWriter, r *http.Request) {
multiplier = &mult
}
- dnssecKeys, ok, err := inf.Vault.GetDNSSECKeys(string(cdnName), inf.Tx.Tx)
+ dnssecKeys, ok, err := inf.Vault.GetDNSSECKeys(string(cdnName), inf.Tx.Tx, r.Context())
if err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("getting CDN DNSSEC keys: "+err.Error()))
return
@@ -109,7 +109,7 @@ func GenerateKSK(w http.ResponseWriter, r *http.Request) {
}
dnssecKeys[string(cdnName)] = newKey
- if err := inf.Vault.PutDNSSECKeys(string(cdnName), dnssecKeys, inf.Tx.Tx); err != nil {
+ if err := inf.Vault.PutDNSSECKeys(string(cdnName), dnssecKeys, inf.Tx.Tx, r.Context()); err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("putting CDN DNSSEC keys: "+err.Error()))
return
}
diff --git a/traffic_ops/traffic_ops_golang/cdn/sslkeys.go b/traffic_ops/traffic_ops_golang/cdn/sslkeys.go
index a14be70..7715c6d 100644
--- a/traffic_ops/traffic_ops_golang/cdn/sslkeys.go
+++ b/traffic_ops/traffic_ops_golang/cdn/sslkeys.go
@@ -37,7 +37,7 @@ func GetSSLKeys(w http.ResponseWriter, r *http.Request) {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("getting CDN SSL keys from Traffic Vault: Traffic Vault is not configured"))
return
}
- keys, err := inf.Vault.GetCDNSSLKeys(inf.Params["name"], inf.Tx.Tx)
+ keys, err := inf.Vault.GetCDNSSLKeys(inf.Params["name"], inf.Tx.Tx, r.Context())
if err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("getting cdn ssl keys from Traffic Vault: "+err.Error()))
return
diff --git a/traffic_ops/traffic_ops_golang/deliveryservice/acme.go b/traffic_ops/traffic_ops_golang/deliveryservice/acme.go
index ce63298..4c4d85e 100644
--- a/traffic_ops/traffic_ops_golang/deliveryservice/acme.go
+++ b/traffic_ops/traffic_ops_golang/deliveryservice/acme.go
@@ -469,7 +469,7 @@ func GetAcmeCertificates(cfg *config.Config, req tc.DeliveryServiceAcmeSSLKeysRe
CSR: string(EncodePEMToLegacyPerlRiakFormat([]byte("ACME Generated"))),
}
- if err := tv.PutDeliveryServiceSSLKeys(dsSSLKeys, tx); err != nil {
+ if err := tv.PutDeliveryServiceSSLKeys(dsSSLKeys, tx, context.Background()); err != nil {
log.Errorf("Error putting ACME certificate in Traffic Vault: %s", err.Error())
api.CreateChangeLogRawTx(api.ApiChange, "DS: "+*req.DeliveryService+", ID: "+strconv.Itoa(dsID)+", ACTION: FAILED to add SSL keys with "+provider, currentUser, logTx)
if asycErr := api.UpdateAsyncStatus(db, api.AsyncFailed, "ACME renewal failed.", asyncStatusId, true); asycErr != nil {
diff --git a/traffic_ops/traffic_ops_golang/deliveryservice/acme_renew.go b/traffic_ops/traffic_ops_golang/deliveryservice/acme_renew.go
index 01d4cf1..685a190 100644
--- a/traffic_ops/traffic_ops_golang/deliveryservice/acme_renew.go
+++ b/traffic_ops/traffic_ops_golang/deliveryservice/acme_renew.go
@@ -58,7 +58,7 @@ func RenewAcmeCertificate(w http.ResponseWriter, r *http.Request) {
ctx, _ := context.WithTimeout(r.Context(), AcmeTimeout)
- userErr, sysErr, statusCode := renewAcmeCerts(inf.Config, xmlID, ctx, inf.User, inf.Vault)
+ userErr, sysErr, statusCode := renewAcmeCerts(inf.Config, xmlID, ctx, r.Context(), inf.User, inf.Vault)
if userErr != nil || sysErr != nil {
api.HandleErr(w, r, inf.Tx.Tx, statusCode, userErr, sysErr)
return
@@ -68,7 +68,7 @@ func RenewAcmeCertificate(w http.ResponseWriter, r *http.Request) {
}
-func renewAcmeCerts(cfg *config.Config, dsName string, ctx context.Context, currentUser *auth.CurrentUser, tv trafficvault.TrafficVault) (error, error, int) {
+func renewAcmeCerts(cfg *config.Config, dsName string, ctx context.Context, httpCtx context.Context, currentUser *auth.CurrentUser, tv trafficvault.TrafficVault) (error, error, int) {
db, err := api.GetDB(ctx)
if err != nil {
log.Errorf(dsName+": Error getting db: %s", err.Error())
@@ -109,7 +109,7 @@ func renewAcmeCerts(cfg *config.Config, dsName string, ctx context.Context, curr
if cfg == nil {
return nil, errors.New("acme: config was nil"), http.StatusInternalServerError
}
- keyObj, ok, err := tv.GetDeliveryServiceSSLKeys(dsName, strconv.Itoa(int(*certVersion)), tx)
+ keyObj, ok, err := tv.GetDeliveryServiceSSLKeys(dsName, strconv.Itoa(int(*certVersion)), tx, httpCtx)
if err != nil {
return nil, errors.New("getting ssl keys for xmlId: " + dsName + " and version: " + strconv.Itoa(int(*certVersion)) + " : " + err.Error()), http.StatusInternalServerError
}
@@ -162,7 +162,7 @@ func renewAcmeCerts(cfg *config.Config, dsName string, ctx context.Context, curr
CSR: string(EncodePEMToLegacyPerlRiakFormat([]byte("ACME Generated"))),
}
- if err := tv.PutDeliveryServiceSSLKeys(newCertObj, tx); err != nil {
+ if err := tv.PutDeliveryServiceSSLKeys(newCertObj, tx, httpCtx); err != nil {
log.Errorf("Error posting acme certificate to Traffic Vault: %s", err.Error())
api.CreateChangeLogRawTx(api.ApiChange, "DS: "+dsName+", ID: "+strconv.Itoa(*dsID)+", ACTION: FAILED to add SSL keys with "+acmeAccount.AcmeProvider, currentUser, logTx)
return nil, errors.New(dsName + ": putting keys in Traffic Vault: " + err.Error()), http.StatusInternalServerError
diff --git a/traffic_ops/traffic_ops_golang/deliveryservice/autorenewcerts.go b/traffic_ops/traffic_ops_golang/deliveryservice/autorenewcerts.go
index 4b5318e..6d59d86 100644
--- a/traffic_ops/traffic_ops_golang/deliveryservice/autorenewcerts.go
+++ b/traffic_ops/traffic_ops_golang/deliveryservice/autorenewcerts.go
@@ -164,7 +164,7 @@ func RunAutorenewal(existingCerts []ExistingCerts, cfg *config.Config, ctx conte
}
dsExpInfo := DsExpirationInfo{}
- keyObj, ok, err := tv.GetDeliveryServiceSSLKeys(ds.XmlId, strconv.Itoa(int(ds.Version.Int64)), tx)
+ keyObj, ok, err := tv.GetDeliveryServiceSSLKeys(ds.XmlId, strconv.Itoa(int(ds.Version.Int64)), tx, ctx)
if err != nil {
log.Errorf("getting ssl keys for xmlId: %s and version: %d : %s", ds.XmlId, ds.Version.Int64, err.Error())
dsExpInfo.XmlId = ds.XmlId
@@ -245,7 +245,9 @@ func RunAutorenewal(existingCerts []ExistingCerts, cfg *config.Config, ctx conte
if acmeAccount == nil {
keysFound.OtherExpirations = append(keysFound.OtherExpirations, dsExpInfo)
} else {
- userErr, sysErr, statusCode := renewAcmeCerts(cfg, keyObj.DeliveryService, ctx, currentUser, tv)
+ // background httpCtx since this is run in a goroutine spawned off the original http request
+ // so the context isn't cancelled when the http connection is closed
+ userErr, sysErr, statusCode := renewAcmeCerts(cfg, keyObj.DeliveryService, ctx, context.Background(), currentUser, tv)
if userErr != nil {
errorCount++
dsExpInfo.Error = userErr
diff --git a/traffic_ops/traffic_ops_golang/deliveryservice/deleteoldcerts.go b/traffic_ops/traffic_ops_golang/deliveryservice/deleteoldcerts.go
index 478c43b..330a48b 100644
--- a/traffic_ops/traffic_ops_golang/deliveryservice/deleteoldcerts.go
+++ b/traffic_ops/traffic_ops_golang/deliveryservice/deleteoldcerts.go
@@ -66,7 +66,7 @@ func deleteOldDSCerts(tx *sql.Tx, cdn tc.CDNName, tv trafficvault.TrafficVault)
return errors.New("getting ds names: " + err.Error())
}
- if err := tv.DeleteOldDeliveryServiceSSLKeys(dses, string(cdn), tx); err != nil {
+ if err := tv.DeleteOldDeliveryServiceSSLKeys(dses, string(cdn), tx, context.Background()); err != nil {
return errors.New("getting ds keys from Traffic Vault: " + err.Error())
}
return nil
diff --git a/traffic_ops/traffic_ops_golang/deliveryservice/deliveryservices.go b/traffic_ops/traffic_ops_golang/deliveryservice/deliveryservices.go
index f4c0978..6a53d69 100644
--- a/traffic_ops/traffic_ops_golang/deliveryservice/deliveryservices.go
+++ b/traffic_ops/traffic_ops_golang/deliveryservice/deliveryservices.go
@@ -482,7 +482,7 @@ func createV40(w http.ResponseWriter, r *http.Request, inf *api.APIInfo, dsV40 t
if !inf.Config.TrafficVaultEnabled {
return nil, http.StatusInternalServerError, nil, errors.New("cannot create DNSSEC keys for delivery service: Traffic Vault is not configured")
}
- if userErr, sysErr, statusCode := PutDNSSecKeys(tx, *ds.XMLID, cdnName, ds.ExampleURLs, inf.Vault); userErr != nil || sysErr != nil {
+ if userErr, sysErr, statusCode := PutDNSSecKeys(tx, *ds.XMLID, cdnName, ds.ExampleURLs, inf.Vault, r.Context()); userErr != nil || sysErr != nil {
return nil, statusCode, userErr, sysErr
}
}
diff --git a/traffic_ops/traffic_ops_golang/deliveryservice/dnssec.go b/traffic_ops/traffic_ops_golang/deliveryservice/dnssec.go
index fbd9096..dfee419 100644
--- a/traffic_ops/traffic_ops_golang/deliveryservice/dnssec.go
+++ b/traffic_ops/traffic_ops_golang/deliveryservice/dnssec.go
@@ -20,6 +20,7 @@ package deliveryservice
*/
import (
+ "context"
"database/sql"
"encoding/base64"
"errors"
@@ -35,8 +36,8 @@ import (
"github.com/miekg/dns"
)
-func PutDNSSecKeys(tx *sql.Tx, xmlID string, cdnName string, exampleURLs []string, tv trafficvault.TrafficVault) (error, error, int) {
- keys, ok, err := tv.GetDNSSECKeys(cdnName, tx)
+func PutDNSSecKeys(tx *sql.Tx, xmlID string, cdnName string, exampleURLs []string, tv trafficvault.TrafficVault, ctx context.Context) (error, error, int) {
+ keys, ok, err := tv.GetDNSSECKeys(cdnName, tx, ctx)
if err != nil {
return nil, errors.New("getting DNSSec keys from Traffic Vault: " + err.Error()), http.StatusInternalServerError
} else if !ok {
@@ -55,7 +56,7 @@ func PutDNSSecKeys(tx *sql.Tx, xmlID string, cdnName string, exampleURLs []strin
return nil, errors.New("creating DNSSEC keys for delivery service '" + xmlID + "': " + err.Error()), http.StatusInternalServerError
}
keys[xmlID] = dsKeys
- if err := tv.PutDNSSECKeys(cdnName, keys, tx); err != nil {
+ if err := tv.PutDNSSECKeys(cdnName, keys, tx, ctx); err != nil {
return nil, errors.New("putting DNSSEC keys in Traffic Vault: " + err.Error()), http.StatusInternalServerError
}
return nil, nil, http.StatusOK
diff --git a/traffic_ops/traffic_ops_golang/deliveryservice/keys.go b/traffic_ops/traffic_ops_golang/deliveryservice/keys.go
index 2ba7c0a..a9d8a07 100644
--- a/traffic_ops/traffic_ops_golang/deliveryservice/keys.go
+++ b/traffic_ops/traffic_ops_golang/deliveryservice/keys.go
@@ -109,7 +109,7 @@ func AddSSLKeys(w http.ResponseWriter, r *http.Request) {
AuthType: authType,
}
- if err := inf.Vault.PutDeliveryServiceSSLKeys(dsSSLKeys, inf.Tx.Tx); err != nil {
+ if err := inf.Vault.PutDeliveryServiceSSLKeys(dsSSLKeys, inf.Tx.Tx, r.Context()); err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("putting SSL keys in Traffic Vault for delivery service '"+*req.DeliveryService+"': "+err.Error()))
return
}
@@ -242,7 +242,7 @@ func getSSLKeysByXMLIDHelper(xmlID string, tv trafficvault.TrafficVault, alerts
api.WriteAlerts(w, r, errCode, alerts)
return
}
- keyObjV15, ok, err := tv.GetDeliveryServiceSSLKeys(xmlID, version, inf.Tx.Tx)
+ keyObjV15, ok, err := tv.GetDeliveryServiceSSLKeys(xmlID, version, inf.Tx.Tx, r.Context())
if err != nil {
userErr := api.LogErr(r, http.StatusInternalServerError, nil, errors.New("getting ssl keys: "+err.Error()))
alerts.AddNewAlert(tc.ErrorLevel, userErr.Error())
@@ -295,7 +295,7 @@ func getSSLKeysByXMLIDHelperV15(xmlID string, alerts tc.Alerts, inf *api.APIInfo
api.WriteAlerts(w, r, errCode, alerts)
return
}
- keyObj, ok, err := inf.Vault.GetDeliveryServiceSSLKeys(xmlID, version, inf.Tx.Tx)
+ keyObj, ok, err := inf.Vault.GetDeliveryServiceSSLKeys(xmlID, version, inf.Tx.Tx, r.Context())
if err != nil {
userErr := api.LogErr(r, http.StatusInternalServerError, nil, errors.New("getting ssl keys: "+err.Error()))
alerts.AddNewAlert(tc.ErrorLevel, userErr.Error())
@@ -410,7 +410,7 @@ func deleteSSLKeys(w http.ResponseWriter, r *http.Request, deprecated bool) {
api.HandleErrOptionalDeprecation(w, r, inf.Tx.Tx, errCode, userErr, sysErr, deprecated, &alt)
return
}
- if err := inf.Vault.DeleteDeliveryServiceSSLKeys(xmlID, inf.Params["version"], inf.Tx.Tx); err != nil {
+ if err := inf.Vault.DeleteDeliveryServiceSSLKeys(xmlID, inf.Params["version"], inf.Tx.Tx, r.Context()); err != nil {
api.HandleErrOptionalDeprecation(w, r, inf.Tx.Tx, http.StatusInternalServerError, userErr, errors.New("deliveryservice.DeleteSSLKeys: deleting SSL keys: "+err.Error()), deprecated, &alt)
return
}
diff --git a/traffic_ops/traffic_ops_golang/deliveryservice/sslkeys.go b/traffic_ops/traffic_ops_golang/deliveryservice/sslkeys.go
index 14a0334..66a5183 100644
--- a/traffic_ops/traffic_ops_golang/deliveryservice/sslkeys.go
+++ b/traffic_ops/traffic_ops_golang/deliveryservice/sslkeys.go
@@ -20,6 +20,7 @@ package deliveryservice
*/
import (
+ "context"
"database/sql"
"errors"
"net/http"
@@ -64,7 +65,7 @@ func GenerateSSLKeys(w http.ResponseWriter, r *http.Request) {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusNotFound, errors.New("no DS with name "+*req.DeliveryService), nil)
return
}
- if err := generatePutRiakKeys(req, inf.Tx.Tx, inf.Vault); err != nil {
+ if err := generatePutRiakKeys(req, inf.Tx.Tx, inf.Vault, r.Context()); err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("generating and putting SSL keys: "+err.Error()))
return
}
@@ -78,7 +79,7 @@ func GenerateSSLKeys(w http.ResponseWriter, r *http.Request) {
// generatePutRiakKeys generates a certificate, csr, and key from the given request, and insert it into the Riak key database.
// The req MUST be validated, ensuring required fields exist.
-func generatePutRiakKeys(req tc.DeliveryServiceGenSSLKeysReq, tx *sql.Tx, tv trafficvault.TrafficVault) error {
+func generatePutRiakKeys(req tc.DeliveryServiceGenSSLKeysReq, tx *sql.Tx, tv trafficvault.TrafficVault, ctx context.Context) error {
dsSSLKeys := tc.DeliveryServiceSSLKeys{
CDN: *req.CDN,
DeliveryService: *req.DeliveryService,
@@ -99,7 +100,7 @@ func generatePutRiakKeys(req tc.DeliveryServiceGenSSLKeysReq, tx *sql.Tx, tv tra
dsSSLKeys.AuthType = tc.SelfSignedCertAuthType
- if err := tv.PutDeliveryServiceSSLKeys(dsSSLKeys, tx); err != nil {
+ if err := tv.PutDeliveryServiceSSLKeys(dsSSLKeys, tx, ctx); err != nil {
return errors.New("putting keys in Traffic Vault: " + err.Error())
}
return nil
diff --git a/traffic_ops/traffic_ops_golang/deliveryservice/urlkey.go b/traffic_ops/traffic_ops_golang/deliveryservice/urlkey.go
index 4d7f16a..974d05a 100644
--- a/traffic_ops/traffic_ops_golang/deliveryservice/urlkey.go
+++ b/traffic_ops/traffic_ops_golang/deliveryservice/urlkey.go
@@ -73,7 +73,7 @@ func GetURLKeysByID(w http.ResponseWriter, r *http.Request) {
return
}
- keys, ok, err := inf.Vault.GetURLSigKeys(string(ds), inf.Tx.Tx)
+ keys, ok, err := inf.Vault.GetURLSigKeys(string(ds), inf.Tx.Tx, r.Context())
if err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("getting URL Sig keys from Traffic Vault: "+err.Error()))
return
@@ -117,7 +117,7 @@ func GetURLKeysByName(w http.ResponseWriter, r *http.Request) {
return
}
- keys, ok, err := inf.Vault.GetURLSigKeys(string(ds), inf.Tx.Tx)
+ keys, ok, err := inf.Vault.GetURLSigKeys(string(ds), inf.Tx.Tx, r.Context())
if err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("getting URL Sig keys from Traffic Vault: "+err.Error()))
return
@@ -190,7 +190,7 @@ func CopyURLKeys(w http.ResponseWriter, r *http.Request) {
return
}
- keys, ok, err := inf.Vault.GetURLSigKeys(string(copyDS), inf.Tx.Tx)
+ keys, ok, err := inf.Vault.GetURLSigKeys(string(copyDS), inf.Tx.Tx, r.Context())
if err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("getting URL Sig keys from Traffic Vault: "+err.Error()))
return
@@ -200,7 +200,7 @@ func CopyURLKeys(w http.ResponseWriter, r *http.Request) {
return
}
- if err := inf.Vault.PutURLSigKeys(string(ds), keys, inf.Tx.Tx); err != nil {
+ if err := inf.Vault.PutURLSigKeys(string(ds), keys, inf.Tx.Tx, r.Context()); err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("setting URL Sig keys for '"+string(ds)+" copied from "+string(copyDS)+": "+err.Error()))
return
}
@@ -255,7 +255,7 @@ func GenerateURLKeys(w http.ResponseWriter, r *http.Request) {
return
}
- if err := inf.Vault.PutURLSigKeys(string(ds), keys, inf.Tx.Tx); err != nil {
+ if err := inf.Vault.PutURLSigKeys(string(ds), keys, inf.Tx.Tx, r.Context()); err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("setting URL Sig keys for '"+string(ds)+": "+err.Error()))
return
}
diff --git a/traffic_ops/traffic_ops_golang/ping/keys.go b/traffic_ops/traffic_ops_golang/ping/keys.go
index 2e3e2aa..d135834 100644
--- a/traffic_ops/traffic_ops_golang/ping/keys.go
+++ b/traffic_ops/traffic_ops_golang/ping/keys.go
@@ -36,7 +36,7 @@ func Keys(w http.ResponseWriter, r *http.Request) {
}
defer inf.Close()
- pingResp, err := inf.Vault.Ping(inf.Tx.Tx)
+ pingResp, err := inf.Vault.Ping(inf.Tx.Tx, r.Context())
if err != nil {
api.HandleDeprecatedErr(w, r, nil, http.StatusInternalServerError, err, nil, util.StrPtr(API_VAULT_PING))
return
diff --git a/traffic_ops/traffic_ops_golang/ping/riak.go b/traffic_ops/traffic_ops_golang/ping/riak.go
index 1d2b403..ae981ff 100644
--- a/traffic_ops/traffic_ops_golang/ping/riak.go
+++ b/traffic_ops/traffic_ops_golang/ping/riak.go
@@ -41,10 +41,10 @@ func Riak(w http.ResponseWriter, r *http.Request) {
defer inf.Close()
- pingResp, err := inf.Vault.Ping(inf.Tx.Tx)
+ pingResp, err := inf.Vault.Ping(inf.Tx.Tx, r.Context())
if err != nil {
- userErr = api.LogErr(r, http.StatusInternalServerError, nil, errors.New("error pinging Riak: "+err.Error()))
+ userErr = api.LogErr(r, http.StatusInternalServerError, nil, errors.New("error pinging Traffic Vault: "+err.Error()))
alerts.AddAlerts(tc.CreateErrorAlerts(userErr))
api.WriteAlerts(w, r, http.StatusInternalServerError, alerts)
return
diff --git a/traffic_ops/traffic_ops_golang/ping/vault.go b/traffic_ops/traffic_ops_golang/ping/vault.go
index f8f4bd5..381260d 100644
--- a/traffic_ops/traffic_ops_golang/ping/vault.go
+++ b/traffic_ops/traffic_ops_golang/ping/vault.go
@@ -34,9 +34,9 @@ func Vault(w http.ResponseWriter, r *http.Request) {
}
defer inf.Close()
- pingResp, err := inf.Vault.Ping(inf.Tx.Tx)
+ pingResp, err := inf.Vault.Ping(inf.Tx.Tx, r.Context())
if err != nil {
- api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("error pinging Riak: "+err.Error()))
+ api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("error pinging Traffic Vault: "+err.Error()))
return
}
api.WriteResp(w, r, pingResp)
diff --git a/traffic_ops/traffic_ops_golang/traffic_ops_golang.go b/traffic_ops/traffic_ops_golang/traffic_ops_golang.go
index f1701e9..25bfcf3 100644
--- a/traffic_ops/traffic_ops_golang/traffic_ops_golang.go
+++ b/traffic_ops/traffic_ops_golang/traffic_ops_golang.go
@@ -41,6 +41,7 @@ import (
"github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/plugin"
"github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/routing"
"github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/trafficvault"
+ _ "github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/trafficvault/backends" // init traffic vault backends
"github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/trafficvault/backends/disabled"
"github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/trafficvault/backends/riaksvc"
diff --git a/traffic_ops/traffic_ops_golang/trafficvault/README.md b/traffic_ops/traffic_ops_golang/trafficvault/README.md
index cbff240..bd46bbc 100644
--- a/traffic_ops/traffic_ops_golang/trafficvault/README.md
+++ b/traffic_ops/traffic_ops_golang/trafficvault/README.md
@@ -34,13 +34,13 @@ type Config struct {
```
4. Implement all the methods required by the `TrafficVault` interface on your new `Foo` struct. Initially, you may want to simply stub out the methods and implement them later:
```go
-func (f *Foo) GetDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx) (tc.DeliveryServiceSSLKeysV15, bool, error) {
+func (f *Foo) GetDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx, ctx context.Context) (tc.DeliveryServiceSSLKeysV15, bool, error) {
return tc.DeliveryServiceSSLKeysV15{}, false, nil
}
... (snip)
-func (f *Foo) Ping(tx *sql.Tx) (tc.TrafficVaultPingResponse, error) {
+func (f *Foo) Ping(tx *sql.Tx, ctx context.Context) (tc.TrafficVaultPingResponse, error) {
return tc.TrafficVaultPingResponse{}, nil
}
```
@@ -58,4 +58,11 @@ func init() {
trafficvault.AddBackend("foo", loadFoo)
}
```
-7. You are now able to test your new Traffic Vault `Foo` backend. First, in `cdn.conf`, you need to set `traffic_vault_backend` to `"foo"` and include your desired `Foo` configuration in `traffic_vault_config`. Once that is done, Traffic Vault is enabled, and you can use Traffic Ops API routes that require Traffic Vault. At this point, you should go back and fully implement the stubbed out `TrafficVault` interface methods on your `Foo` type.
+7. In `./backends/backends.go`, import your new package:
+```go
+import (
+ _ "github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/trafficvault/backends/foo"
+)
+```
+This is required for the package `init()` function to run and register the new backend.
+8. You are now able to test your new Traffic Vault `Foo` backend. First, in `cdn.conf`, you need to set `traffic_vault_backend` to `"foo"` and include your desired `Foo` configuration in `traffic_vault_config`. Once that is done, Traffic Vault is enabled, and you can use Traffic Ops API routes that require Traffic Vault. At this point, you should go back and fully implement the stubbed out `TrafficVault` interface methods on your `Foo` type.
diff --git a/traffic_ops/traffic_ops_golang/ping/vault.go b/traffic_ops/traffic_ops_golang/trafficvault/backends/backends.go
similarity index 58%
copy from traffic_ops/traffic_ops_golang/ping/vault.go
copy to traffic_ops/traffic_ops_golang/trafficvault/backends/backends.go
index f8f4bd5..b9ff26c 100644
--- a/traffic_ops/traffic_ops_golang/ping/vault.go
+++ b/traffic_ops/traffic_ops_golang/trafficvault/backends/backends.go
@@ -1,4 +1,5 @@
-package ping
+// Package backends is simply for importing the traffic vault backend packages so they can initialize.
+package backends
/*
* Licensed to the Apache Software Foundation (ASF) under one
@@ -20,24 +21,5 @@ package ping
*/
import (
- "errors"
- "net/http"
-
- "github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/api"
+ _ "github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/trafficvault/backends/postgres"
)
-
-func Vault(w http.ResponseWriter, r *http.Request) {
- inf, userErr, sysErr, errCode := api.NewInfo(r, nil, nil)
- if userErr != nil || sysErr != nil {
- api.HandleErr(w, r, inf.Tx.Tx, errCode, userErr, sysErr)
- return
- }
- defer inf.Close()
-
- pingResp, err := inf.Vault.Ping(inf.Tx.Tx)
- if err != nil {
- api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("error pinging Riak: "+err.Error()))
- return
- }
- api.WriteResp(w, r, pingResp)
-}
diff --git a/traffic_ops/traffic_ops_golang/trafficvault/backends/disabled/disabled.go b/traffic_ops/traffic_ops_golang/trafficvault/backends/disabled/disabled.go
index 4e615dc..81dda37 100644
--- a/traffic_ops/traffic_ops_golang/trafficvault/backends/disabled/disabled.go
+++ b/traffic_ops/traffic_ops_golang/trafficvault/backends/disabled/disabled.go
@@ -24,6 +24,7 @@ package disabled
*/
import (
+ "context"
"database/sql"
"github.com/apache/trafficcontrol/lib/go-tc"
@@ -40,59 +41,59 @@ const disabledErr = Error("traffic vault is disabled")
type Disabled struct {
}
-func (d *Disabled) GetDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx) (tc.DeliveryServiceSSLKeysV15, bool, error) {
+func (d *Disabled) GetDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx, ctx context.Context) (tc.DeliveryServiceSSLKeysV15, bool, error) {
return tc.DeliveryServiceSSLKeysV15{}, false, disabledErr
}
-func (d *Disabled) PutDeliveryServiceSSLKeys(key tc.DeliveryServiceSSLKeys, tx *sql.Tx) error {
+func (d *Disabled) PutDeliveryServiceSSLKeys(key tc.DeliveryServiceSSLKeys, tx *sql.Tx, ctx context.Context) error {
return disabledErr
}
-func (d *Disabled) DeleteDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx) error {
+func (d *Disabled) DeleteDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx, ctx context.Context) error {
return disabledErr
}
-func (d *Disabled) DeleteOldDeliveryServiceSSLKeys(existingXMLIDs map[string]struct{}, cdnName string, tx *sql.Tx) error {
+func (d *Disabled) DeleteOldDeliveryServiceSSLKeys(existingXMLIDs map[string]struct{}, cdnName string, tx *sql.Tx, ctx context.Context) error {
return disabledErr
}
-func (d *Disabled) GetCDNSSLKeys(cdnName string, tx *sql.Tx) ([]tc.CDNSSLKey, error) {
+func (d *Disabled) GetCDNSSLKeys(cdnName string, tx *sql.Tx, ctx context.Context) ([]tc.CDNSSLKey, error) {
return nil, disabledErr
}
-func (d *Disabled) GetDNSSECKeys(cdnName string, tx *sql.Tx) (tc.DNSSECKeysTrafficVault, bool, error) {
+func (d *Disabled) GetDNSSECKeys(cdnName string, tx *sql.Tx, ctx context.Context) (tc.DNSSECKeysTrafficVault, bool, error) {
return nil, false, disabledErr
}
-func (d *Disabled) PutDNSSECKeys(cdnName string, keys tc.DNSSECKeysTrafficVault, tx *sql.Tx) error {
+func (d *Disabled) PutDNSSECKeys(cdnName string, keys tc.DNSSECKeysTrafficVault, tx *sql.Tx, ctx context.Context) error {
return disabledErr
}
-func (d *Disabled) DeleteDNSSECKeys(cdnName string, tx *sql.Tx) error {
+func (d *Disabled) DeleteDNSSECKeys(cdnName string, tx *sql.Tx, ctx context.Context) error {
return disabledErr
}
-func (d *Disabled) GetURLSigKeys(xmlID string, tx *sql.Tx) (tc.URLSigKeys, bool, error) {
+func (d *Disabled) GetURLSigKeys(xmlID string, tx *sql.Tx, ctx context.Context) (tc.URLSigKeys, bool, error) {
return nil, false, disabledErr
}
-func (d *Disabled) PutURLSigKeys(xmlID string, keys tc.URLSigKeys, tx *sql.Tx) error {
+func (d *Disabled) PutURLSigKeys(xmlID string, keys tc.URLSigKeys, tx *sql.Tx, ctx context.Context) error {
return disabledErr
}
-func (d *Disabled) GetURISigningKeys(xmlID string, tx *sql.Tx) ([]byte, bool, error) {
+func (d *Disabled) GetURISigningKeys(xmlID string, tx *sql.Tx, ctx context.Context) ([]byte, bool, error) {
return nil, false, disabledErr
}
-func (d *Disabled) PutURISigningKeys(xmlID string, keysJson []byte, tx *sql.Tx) error {
+func (d *Disabled) PutURISigningKeys(xmlID string, keysJson []byte, tx *sql.Tx, ctx context.Context) error {
return disabledErr
}
-func (d *Disabled) DeleteURISigningKeys(xmlID string, tx *sql.Tx) error {
+func (d *Disabled) DeleteURISigningKeys(xmlID string, tx *sql.Tx, ctx context.Context) error {
return disabledErr
}
-func (d *Disabled) Ping(tx *sql.Tx) (tc.TrafficVaultPingResponse, error) {
- return tc.TrafficVaultPingResponse{}, disabledErr
+func (d *Disabled) Ping(tx *sql.Tx, ctx context.Context) (tc.TrafficVaultPing, error) {
+ return tc.TrafficVaultPing{}, disabledErr
}
func (d *Disabled) GetBucketKey(bucket string, key string, tx *sql.Tx) ([]byte, bool, error) {
diff --git a/traffic_ops/traffic_ops_golang/trafficvault/backends/postgres/postgres.go b/traffic_ops/traffic_ops_golang/trafficvault/backends/postgres/postgres.go
new file mode 100644
index 0000000..6dea910
--- /dev/null
+++ b/traffic_ops/traffic_ops_golang/trafficvault/backends/postgres/postgres.go
@@ -0,0 +1,236 @@
+// Package postgres provides a TrafficVault implementation which uses PostgreSQL as the backend.
+package postgres
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import (
+ "context"
+ "database/sql"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "strconv"
+ "time"
+
+ "github.com/apache/trafficcontrol/lib/go-log"
+ "github.com/apache/trafficcontrol/lib/go-tc"
+ "github.com/apache/trafficcontrol/lib/go-tc/tovalidate"
+ "github.com/apache/trafficcontrol/lib/go-util"
+ "github.com/apache/trafficcontrol/traffic_ops/traffic_ops_golang/trafficvault"
+
+ validation "github.com/go-ozzo/ozzo-validation"
+ "github.com/jmoiron/sqlx"
+)
+
+type Error string
+
+func (e Error) Error() string {
+ return string(e)
+}
+
+const (
+ notImplementedErr = Error("this Traffic Vault functionality is not implemented for the postgres backend")
+
+ postgresBackendName = "postgres"
+
+ defaultMaxIdleConnections = 10 // if this is higher than MaxDBConnections it will be automatically adjusted below it by the db/sql library
+ defaultConnMaxLifetimeSeconds = 60
+ defaultDBQueryTimeoutSecs = 30
+)
+
+type Config struct {
+ DBName string `json:"dbname"`
+ Hostname string `json:"hostname"`
+ User string `json:"user"`
+ Password string `json:"password"`
+ Port int `json:"port"`
+ SSL bool `json:"ssl"`
+ MaxConnections int `json:"max_connections"`
+ MaxIdleConnections int `json:"max_idle_connections"`
+ ConnMaxLifetimeSeconds int `json:"conn_max_lifetime_seconds"`
+ QueryTimeoutSeconds int `json:"query_timeout_seconds"`
+}
+
+type Postgres struct {
+ cfg Config
+ db *sqlx.DB
+}
+
+func checkErrWithContext(prefix string, err error, ctxErr error) error {
+ e := prefix + err.Error()
+ if ctxErr != nil {
+ e = fmt.Sprintf("%s: %s: %s", prefix, ctxErr.Error(), err.Error())
+ }
+ return errors.New(e)
+}
+
+func (p *Postgres) beginTransaction(ctx context.Context) (*sqlx.Tx, context.Context, context.CancelFunc, error) {
+ dbCtx, cancelFunc := context.WithTimeout(ctx, time.Duration(p.cfg.QueryTimeoutSeconds)*time.Second)
+ tx, err := p.db.BeginTxx(dbCtx, nil)
+ if err != nil {
+ e := checkErrWithContext("could not begin Traffic Vault PostgreSQL transaction", err, ctx.Err())
+ cancelFunc()
+ return nil, nil, nil, e
+ }
+ return tx, dbCtx, cancelFunc, nil
+}
+
+func (p *Postgres) commitTransaction(tx *sqlx.Tx, ctx context.Context, cancelFunc context.CancelFunc) {
+ if err := tx.Commit(); err != nil {
+ e := checkErrWithContext("Traffic Vault PostgreSQL: committing transaction", err, ctx.Err())
+ log.Errorln(e)
+ }
+ cancelFunc()
+}
+
+func (p *Postgres) GetDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx, ctx context.Context) (tc.DeliveryServiceSSLKeysV15, bool, error) {
+ return tc.DeliveryServiceSSLKeysV15{}, false, notImplementedErr
+}
+
+func (p *Postgres) PutDeliveryServiceSSLKeys(key tc.DeliveryServiceSSLKeys, tx *sql.Tx, ctx context.Context) error {
+ return notImplementedErr
+}
+
+func (p *Postgres) DeleteDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx, ctx context.Context) error {
+ return notImplementedErr
+}
+
+func (p *Postgres) DeleteOldDeliveryServiceSSLKeys(existingXMLIDs map[string]struct{}, cdnName string, tx *sql.Tx, ctx context.Context) error {
+ return notImplementedErr
+}
+
+func (p *Postgres) GetCDNSSLKeys(cdnName string, tx *sql.Tx, ctx context.Context) ([]tc.CDNSSLKey, error) {
+ return nil, notImplementedErr
+}
+
+func (p *Postgres) GetDNSSECKeys(cdnName string, tx *sql.Tx, ctx context.Context) (tc.DNSSECKeysTrafficVault, bool, error) {
+ return tc.DNSSECKeysTrafficVault{}, false, notImplementedErr
+}
+
+func (p *Postgres) PutDNSSECKeys(cdnName string, keys tc.DNSSECKeysTrafficVault, tx *sql.Tx, ctx context.Context) error {
+ return notImplementedErr
+}
+
+func (p *Postgres) DeleteDNSSECKeys(cdnName string, tx *sql.Tx, ctx context.Context) error {
+ return notImplementedErr
+}
+
+func (p *Postgres) GetURLSigKeys(xmlID string, tx *sql.Tx, ctx context.Context) (tc.URLSigKeys, bool, error) {
+ return tc.URLSigKeys{}, false, notImplementedErr
+}
+
+func (p *Postgres) PutURLSigKeys(xmlID string, keys tc.URLSigKeys, tx *sql.Tx, ctx context.Context) error {
+ return notImplementedErr
+}
+
+func (p *Postgres) GetURISigningKeys(xmlID string, tx *sql.Tx, ctx context.Context) ([]byte, bool, error) {
+ return nil, false, notImplementedErr
+}
+
+func (p *Postgres) PutURISigningKeys(xmlID string, keysJson []byte, tx *sql.Tx, ctx context.Context) error {
+ return notImplementedErr
+}
+
+func (p *Postgres) DeleteURISigningKeys(xmlID string, tx *sql.Tx, ctx context.Context) error {
+ return notImplementedErr
+}
+
+func (p *Postgres) Ping(tx *sql.Tx, ctx context.Context) (tc.TrafficVaultPing, error) {
+ tvTx, dbCtx, cancelFunc, err := p.beginTransaction(ctx)
+ if err != nil {
+ return tc.TrafficVaultPing{}, err
+ }
+ defer p.commitTransaction(tvTx, dbCtx, cancelFunc)
+ n := 0
+ if err := tvTx.QueryRow("SELECT 1").Scan(&n); err != nil {
+ e := checkErrWithContext("Traffic Vault PostgreSQL: executing ping query", err, dbCtx.Err())
+ return tc.TrafficVaultPing{}, e
+ }
+ if n != 1 {
+ return tc.TrafficVaultPing{}, fmt.Errorf("Traffic Vault PostgreSQL: executing ping query: expected scanned value 1 but got %d instead", n)
+ }
+ return tc.TrafficVaultPing{Status: "OK", Server: p.cfg.Hostname + ":" + strconv.Itoa(p.cfg.Port)}, nil
+}
+
+func (p *Postgres) GetBucketKey(bucket string, key string, tx *sql.Tx) ([]byte, bool, error) {
+ return nil, false, notImplementedErr
+}
+
+func init() {
+ trafficvault.AddBackend(postgresBackendName, postgresLoad)
+}
+
+func postgresLoad(b json.RawMessage) (trafficvault.TrafficVault, error) {
+ pgCfg := Config{}
+ if err := json.Unmarshal(b, &pgCfg); err != nil {
+ return nil, errors.New("unmarshalling Postgres config: " + err.Error())
+ }
+ if err := validateConfig(pgCfg); err != nil {
+ return nil, errors.New("validating Postgres config: " + err.Error())
+ }
+ if pgCfg.MaxIdleConnections == 0 {
+ pgCfg.MaxIdleConnections = defaultMaxIdleConnections
+ }
+ if pgCfg.ConnMaxLifetimeSeconds == 0 {
+ pgCfg.ConnMaxLifetimeSeconds = defaultConnMaxLifetimeSeconds
+ }
+ if pgCfg.QueryTimeoutSeconds == 0 {
+ pgCfg.QueryTimeoutSeconds = defaultDBQueryTimeoutSecs
+ }
+
+ sslStr := "require"
+ if !pgCfg.SSL {
+ sslStr = "disable"
+ }
+ db, err := sqlx.Open("postgres", fmt.Sprintf("postgres://%s:%s@%s:%d/%s?sslmode=%s&fallback_application_name=trafficvault", pgCfg.User, pgCfg.Password, pgCfg.Hostname, pgCfg.Port, pgCfg.DBName, sslStr))
+ if err != nil {
+ return nil, errors.New("opening database: " + err.Error())
+ }
+ db.SetMaxOpenConns(pgCfg.MaxConnections)
+ db.SetMaxIdleConns(pgCfg.MaxIdleConnections)
+ db.SetConnMaxLifetime(time.Duration(pgCfg.ConnMaxLifetimeSeconds) * time.Second)
+
+ ctx, cancel := context.WithTimeout(context.Background(), time.Duration(pgCfg.QueryTimeoutSeconds)*time.Second)
+ defer cancel()
+ if err := db.PingContext(ctx); err != nil {
+ // NOTE: not fatal since Traffic Vault not being available at startup shouldn't be fatal
+ log.Errorln("pinging the Traffic Vault database: " + err.Error())
+ } else {
+ log.Infoln("successfully pinged the Traffic Vault database")
+ }
+
+ return &Postgres{cfg: pgCfg, db: db}, nil
+}
+
+func validateConfig(cfg Config) error {
+ errs := tovalidate.ToErrors(validation.Errors{
+ "user": validation.Validate(cfg.User, validation.Required),
+ "password": validation.Validate(cfg.Password, validation.Required),
+ "hostname": validation.Validate(cfg.Hostname, validation.Required),
+ "dbname": validation.Validate(cfg.DBName, validation.Required),
+ "port": validation.Validate(cfg.Port, validation.By(tovalidate.IsValidPortNumber)),
+ "max_connections": validation.Validate(cfg.MaxConnections, validation.Min(0)),
+ "query_timeout_seconds": validation.Validate(cfg.QueryTimeoutSeconds, validation.Min(0)),
+ })
+ if len(errs) == 0 {
+ return nil
+ }
+ return util.JoinErrs(errs)
+}
diff --git a/traffic_ops/traffic_ops_golang/trafficvault/backends/riaksvc/riak.go b/traffic_ops/traffic_ops_golang/trafficvault/backends/riaksvc/riak.go
index fd0dc85..06069b4 100644
--- a/traffic_ops/traffic_ops_golang/trafficvault/backends/riaksvc/riak.go
+++ b/traffic_ops/traffic_ops_golang/trafficvault/backends/riaksvc/riak.go
@@ -21,6 +21,7 @@ package riaksvc
*/
import (
+ "context"
"database/sql"
"encoding/json"
"errors"
@@ -39,62 +40,62 @@ type Riak struct {
cfg Config
}
-func (r *Riak) GetDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx) (tc.DeliveryServiceSSLKeysV15, bool, error) {
+func (r *Riak) GetDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx, ctx context.Context) (tc.DeliveryServiceSSLKeysV15, bool, error) {
return getDeliveryServiceSSLKeysObjV15(xmlID, version, tx, &r.cfg.AuthOptions, &r.cfg.Port)
}
-func (r *Riak) PutDeliveryServiceSSLKeys(key tc.DeliveryServiceSSLKeys, tx *sql.Tx) error {
+func (r *Riak) PutDeliveryServiceSSLKeys(key tc.DeliveryServiceSSLKeys, tx *sql.Tx, ctx context.Context) error {
return putDeliveryServiceSSLKeysObj(key, tx, &r.cfg.AuthOptions, &r.cfg.Port)
}
-func (r *Riak) DeleteDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx) error {
+func (r *Riak) DeleteDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx, ctx context.Context) error {
return deleteDSSSLKeys(tx, &r.cfg.AuthOptions, &r.cfg.Port, xmlID, version)
}
-func (r *Riak) DeleteOldDeliveryServiceSSLKeys(existingXMLIDs map[string]struct{}, cdnName string, tx *sql.Tx) error {
+func (r *Riak) DeleteOldDeliveryServiceSSLKeys(existingXMLIDs map[string]struct{}, cdnName string, tx *sql.Tx, ctx context.Context) error {
return deleteOldDeliveryServiceSSLKeys(tx, &r.cfg.AuthOptions, &r.cfg.Port, tc.CDNName(cdnName), existingXMLIDs)
}
-func (r *Riak) GetCDNSSLKeys(cdnName string, tx *sql.Tx) ([]tc.CDNSSLKey, error) {
+func (r *Riak) GetCDNSSLKeys(cdnName string, tx *sql.Tx, ctx context.Context) ([]tc.CDNSSLKey, error) {
return getCDNSSLKeysObj(tx, &r.cfg.AuthOptions, &r.cfg.Port, cdnName)
}
-func (r *Riak) GetDNSSECKeys(cdnName string, tx *sql.Tx) (tc.DNSSECKeysTrafficVault, bool, error) {
+func (r *Riak) GetDNSSECKeys(cdnName string, tx *sql.Tx, ctx context.Context) (tc.DNSSECKeysTrafficVault, bool, error) {
keys, exists, err := getDNSSECKeys(cdnName, tx, &r.cfg.AuthOptions, &r.cfg.Port)
return tc.DNSSECKeysTrafficVault(keys), exists, err
}
-func (r *Riak) PutDNSSECKeys(cdnName string, keys tc.DNSSECKeysTrafficVault, tx *sql.Tx) error {
+func (r *Riak) PutDNSSECKeys(cdnName string, keys tc.DNSSECKeysTrafficVault, tx *sql.Tx, ctx context.Context) error {
return putDNSSECKeys(tc.DNSSECKeysRiak(keys), cdnName, tx, &r.cfg.AuthOptions, &r.cfg.Port)
}
-func (r *Riak) DeleteDNSSECKeys(cdnName string, tx *sql.Tx) error {
+func (r *Riak) DeleteDNSSECKeys(cdnName string, tx *sql.Tx, ctx context.Context) error {
return deleteDNSSECKeys(cdnName, tx, &r.cfg.AuthOptions, &r.cfg.Port)
}
-func (r *Riak) GetURLSigKeys(xmlID string, tx *sql.Tx) (tc.URLSigKeys, bool, error) {
+func (r *Riak) GetURLSigKeys(xmlID string, tx *sql.Tx, ctx context.Context) (tc.URLSigKeys, bool, error) {
return getURLSigKeys(tx, &r.cfg.AuthOptions, &r.cfg.Port, tc.DeliveryServiceName(xmlID))
}
-func (r *Riak) PutURLSigKeys(xmlID string, keys tc.URLSigKeys, tx *sql.Tx) error {
+func (r *Riak) PutURLSigKeys(xmlID string, keys tc.URLSigKeys, tx *sql.Tx, ctx context.Context) error {
return putURLSigKeys(tx, &r.cfg.AuthOptions, &r.cfg.Port, tc.DeliveryServiceName(xmlID), keys)
}
-func (r *Riak) GetURISigningKeys(xmlID string, tx *sql.Tx) ([]byte, bool, error) {
+func (r *Riak) GetURISigningKeys(xmlID string, tx *sql.Tx, ctx context.Context) ([]byte, bool, error) {
return getURISigningKeys(tx, &r.cfg.AuthOptions, &r.cfg.Port, xmlID)
}
-func (r *Riak) PutURISigningKeys(xmlID string, keysJson []byte, tx *sql.Tx) error {
+func (r *Riak) PutURISigningKeys(xmlID string, keysJson []byte, tx *sql.Tx, ctx context.Context) error {
return putURISigningKeys(tx, &r.cfg.AuthOptions, &r.cfg.Port, xmlID, keysJson)
}
-func (r *Riak) DeleteURISigningKeys(xmlID string, tx *sql.Tx) error {
+func (r *Riak) DeleteURISigningKeys(xmlID string, tx *sql.Tx, ctx context.Context) error {
return deleteURISigningKeys(tx, &r.cfg.AuthOptions, &r.cfg.Port, xmlID)
}
-func (r *Riak) Ping(tx *sql.Tx) (tc.TrafficVaultPingResponse, error) {
+func (r *Riak) Ping(tx *sql.Tx, ctx context.Context) (tc.TrafficVaultPing, error) {
resp, err := ping(tx, &r.cfg.AuthOptions, &r.cfg.Port)
- return tc.TrafficVaultPingResponse(resp), err
+ return tc.TrafficVaultPing(resp), err
}
func (r *Riak) GetBucketKey(bucket string, key string, tx *sql.Tx) ([]byte, bool, error) {
diff --git a/traffic_ops/traffic_ops_golang/trafficvault/trafficvault.go b/traffic_ops/traffic_ops_golang/trafficvault/trafficvault.go
index e788587..fb63067 100644
--- a/traffic_ops/traffic_ops_golang/trafficvault/trafficvault.go
+++ b/traffic_ops/traffic_ops_golang/trafficvault/trafficvault.go
@@ -22,6 +22,7 @@ package trafficvault
*/
import (
+ "context"
"database/sql"
"encoding/json"
"fmt"
@@ -32,51 +33,57 @@ import (
// TrafficVault defines the methods necessary for a struct to implement in order to
// provide all the necessary functionality required of a Traffic Vault backend.
type TrafficVault interface {
+ // NOTE: the ctx context.Context in these methods is for the HTTP request context in order to cancel the request
+ // if the HTTP connection is closed. If the method is called asynchronously in a goroutine that is spawned while
+ // handling the original HTTP request, you should use context.Background() so that the context isn't cancelled
+ // when the original HTTP connection is closed.
+
// GetDeliveryServiceSSLKeys retrieves the SSL keys of the given version for
// the delivery service identified by the given xmlID. If version is empty,
// the implementation should return the latest version.
- GetDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx) (tc.DeliveryServiceSSLKeysV15, bool, error)
+ GetDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx, ctx context.Context) (tc.DeliveryServiceSSLKeysV15, bool, error)
// PutDeliveryServiceSSLKeys stores the given SSL keys for a delivery service.
- PutDeliveryServiceSSLKeys(key tc.DeliveryServiceSSLKeys, tx *sql.Tx) error
+ PutDeliveryServiceSSLKeys(key tc.DeliveryServiceSSLKeys, tx *sql.Tx, ctx context.Context) error
// DeleteDeliveryServiceSSLKeys removes the SSL keys of the given version (or latest
// if version is empty) for the delivery service identified by the given xmlID.
- DeleteDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx) error
+ DeleteDeliveryServiceSSLKeys(xmlID string, version string, tx *sql.Tx, ctx context.Context) error
// DeleteOldDeliveryServiceSSLKeys takes a set of existingXMLIDs as input and will remove
// all SSL keys for delivery services in the CDN identified by the given cdnName that
// do not contain an xmlID in the given set of existingXMLIDs. This method is called
// during a snapshot operation in order to delete SSL keys for delivery services that
// no longer exist.
- DeleteOldDeliveryServiceSSLKeys(existingXMLIDs map[string]struct{}, cdnName string, tx *sql.Tx) error
+ DeleteOldDeliveryServiceSSLKeys(existingXMLIDs map[string]struct{}, cdnName string, tx *sql.Tx, ctx context.Context) error
// GetCDNSSLKeys retrieves all the SSL keys for delivery services in the CDN identified
// by the given cdnName.
- GetCDNSSLKeys(cdnName string, tx *sql.Tx) ([]tc.CDNSSLKey, error)
+ GetCDNSSLKeys(cdnName string, tx *sql.Tx, ctx context.Context) ([]tc.CDNSSLKey, error)
// GetDNSSECKeys retrieves all the DNSSEC keys associated with the CDN identified by the
// given cdnName.
- GetDNSSECKeys(cdnName string, tx *sql.Tx) (tc.DNSSECKeysTrafficVault, bool, error)
+ GetDNSSECKeys(cdnName string, tx *sql.Tx, ctx context.Context) (tc.DNSSECKeysTrafficVault, bool, error)
// PutDNSSECKeys stores all the DNSSEC keys for the CDN identified by the given cdnName.
- PutDNSSECKeys(cdnName string, keys tc.DNSSECKeysTrafficVault, tx *sql.Tx) error
+ PutDNSSECKeys(cdnName string, keys tc.DNSSECKeysTrafficVault, tx *sql.Tx, ctx context.Context) error
// DeleteDNSSECKeys removes all the DNSSEC keys for the CDN identified by the given cdnName.
- DeleteDNSSECKeys(cdnName string, tx *sql.Tx) error
+ DeleteDNSSECKeys(cdnName string, tx *sql.Tx, ctx context.Context) error
// GetURLSigKeys retrieves the URL sig keys for the delivery service identified by the
// given xmlID.
- GetURLSigKeys(xmlID string, tx *sql.Tx) (tc.URLSigKeys, bool, error)
+ GetURLSigKeys(xmlID string, tx *sql.Tx, ctx context.Context) (tc.URLSigKeys, bool, error)
// PutURLSigKeys stores the given URL sig keys for the delivery service identified by
// the given xmlID.
- PutURLSigKeys(xmlID string, keys tc.URLSigKeys, tx *sql.Tx) error
+ PutURLSigKeys(xmlID string, keys tc.URLSigKeys, tx *sql.Tx, ctx context.Context) error
// GetURISigningKeys retrieves the URI signing keys (as raw JSON bytes) for the delivery
// service identified by the given xmlID.
- GetURISigningKeys(xmlID string, tx *sql.Tx) ([]byte, bool, error)
+ GetURISigningKeys(xmlID string, tx *sql.Tx, ctx context.Context) ([]byte, bool, error)
// PutURISigningKeys stores the given URI signing keys (as raw JSON bytes) for the delivery
// service identified by the given xmlID.
- PutURISigningKeys(xmlID string, keysJson []byte, tx *sql.Tx) error
+ PutURISigningKeys(xmlID string, keysJson []byte, tx *sql.Tx, ctx context.Context) error
// DeleteURISigningKeys removes the URI signing keys for the delivery service identified by
// the given xmlID.
- DeleteURISigningKeys(xmlID string, tx *sql.Tx) error
+ DeleteURISigningKeys(xmlID string, tx *sql.Tx, ctx context.Context) error
// Ping simply checks the health of the Traffic Vault backend, returning a status and which
// server hostname the status was returned by.
- Ping(tx *sql.Tx) (tc.TrafficVaultPingResponse, error)
+ Ping(tx *sql.Tx, ctx context.Context) (tc.TrafficVaultPing, error)
// GetBucketKey returns the raw bytes identified by the given bucket and key. This may not
// apply to every Traffic Vault backend implementation.
+ // Deprecated: this method and associated API routes will be removed in the future.
GetBucketKey(bucket string, key string, tx *sql.Tx) ([]byte, bool, error)
}
@@ -102,7 +109,7 @@ func AddBackend(name string, loadConfig LoadFunc) {
func GetBackend(name string, cfgJson json.RawMessage) (TrafficVault, error) {
loader, ok := backends[name]
if !ok {
- return nil, fmt.Errorf("no support Traffic Vault backend named '%s' was found", name)
+ return nil, fmt.Errorf("no supported Traffic Vault backend named '%s' was found", name)
}
backend, err := loader(cfgJson)
if err != nil {
diff --git a/traffic_ops/traffic_ops_golang/urisigning/urisigning.go b/traffic_ops/traffic_ops_golang/urisigning/urisigning.go
index 5341d6f..71fafcb 100644
--- a/traffic_ops/traffic_ops_golang/urisigning/urisigning.go
+++ b/traffic_ops/traffic_ops_golang/urisigning/urisigning.go
@@ -55,7 +55,7 @@ func GetURIsignkeysHandler(w http.ResponseWriter, r *http.Request) {
api.HandleErr(w, r, inf.Tx.Tx, errCode, userErr, sysErr)
return
}
- ro, _, err := inf.Vault.GetURISigningKeys(xmlID, inf.Tx.Tx)
+ ro, _, err := inf.Vault.GetURISigningKeys(xmlID, inf.Tx.Tx, r.Context())
if err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("getting URI signing keys: "+err.Error()))
return
@@ -92,7 +92,7 @@ func RemoveDeliveryServiceURIKeysHandler(w http.ResponseWriter, r *http.Request)
return
}
- _, found, err := inf.Vault.GetURISigningKeys(xmlID, inf.Tx.Tx)
+ _, found, err := inf.Vault.GetURISigningKeys(xmlID, inf.Tx.Tx, r.Context())
if err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("removing URI signing keys: "+err.Error()))
return
@@ -102,7 +102,7 @@ func RemoveDeliveryServiceURIKeysHandler(w http.ResponseWriter, r *http.Request)
api.WriteRespAlert(w, r, tc.InfoLevel, "not deleted, no object found to delete")
return
}
- if err := inf.Vault.DeleteURISigningKeys(xmlID, inf.Tx.Tx); err != nil {
+ if err := inf.Vault.DeleteURISigningKeys(xmlID, inf.Tx.Tx, r.Context()); err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("removing URI signing keys: "+err.Error()))
return
}
@@ -154,7 +154,7 @@ func SaveDeliveryServiceURIKeysHandler(w http.ResponseWriter, r *http.Request) {
return
}
- if err := inf.Vault.PutURISigningKeys(xmlID, data, inf.Tx.Tx); err != nil {
+ if err := inf.Vault.PutURISigningKeys(xmlID, data, inf.Tx.Tx, r.Context()); err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("saving URI signing keys: "+err.Error()))
return
}
diff --git a/traffic_ops/v4-client/vault.go b/traffic_ops/v4-client/vault.go
new file mode 100644
index 0000000..23b57d0
--- /dev/null
+++ b/traffic_ops/v4-client/vault.go
@@ -0,0 +1,32 @@
+package client
+
+/*
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+import (
+ "github.com/apache/trafficcontrol/lib/go-tc"
+ "github.com/apache/trafficcontrol/traffic_ops/toclientlib"
+)
+
+const (
+ // APIVaultPing is the partial path (excluding the /api/<version> prefix) to the /vault/ping API endpoint.
+ APIVaultPing = "/vault/ping"
+)
+
+// TrafficVaultPing returns a response indicating whether or not Traffic Vault is responsive.
+func (to *Session) TrafficVaultPing() (tc.TrafficVaultPingResponse, toclientlib.ReqInf, error) {
+ var data tc.TrafficVaultPingResponse
+ reqInf, err := to.get(APIVaultPing, nil, &data)
+ return data, reqInf, err
+}