You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficcontrol.apache.org by GitBox <gi...@apache.org> on 2021/05/12 21:48:53 UTC
[GitHub] [trafficcontrol] mattjackson220 opened a new pull request #5843: Updated TV Postgres to use AES encryption for URL Sig Keys
mattjackson220 opened a new pull request #5843:
URL: https://github.com/apache/trafficcontrol/pull/5843
<!--
************ STOP!! ************
If this Pull Request is intended to fix a security vulnerability, DO NOT submit it! Instead, contact
the Apache Software Foundation Security Team at security@trafficcontrol.apache.org and follow the
guidelines at https://www.apache.org/security/ regarding vulnerability disclosure.
-->
## What does this PR (Pull Request) do?
<!-- Explain the changes you made here. If this fixes an Issue, identify it by
replacing the text in the checkbox item with the Issue number e.g.
- [x] This PR fixes #9001 OR is not related to any Issue
^ This will automatically close Issue number 9001 when the Pull Request is
merged (The '#' is important).
Be sure you check the box properly, see the "The following criteria are ALL
met by this PR" section for details.
-->
- [x] This PR is not related to any Issue <!-- You can check for an issue here: https://github.com/apache/trafficcontrol/issues -->
## Which Traffic Control components are affected by this PR?
<!-- Please delete all components from this list that are NOT affected by this
Pull Request. Also, feel free to add the name of a tool or script that is
affected but not on the list.
Additionally, if this Pull Request does NOT affect documentation, please
explain why documentation is not required. -->
- Documentation
- Traffic Ops
- Traffic Vault
## What is the best way to verify this PR?
<!-- Please include here ALL the steps necessary to test your Pull Request. If
it includes tests (and most should), outline here the steps needed to run the
tests. If not, lay out the manual testing procedure and please explain why
tests are unnecessary for this Pull Request. -->
Create a key locally and put the location into the cdn.conf field
Run TO with TV setup to use Postgres
Verify that GET, POST, PUT, DELETE all work as expected
Verify that the data in Postgres is encrypted successfully
## If this is a bug fix, what versions of Traffic Control are affected?
<!-- If this PR fixes a bug, please list here all of the affected versions - to
the best of your knowledge. It's also pretty helpful to include a commit hash
of where 'master' is at the time this PR is opened (if it affects master),
because what 'master' means will change over time. For example, if this PR
fixes a bug that's present in master (at commit hash '1df853c8'), in v4.0.0,
and in the current 4.0.1 Release candidate (e.g. RC1), then this list would
look like:
- master (1df853c8)
- 4.0.0
- 4.0.1 (RC1)
If you don't know what other versions might have this bug, AND don't know how
to find the commit hash of 'master', then feel free to leave this section
blank (or, preferably, delete it entirely).
-->
## The following criteria are ALL met by this PR
<!-- Check the boxes to signify that the associated statement is true. To
"check a box", replace the space inside of the square brackets with an 'x'.
e.g.
- [ x] <- Wrong
- [x ] <- Wrong
- [] <- Wrong
- [*] <- Wrong
- [x] <- Correct!
-->
No test updates, changelog, or license header updates needed since this is a backend change and is covered by a previous changelog entry
- [x] This PR includes tests OR I have explained why tests are unnecessary
- [x] This PR includes documentation OR I have explained why documentation is unnecessary
- [x] This PR includes an update to CHANGELOG.md OR such an update is not necessary
- [x] This PR includes any and all required license headers
- [x] This PR **DOES NOT FIX A SERIOUS SECURITY VULNERABILITY** (see [the Apache Software Foundation's security guidelines](https://www.apache.org/security/) for details)
## Additional Information
<!-- If you would like to include any additional information on the PR for
potential reviewers please put it here.
Some examples of this would be:
- Before and after screenshots/gifs of the Traffic Portal if it is affected
- Links to other dependent Pull Requests
- References to relevant context (e.g. new/updates to dependent libraries,
mailing list records, blueprints)
Feel free to leave this section blank (or, preferably, delete it entirely).
-->
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] mattjackson220 commented on a change in pull request #5843: Updated TV Postgres to use AES encryption for URL Sig Keys
Posted by GitBox <gi...@apache.org>.
mattjackson220 commented on a change in pull request #5843:
URL: https://github.com/apache/trafficcontrol/pull/5843#discussion_r631959019
##########
File path: traffic_ops/traffic_ops_golang/trafficvault/backends/postgres/encrypt.go
##########
@@ -0,0 +1,92 @@
+package postgres
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import (
+ "crypto/aes"
+ "crypto/cipher"
+ "crypto/rand"
+ "errors"
+ "io"
+ "io/ioutil"
+)
+
+func encryptString(stringToEncrypt string, encryptionKeyLocation string) (string, error) {
+ key, err := readKeyFromFile(encryptionKeyLocation)
+ if err != nil {
+ return "", err
+ }
+
+ cipherBlock, err := aes.NewCipher(key)
+ if err != nil {
+ return "", err
+ }
+
+ gcm, err := cipher.NewGCM(cipherBlock)
+ if err != nil {
+ return "", err
+ }
+
+ nonce := make([]byte, gcm.NonceSize())
+ if _, err = io.ReadFull(rand.Reader, nonce); err != nil {
+ return "", err
+ }
+
+ return string(gcm.Seal(nonce, nonce, []byte(stringToEncrypt), nil)), nil
Review comment:
yea just to prepend it there since we dont want to save it elsewhere
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp merged pull request #5843: Updated TV Postgres to use AES encryption for URL Sig Keys
Posted by GitBox <gi...@apache.org>.
rawlinp merged pull request #5843:
URL: https://github.com/apache/trafficcontrol/pull/5843
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #5843: Updated TV Postgres to use AES encryption for URL Sig Keys
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #5843:
URL: https://github.com/apache/trafficcontrol/pull/5843#discussion_r631439898
##########
File path: docs/source/admin/traffic_vault.rst
##########
@@ -33,6 +33,7 @@ In order to use the PostgreSQL backend for Traffic Vault, you will need to set t
:password: The password to use when connecting to the database
:port: The port number that the database listens for new connections on (NOTE: the PostgreSQL default is 5432)
:user: The username to use when connecting to the database
+:encryption_key_location: The location on-disk for the key used to encrypt secrets before they are stored.
Review comment:
instead of calling this `encryption_key_location`, which might be confused w/ a public key, we should call it `aes_key_location` and note that it must be an AES key which provides both encryption and decryption
##########
File path: traffic_ops/traffic_ops_golang/trafficvault/backends/postgres/encrypt.go
##########
@@ -0,0 +1,92 @@
+package postgres
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import (
+ "crypto/aes"
+ "crypto/cipher"
+ "crypto/rand"
+ "errors"
+ "io"
+ "io/ioutil"
+)
+
+func encryptString(stringToEncrypt string, encryptionKeyLocation string) (string, error) {
+ key, err := readKeyFromFile(encryptionKeyLocation)
Review comment:
Thinking about fetching this key file from HashiCorp Vault, we probably wouldn't want to fetch it every time we needed to encrypt/decrypt. I think the same would apply to on-disk storage. We probably just want to read it into memory during startup (in `postgresLoad()`) and save it as a private field in the `Postgres` struct in order to reuse it.
##########
File path: traffic_ops/traffic_ops_golang/trafficvault/backends/postgres/url_sig_keys.go
##########
@@ -30,26 +30,31 @@ import (
"github.com/jmoiron/sqlx"
)
-func getURLSigKeys(xmlID string, tvTx *sqlx.Tx, ctx context.Context) (tc.URLSigKeys, bool, error) {
- var jsonUrlKeys string
- if err := tvTx.QueryRow("SELECT data FROM url_sig_key WHERE deliveryservice = $1", xmlID).Scan(&jsonUrlKeys); err != nil {
+func getURLSigKeys(xmlID string, tvTx *sqlx.Tx, ctx context.Context, encryptionKeyLocation string) (tc.URLSigKeys, bool, error) {
+ var encryptedUrlSigKey string
Review comment:
Would it make sense to have this be `[]byte` instead of a `string`? Since the encryption/decryption works w/ `[]byte` and the DB type is `bytea`?
##########
File path: traffic_ops/traffic_ops_golang/trafficvault/backends/postgres/encrypt.go
##########
@@ -0,0 +1,92 @@
+package postgres
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import (
+ "crypto/aes"
+ "crypto/cipher"
+ "crypto/rand"
+ "errors"
+ "io"
+ "io/ioutil"
+)
+
+func encryptString(stringToEncrypt string, encryptionKeyLocation string) (string, error) {
+ key, err := readKeyFromFile(encryptionKeyLocation)
+ if err != nil {
+ return "", err
+ }
+
+ cipherBlock, err := aes.NewCipher(key)
+ if err != nil {
+ return "", err
+ }
+
+ gcm, err := cipher.NewGCM(cipherBlock)
+ if err != nil {
+ return "", err
+ }
+
+ nonce := make([]byte, gcm.NonceSize())
+ if _, err = io.ReadFull(rand.Reader, nonce); err != nil {
+ return "", err
+ }
+
+ return string(gcm.Seal(nonce, nonce, []byte(stringToEncrypt), nil)), nil
Review comment:
Just for my understanding: in the [example](https://golang.org/pkg/crypto/cipher/#example_NewGCM_encrypt), `dst` is `nil`, but here it is `nonce`. I assume that's just to prepend the nonce to the resulting ciphertext?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #5843: Updated TV Postgres to use AES encryption for URL Sig Keys
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #5843:
URL: https://github.com/apache/trafficcontrol/pull/5843#discussion_r632601689
##########
File path: docs/source/admin/traffic_vault.rst
##########
@@ -33,6 +33,7 @@ In order to use the PostgreSQL backend for Traffic Vault, you will need to set t
:password: The password to use when connecting to the database
:port: The port number that the database listens for new connections on (NOTE: the PostgreSQL default is 5432)
:user: The username to use when connecting to the database
+:aes_key_location: The location on-disk for an AES key used to encrypt secrets before they are stored.
Review comment:
here we should also state that the key must be stored in base64 encoding
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #5843: Updated TV Postgres to use AES encryption for URL Sig Keys
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #5843:
URL: https://github.com/apache/trafficcontrol/pull/5843#discussion_r632107368
##########
File path: traffic_ops/traffic_ops_golang/trafficvault/backends/postgres/postgres.go
##########
@@ -433,7 +435,12 @@ func postgresLoad(b json.RawMessage) (trafficvault.TrafficVault, error) {
log.Infoln("successfully pinged the Traffic Vault database")
}
- return &Postgres{cfg: pgCfg, db: db}, nil
+ aesKey, err := readKeyFromFile(pgCfg.AesKeyLocation)
+ if err != nil {
+ return nil, err
+ }
+
Review comment:
We should call `aes.NewCipher` here with the key contents just to ensure it's a valid key. Also, I was thinking we might want to store the key data in base64 encoding rather than raw bytes. That would make it a little easier to pass around (for instance in HashiCorp Vault we might want to store it that way as well).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org