You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficcontrol.apache.org by GitBox <gi...@apache.org> on 2021/05/13 13:04:41 UTC
[GitHub] [trafficcontrol] zrhoffman opened a new pull request #5844: Circumvent copylocks `go vet` check using unsafe Go
zrhoffman opened a new pull request #5844:
URL: https://github.com/apache/trafficcontrol/pull/5844
<!--
************ STOP!! ************
If this Pull Request is intended to fix a security vulnerability, DO NOT submit it! Instead, contact
the Apache Software Foundation Security Team at security@trafficcontrol.apache.org and follow the
guidelines at https://www.apache.org/security/ regarding vulnerability disclosure.
-->
## What does this PR (Pull Request) do?
<!-- Explain the changes you made here. If this fixes an Issue, identify it by
replacing the text in the checkbox item with the Issue number e.g.
- [x] This PR fixes #9001 OR is not related to any Issue
^ This will automatically close Issue number 9001 when the Pull Request is
merged (The '#' is important).
Be sure you check the box properly, see the "The following criteria are ALL
met by this PR" section for details.
-->
- [x] This PR fixes #5758 by copying a struct containing locks using unsafe Go.<!-- You can check for an issue here: https://github.com/apache/trafficcontrol/issues -->
## Which Traffic Control components are affected by this PR?
<!-- Please delete all components from this list that are NOT affected by this
Pull Request. Also, feel free to add the name of a tool or script that is
affected but not on the list.
Additionally, if this Pull Request does NOT affect documentation, please
explain why documentation is not required. -->
- Grove
## What is the best way to verify this PR?
<!-- Please include here ALL the steps necessary to test your Pull Request. If
it includes tests (and most should), outline here the steps needed to run the
tests. If not, lay out the manual testing procedure and please explain why
tests are unnecessary for this Pull Request. -->
Run `go vet -copylocks ./...`
## If this is a bug fix, what versions of Traffic Control are affected?
<!-- If this PR fixes a bug, please list here all of the affected versions - to
the best of your knowledge. It's also pretty helpful to include a commit hash
of where 'master' is at the time this PR is opened (if it affects master),
because what 'master' means will change over time. For example, if this PR
fixes a bug that's present in master (at commit hash '1df853c8'), in v4.0.0,
and in the current 4.0.1 Release candidate (e.g. RC1), then this list would
look like:
- master (1df853c8)
- 4.0.0
- 4.0.1 (RC1)
If you don't know what other versions might have this bug, AND don't know how
to find the commit hash of 'master', then feel free to leave this section
blank (or, preferably, delete it entirely).
-->
- master (5b9e199244)
- 5.1.1
- 5.1.2 (RC3)
## The following criteria are ALL met by this PR
<!-- Check the boxes to signify that the associated statement is true. To
"check a box", replace the space inside of the square brackets with an 'x'.
e.g.
- [ x] <- Wrong
- [x ] <- Wrong
- [] <- Wrong
- [*] <- Wrong
- [x] <- Correct!
-->
- [x] Effect of struct copy is unchanged, tests are unnecessary
- [x] Behavior is unchanged, documentation is unnecessary
- [x] An update to CHANGELOG.md is not necessary
- [x] This PR includes any and all required license headers
- [x] This PR **DOES NOT FIX A SERIOUS SECURITY VULNERABILITY** (see [the Apache Software Foundation's security guidelines](https://www.apache.org/security/) for details)
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on a change in pull request #5844: Circumvent copylocks `go vet` check using unsafe Go
Posted by GitBox <gi...@apache.org>.
rob05c commented on a change in pull request #5844:
URL: https://github.com/apache/trafficcontrol/pull/5844#discussion_r631848967
##########
File path: grove/remap/remap.go
##########
@@ -497,7 +498,11 @@ func makeTo(tosJSON []RemapRuleToJSON, rule remapdata.RemapRule, baseTransport *
return nil, fmt.Errorf("error parsing to %v proxy_url: %v", to.URL, toJSON.ProxyURL)
}
to.ProxyURL = proxyURL
- newTransport := *baseTransport
+ newTransport := http.Transport{}
+ const transportSize = unsafe.Sizeof(*baseTransport)
+ // Copy baseTransport to newTransport without failing the copylocks `go vet` check. See
+ // b7953cc239 for the reasoning behind copying the Transport.
+ copy((*[transportSize]byte)(unsafe.Pointer(&newTransport))[:], (*[transportSize]byte)(unsafe.Pointer(baseTransport))[:])
Review comment:
I assume this is the only way to make `go vet` happy with the copy?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on pull request #5844: Only copy Transport exported fields
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on pull request #5844:
URL: https://github.com/apache/trafficcontrol/pull/5844#issuecomment-842463944
Rebased to resolve merge conflicts in `.github/workflows/go.vet.yml` and `.golangci.yml` now that #5854 is merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on pull request #5844: Only copy Transport exported fields
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on pull request #5844:
URL: https://github.com/apache/trafficcontrol/pull/5844#issuecomment-840722792
Rebased to remove the commit message about unsafe Go.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on pull request #5844: Only copy Transport exported fields
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on pull request #5844:
URL: https://github.com/apache/trafficcontrol/pull/5844#issuecomment-842463944
Rebased to resolve merge conflicts in `.github/workflows/go.vet.yml` and `.golangci.yml` now that #5854 is merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #5844: Only copy Transport exported fields
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #5844:
URL: https://github.com/apache/trafficcontrol/pull/5844#discussion_r631990085
##########
File path: grove/remap/remap.go
##########
@@ -497,7 +498,11 @@ func makeTo(tosJSON []RemapRuleToJSON, rule remapdata.RemapRule, baseTransport *
return nil, fmt.Errorf("error parsing to %v proxy_url: %v", to.URL, toJSON.ProxyURL)
}
to.ProxyURL = proxyURL
- newTransport := *baseTransport
+ newTransport := http.Transport{}
+ const transportSize = unsafe.Sizeof(*baseTransport)
+ // Copy baseTransport to newTransport without failing the copylocks `go vet` check. See
+ // b7953cc239 for the reasoning behind copying the Transport.
+ copy((*[transportSize]byte)(unsafe.Pointer(&newTransport))[:], (*[transportSize]byte)(unsafe.Pointer(baseTransport))[:])
Review comment:
Using `Clone` in da61fe67d0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on a change in pull request #5844: Circumvent copylocks `go vet` check using unsafe Go
Posted by GitBox <gi...@apache.org>.
rob05c commented on a change in pull request #5844:
URL: https://github.com/apache/trafficcontrol/pull/5844#discussion_r631942428
##########
File path: grove/remap/remap.go
##########
@@ -497,7 +498,11 @@ func makeTo(tosJSON []RemapRuleToJSON, rule remapdata.RemapRule, baseTransport *
return nil, fmt.Errorf("error parsing to %v proxy_url: %v", to.URL, toJSON.ProxyURL)
}
to.ProxyURL = proxyURL
- newTransport := *baseTransport
+ newTransport := http.Transport{}
+ const transportSize = unsafe.Sizeof(*baseTransport)
+ // Copy baseTransport to newTransport without failing the copylocks `go vet` check. See
+ // b7953cc239 for the reasoning behind copying the Transport.
+ copy((*[transportSize]byte)(unsafe.Pointer(&newTransport))[:], (*[transportSize]byte)(unsafe.Pointer(baseTransport))[:])
Review comment:
We don't _need_ to copy the lock, we just need to copy the other fields we set from config, and we need the lock to not be the same as the other object. Default-initialization of the lock is fine.
We could also manually record and set all the transport fields we care about. I don't like that option, it's error-prone, it would silently fail if someone added a new option and forgot to add it to the "copy record."
I think `Clone` will work. All the fields we're setting in config are exported, and unless some function is added to Transport to set a private field, that shouldn't cause subtle bugs in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c merged pull request #5844: Only copy Transport exported fields
Posted by GitBox <gi...@apache.org>.
rob05c merged pull request #5844:
URL: https://github.com/apache/trafficcontrol/pull/5844
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #5844: Circumvent copylocks `go vet` check using unsafe Go
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #5844:
URL: https://github.com/apache/trafficcontrol/pull/5844#discussion_r631931430
##########
File path: grove/remap/remap.go
##########
@@ -497,7 +498,11 @@ func makeTo(tosJSON []RemapRuleToJSON, rule remapdata.RemapRule, baseTransport *
return nil, fmt.Errorf("error parsing to %v proxy_url: %v", to.URL, toJSON.ProxyURL)
}
to.ProxyURL = proxyURL
- newTransport := *baseTransport
+ newTransport := http.Transport{}
+ const transportSize = unsafe.Sizeof(*baseTransport)
+ // Copy baseTransport to newTransport without failing the copylocks `go vet` check. See
+ // b7953cc239 for the reasoning behind copying the Transport.
+ copy((*[transportSize]byte)(unsafe.Pointer(&newTransport))[:], (*[transportSize]byte)(unsafe.Pointer(baseTransport))[:])
Review comment:
We could do
```go
newTransport := *baseTransport.Clone()
```
but it would skip unexported fields like locks. We could use `reflect` to copy and set the unexported fields, but that would also involve using `unsafe`.
As for silencing the check on this line, Go offers no such option. From [go/issues#17058](https://github.com/golang/go/issues/17058#issuecomment-259012764):
> I am adamant that source code not be annotated to silence vet. That will reduce the options available but that is my position.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5844: Circumvent copylocks `go vet` check using unsafe Go
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5844:
URL: https://github.com/apache/trafficcontrol/pull/5844#discussion_r631876688
##########
File path: grove/remap/remap.go
##########
@@ -497,7 +498,11 @@ func makeTo(tosJSON []RemapRuleToJSON, rule remapdata.RemapRule, baseTransport *
return nil, fmt.Errorf("error parsing to %v proxy_url: %v", to.URL, toJSON.ProxyURL)
}
to.ProxyURL = proxyURL
- newTransport := *baseTransport
+ newTransport := http.Transport{}
+ const transportSize = unsafe.Sizeof(*baseTransport)
+ // Copy baseTransport to newTransport without failing the copylocks `go vet` check. See
+ // b7953cc239 for the reasoning behind copying the Transport.
+ copy((*[transportSize]byte)(unsafe.Pointer(&newTransport))[:], (*[transportSize]byte)(unsafe.Pointer(baseTransport))[:])
Review comment:
I, too, was hoping there was a way to do this without `unsafe`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #5844: Only copy Transport exported fields
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #5844:
URL: https://github.com/apache/trafficcontrol/pull/5844#discussion_r631990085
##########
File path: grove/remap/remap.go
##########
@@ -497,7 +498,11 @@ func makeTo(tosJSON []RemapRuleToJSON, rule remapdata.RemapRule, baseTransport *
return nil, fmt.Errorf("error parsing to %v proxy_url: %v", to.URL, toJSON.ProxyURL)
}
to.ProxyURL = proxyURL
- newTransport := *baseTransport
+ newTransport := http.Transport{}
+ const transportSize = unsafe.Sizeof(*baseTransport)
+ // Copy baseTransport to newTransport without failing the copylocks `go vet` check. See
+ // b7953cc239 for the reasoning behind copying the Transport.
+ copy((*[transportSize]byte)(unsafe.Pointer(&newTransport))[:], (*[transportSize]byte)(unsafe.Pointer(baseTransport))[:])
Review comment:
Using `Clone` in 476e501e87.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on pull request #5844: Only copy Transport exported fields
Posted by GitBox <gi...@apache.org>.
rob05c commented on pull request #5844:
URL: https://github.com/apache/trafficcontrol/pull/5844#issuecomment-849808094
I'll try to find time to review. But whoever reviews, just running `go vet -copylocks` isn't enough, it really needs verified that running Grove, all remaps use the correct Proxy in their upstream request. I'm _pretty sure_ they will from the code, but it's a pretty critical thing, it really needs tested to be sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org