You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by John Sirois <js...@apache.org> on 2016/03/23 03:20:01 UTC
Review Request 45193: Treat empty and null collections equivalently
in task queries.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/
-----------------------------------------------------------
Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
Repository: aurora
Description
-------
Previously, `null` was handled differently from an empty collection in
task queries. For the Go thrift bindings, this was problematic since
zero types in Go are useful in almost all cases and in particular in the
case of slices (collections). In these cases unset `TaskQuery`
collection parameters are represented as empty collections (empty
slices[]) instead of `nil` (`null`), leading to the inability to use the
query API in any natural way.
src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
6 files changed, 27 insertions(+), 11 deletions(-)
Diffs
-----
src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
Diff: https://reviews.apache.org/r/45193/diff/
Testing
-------
NB: This change was broken out of https://reviews.apache.org/r/42756/
since it stands on its own (although its slightly more awkward in the
mutable thrift world) and the case of the Go Aurora API client forces the
issue.
Locally green:
```
./build-support/jenkins/build.sh
./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
```
Thanks,
John Sirois
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by John Sirois <js...@apache.org>.
> On March 23, 2016, 8:35 a.m., John Sirois wrote:
> > I think this change stands on its own aside from the current state of the generated Go thrift bindings, but there has been a good deal of discussion about those bindings offline. Some homework below.
> >
> > For the case of the `TaskQuery` thrift struct, thrift 0.9.3 generates the following Go struct:
> > ```go
> > type TaskQuery struct {
> > // unused field # 1
> > JobName string `thrift:"jobName,2" json:"jobName"`
> > // unused field # 3
> > TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
> > Statuses map[ScheduleStatus]bool `thrift:"statuses,5" json:"statuses"`
> > // unused field # 6
> > InstanceIds map[int32]bool `thrift:"instanceIds,7" json:"instanceIds"`
> > // unused field # 8
> > Environment string `thrift:"environment,9" json:"environment"`
> > SlaveHosts map[string]bool `thrift:"slaveHosts,10" json:"slaveHosts"`
> > JobKeys map[*JobKey]bool `thrift:"jobKeys,11" json:"jobKeys"`
> > Offset int32 `thrift:"offset,12" json:"offset"`
> > Limit int32 `thrift:"limit,13" json:"limit"`
> > Role string `thrift:"role,14" json:"role"`
> > }
> > ```
> >
> > This is reasonable since `api.TaskQuery{}.TaskIds == nil` is true; ie the collections (maps represent sets here) zero to nil.
> > The issue comes in the serialization for these fields, `taskIds` is shown below as an example:
> > ```go
> > func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
> > if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
> > return thrift.PrependError(fmt.Sprintf("%T write field begin error 4:taskIds: ", p), err)
> > }
> > if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err != nil {
> > return thrift.PrependError("error writing set begin: ", err)
> > }
> > for v, _ := range p.TaskIds {
> > if err := oprot.WriteString(string(v)); err != nil {
> > return thrift.PrependError(fmt.Sprintf("%T. (0) field write error: ", p), err)
> > }
> > }
> > if err := oprot.WriteSetEnd(); err != nil {
> > return thrift.PrependError("error writing set end: ", err)
> > }
> > if err := oprot.WriteFieldEnd(); err != nil {
> > return thrift.PrependError(fmt.Sprintf("%T write field end error 4:taskIds: ", p), err)
> > }
> > return err
> > }
> > ```
> >
> > So, since its safe to do so in Go (`len(p.TaskIds) == 0` and `for v, _ := range p.TaskIds {` loops 0 times for `p.TaskIds == nil`), the code always emits the `taskIds` field, whether nil or not, which presents on the other end of the wire as an empty set (as opposed to a null or un-set set). This does seem like a clear bug in the thrift compiler. https://issues.apache.org/jira/browse/THRIFT-3700 is similar, but on the deserialization side of things so I've filed https://issues.apache.org/jira/browse/THRIFT-3752.
Hrm, so this may not be a thrift compiler bug per-se. Marking all TaskQuery thrift fields as optional yields:
```go
type TaskQuery struct {
// unused field # 1
JobName *string `thrift:"jobName,2" json:"jobName,omitempty"`
// unused field # 3
TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds,omitempty"`
Statuses map[ScheduleStatus]bool `thrift:"statuses,5" json:"statuses,omitempty"`
// unused field # 6
InstanceIds map[int32]bool `thrift:"instanceIds,7" json:"instanceIds,omitempty"`
// unused field # 8
Environment *string `thrift:"environment,9" json:"environment,omitempty"`
SlaveHosts map[string]bool `thrift:"slaveHosts,10" json:"slaveHosts,omitempty"`
JobKeys map[*JobKey]bool `thrift:"jobKeys,11" json:"jobKeys,omitempty"`
Offset *int32 `thrift:"offset,12" json:"offset,omitempty"`
Limit *int32 `thrift:"limit,13" json:"limit,omitempty"`
Role *string `thrift:"role,14" json:"role,omitempty"`
}
```
So map (set) fields are unchanged (still `nil`able), but primitives - not `nil`able before, are now represented
as `nil`able pointers. This also has the effect of emitting `IsSet*` methods and respecting these methods as a
gate for field serialization:
```go
func (p *TaskQuery) IsSetTaskIds() bool {
return p.TaskIds != nil
}
func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
if p.IsSetTaskIds() {
if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field begin error 4:taskIds: ", p), err)
}
if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err != nil {
return thrift.PrependError("error writing set begin: ", err)
}
for v, _ := range p.TaskIds {
if err := oprot.WriteString(string(v)); err != nil {
return thrift.PrependError(fmt.Sprintf("%T. (0) field write error: ", p), err)
}
}
if err := oprot.WriteSetEnd(); err != nil {
return thrift.PrependError("error writing set end: ", err)
}
if err := oprot.WriteFieldEnd(); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field end error 4:taskIds: ", p), err)
}
}
return err
}
```
In other words, the thirft compiler for Go has a different notion of unset requiredness than the java compiler - and the perils of the global lack of specificity as to how to handle optional vs required vs <none> is well documented :/
- John
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125024
-----------------------------------------------------------
On March 23, 2016, 10:35 a.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 23, 2016, 10:35 a.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets). In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by John Sirois <js...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125024
-----------------------------------------------------------
I think this change stands on its own aside from the current state of the generated Go thrift bindings, but there has been a good deal of discussion about those bindings offline. Some homework below.
For the case of the `TaskQuery` thrift struct, thrift 0.9.3 generates the following Go struct:
```go
type TaskQuery struct {
// unused field # 1
JobName string `thrift:"jobName,2" json:"jobName"`
// unused field # 3
TaskIds map[string]bool `thrift:"taskIds,4" json:"taskIds"`
Statuses map[ScheduleStatus]bool `thrift:"statuses,5" json:"statuses"`
// unused field # 6
InstanceIds map[int32]bool `thrift:"instanceIds,7" json:"instanceIds"`
// unused field # 8
Environment string `thrift:"environment,9" json:"environment"`
SlaveHosts map[string]bool `thrift:"slaveHosts,10" json:"slaveHosts"`
JobKeys map[*JobKey]bool `thrift:"jobKeys,11" json:"jobKeys"`
Offset int32 `thrift:"offset,12" json:"offset"`
Limit int32 `thrift:"limit,13" json:"limit"`
Role string `thrift:"role,14" json:"role"`
}
```
This is reasonable since `api.TaskQuery{}.TaskIds == nil` is true; ie the collections (maps represent sets here) zero to nil.
The issue comes in the serialization for these fields, `taskIds` is shown below as an example:
```go
func (p *TaskQuery) writeField4(oprot thrift.TProtocol) (err error) {
if err := oprot.WriteFieldBegin("taskIds", thrift.SET, 4); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field begin error 4:taskIds: ", p), err)
}
if err := oprot.WriteSetBegin(thrift.STRING, len(p.TaskIds)); err != nil {
return thrift.PrependError("error writing set begin: ", err)
}
for v, _ := range p.TaskIds {
if err := oprot.WriteString(string(v)); err != nil {
return thrift.PrependError(fmt.Sprintf("%T. (0) field write error: ", p), err)
}
}
if err := oprot.WriteSetEnd(); err != nil {
return thrift.PrependError("error writing set end: ", err)
}
if err := oprot.WriteFieldEnd(); err != nil {
return thrift.PrependError(fmt.Sprintf("%T write field end error 4:taskIds: ", p), err)
}
return err
}
```
So, since its safe to do so in Go (`len(p.TaskIds) == 0` and `for v, _ := range p.TaskIds {` loops 0 times for `p.TaskIds == nil`), the code always emits the `taskIds` field, whether nil or not, which presents on the other end of the wire as an empty set (as opposed to a null or un-set set). This does seem like a clear bug in the thrift compiler. https://issues.apache.org/jira/browse/THRIFT-3700 is similar, but on the deserialization side of things so I've filed https://issues.apache.org/jira/browse/THRIFT-3752.
- John Sirois
On March 22, 2016, 8:32 p.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 22, 2016, 8:32 p.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of slices (collections). In these cases unset `TaskQuery`
> collection parameters are represented as empty collections (empty
> slices[]) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review124954
-----------------------------------------------------------
Ship it!
Ship It!
- Bill Farner
On March 22, 2016, 7:32 p.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 22, 2016, 7:32 p.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of slices (collections). In these cases unset `TaskQuery`
> collection parameters are represented as empty collections (empty
> slices[]) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by John Sirois <js...@apache.org>.
> On March 23, 2016, 1:36 p.m., Zameer Manji wrote:
> > I am in favor of making this change. However, it seems this patch could be
> > improved because the storage layer has to now check for both `null` and
> > empty collection.
> >
> > I think a better solution would be to change Query.Builder to be a Supplier of
> > `ITaskQuery` instead of `TaskQuery`. The `I*` classes always return empty
> > collections and never null. By doing this the storage layer can be simplified to
> > only check for empty collections. This is also more future proof because the
> > storage/query layer can never be given `null` which means the next time we add a
> > new field to `TaskQuery` (such as image id?) we don't have to risk regressing on
> > this behaviour.
> >
> > I have scanned the code and it doesn't seem to be a lot of work, it seems mostly
> > places where there are entries like `query.getStatusesSize()` would need to be
> > replaced with `query.getStatus().size()`.
> >
> > I don't feel super strong about my suggestion here so if you do object or if it
> > does seem like a lot of work, I will just ship this change as is.
>
> John Sirois wrote:
> I'll give it a spike and see how it works out. I will note though that the storage layer (db) - was already doing this for 2/5 collection fields in the TaskMapper.select. IOW: the scope here is expanding from making things consistent to also paying down more fundamental existing debt. I'm all for that, but I also want to call out I tried paying down that debt and then some by eliminating I* alltogether and using a single consistent immutable model where no collections were null, but instead empty. So this is make it nice - but in a middling way.
The `TaskMapper.xml` does of course get cleaner, but the scheduler code gets trickier due to some I* isSet issues (https://issues.apache.org/jira/browse/AURORA-1650 https://issues.apache.org/jira/browse/AURORA-1651).
See what you think.
- John
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125102
-----------------------------------------------------------
On March 23, 2016, 10:35 a.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 23, 2016, 10:35 a.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets). In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by John Sirois <js...@apache.org>.
> On March 23, 2016, 1:36 p.m., Zameer Manji wrote:
> > I am in favor of making this change. However, it seems this patch could be
> > improved because the storage layer has to now check for both `null` and
> > empty collection.
> >
> > I think a better solution would be to change Query.Builder to be a Supplier of
> > `ITaskQuery` instead of `TaskQuery`. The `I*` classes always return empty
> > collections and never null. By doing this the storage layer can be simplified to
> > only check for empty collections. This is also more future proof because the
> > storage/query layer can never be given `null` which means the next time we add a
> > new field to `TaskQuery` (such as image id?) we don't have to risk regressing on
> > this behaviour.
> >
> > I have scanned the code and it doesn't seem to be a lot of work, it seems mostly
> > places where there are entries like `query.getStatusesSize()` would need to be
> > replaced with `query.getStatus().size()`.
> >
> > I don't feel super strong about my suggestion here so if you do object or if it
> > does seem like a lot of work, I will just ship this change as is.
I'll give it a spike and see how it works out. I will note though that the storage layer (db) - was already doing this for 2/5 collection fields in the TaskMapper.select. IOW: the scope here is expanding from making things consistent to also paying down more fundamental existing debt. I'm all for that, but I also want to call out I tried paying down that debt and then some by eliminating I* alltogether and using a single consistent immutable model where no collections were null, but instead empty. So this is make it nice - but in a middling way.
- John
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125102
-----------------------------------------------------------
On March 23, 2016, 10:35 a.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 23, 2016, 10:35 a.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets). In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by Zameer Manji <zm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125102
-----------------------------------------------------------
I am in favor of making this change. However, it seems this patch could be
improved because the storage layer has to now check for both `null` and
empty collection.
I think a better solution would be to change Query.Builder to be a Supplier of
`ITaskQuery` instead of `TaskQuery`. The `I*` classes always return empty
collections and never null. By doing this the storage layer can be simplified to
only check for empty collections. This is also more future proof because the
storage/query layer can never be given `null` which means the next time we add a
new field to `TaskQuery` (such as image id?) we don't have to risk regressing on
this behaviour.
I have scanned the code and it doesn't seem to be a lot of work, it seems mostly
places where there are entries like `query.getStatusesSize()` would need to be
replaced with `query.getStatus().size()`.
I don't feel super strong about my suggestion here so if you do object or if it
does seem like a lot of work, I will just ship this change as is.
- Zameer Manji
On March 23, 2016, 9:35 a.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 23, 2016, 9:35 a.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets). In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by Zameer Manji <zm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125262
-----------------------------------------------------------
Ship it!
The ripples into the scheduler code was a little bit more than I expected, but I think this is the better solution. I think whenever we get around to fixing AURORA-1650 and AURORA-1651 the code should end up a lot cleaner.
- Zameer Manji
On March 24, 2016, 7:52 a.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 24, 2016, 7:52 a.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets). In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 078dd8b63fdca192c735f9097edd030ee315a021
> src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 4bf40047e105389ac7139edc449857889d390106
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java d326d24dd527d084bce1b300f1818d3b1d94c036
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 5d246bee1a4dabc563a23c542384205537719f6a
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 3ba03429748448642571cfe0858278a50148745a
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by John Sirois <js...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125265
-----------------------------------------------------------
@ReviewBot retry
- John Sirois
On March 24, 2016, 8:52 a.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 24, 2016, 8:52 a.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets). In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 078dd8b63fdca192c735f9097edd030ee315a021
> src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 4bf40047e105389ac7139edc449857889d390106
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java d326d24dd527d084bce1b300f1818d3b1d94c036
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 5d246bee1a4dabc563a23c542384205537719f6a
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 3ba03429748448642571cfe0858278a50148745a
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by John Sirois <js...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125643
-----------------------------------------------------------
Rebasing to pick up https://reviews.apache.org/r/45366/
- John Sirois
On March 24, 2016, 8:52 a.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 24, 2016, 8:52 a.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets). In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 078dd8b63fdca192c735f9097edd030ee315a021
> src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 4bf40047e105389ac7139edc449857889d390106
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java d326d24dd527d084bce1b300f1818d3b1d94c036
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 5d246bee1a4dabc563a23c542384205537719f6a
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 3ba03429748448642571cfe0858278a50148745a
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125270
-----------------------------------------------------------
Ship it!
Master (9927231) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On March 24, 2016, 2:52 p.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 24, 2016, 2:52 p.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets). In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 078dd8b63fdca192c735f9097edd030ee315a021
> src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 4bf40047e105389ac7139edc449857889d390106
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java d326d24dd527d084bce1b300f1818d3b1d94c036
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 5d246bee1a4dabc563a23c542384205537719f6a
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 3ba03429748448642571cfe0858278a50148745a
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by John Sirois <js...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125271
-----------------------------------------------------------
Bill - this has changed enough that you should re-assess your ship.
- John Sirois
On March 24, 2016, 8:52 a.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 24, 2016, 8:52 a.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets). In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 078dd8b63fdca192c735f9097edd030ee315a021
> src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 4bf40047e105389ac7139edc449857889d390106
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java d326d24dd527d084bce1b300f1818d3b1d94c036
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 5d246bee1a4dabc563a23c542384205537719f6a
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 3ba03429748448642571cfe0858278a50148745a
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125678
-----------------------------------------------------------
Ship it!
- Bill Farner
On March 28, 2016, 8:33 a.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 28, 2016, 8:33 a.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets). In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 078dd8b63fdca192c735f9097edd030ee315a021
> src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 4bf40047e105389ac7139edc449857889d390106
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java d326d24dd527d084bce1b300f1818d3b1d94c036
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 5d246bee1a4dabc563a23c542384205537719f6a
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 3ba03429748448642571cfe0858278a50148745a
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by John Sirois <js...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125644
-----------------------------------------------------------
Bill - I'll take silence as consent and merge this today.
- John Sirois
On March 28, 2016, 9:33 a.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 28, 2016, 9:33 a.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets). In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 078dd8b63fdca192c735f9097edd030ee315a021
> src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 4bf40047e105389ac7139edc449857889d390106
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java d326d24dd527d084bce1b300f1818d3b1d94c036
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 5d246bee1a4dabc563a23c542384205537719f6a
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 3ba03429748448642571cfe0858278a50148745a
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125657
-----------------------------------------------------------
Ship it!
Master (83a078b) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On March 28, 2016, 3:33 p.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 28, 2016, 3:33 p.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets). In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 078dd8b63fdca192c735f9097edd030ee315a021
> src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 4bf40047e105389ac7139edc449857889d390106
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java d326d24dd527d084bce1b300f1818d3b1d94c036
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 5d246bee1a4dabc563a23c542384205537719f6a
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 3ba03429748448642571cfe0858278a50148745a
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by John Sirois <js...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/
-----------------------------------------------------------
(Updated March 28, 2016, 9:33 a.m.)
Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
Changes
-------
Back task queries with `ITaskQuery`.
This avoids special null checks for collection query criteria although
it does require more care in struct handling code where `TaskQuery` <->
`ITaskQuery` are lossy for `isSet` query methods.
src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 8 +++-----
src/main/java/org/apache/aurora/scheduler/base/Query.java | 19 +++++++++----------
src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 15 +++++++--------
src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java | 2 +-
src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java | 4 ++--
src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 14 +++++++-------
src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java | 12 ++++++++----
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java | 2 +-
src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 10 +++++-----
src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java | 18 ++++++++++--------
src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java | 6 +++---
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java | 20 ++++++++++----------
12 files changed, 66 insertions(+), 64 deletions(-)
Repository: aurora
Description
-------
Previously, `null` was handled differently from an empty collection in
task queries. For the Go thrift bindings, this was problematic since
zero values in Go are useful in almost all cases and in particular in the
case of maps (used to represent sets). In these cases unset `TaskQuery`
collection parameters are serialized as empty collections (empty
maps) instead of `nil` (`null`), leading to the inability to use the
query API in any natural way.
src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
6 files changed, 27 insertions(+), 11 deletions(-)
Diffs (updated)
-----
src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 078dd8b63fdca192c735f9097edd030ee315a021
src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 4bf40047e105389ac7139edc449857889d390106
src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java d326d24dd527d084bce1b300f1818d3b1d94c036
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 5d246bee1a4dabc563a23c542384205537719f6a
src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99
src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 3ba03429748448642571cfe0858278a50148745a
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb
Diff: https://reviews.apache.org/r/45193/diff/
Testing
-------
NB: This change was broken out of https://reviews.apache.org/r/42756/
since it stands on its own (although its slightly more awkward in the
mutable thrift world) and the case of the Go Aurora API client forces the
issue.
Locally green:
```
./build-support/jenkins/build.sh
./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
```
Thanks,
John Sirois
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review125261
-----------------------------------------------------------
Master (9927231) is red with this patch.
./build-support/jenkins/build.sh
[1m self._clock.converge(threads=[hct.threaded_health_checker])[0m
[1m self._clock.assert_waiting(hct.threaded_health_checker, amount=1)[0m
[1m [0m
[1m assert hct._total_latency == 0[0m
[1m assert hct.metrics.sample()['total_latency_secs'] == 0[0m
[1m [0m
[1m # start the health check (during health check it is still 0)[0m
[1m epsilon = 0.001[0m
[1m self._clock.tick(1.0 + epsilon)[0m
[1m self._clock.converge(threads=[hct.threaded_health_checker])[0m
[1m self._clock.assert_waiting(hct.threaded_health_checker, amount=0.5)[0m
[1m assert hct._total_latency == 0[0m
[1m assert hct.metrics.sample()['total_latency_secs'] == 0[0m
[1m assert hct.metrics.sample()['checks'] == 0[0m
[1m [0m
[1m # finish the health check[0m
[1m self._clock.tick(0.5 + epsilon)[0m
[1m self._clock.converge(threads=[hct.threaded_health_checker])[0m
[1m self._clock.assert_waiting(hct.threaded_health_checker, amount=1) # interval_secs[0m
[1m> assert hct._total_latency == 0.5[0m
[1m[31mE AssertionError: assert 0.5009999999999999 == 0.5[0m
[1m[31mE + where 0.5009999999999999 = <apache.aurora.executor.common.health_checker.HealthChecker object at 0x7fcda9901c10>._total_latency[0m
src/test/python/apache/aurora/executor/common/test_health_checker.py:174: AssertionError
-------------- Captured stderr call --------------
[<twitter.common.testing.clock.ThreadedClock object at 0x7fcda9901250>] Time now: 0.0
[<twitter.common.testing.clock.ThreadedClock object at 0x7fcda9901250>] Time now: 0.0
[<twitter.common.testing.clock.ThreadedClock object at 0x7fcda9901250>] Time now: 1.0
[<twitter.common.testing.clock.ThreadedClock object at 0x7fcda9901250>] Time now: 1.001
[<twitter.common.testing.clock.ThreadedClock object at 0x7fcda9901250>] Time now: 1.001
[<twitter.common.testing.clock.ThreadedClock object at 0x7fcda9901250>] Time now: 1.501
[<twitter.common.testing.clock.ThreadedClock object at 0x7fcda9901250>] Time now: 1.502
generated xml file: /home/jenkins/jenkins-slave/workspace/AuroraBot/dist/test-results/415337499eb72578eab327a6487c1f5c9452b3d6.xml
[1m[31m 1 failed, 663 passed, 5 skipped, 1 warnings in 313.43 seconds [0m
FAILURE
15:26:21 07:04 [complete][31m
FAILURE[0m
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On March 24, 2016, 2:52 p.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 24, 2016, 2:52 p.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of maps (used to represent sets). In these cases unset `TaskQuery`
> collection parameters are serialized as empty collections (empty
> maps) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 078dd8b63fdca192c735f9097edd030ee315a021
> src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 4bf40047e105389ac7139edc449857889d390106
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java d326d24dd527d084bce1b300f1818d3b1d94c036
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 5d246bee1a4dabc563a23c542384205537719f6a
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
> src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 3ba03429748448642571cfe0858278a50148745a
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by John Sirois <js...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/
-----------------------------------------------------------
(Updated March 24, 2016, 8:52 a.m.)
Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
Changes
-------
Back task queries with `ITaskQuery`.
This avoids special null checks for collection query criteria although
it does require more care in struct handling code where `TaskQuery` <->
`ITaskQuery` are lossy for `isSet` query methods.
src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 8 +++-----
src/main/java/org/apache/aurora/scheduler/base/Query.java | 19 +++++++++----------
src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 15 +++++++--------
src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java | 2 +-
src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java | 4 ++--
src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 14 +++++++-------
src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java | 12 ++++++++----
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java | 2 +-
src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 10 +++++-----
src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java | 18 ++++++++++--------
src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java | 6 +++---
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java | 20 ++++++++++----------
12 files changed, 66 insertions(+), 64 deletions(-)
Repository: aurora
Description
-------
Previously, `null` was handled differently from an empty collection in
task queries. For the Go thrift bindings, this was problematic since
zero values in Go are useful in almost all cases and in particular in the
case of maps (used to represent sets). In these cases unset `TaskQuery`
collection parameters are serialized as empty collections (empty
maps) instead of `nil` (`null`), leading to the inability to use the
query API in any natural way.
src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
6 files changed, 27 insertions(+), 11 deletions(-)
Diffs (updated)
-----
src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java 078dd8b63fdca192c735f9097edd030ee315a021
src/main/java/org/apache/aurora/scheduler/storage/db/TaskMapper.java 4bf40047e105389ac7139edc449857889d390106
src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
src/main/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImpl.java d326d24dd527d084bce1b300f1818d3b1d94c036
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java 5d246bee1a4dabc563a23c542384205537719f6a
src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
src/test/java/org/apache/aurora/scheduler/http/api/security/HttpSecurityIT.java dfe94d3fadc3f5e3322dd5a3a367ad6ef22c2a99
src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
src/test/java/org/apache/aurora/scheduler/thrift/ReadOnlySchedulerImplTest.java 3ba03429748448642571cfe0858278a50148745a
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0a7b518578f4fd62c22e3ba52d8beae7958dc9eb
Diff: https://reviews.apache.org/r/45193/diff/
Testing
-------
NB: This change was broken out of https://reviews.apache.org/r/42756/
since it stands on its own (although its slightly more awkward in the
mutable thrift world) and the case of the Go Aurora API client forces the
issue.
Locally green:
```
./build-support/jenkins/build.sh
./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
```
Thanks,
John Sirois
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by John Sirois <js...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/
-----------------------------------------------------------
(Updated March 23, 2016, 10:35 a.m.)
Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
Changes
-------
Update the description to reflect homework.
Repository: aurora
Description (updated)
-------
Previously, `null` was handled differently from an empty collection in
task queries. For the Go thrift bindings, this was problematic since
zero values in Go are useful in almost all cases and in particular in the
case of maps (used to represent sets). In these cases unset `TaskQuery`
collection parameters are serialized as empty collections (empty
maps) instead of `nil` (`null`), leading to the inability to use the
query API in any natural way.
src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
6 files changed, 27 insertions(+), 11 deletions(-)
Diffs
-----
src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
Diff: https://reviews.apache.org/r/45193/diff/
Testing
-------
NB: This change was broken out of https://reviews.apache.org/r/42756/
since it stands on its own (although its slightly more awkward in the
mutable thrift world) and the case of the Go Aurora API client forces the
issue.
Locally green:
```
./build-support/jenkins/build.sh
./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
```
Thanks,
John Sirois
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/#review124951
-----------------------------------------------------------
Ship it!
Master (c66a9ee) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On March 23, 2016, 2:32 a.m., John Sirois wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45193/
> -----------------------------------------------------------
>
> (Updated March 23, 2016, 2:32 a.m.)
>
>
> Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Previously, `null` was handled differently from an empty collection in
> task queries. For the Go thrift bindings, this was problematic since
> zero values in Go are useful in almost all cases and in particular in the
> case of slices (collections). In these cases unset `TaskQuery`
> collection parameters are represented as empty collections (empty
> slices[]) instead of `nil` (`null`), leading to the inability to use the
> query API in any natural way.
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
> 6 files changed, 27 insertions(+), 11 deletions(-)
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
> src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
> src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
> src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
> src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
>
> Diff: https://reviews.apache.org/r/45193/diff/
>
>
> Testing
> -------
>
> NB: This change was broken out of https://reviews.apache.org/r/42756/
> since it stands on its own (although its slightly more awkward in the
> mutable thrift world) and the case of the Go Aurora API client forces the
> issue.
>
> Locally green:
> ```
> ./build-support/jenkins/build.sh
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> ```
>
>
> Thanks,
>
> John Sirois
>
>
Re: Review Request 45193: Treat empty and null collections
equivalently in task queries.
Posted by John Sirois <js...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45193/
-----------------------------------------------------------
(Updated March 22, 2016, 8:32 p.m.)
Review request for Aurora, David Chung, Bill Farner, and Zameer Manji.
Repository: aurora
Description (updated)
-------
Previously, `null` was handled differently from an empty collection in
task queries. For the Go thrift bindings, this was problematic since
zero values in Go are useful in almost all cases and in particular in the
case of slices (collections). In these cases unset `TaskQuery`
collection parameters are represented as empty collections (empty
slices[]) instead of `nil` (`null`), leading to the inability to use the
query API in any natural way.
src/main/java/org/apache/aurora/scheduler/base/JobKeys.java | 2 +-
src/main/java/org/apache/aurora/scheduler/base/Query.java | 2 +-
src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java | 2 +-
src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java | 6 ++++--
src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml | 6 +++---
src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java | 20 +++++++++++++++++---
6 files changed, 27 insertions(+), 11 deletions(-)
Diffs
-----
src/main/java/org/apache/aurora/scheduler/base/JobKeys.java 8f5bf58b963ae5f76aad7dfa34bae5b9e67d6242
src/main/java/org/apache/aurora/scheduler/base/Query.java ee01eaa4d0230d6bf0909b6460f27a74f03240db
src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java ac0bb374842741d7ccb7a83c574a90ac156af0f9
src/main/java/org/apache/aurora/scheduler/storage/mem/MemTaskStore.java 231a55615abfbb483667f5f8ef71d2709fc16a88
src/main/resources/org/apache/aurora/scheduler/storage/db/TaskMapper.xml 684614ffc42dd6778c7675a6c2f81cb72c106c0e
src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java e56fed2e6c0cdb47737cf1a9b637c44c5e5b9815
Diff: https://reviews.apache.org/r/45193/diff/
Testing
-------
NB: This change was broken out of https://reviews.apache.org/r/42756/
since it stands on its own (although its slightly more awkward in the
mutable thrift world) and the case of the Go Aurora API client forces the
issue.
Locally green:
```
./build-support/jenkins/build.sh
./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
```
Thanks,
John Sirois