You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Zameer Manji <zm...@apache.org> on 2016/02/17 04:41:45 UTC
Review Request 43643: Do not materialize set inside
DbTaskStore#fetchTasks
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43643/
-----------------------------------------------------------
Review request for Aurora, John Sirois and Maxim Khutornenko.
Repository: aurora
Description
-------
The TaskStore interface requires an Interable to be returned. I have modified DbTaskStore to take advantage of this and instead just return the List MyBatis populates.
Diffs
-----
build.gradle 3d37187429deddc74eceb73dfb849ae60d58b206
src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java b899d0f11ea499e40298ea9020414997c4527821
src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java dee34e29465d58adbc805cbb9c400e0328012ab8
Diff: https://reviews.apache.org/r/43643/diff/
Testing
-------
Master:
Benchmark (numTasks) Mode Cnt Score Error Units
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 51.531 ± 7.236 ops/s
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 7.370 ± 1.320 ops/s
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 2.143 ± 1.234 ops/s
This patch:
Benchmark (numTasks) Mode Cnt Score Error Units
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 93344.488 ± 21422.498 ops/s
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 70160.553 ± 20418.670 ops/s
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 29834.284 ± 10440.180 ops/s
Thanks,
Zameer Manji
Re: Review Request 43643: Do not materialize set inside
DbTaskStore#fetchTasks
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43643/#review119484
-----------------------------------------------------------
Ship it!
Ship It!
- Maxim Khutornenko
On Feb. 17, 2016, 3:42 a.m., Zameer Manji wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43643/
> -----------------------------------------------------------
>
> (Updated Feb. 17, 2016, 3:42 a.m.)
>
>
> Review request for Aurora, John Sirois and Maxim Khutornenko.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> The TaskStore interface requires an Interable to be returned. I have modified DbTaskStore to take advantage of this and instead just return the List MyBatis populates.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java b899d0f11ea499e40298ea9020414997c4527821
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java dee34e29465d58adbc805cbb9c400e0328012ab8
>
> Diff: https://reviews.apache.org/r/43643/diff/
>
>
> Testing
> -------
>
> Master:
> Benchmark (numTasks) Mode Cnt Score Error Units
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 51.531 ± 7.236 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 7.370 ± 1.320 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 2.143 ± 1.234 ops/s
>
> This patch:
> Benchmark (numTasks) Mode Cnt Score Error Units
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 93344.488 ± 21422.498 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 70160.553 ± 20418.670 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 29834.284 ± 10440.180 ops/s
>
>
> Thanks,
>
> Zameer Manji
>
>
Re: Review Request 43643: Do not materialize set inside
DbTaskStore#fetchTasks
Posted by John Sirois <js...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43643/#review119457
-----------------------------------------------------------
Ship it!
Ship It!
- John Sirois
On Feb. 16, 2016, 8:42 p.m., Zameer Manji wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43643/
> -----------------------------------------------------------
>
> (Updated Feb. 16, 2016, 8:42 p.m.)
>
>
> Review request for Aurora, John Sirois and Maxim Khutornenko.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> The TaskStore interface requires an Interable to be returned. I have modified DbTaskStore to take advantage of this and instead just return the List MyBatis populates.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java b899d0f11ea499e40298ea9020414997c4527821
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java dee34e29465d58adbc805cbb9c400e0328012ab8
>
> Diff: https://reviews.apache.org/r/43643/diff/
>
>
> Testing
> -------
>
> Master:
> Benchmark (numTasks) Mode Cnt Score Error Units
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 51.531 ± 7.236 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 7.370 ± 1.320 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 2.143 ± 1.234 ops/s
>
> This patch:
> Benchmark (numTasks) Mode Cnt Score Error Units
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 93344.488 ± 21422.498 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 70160.553 ± 20418.670 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 29834.284 ± 10440.180 ops/s
>
>
> Thanks,
>
> Zameer Manji
>
>
Re: Review Request 43643: Do not materialize set inside
DbTaskStore#fetchTasks
Posted by Zameer Manji <zm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43643/
-----------------------------------------------------------
(Updated Feb. 19, 2016, 12:44 p.m.)
Review request for Aurora, John Sirois and Maxim Khutornenko.
Changes
-------
Stephan Erb's feedback.
Repository: aurora
Description
-------
The TaskStore interface requires an Interable to be returned. I have modified DbTaskStore to take advantage of this and instead just return the List MyBatis populates.
Diffs (updated)
-----
src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java b899d0f11ea499e40298ea9020414997c4527821
src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java dee34e29465d58adbc805cbb9c400e0328012ab8
Diff: https://reviews.apache.org/r/43643/diff/
Testing
-------
Master:
Benchmark (numTasks) Mode Cnt Score Error Units
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 51.531 ± 7.236 ops/s
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 7.370 ± 1.320 ops/s
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 2.143 ± 1.234 ops/s
This patch:
Benchmark (numTasks) Mode Cnt Score Error Units
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 93344.488 ± 21422.498 ops/s
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 70160.553 ± 20418.670 ops/s
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 29834.284 ± 10440.180 ops/s
Thanks,
Zameer Manji
Re: Review Request 43643: Do not materialize set inside
DbTaskStore#fetchTasks
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43643/#review119409
-----------------------------------------------------------
Ship it!
Master (9d642dc) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 17, 2016, 3:42 a.m., Zameer Manji wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43643/
> -----------------------------------------------------------
>
> (Updated Feb. 17, 2016, 3:42 a.m.)
>
>
> Review request for Aurora, John Sirois and Maxim Khutornenko.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> The TaskStore interface requires an Interable to be returned. I have modified DbTaskStore to take advantage of this and instead just return the List MyBatis populates.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java b899d0f11ea499e40298ea9020414997c4527821
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java dee34e29465d58adbc805cbb9c400e0328012ab8
>
> Diff: https://reviews.apache.org/r/43643/diff/
>
>
> Testing
> -------
>
> Master:
> Benchmark (numTasks) Mode Cnt Score Error Units
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 51.531 ± 7.236 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 7.370 ± 1.320 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 2.143 ± 1.234 ops/s
>
> This patch:
> Benchmark (numTasks) Mode Cnt Score Error Units
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 93344.488 ± 21422.498 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 70160.553 ± 20418.670 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 29834.284 ± 10440.180 ops/s
>
>
> Thanks,
>
> Zameer Manji
>
>
Re: Review Request 43643: Do not materialize set inside
DbTaskStore#fetchTasks
Posted by Zameer Manji <zm...@apache.org>.
> On Feb. 17, 2016, 1:39 a.m., Stephan Erb wrote:
> > src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java, line 567
> > <https://reviews.apache.org/r/43643/diff/2/?file=1252301#file1252301line567>
> >
> > Would you mind changing this test slightly and assert the number of items before you wrap it in a set? This would allow us to catch duplicate elements without silently discarding those in the set constructor.
> >
> > That's a rather unlikely scenario, but the impact would be quite severe.
Good catch, I have updated the test to ensure the query has no duplicates.
- Zameer
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43643/#review119428
-----------------------------------------------------------
On Feb. 16, 2016, 7:42 p.m., Zameer Manji wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43643/
> -----------------------------------------------------------
>
> (Updated Feb. 16, 2016, 7:42 p.m.)
>
>
> Review request for Aurora, John Sirois and Maxim Khutornenko.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> The TaskStore interface requires an Interable to be returned. I have modified DbTaskStore to take advantage of this and instead just return the List MyBatis populates.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java b899d0f11ea499e40298ea9020414997c4527821
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java dee34e29465d58adbc805cbb9c400e0328012ab8
>
> Diff: https://reviews.apache.org/r/43643/diff/
>
>
> Testing
> -------
>
> Master:
> Benchmark (numTasks) Mode Cnt Score Error Units
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 51.531 ± 7.236 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 7.370 ± 1.320 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 2.143 ± 1.234 ops/s
>
> This patch:
> Benchmark (numTasks) Mode Cnt Score Error Units
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 93344.488 ± 21422.498 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 70160.553 ± 20418.670 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 29834.284 ± 10440.180 ops/s
>
>
> Thanks,
>
> Zameer Manji
>
>
Re: Review Request 43643: Do not materialize set inside
DbTaskStore#fetchTasks
Posted by Stephan Erb <se...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43643/#review119428
-----------------------------------------------------------
Ship it!
Great catch! One minor nitpick below.
src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java (line 567)
<https://reviews.apache.org/r/43643/#comment180803>
Would you mind changing this test slightly and assert the number of items before you wrap it in a set? This would allow us to catch duplicate elements without silently discarding those in the set constructor.
That's a rather unlikely scenario, but the impact would be quite severe.
- Stephan Erb
On Feb. 17, 2016, 4:42 a.m., Zameer Manji wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43643/
> -----------------------------------------------------------
>
> (Updated Feb. 17, 2016, 4:42 a.m.)
>
>
> Review request for Aurora, John Sirois and Maxim Khutornenko.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> The TaskStore interface requires an Interable to be returned. I have modified DbTaskStore to take advantage of this and instead just return the List MyBatis populates.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java b899d0f11ea499e40298ea9020414997c4527821
> src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java dee34e29465d58adbc805cbb9c400e0328012ab8
>
> Diff: https://reviews.apache.org/r/43643/diff/
>
>
> Testing
> -------
>
> Master:
> Benchmark (numTasks) Mode Cnt Score Error Units
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 51.531 ± 7.236 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 7.370 ± 1.320 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 2.143 ± 1.234 ops/s
>
> This patch:
> Benchmark (numTasks) Mode Cnt Score Error Units
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 93344.488 ± 21422.498 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 70160.553 ± 20418.670 ops/s
> TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 29834.284 ± 10440.180 ops/s
>
>
> Thanks,
>
> Zameer Manji
>
>
Re: Review Request 43643: Do not materialize set inside
DbTaskStore#fetchTasks
Posted by Zameer Manji <zm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43643/
-----------------------------------------------------------
(Updated Feb. 16, 2016, 7:42 p.m.)
Review request for Aurora, John Sirois and Maxim Khutornenko.
Repository: aurora
Description
-------
The TaskStore interface requires an Interable to be returned. I have modified DbTaskStore to take advantage of this and instead just return the List MyBatis populates.
Diffs (updated)
-----
src/main/java/org/apache/aurora/scheduler/storage/db/DbTaskStore.java b899d0f11ea499e40298ea9020414997c4527821
src/test/java/org/apache/aurora/scheduler/storage/AbstractTaskStoreTest.java dee34e29465d58adbc805cbb9c400e0328012ab8
Diff: https://reviews.apache.org/r/43643/diff/
Testing
-------
Master:
Benchmark (numTasks) Mode Cnt Score Error Units
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 51.531 ± 7.236 ops/s
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 7.370 ± 1.320 ops/s
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 2.143 ± 1.234 ops/s
This patch:
Benchmark (numTasks) Mode Cnt Score Error Units
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 10000 thrpt 5 93344.488 ± 21422.498 ops/s
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 50000 thrpt 5 70160.553 ± 20418.670 ops/s
TaskStoreBenchmarks.DBFetchTasksBenchmark.run 100000 thrpt 5 29834.284 ± 10440.180 ops/s
Thanks,
Zameer Manji