You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Maxim Khutornenko <ma...@apache.org> on 2016/01/28 01:24:57 UTC
Review Request 42882: Improving job update query performance.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42882/
-----------------------------------------------------------
Review request for Aurora, Bill Farner and Zameer Manji.
Repository: aurora
Description
-------
The solution is to do sub-selects instead of relying on a large join. For a given update this approach reduced the overall number of rows returned by h2 to mybatis from 531441 to just 81.
Diffs
-----
src/jmh/java/org/apache/aurora/benchmark/JobUpdates.java 0c9dbaee81d27bacee1b6bf51b33b51509e28a63
src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java 1d8986ba117f32288510c299ea5634f90a9311e7
src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml fba7d4f39dffb0f431e957993b9ae7c03eba2c60
Diff: https://reviews.apache.org/r/42882/diff/
Testing
-------
Before this change:
```
Benchmark (instanceOverrides) Mode Cnt Score Error Units
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1 thrpt 5 39.907 ± 4.694 ops/s
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 10 thrpt 5 24.641 ± 4.187 ops/s
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 100 thrpt 5 0.654 ± 0.080 ops/s
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1000 thrpt 5 0.007 ± 0.001 ops/s
```
After this change:
```
Benchmark (instanceOverrides) Mode Cnt Score Error Units
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1 thrpt 5 41.838 ± 3.060 ops/s
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 10 thrpt 5 41.812 ± 2.373 ops/s
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 100 thrpt 5 38.281 ± 2.034 ops/s
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1000 thrpt 5 20.459 ± 2.886 ops/s
```
Thanks,
Maxim Khutornenko
Re: Review Request 42882: Improving job update query performance.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42882/#review116671
-----------------------------------------------------------
Ship it!
This is one of the places where i'm impressed with mybatis configuration design - zero code changes required here!
- Bill Farner
On Jan. 27, 2016, 4:25 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42882/
> -----------------------------------------------------------
>
> (Updated Jan. 27, 2016, 4:25 p.m.)
>
>
> Review request for Aurora, Bill Farner and Zameer Manji.
>
>
> Bugs: AURORA-1600
> https://issues.apache.org/jira/browse/AURORA-1600
>
>
> Repository: aurora
>
>
> Description
> -------
>
> The solution is to do sub-selects instead of relying on a large join. For a given update this approach reduced the overall number of rows returned by h2 to mybatis from 531441 to just 81.
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/JobUpdates.java 0c9dbaee81d27bacee1b6bf51b33b51509e28a63
> src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java 1d8986ba117f32288510c299ea5634f90a9311e7
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml fba7d4f39dffb0f431e957993b9ae7c03eba2c60
>
> Diff: https://reviews.apache.org/r/42882/diff/
>
>
> Testing
> -------
>
> Before this change:
> ```
> Benchmark (instanceOverrides) Mode Cnt Score Error Units
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1 thrpt 5 39.907 ± 4.694 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 10 thrpt 5 24.641 ± 4.187 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 100 thrpt 5 0.654 ± 0.080 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1000 thrpt 5 0.007 ± 0.001 ops/s
> ```
>
> After this change:
> ```
> Benchmark (instanceOverrides) Mode Cnt Score Error Units
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1 thrpt 5 41.838 ± 3.060 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 10 thrpt 5 41.812 ± 2.373 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 100 thrpt 5 38.281 ± 2.034 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1000 thrpt 5 20.459 ± 2.886 ops/s
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 42882: Improving job update query performance.
Posted by Zameer Manji <zm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42882/#review116678
-----------------------------------------------------------
Ship it!
Ship It!
- Zameer Manji
On Jan. 27, 2016, 4:25 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42882/
> -----------------------------------------------------------
>
> (Updated Jan. 27, 2016, 4:25 p.m.)
>
>
> Review request for Aurora, Bill Farner and Zameer Manji.
>
>
> Bugs: AURORA-1600
> https://issues.apache.org/jira/browse/AURORA-1600
>
>
> Repository: aurora
>
>
> Description
> -------
>
> The solution is to do sub-selects instead of relying on a large join. For a given update this approach reduced the overall number of rows returned by h2 to mybatis from 531441 to just 81.
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/JobUpdates.java 0c9dbaee81d27bacee1b6bf51b33b51509e28a63
> src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java 1d8986ba117f32288510c299ea5634f90a9311e7
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml fba7d4f39dffb0f431e957993b9ae7c03eba2c60
>
> Diff: https://reviews.apache.org/r/42882/diff/
>
>
> Testing
> -------
>
> Before this change:
> ```
> Benchmark (instanceOverrides) Mode Cnt Score Error Units
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1 thrpt 5 39.907 ± 4.694 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 10 thrpt 5 24.641 ± 4.187 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 100 thrpt 5 0.654 ± 0.080 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1000 thrpt 5 0.007 ± 0.001 ops/s
> ```
>
> After this change:
> ```
> Benchmark (instanceOverrides) Mode Cnt Score Error Units
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1 thrpt 5 41.838 ± 3.060 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 10 thrpt 5 41.812 ± 2.373 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 100 thrpt 5 38.281 ± 2.034 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1000 thrpt 5 20.459 ± 2.886 ops/s
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 42882: Improving job update query performance.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42882/#review116675
-----------------------------------------------------------
Master (e240fdc) is green with this patch.
./build-support/jenkins/build.sh
However, it appears that it might lack test coverage.
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Jan. 28, 2016, 12:25 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42882/
> -----------------------------------------------------------
>
> (Updated Jan. 28, 2016, 12:25 a.m.)
>
>
> Review request for Aurora, Bill Farner and Zameer Manji.
>
>
> Bugs: AURORA-1600
> https://issues.apache.org/jira/browse/AURORA-1600
>
>
> Repository: aurora
>
>
> Description
> -------
>
> The solution is to do sub-selects instead of relying on a large join. For a given update this approach reduced the overall number of rows returned by h2 to mybatis from 531441 to just 81.
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/JobUpdates.java 0c9dbaee81d27bacee1b6bf51b33b51509e28a63
> src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java 1d8986ba117f32288510c299ea5634f90a9311e7
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml fba7d4f39dffb0f431e957993b9ae7c03eba2c60
>
> Diff: https://reviews.apache.org/r/42882/diff/
>
>
> Testing
> -------
>
> Before this change:
> ```
> Benchmark (instanceOverrides) Mode Cnt Score Error Units
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1 thrpt 5 39.907 ± 4.694 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 10 thrpt 5 24.641 ± 4.187 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 100 thrpt 5 0.654 ± 0.080 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1000 thrpt 5 0.007 ± 0.001 ops/s
> ```
>
> After this change:
> ```
> Benchmark (instanceOverrides) Mode Cnt Score Error Units
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1 thrpt 5 41.838 ± 3.060 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 10 thrpt 5 41.812 ± 2.373 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 100 thrpt 5 38.281 ± 2.034 ops/s
> UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1000 thrpt 5 20.459 ± 2.886 ops/s
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 42882: Improving job update query performance.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42882/
-----------------------------------------------------------
(Updated Jan. 28, 2016, 12:25 a.m.)
Review request for Aurora, Bill Farner and Zameer Manji.
Changes
-------
+ticket
Bugs: AURORA-1600
https://issues.apache.org/jira/browse/AURORA-1600
Repository: aurora
Description
-------
The solution is to do sub-selects instead of relying on a large join. For a given update this approach reduced the overall number of rows returned by h2 to mybatis from 531441 to just 81.
Diffs
-----
src/jmh/java/org/apache/aurora/benchmark/JobUpdates.java 0c9dbaee81d27bacee1b6bf51b33b51509e28a63
src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java 1d8986ba117f32288510c299ea5634f90a9311e7
src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml fba7d4f39dffb0f431e957993b9ae7c03eba2c60
Diff: https://reviews.apache.org/r/42882/diff/
Testing
-------
Before this change:
```
Benchmark (instanceOverrides) Mode Cnt Score Error Units
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1 thrpt 5 39.907 ± 4.694 ops/s
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 10 thrpt 5 24.641 ± 4.187 ops/s
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 100 thrpt 5 0.654 ± 0.080 ops/s
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1000 thrpt 5 0.007 ± 0.001 ops/s
```
After this change:
```
Benchmark (instanceOverrides) Mode Cnt Score Error Units
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1 thrpt 5 41.838 ± 3.060 ops/s
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 10 thrpt 5 41.812 ± 2.373 ops/s
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 100 thrpt 5 38.281 ± 2.034 ops/s
UpdateStoreBenchmarks.JobInstructionsBenchmark.run 1000 thrpt 5 20.459 ± 2.886 ops/s
```
Thanks,
Maxim Khutornenko