You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Maxim Khutornenko <ma...@apache.org> on 2015/04/08 23:45:54 UTC
Review Request 32990: Improving JobUpdateDetails fetch performance.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32990/
-----------------------------------------------------------
Review request for Aurora and Bill Farner.
Bugs: AURORA-1263
https://issues.apache.org/jira/browse/AURORA-1263
Repository: aurora
Description
-------
Using event collections with "select" avoids unnecessary row explosion and provdes subsecond execution time for a large job update that used to take half a minute.
Diffs
-----
src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d
Diff: https://reviews.apache.org/r/32990/diff/
Testing
-------
./gradlew -Pq build
Manual testing in vagrant.
Thanks,
Maxim Khutornenko
Re: Review Request 32990: Improving JobUpdateDetails fetch
performance.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32990/#review79556
-----------------------------------------------------------
Master (7e3e7c9) is red with this patch.
./build-support/jenkins/build.sh
make[1]: Leaving directory `/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/thrift/thrift-0.9.1'
make: Leaving directory `/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/thrift'
:api:classesThriftNote: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:api:checkPython
:api:generateThriftEntitiesJava
:api:classesThriftEntities
:api:compileJava UP-TO-DATE
:api:generateThriftResources
:api:processResources UP-TO-DATE
:api:classes
:api:jar
:compileJavaNote: Writing file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/com/twitter/common/args/apt/cmdline.arg.info.txt.2
:processResources
:classes
:jar
:startScripts
:distTar
:distZip
:assemble
:compileJmhJava
:processJmhResources UP-TO-DATE
:jmhClasses
:checkstyleJmh[ant:checkstyle] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java:67: Wrong order for 'com.twitter.common.inject.Bindings' import. Order should be: java, javax, scala, com, net, org. Each group should be separated by a single blank line.
FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':checkstyleJmh'.
> Checkstyle rule violations were found. See the report at: file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/jmh.xml
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 1 mins 39.707 secs
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On April 9, 2015, 6:43 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32990/
> -----------------------------------------------------------
>
> (Updated April 9, 2015, 6:43 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Bugs: AURORA-1263
> https://issues.apache.org/jira/browse/AURORA-1263
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Using event collections with "select" avoids unnecessary row explosion and provdes subsecond execution time for a large job update that used to take half a minute.
>
> Before:
> ```
> Benchmark (instances) Mode Cnt Score Error Units
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt 5 15.595 ± 1.056 ops/s
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt 5 3.143 ± 0.259 ops/s
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 10000 thrpt 5 1.539 ± 0.075 ops/s
> ```
>
> After:
> ```
> Benchmark (instances) Mode Cnt Score Error Units
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt 5 278.405 ± 57.311 ops/s
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt 5 65.386 ± 6.455 ops/s
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 10000 thrpt 5 33.353 ± 1.974 ops/s
> ```
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java PRE-CREATION
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d
>
> Diff: https://reviews.apache.org/r/32990/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
> Manual testing in vagrant.
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 32990: Improving JobUpdateDetails fetch
performance.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32990/#review79562
-----------------------------------------------------------
Ship it!
Ship It!
- Bill Farner
On April 9, 2015, 7:15 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32990/
> -----------------------------------------------------------
>
> (Updated April 9, 2015, 7:15 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Bugs: AURORA-1263
> https://issues.apache.org/jira/browse/AURORA-1263
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Using event collections with "select" avoids unnecessary row explosion and provdes subsecond execution time for a large job update that used to take half a minute.
>
> Before:
> ```
> Benchmark (instances) Mode Cnt Score Error Units
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt 5 15.595 ± 1.056 ops/s
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt 5 3.143 ± 0.259 ops/s
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 10000 thrpt 5 1.539 ± 0.075 ops/s
> ```
>
> After:
> ```
> Benchmark (instances) Mode Cnt Score Error Units
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt 5 278.405 ± 57.311 ops/s
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt 5 65.386 ± 6.455 ops/s
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 10000 thrpt 5 33.353 ± 1.974 ops/s
> ```
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java PRE-CREATION
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d
>
> Diff: https://reviews.apache.org/r/32990/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
> Manual testing in vagrant.
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 32990: Improving JobUpdateDetails fetch
performance.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32990/#review79566
-----------------------------------------------------------
Master (f48d0e1) is green with this patch.
./build-support/jenkins/build.sh
However, it appears that it might lack test coverage.
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On April 9, 2015, 7:15 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32990/
> -----------------------------------------------------------
>
> (Updated April 9, 2015, 7:15 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Bugs: AURORA-1263
> https://issues.apache.org/jira/browse/AURORA-1263
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Using event collections with "select" avoids unnecessary row explosion and provdes subsecond execution time for a large job update that used to take half a minute.
>
> Before:
> ```
> Benchmark (instances) Mode Cnt Score Error Units
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt 5 15.595 ± 1.056 ops/s
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt 5 3.143 ± 0.259 ops/s
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 10000 thrpt 5 1.539 ± 0.075 ops/s
> ```
>
> After:
> ```
> Benchmark (instances) Mode Cnt Score Error Units
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt 5 278.405 ± 57.311 ops/s
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt 5 65.386 ± 6.455 ops/s
> UpdateStoreBenchmarks.JobDetailsBenchmark.run 10000 thrpt 5 33.353 ± 1.974 ops/s
> ```
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java PRE-CREATION
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d
>
> Diff: https://reviews.apache.org/r/32990/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
> Manual testing in vagrant.
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 32990: Improving JobUpdateDetails fetch
performance.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32990/
-----------------------------------------------------------
(Updated April 9, 2015, 7:15 p.m.)
Review request for Aurora and Bill Farner.
Changes
-------
Fixed import ordering.
Bugs: AURORA-1263
https://issues.apache.org/jira/browse/AURORA-1263
Repository: aurora
Description
-------
Using event collections with "select" avoids unnecessary row explosion and provdes subsecond execution time for a large job update that used to take half a minute.
Before:
```
Benchmark (instances) Mode Cnt Score Error Units
UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt 5 15.595 ± 1.056 ops/s
UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt 5 3.143 ± 0.259 ops/s
UpdateStoreBenchmarks.JobDetailsBenchmark.run 10000 thrpt 5 1.539 ± 0.075 ops/s
```
After:
```
Benchmark (instances) Mode Cnt Score Error Units
UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt 5 278.405 ± 57.311 ops/s
UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt 5 65.386 ± 6.455 ops/s
UpdateStoreBenchmarks.JobDetailsBenchmark.run 10000 thrpt 5 33.353 ± 1.974 ops/s
```
Diffs (updated)
-----
src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java PRE-CREATION
src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d
Diff: https://reviews.apache.org/r/32990/diff/
Testing
-------
./gradlew -Pq build
Manual testing in vagrant.
Thanks,
Maxim Khutornenko
Re: Review Request 32990: Improving JobUpdateDetails fetch
performance.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32990/
-----------------------------------------------------------
(Updated April 9, 2015, 6:43 p.m.)
Review request for Aurora and Bill Farner.
Changes
-------
Adding jmh benchmark.
Bugs: AURORA-1263
https://issues.apache.org/jira/browse/AURORA-1263
Repository: aurora
Description (updated)
-------
Using event collections with "select" avoids unnecessary row explosion and provdes subsecond execution time for a large job update that used to take half a minute.
Before:
```
Benchmark (instances) Mode Cnt Score Error Units
UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt 5 15.595 ± 1.056 ops/s
UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt 5 3.143 ± 0.259 ops/s
UpdateStoreBenchmarks.JobDetailsBenchmark.run 10000 thrpt 5 1.539 ± 0.075 ops/s
```
After:
```
Benchmark (instances) Mode Cnt Score Error Units
UpdateStoreBenchmarks.JobDetailsBenchmark.run 1000 thrpt 5 278.405 ± 57.311 ops/s
UpdateStoreBenchmarks.JobDetailsBenchmark.run 5000 thrpt 5 65.386 ± 6.455 ops/s
UpdateStoreBenchmarks.JobDetailsBenchmark.run 10000 thrpt 5 33.353 ± 1.974 ops/s
```
Diffs (updated)
-----
src/jmh/java/org/apache/aurora/benchmark/UpdateStoreBenchmarks.java PRE-CREATION
src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d
Diff: https://reviews.apache.org/r/32990/diff/
Testing
-------
./gradlew -Pq build
Manual testing in vagrant.
Thanks,
Maxim Khutornenko
Re: Review Request 32990: Improving JobUpdateDetails fetch
performance.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On April 9, 2015, 2:29 a.m., Bill Farner wrote:
> > Actually - can you add a benchmark to ThriftApiBenchmarks and include results in the description? Should be pretty easy to prove the improvement of this.
Done.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32990/#review79467
-----------------------------------------------------------
On April 8, 2015, 9:45 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32990/
> -----------------------------------------------------------
>
> (Updated April 8, 2015, 9:45 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Bugs: AURORA-1263
> https://issues.apache.org/jira/browse/AURORA-1263
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Using event collections with "select" avoids unnecessary row explosion and provdes subsecond execution time for a large job update that used to take half a minute.
>
>
> Diffs
> -----
>
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d
>
> Diff: https://reviews.apache.org/r/32990/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
> Manual testing in vagrant.
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 32990: Improving JobUpdateDetails fetch
performance.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32990/#review79467
-----------------------------------------------------------
Actually - can you add a benchmark to ThriftApiBenchmarks and include results in the description? Should be pretty easy to prove the improvement of this.
- Bill Farner
On April 8, 2015, 9:45 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32990/
> -----------------------------------------------------------
>
> (Updated April 8, 2015, 9:45 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Bugs: AURORA-1263
> https://issues.apache.org/jira/browse/AURORA-1263
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Using event collections with "select" avoids unnecessary row explosion and provdes subsecond execution time for a large job update that used to take half a minute.
>
>
> Diffs
> -----
>
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d
>
> Diff: https://reviews.apache.org/r/32990/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
> Manual testing in vagrant.
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 32990: Improving JobUpdateDetails fetch
performance.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32990/#review79466
-----------------------------------------------------------
Ship it!
Great work!
- Bill Farner
On April 8, 2015, 9:45 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32990/
> -----------------------------------------------------------
>
> (Updated April 8, 2015, 9:45 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Bugs: AURORA-1263
> https://issues.apache.org/jira/browse/AURORA-1263
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Using event collections with "select" avoids unnecessary row explosion and provdes subsecond execution time for a large job update that used to take half a minute.
>
>
> Diffs
> -----
>
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d
>
> Diff: https://reviews.apache.org/r/32990/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
> Manual testing in vagrant.
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 32990: Improving JobUpdateDetails fetch
performance.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32990/#review79441
-----------------------------------------------------------
Master (35f3e8d) is green with this patch.
./build-support/jenkins/build.sh
However, it appears that it might lack test coverage.
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On April 8, 2015, 9:45 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32990/
> -----------------------------------------------------------
>
> (Updated April 8, 2015, 9:45 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Bugs: AURORA-1263
> https://issues.apache.org/jira/browse/AURORA-1263
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Using event collections with "select" avoids unnecessary row explosion and provdes subsecond execution time for a large job update that used to take half a minute.
>
>
> Diffs
> -----
>
> src/main/resources/org/apache/aurora/scheduler/storage/db/JobUpdateDetailsMapper.xml 4536cd35ffc231a5d2f6c1b5bf2aaa084ce3ad1d
>
> Diff: https://reviews.apache.org/r/32990/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
> Manual testing in vagrant.
>
>
> Thanks,
>
> Maxim Khutornenko
>
>