You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kvrocks.apache.org by GitBox <gi...@apache.org> on 2022/06/08 15:06:19 UTC
[GitHub] [incubator-kvrocks] PragmaTwice opened a new issue, #624: Benchmark between make and ninja build for kvrocks
PragmaTwice opened a new issue, #624:
URL: https://github.com/apache/incubator-kvrocks/issues/624
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-kvrocks/issues) and found no similar issues.
### Motivation
In #621, @tisonkun came up with a great idea: benchmark ninja build and make build of kvrocks.
I did this test after making some fixes to make cmake configuration of kvrocks support ninja.
A PR for that will follow.
### Preparation
```sh
mkdir bm-make-build && cd bm-make-build && cmake .. -DCMAKE_BUILD_TYPE=RelWithDbgInfo
mkdir bm-ninja-build && cd bm-ninja-build && cmake .. -DCMAKE_BUILD_TYPE=RelWithDbgInfo -G Ninja
```
### Clean Build
```sh
$ hyperfine -r 1 "cd bm-make-build; make -j8 kvrocks kvrocks2redis unittest" "cd bm-ninja-build; ninja -j8 kvrocks kvrocks2redis unittest"
Benchmark 1: cd bm-make-build; make -j8 kvrocks kvrocks2redis unittest
Time (abs ≡): 186.883 s [User: 1139.316 s, System: 62.312 s]
Benchmark 2: cd bm-ninja-build; ninja -j8 kvrocks kvrocks2redis unittest
Time (abs ≡): 212.749 s [User: 1129.823 s, System: 60.877 s]
Summary
'cd bm-make-build; make -j8 kvrocks kvrocks2redis unittest' ran
1.14 times faster than 'cd bm-ninja-build; ninja -j8 kvrocks kvrocks2redis unittest'
```
### No-op Incremental Build
```sh
$ hyperfine -r 1 "cd bm-make-build; make -j8 kvrocks kvrocks2redis unittest" "cd bm-ninja-build; ninja -j8 kvrocks kvrocks2redis unittest"
Benchmark 1: cd bm-make-build; make -j8 kvrocks kvrocks2redis unittest
Time (abs ≡): 657.1 ms [User: 707.8 ms, System: 224.1 ms]
Benchmark 2: cd bm-ninja-build; ninja -j8 kvrocks kvrocks2redis unittest
Time (abs ≡): 39.0 ms [User: 25.7 ms, System: 17.7 ms]
Summary
'cd bm-ninja-build; ninja -j8 kvrocks kvrocks2redis unittest' ran
16.86 times faster than 'cd bm-make-build; make -j8 kvrocks kvrocks2redis unittest
```
### Incremental Build after modifying a source file
```sh
$ hyperfine -r 1 "cd bm-make-build; make -j8 kvrocks kvrocks2redis unittest" "cd bm-ninja-build; ninja -j8 kvrocks kvrocks2redis unittest"
Benchmark 1: cd bm-make-build; make -j8 kvrocks kvrocks2redis unittest
Time (abs ≡): 16.158 s [User: 14.762 s, System: 1.640 s]
Benchmark 2: cd bm-ninja-build; ninja -j8 kvrocks kvrocks2redis unittest
Time (abs ≡): 6.797 s [User: 16.019 s, System: 1.530 s]
Summary
'cd bm-ninja-build; ninja -j8 kvrocks kvrocks2redis unittest' ran
2.38 times faster than 'cd bm-make-build; make -j8 kvrocks kvrocks2redis unittest'
```
### Incremental Build after modifying a header file
```sh
$ hyperfine -r 1 "cd bm-make-build; make -j8 kvrocks kvrocks2redis unittest" "cd bm-ninja-build; ninja -j8 kvrocks kvrocks2redis unittest"
Benchmark 1: cd bm-make-build; make -j8 kvrocks kvrocks2redis unittest
Time (abs ≡): 39.906 s [User: 143.227 s, System: 7.891 s]
Benchmark 2: cd bm-ninja-build; ninja -j8 kvrocks kvrocks2redis unittest
Time (abs ≡): 27.103 s [User: 163.895 s, System: 8.882 s]
Summary
'cd bm-ninja-build; ninja -j8 kvrocks kvrocks2redis unittest' ran
1.47 times faster than 'cd bm-make-build; make -j8 kvrocks kvrocks2redis unittest'
```
### Conclusion
As you can see, ninja is slightly slower than make build in clean build, but has huge advantages in incremental build, which is very friendly to developers who need to do frequent incremental builds.
I would like to make kvrocks use make by default, but have a `--ninja` option to enable ninja.
### Solution
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kvrocks.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] PragmaTwice commented on issue #624: Benchmark between make and ninja build for kvrocks
Posted by GitBox <gi...@apache.org>.
PragmaTwice commented on issue #624:
URL: https://github.com/apache/incubator-kvrocks/issues/624#issuecomment-1150054042
> @PragmaTwice I guess this can be a reply on #621 or since this issue has been created, we can move it to Discussions?
>
> It doesn't seem like an issue to resolve but a general topic about benchmark.
Oh yeah, I think moving to Discussions is better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] tisonkun commented on issue #624: Benchmark between make and ninja build for kvrocks
Posted by GitBox <gi...@apache.org>.
tisonkun commented on issue #624:
URL: https://github.com/apache/incubator-kvrocks/issues/624#issuecomment-1150049112
@PragmaTwice I guess this can be a reply on #621 or since this issue has been created, we can move it to Discussions?
It doesn't seem like an issue to resolve but a general topic about benchmark.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org