You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/06/18 22:06:03 UTC
[GitHub] [pulsar-client-go] merlimat opened a new pull request #287: Switched to DataDog zstd wrapper, reusing the compression ctx
merlimat opened a new pull request #287:
URL: https://github.com/apache/pulsar-client-go/pull/287
### Motivation
The Zstd wrapper we're using (`valyala/gozstd`) is shipping with pre-built binaries as a well. That is nice but it makes using CGo in Alpine very complicated (since `go build` tries to reuse the pre-built binaries but fails at it).
Switched to use `DataDog/zstd` which doesn't have pre-built binaries and use `ctx` API from zstd (merged in https://github.com/DataDog/zstd/pull/83) to improve internal memory usage.
#### Before
```
BenchmarkCompression/zstd-cgo-level-fastest-16 2272 513861 ns/op 195.78 MB/s
BenchmarkCompression/zstd-cgo-level-default-16 1477 772904 ns/op 130.17 MB/s
BenchmarkCompression/zstd-cgo-level-best-16 254 4670399 ns/op 21.54 MB/s
BenchmarkDecompression/zstd-cgo-level-fastest-16 8382 154040 ns/op 653.11 MB/s
BenchmarkDecompression/zstd-cgo-level-default-16 7524 156934 ns/op 641.07 MB/s
BenchmarkDecompression/zstd-cgo-level-best-16 7748 162531 ns/op 619.00 MB/s
BenchmarkCompressionParallel/zstd-cgo-level-fastest-16 22719 54002 ns/op 1862.99 MB/s
BenchmarkCompressionParallel/zstd-cgo-level-default-16 13485 88328 ns/op 1139.01 MB/s
BenchmarkCompressionParallel/zstd-cgo-level-best-16 2161 561107 ns/op 179.30 MB/s
```
#### After
```
BenchmarkCompression/zstd-cgo-level-fastest-16 2366 472138 ns/op 213.09 MB/s
BenchmarkCompression/zstd-cgo-level-default-16 1576 756111 ns/op 133.06 MB/s
BenchmarkCompression/zstd-cgo-level-best-16 271 4452809 ns/op 22.59 MB/s
BenchmarkDecompression/zstd-cgo-level-fastest-16 9352 132880 ns/op 757.12 MB/s
BenchmarkDecompression/zstd-cgo-level-default-16 8473 142471 ns/op 706.15 MB/s
BenchmarkDecompression/zstd-cgo-level-best-16 7413 147722 ns/op 681.05 MB/s
BenchmarkCompressionParallel/zstd-cgo-level-fastest-16 23857 50493 ns/op 1992.49 MB/s
BenchmarkCompressionParallel/zstd-cgo-level-default-16 13832 87510 ns/op 1149.66 MB/s
BenchmarkCompressionParallel/zstd-cgo-level-best-16 2220 526170 ns/op 191.20 MB/s
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] merlimat merged pull request #287: Switched to DataDog zstd wrapper, reusing the compression ctx
Posted by GitBox <gi...@apache.org>.
merlimat merged pull request #287:
URL: https://github.com/apache/pulsar-client-go/pull/287
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org