You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/12/24 07:16:33 UTC
[GitHub] [pulsar-client-go] dferstay opened a new pull request #690: Upgrade DataDog/zstd to v1.5.0
dferstay opened a new pull request #690:
URL: https://github.com/apache/pulsar-client-go/pull/690
### Motivation
This updates zstd to 1.5.0 which supports some major performance
improvements. For details see:
https://github.com/facebook/zstd/releases/tag/v1.5.0
Results from `pulsar/internal/compression/compression_bench_test.go` show the speedup.
```
name old time/op new time/op delta
CompressionParallel/zstd-cgo-level-fastest 1.30ms ±13% 0.95ms ± 0% -27.09% (p=0.000 n=9+8)
CompressionParallel/zstd-cgo-level-fastest-2 672µs ± 2% 517µs ± 4% -23.12% (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-fastest-4 421µs ± 6% 323µs ± 3% -23.10% (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-fastest-8 241µs ± 1% 187µs ± 1% -22.25% (p=0.000 n=8+8)
CompressionParallel/zstd-cgo-level-default 1.34ms ±15% 1.00ms ± 2% -25.02% (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-default-2 676µs ± 1% 525µs ± 1% -22.30% (p=0.000 n=10+9)
CompressionParallel/zstd-cgo-level-default-4 389µs ± 1% 317µs ± 5% -18.39% (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-default-8 245µs ±11% 210µs ± 2% -14.36% (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-best 1.29ms ± 2% 0.97ms ± 0% -24.63% (p=0.000 n=8+8)
CompressionParallel/zstd-cgo-level-best-2 684µs ± 1% 528µs ± 2% -22.75% (p=0.000 n=8+10)
CompressionParallel/zstd-cgo-level-best-4 434µs ± 4% 323µs ± 2% -25.64% (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-best-8 260µs ± 7% 211µs ± 1% -19.05% (p=0.000 n=10+9)
name old speed new speed delta
CompressionParallel/zstd-cgo-level-fastest 77.4MB/s ±12% 105.9MB/s ± 0% +36.85% (p=0.000 n=9+8)
CompressionParallel/zstd-cgo-level-fastest-2 150MB/s ± 2% 195MB/s ± 4% +30.11% (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-fastest-4 240MB/s ± 6% 311MB/s ± 3% +29.93% (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-fastest-8 418MB/s ± 1% 537MB/s ± 1% +28.62% (p=0.000 n=8+8)
CompressionParallel/zstd-cgo-level-default 75.6MB/s ±14% 100.4MB/s ± 2% +32.82% (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-default-2 149MB/s ± 1% 191MB/s ± 1% +28.71% (p=0.000 n=10+9)
CompressionParallel/zstd-cgo-level-default-4 259MB/s ± 1% 318MB/s ± 5% +22.66% (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-default-8 411MB/s ±10% 479MB/s ± 2% +16.39% (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-best 77.8MB/s ± 2% 103.2MB/s ± 0% +32.66% (p=0.000 n=8+8)
CompressionParallel/zstd-cgo-level-best-2 147MB/s ± 1% 190MB/s ± 2% +29.47% (p=0.000 n=8+10)
CompressionParallel/zstd-cgo-level-best-4 232MB/s ± 4% 312MB/s ± 2% +34.44% (p=0.000 n=10+10)
CompressionParallel/zstd-cgo-level-best-8 388MB/s ± 8% 478MB/s ± 1% +23.27% (p=0.000 n=10+9)
```
```
name old time/op new time/op delta
Compression/zstd-cgo-level-fastest 369µs ± 0% 371µs ± 0% +0.65% (p=0.000 n=8+9)
Compression/zstd-cgo-level-fastest-2 369µs ± 0% 370µs ± 1% ~ (p=0.796 n=9+9)
Compression/zstd-cgo-level-fastest-4 369µs ± 0% 373µs ± 3% ~ (p=0.101 n=8+10)
Compression/zstd-cgo-level-fastest-8 371µs ± 2% 374µs ± 2% ~ (p=0.053 n=9+10)
Compression/zstd-cgo-level-default 1.28ms ± 0% 0.96ms ± 2% -25.24% (p=0.000 n=9+10)
Compression/zstd-cgo-level-default-2 1.29ms ± 2% 0.96ms ± 2% -26.05% (p=0.000 n=10+10)
Compression/zstd-cgo-level-default-4 1.30ms ± 2% 0.96ms ± 1% -25.88% (p=0.000 n=10+10)
Compression/zstd-cgo-level-default-8 1.28ms ± 3% 0.96ms ± 1% -24.63% (p=0.000 n=10+10)
Compression/zstd-cgo-level-best 3.90ms ± 0% 2.32ms ± 1% -40.39% (p=0.000 n=10+9)
Compression/zstd-cgo-level-best-2 3.90ms ± 0% 2.36ms ± 0% -39.63% (p=0.000 n=9+8)
Compression/zstd-cgo-level-best-4 3.94ms ± 2% 2.33ms ± 1% -40.88% (p=0.000 n=10+10)
Compression/zstd-cgo-level-best-8 3.97ms ± 2% 2.36ms ± 0% -40.59% (p=0.000 n=10+9)
name old speed new speed delta
Compression/zstd-cgo-level-fastest 273MB/s ± 0% 271MB/s ± 0% -0.64% (p=0.000 n=8+9)
Compression/zstd-cgo-level-fastest-2 272MB/s ± 0% 272MB/s ± 1% ~ (p=0.779 n=9+9)
Compression/zstd-cgo-level-fastest-4 272MB/s ± 0% 270MB/s ± 3% ~ (p=0.101 n=8+10)
Compression/zstd-cgo-level-fastest-8 271MB/s ± 2% 269MB/s ± 2% ~ (p=0.053 n=9+10)
Compression/zstd-cgo-level-default 78.7MB/s ± 0% 105.2MB/s ± 2% +33.77% (p=0.000 n=9+10)
Compression/zstd-cgo-level-default-2 77.8MB/s ± 2% 105.1MB/s ± 2% +35.21% (p=0.000 n=10+10)
Compression/zstd-cgo-level-default-4 77.7MB/s ± 2% 104.8MB/s ± 1% +34.91% (p=0.000 n=10+10)
Compression/zstd-cgo-level-default-8 78.9MB/s ± 3% 104.7MB/s ± 1% +32.66% (p=0.000 n=10+10)
Compression/zstd-cgo-level-best 25.8MB/s ± 0% 43.3MB/s ± 1% +67.77% (p=0.000 n=10+9)
Compression/zstd-cgo-level-best-2 25.8MB/s ± 0% 42.7MB/s ± 0% +65.63% (p=0.000 n=9+8)
Compression/zstd-cgo-level-best-4 25.5MB/s ± 2% 43.1MB/s ± 1% +69.12% (p=0.000 n=10+10)
Compression/zstd-cgo-level-best-8 25.4MB/s ± 2% 42.7MB/s ± 0% +68.30% (p=0.000 n=10+9)
```
```
name old time/op new time/op delta
Decompression/zstd-cgo-level-fastest 121µs ± 1% 111µs ±20% ~ (p=0.133 n=9+10)
Decompression/zstd-cgo-level-fastest-2 119µs ±10% 98µs ± 6% -17.77% (p=0.000 n=9+10)
Decompression/zstd-cgo-level-fastest-4 108µs ± 2% 96µs ± 6% -10.93% (p=0.000 n=10+10)
Decompression/zstd-cgo-level-fastest-8 113µs ± 5% 103µs ± 4% -9.03% (p=0.000 n=10+10)
Decompression/zstd-cgo-level-default 155µs ± 7% 135µs ± 4% -12.99% (p=0.000 n=10+10)
Decompression/zstd-cgo-level-default-2 203µs ±51% 119µs ± 0% -41.63% (p=0.000 n=10+9)
Decompression/zstd-cgo-level-default-4 138µs ± 2% 122µs ± 2% -11.63% (p=0.000 n=10+9)
Decompression/zstd-cgo-level-default-8 143µs ± 1% 127µs ± 2% -11.16% (p=0.000 n=10+9)
Decompression/zstd-cgo-level-best 137µs ± 1% 125µs ± 0% -8.98% (p=0.000 n=9+8)
Decompression/zstd-cgo-level-best-2 139µs ±15% 122µs ±13% -12.42% (p=0.034 n=8+10)
Decompression/zstd-cgo-level-best-4 124µs ± 1% 110µs ± 2% -11.14% (p=0.000 n=9+10)
Decompression/zstd-cgo-level-best-8 129µs ± 3% 114µs ± 1% -11.17% (p=0.000 n=10+8)
name old speed new speed delta
Decompression/zstd-cgo-level-fastest 832MB/s ± 1% 914MB/s ±17% ~ (p=0.133 n=9+10)
Decompression/zstd-cgo-level-fastest-2 848MB/s ±11% 1029MB/s ± 6% +21.29% (p=0.000 n=9+10)
Decompression/zstd-cgo-level-fastest-4 930MB/s ± 2% 1044MB/s ± 6% +12.36% (p=0.000 n=10+10)
Decompression/zstd-cgo-level-fastest-8 889MB/s ± 4% 977MB/s ± 4% +9.91% (p=0.000 n=10+10)
Decompression/zstd-cgo-level-default 652MB/s ± 7% 748MB/s ± 5% +14.83% (p=0.000 n=10+10)
Decompression/zstd-cgo-level-default-2 557MB/s ±41% 849MB/s ± 0% +52.42% (p=0.000 n=10+9)
Decompression/zstd-cgo-level-default-4 730MB/s ± 2% 826MB/s ± 2% +13.16% (p=0.000 n=10+9)
Decompression/zstd-cgo-level-default-8 705MB/s ± 1% 794MB/s ± 2% +12.56% (p=0.000 n=10+9)
Decompression/zstd-cgo-level-best 734MB/s ± 1% 806MB/s ± 0% +9.87% (p=0.000 n=9+8)
Decompression/zstd-cgo-level-best-2 660MB/s ±45% 815MB/s ±12% +23.45% (p=0.017 n=10+9)
Decompression/zstd-cgo-level-best-4 811MB/s ± 1% 912MB/s ± 2% +12.54% (p=0.000 n=9+10)
Decompression/zstd-cgo-level-best-8 782MB/s ± 3% 880MB/s ± 1% +12.56% (p=0.000 n=10+8)
```
### Modifications
The version of DataDog/zstd has been updated to tag v1.5.0
### Verifying this change
This change is already covered by existing tests, such as:.
- pulsar/internal/compression/compression_test.go
- pulsar/internal/compression/compression_bench_test.go
### Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): Yes, it moves to a newer version of `github.com/DataDog/zstd`
- The public API: No
- The schema: No
- The default values of configurations: No
- The wire protocol: No
### Documentation
- Does this pull request introduce a new feature? No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] wolfstudy merged pull request #690: Upgrade DataDog/zstd to v1.5.0
Posted by GitBox <gi...@apache.org>.
wolfstudy merged pull request #690:
URL: https://github.com/apache/pulsar-client-go/pull/690
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org