You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "aokolnychyi (via GitHub)" <gi...@apache.org> on 2023/08/22 04:31:34 UTC
[GitHub] [iceberg] aokolnychyi commented on pull request #8346: Core: Optimize computing user-facing state in data task
aokolnychyi commented on PR #8346:
URL: https://github.com/apache/iceberg/pull/8346#issuecomment-1687400934
Here are some numbers with #8336 and this change.
After the changes:
```
Benchmark Mode Cnt Score Error Units
TaskGroupPlanningBenchmark.planTaskGroups ss 5 1.989 ± 0.097 s/op
TaskGroupPlanningBenchmark.planTaskGroups:·async ss NaN ---
TaskGroupPlanningBenchmark.planTaskGroups:·gc.alloc.rate ss 5 569.803 ± 30.506 MB/sec
TaskGroupPlanningBenchmark.planTaskGroups:·gc.alloc.rate.norm ss 5 1499818088.000 ± 3617083.698 B/op
TaskGroupPlanningBenchmark.planTaskGroups:·gc.churn.PS_Eden_Space ss 5 603.209 ± 3180.736 MB/sec
TaskGroupPlanningBenchmark.planTaskGroups:·gc.churn.PS_Eden_Space.norm ss 5 1596666675.200 ± 8422157507.465 B/op
TaskGroupPlanningBenchmark.planTaskGroups:·gc.churn.PS_Survivor_Space ss 5 25.930 ± 149.334 MB/sec
TaskGroupPlanningBenchmark.planTaskGroups:·gc.churn.PS_Survivor_Space.norm ss 5 68311168.000 ± 391184976.025 B/op
TaskGroupPlanningBenchmark.planTaskGroups:·gc.count ss 5 2.000 counts
TaskGroupPlanningBenchmark.planTaskGroups:·gc.time ss 5 63.000 ms
```
Before the changes:
```
Benchmark Mode Cnt Score Error Units
TaskGroupPlanningBenchmark.planTaskGroups ss 5 6.659 ± 0.214 s/op
TaskGroupPlanningBenchmark.planTaskGroups:·async ss NaN ---
TaskGroupPlanningBenchmark.planTaskGroups:·gc.alloc.rate ss 5 2529.137 ± 70.078 MB/sec
TaskGroupPlanningBenchmark.planTaskGroups:·gc.alloc.rate.norm ss 5 19052489617.600 ± 3591565.223 B/op
TaskGroupPlanningBenchmark.planTaskGroups:·gc.churn.PS_Eden_Space ss 5 2559.777 ± 1165.034 MB/sec
TaskGroupPlanningBenchmark.planTaskGroups:·gc.churn.PS_Eden_Space.norm ss 5 19285409792.000 ± 8829460580.255 B/op
TaskGroupPlanningBenchmark.planTaskGroups:·gc.churn.PS_Survivor_Space ss 5 38.440 ± 29.535 MB/sec
TaskGroupPlanningBenchmark.planTaskGroups:·gc.churn.PS_Survivor_Space.norm ss 5 289638648.000 ± 225080948.753 B/op
TaskGroupPlanningBenchmark.planTaskGroups:·gc.count ss 5 22.000 counts
TaskGroupPlanningBenchmark.planTaskGroups:·gc.time ss 5 908.000 ms
```
Apart from time, there is 10+ times reduction in allocation rate and only 2 vs 22 garbage collections, which would make a big difference if we are running low on memory.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org