You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/11/25 03:38:28 UTC
[GitHub] [druid] kfaraz commented on pull request #11950: Improve on-heap aggregator footprint estimates.
kfaraz commented on pull request #11950:
URL: https://github.com/apache/druid/pull/11950#issuecomment-978796502
Thanks for the changes, @gianm .
On running ingestion of sample "wikipedia" data, I got the following persisted stats in the indexing logs.
Before the changes
(total 38 persistence cycles, 652 rows persisted in each cycle):
```
...
2021-11-24T15:55:26,139 INFO [[index_parallel_test_sketch_lolhhmjb_2021-11-24T15:54:54.532Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Persisted stats: processed rows: [1304], persisted rows[652], sinks: [1], total fireHydrants (across sinks): [1], persisted fireHydrants (across sinks): [1]
2021-11-24T15:55:26,244 INFO [[index_parallel_test_sketch_lolhhmjb_2021-11-24T15:54:54.532Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Persisted stats: processed rows: [1956], persisted rows[652], sinks: [1], total fireHydrants (across sinks): [2], persisted fireHydrants (across sinks): [2]
2021-11-24T15:55:26,353 INFO [[index_parallel_test_sketch_lolhhmjb_2021-11-24T15:54:54.532Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Persisted stats: processed rows: [2608], persisted rows[652], sinks: [1], total fireHydrants (across sinks): [3], persisted fireHydrants (across sinks): [2]
2021-11-24T15:55:26,436 INFO [[index_parallel_test_sketch_lolhhmjb_2021-11-24T15:54:54.532Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Persisted stats: processed rows: [3260], persisted rows[652], sinks: [1], total fireHydrants (across sinks): [4], persisted fireHydrants (across sinks): [2]
2021-11-24T15:55:26,509 INFO [[index_parallel_test_sketch_lolhhmjb_2021-11-24T15:54:54.532Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Persisted stats: processed rows: [3912], persisted rows[652], sinks: [1], total fireHydrants (across sinks): [5], persisted fireHydrants (across sinks): [2]
2021-11-24T15:55:26,578 INFO [[index_parallel_test_sketch_lolhhmjb_2021-11-24T15:54:54.532Z]-appenderator-persist]
...
```
After the changes
(total 1 persistence cycle, ~24k rows persisted in one cycle):
```
2021-11-25T03:16:30,200 INFO [[index_parallel_test_sketchy_ghobnhcj_2021-11-25T03:15:57.793Z]-appenderator-persist] org.apache.druid.segment.realtime.appenderator.AppenderatorImpl - Persisted stats: processed rows: [24433], persisted rows[24433], sinks: [1], total fireHydrants (across sinks): [1], persisted fireHydrants (across sinks): [1]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org