You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/02/22 15:12:45 UTC
[GitHub] [incubator-doris] blackfox1983 opened a new issue #2974: 精准uv查询性能过长
blackfox1983 opened a new issue #2974: 精准uv查询性能过长
URL: https://github.com/apache/incubator-doris/issues/2974
版本：2.13左右更新的master版本代码
测试表：
CREATE TABLE `uv_bitmap` (
`sub_trade_id` int(11) NULL COMMENT "",
`trade_id` int(11) NULL COMMENT "",
`course_id` int(11) NULL COMMENT "",
`user_id` bitmap BITMAP_UNION NOT NULL COMMENT ""
) ENGINE=OLAP
AGGREGATE KEY(`sub_trade_id`, `trade_id`, `course_id`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`sub_trade_id`) BUCKETS 2
PROPERTIES (
"storage_type" = "COLUMN"
); |
数据规模：~2亿
sql : select bitmap_count(bitmap_union(user_id)) from uv_bitmap;
耗时 90s-120s
explain select bitmap_count(bitmap_union(user_id)) from uv_bitmap;
+------------------------------------------------------------------------------------------------------------+
| Explain String |
+------------------------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS:bitmap_count(<slot 1>) | |
| PARTITION: UNPARTITIONED |
| |
| RESULT SINK |
| |
| 3:AGGREGATE (merge finalize) |
| | output: bitmap_union(<slot 1>) |
| | group by: |
| | tuple ids: 1 |
| | |
| 2:EXCHANGE |
| tuple ids: 1 |
| |
| PLAN FRAGMENT 1 |
| OUTPUT EXPRS: |
| PARTITION: RANDOM |
| |
| STREAM DATA SINK |
| EXCHANGE ID: 02 |
| UNPARTITIONED |
| |
| 1:AGGREGATE (update serialize) |
| | output: bitmap_union(`user_id`) |
| | group by: |
| | tuple ids: 1 |
| | |
| 0:OlapScanNode |
| TABLE: uv_bitmap |
| **PREAGGREGATION: OFF**. Reason: aggExpr.getChild(0)[aggExpr.getChild(0).toSql()] is not Numeric CastExpr |
| partitions=1/1 |
| rollup: uv_bitmap |
| buckets=2/2 |
| cardinality=234779482 |
| avgRowSize=10.024232 |
| numNodes=4 |
| tuple ids: 0
这个查询模式是否有点过长了。我理解doris的性能这个级别的数据应该在ms级。
（druid 25亿数据，1秒左右返回的）
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #2974: 精准uv查询性能过长
Posted by GitBox <gi...@apache.org>.
morningman commented on issue #2974: 精准uv查询性能过长
URL: https://github.com/apache/incubator-doris/issues/2974#issuecomment-590816129
Like we discussed in WeChat group, `PREAGGREGATION: OFF` is a bug that is already fixed in latest master branch.
Hope you next test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] blackfox1983 commented on issue #2974: 精准uv查询性能过长
Posted by GitBox <gi...@apache.org>.
blackfox1983 commented on issue #2974: 精准uv查询性能过长
URL: https://github.com/apache/incubator-doris/issues/2974#issuecomment-592297343
@morningman
The conclusion is very surprising.
it take ~1s using the newest version.
It gives me great confidence to use doris instead of druid（druid take ~3s）in the future.
finally thanks very much for @imay 's advice on how to use rollup~
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] blackfox1983 closed issue #2974: 精准uv查询性能过长
Posted by GitBox <gi...@apache.org>.
blackfox1983 closed issue #2974: 精准uv查询性能过长
URL: https://github.com/apache/incubator-doris/issues/2974
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] blackfox1983 edited a comment on issue #2974: 精准uv查询性能过长
Posted by GitBox <gi...@apache.org>.
blackfox1983 edited a comment on issue #2974: 精准uv查询性能过长
URL: https://github.com/apache/incubator-doris/issues/2974#issuecomment-592297343
@morningman
The conclusion is very surprising.
it take ~1s using the newest version.
It gives me great confidence to use doris instead of druid（druid take ~3s）in the future.
finally thanks very much for @imay 's advice on how to use rollup
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org