You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/08/03 12:14:13 UTC
[GitHub] [incubator-doris] Astralidea opened a new issue #4240: 查询的时候移除HLL_UNION_AGG后一查就把BE打挂了
Astralidea opened a new issue #4240:
URL: https://github.com/apache/incubator-doris/issues/4240
**Describe the bug**
一个稍复杂一点的查询，把HLL_UNION_AGG拿走后查导致BE挂掉。
**To Reproduce**
Steps to reproduce the behavior:
1. create table with following SQL:
CREATE TABLE `app_rt_flow_poi_intent_uv_price_v2` (
`datekey` int(11) NULL COMMENT "datekey",
`poi_id` bigint(20) NULL COMMENT "poi_id",
`area_id` int(11) NULL COMMENT "area_id",
`device_id_hll` hll HLL_UNION NULL COMMENT "uv"
) ENGINE=OLAP
AGGREGATE KEY(`datekey`, `poi_id`, `area_id`)
COMMENT "OLAP"
PARTITION BY RANGE(`datekey`)
(PARTITION p20191110 VALUES [("-2147483648"), ("20200726")),
PARTITION p20200727 VALUES [("20200726"), ("20200727")),
PARTITION p20200728 VALUES [("20200727"), ("20200728")),
PARTITION p20200729 VALUES [("20200728"), ("20200729")),
PARTITION p20200730 VALUES [("20200729"), ("20200730")),
PARTITION p20200731 VALUES [("20200730"), ("20200731")),
PARTITION p20200801 VALUES [("20200731"), ("20200801")),
PARTITION p20200802 VALUES [("20200801"), ("20200802")),
PARTITION p20200803 VALUES [("20200802"), ("20200803")),
PARTITION p20200804 VALUES [("20200803"), ("20200804")),
PARTITION p20200805 VALUES [("20200804"), ("20200805")))
DISTRIBUTED BY HASH(`poi_id`) BUCKETS 20
PROPERTIES (
"storage_type" = "COLUMN",
"replication_num" = "3"
);
2. then query:
sql> select * from (
select poi_id,
uv,
datekey,
rank() over(partition by datekey, area_id order by uv desc) rank,
sum(flag) over (partition by area_id) poi_cnt
from (
select datekey,
poi_id,
area_id,
case when area_id = -1 then 0 else 1 end as flag,
device_id_hll uv
from app_rt_flow_poi_intent_uv_price_v2
where datekey >= 20200727
and datekey <= 20200803
and area_id in (1001781)
) core
) final
where poi_id in (1531282)
[2020-08-03 19:41:54] [42000][1064] rpc failed, host: xx.xx.xx.xx
**Expected behavior**
希望能预知这个查询能够把BE打挂，用户这么查的时候直接给出提示不合法。
或者查询有限度可以保证BE不挂。
**Screenshots**
**Additional context**
如果把device_id_hll改为 HLL_UNION_AGG(device_id_hll) uv则查询正常，没啥问题。
数据量1天大概4千万。集群20台。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] wuyunfeng removed a comment on issue #4240: 查询的时候移除HLL_UNION_AGG后一查就把BE打挂了
Posted by GitBox <gi...@apache.org>.
wuyunfeng removed a comment on issue #4240:
URL: https://github.com/apache/incubator-doris/issues/4240#issuecomment-669192010
....
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] imay commented on issue #4240: 查询的时候移除HLL_UNION_AGG后一查就把BE打挂了
Posted by GitBox <gi...@apache.org>.
imay commented on issue #4240:
URL: https://github.com/apache/incubator-doris/issues/4240#issuecomment-668000551
Can you tell me why be crash? OOM or segment fault?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] wuyunfeng commented on issue #4240: 查询的时候移除HLL_UNION_AGG后一查就把BE打挂了
Posted by GitBox <gi...@apache.org>.
wuyunfeng commented on issue #4240:
URL: https://github.com/apache/incubator-doris/issues/4240#issuecomment-669192010
....
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] Astralidea commented on issue #4240: 查询的时候移除HLL_UNION_AGG后一查就把BE打挂了
Posted by GitBox <gi...@apache.org>.
Astralidea commented on issue #4240:
URL: https://github.com/apache/incubator-doris/issues/4240#issuecomment-668050675
我这边儿看不到日志，但是从监控上来看应该是BE OOM了。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org