You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/06/11 07:41:03 UTC
[GitHub] [incubator-doris] WingsGo opened a new issue #3832: [Bug]BE core due to invalid bitmap aggragate column
WingsGo opened a new issue #3832:
URL: https://github.com/apache/incubator-doris/issues/3832
**Describe the bug**
If we create a table with column type is bitmap and aggregate type is replace.BE will core during load data.
```
Thu Jun 11 15:23:42 CST 2020: *** Aborted at 1591860222 (unix time) try "date -d @1591860222" if you are using GNU date ***
Thu Jun 11 15:23:42 CST 2020: PC: @ 0xe974e1 doris::Field::consume()
Thu Jun 11 15:23:42 CST 2020: *** SIGSEGV (@0x18) received by PID 28285 (TID 0x7fb89f22b700) from PID 24; stack trace: ***
Thu Jun 11 15:23:42 CST 2020: @ 0x7fb9222a9250 (unknown)
Thu Jun 11 15:23:42 CST 2020: @ 0xe974e1 doris::Field::consume()
Thu Jun 11 15:23:42 CST 2020: @ 0xec990f doris::MemTable::insert()
Thu Jun 11 15:23:42 CST 2020: @ 0x155ddd6 doris::DeltaWriter::write()
Thu Jun 11 15:23:42 CST 2020: @ 0xfedd76 doris::TabletsChannel::add_batch()
Thu Jun 11 15:23:42 CST 2020: @ 0xfea5c6 doris::LoadChannel::add_batch()
Thu Jun 11 15:23:42 CST 2020: @ 0xfe5bc1 doris::LoadChannelMgr::add_batch()
Thu Jun 11 15:23:42 CST 2020: @ 0x106a5b5 _ZZN5doris20PInternalServiceImplIN4palo16PInternalServiceEE23tablet_writer_add_batchEPN6google8protobuf13RpcControllerEPKNS_28PTabletWriterAddBatchRequestEPNS_27PTabletWriterAddBatchResultEPNS5_7ClosureEENKUlvE_clEv
Thu Jun 11 15:23:42 CST 2020: @ 0xf80105 doris::PriorityThreadPool::work_thread()
Thu Jun 11 15:23:42 CST 2020: @ 0x1a2096d thread_proxy
Thu Jun 11 15:23:42 CST 2020: @ 0x7fb92205fdc5 start_thread
Thu Jun 11 15:23:42 CST 2020: @ 0x7fb92236b73d __clone
```
**To Reproduce**
Steps to reproduce the behavior:
1. create a table
```
CREATE TABLE `xm_stat_addcart_goods_user_minute` (
`stat_date` date NOT NULL DEFAULT "1970-01-01" ,
`c1` bitmap REPLACE NOT NULL,
`c2` datetime REPLACE NOT NULL DEFAULT "1970-01-01 00:00:00"
) ENGINE=OLAP
AGGREGATE KEY(`stat_date`)
DISTRIBUTED BY HASH(`stat_date`) BUCKETS 9
PROPERTIES (
"replication_num" = "3",
"in_memory" = "false",
"storage_format" = "DEFAULT"
);
```
2. load data into this table.
**reason**
https://github.com/apache/incubator-doris/blob/2ce2cf78acc298a094771d1b6df602aeddb3e8d1/be/src/olap/field.h#L41-L76
In class `Field`, the member variable `_agg_info` is initialized by function `get_aggregate_info`, and if the agg_method is `OLAP_FIELD_AGGREGATION_REPLACE` and `OLAP_FIELD_TYPE_OBJECT`, it will return `nullptr`, because the pair is not initialize in _infos_mapping, so we should forbidden the behavior and add more unit test to test the fucntion `get_aggregate_info` to avoid get nullptr, I will fix it later.
https://github.com/apache/incubator-doris/blob/2ce2cf78acc298a094771d1b6df602aeddb3e8d1/be/src/olap/aggregate_func.cpp#L39-L47
**Expected behavior**
be not core.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #3832: [Bug]BE core due to invalid bitmap aggragate column
Posted by GitBox <gi...@apache.org>.
morningman commented on issue #3832:
URL: https://github.com/apache/incubator-doris/issues/3832#issuecomment-642515292
> > I think this problem is fixed by this PR #3768
>
> It look great!!! Maybe I should look at the commit log first at next time. By the way, I think we should add more UT in BE and more annotation in FE to avoid incompatible code between FE and BE, just like schema change check, If we change the FE's contraint, we should also check the BE's logic if it is right.
I agree. BE needs a self-protection mechanism to ensure that it will not crash when it receives a wrong request.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WingsGo commented on issue #3832: [Bug]BE core due to invalid bitmap aggragate column
Posted by GitBox <gi...@apache.org>.
WingsGo commented on issue #3832:
URL: https://github.com/apache/incubator-doris/issues/3832#issuecomment-642513744
> I think this problem is fixed by this PR #3768
It look great!!! Maybe I should look at the commit log first at next time, by the way, I think we should add more UT in BE and more annotation in FE to avoid incompatible code between FE and BE, just like schema change check, If we change the FE's contraint, we should also check the BE's logic if it is right.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #3832: [Bug]BE core due to invalid bitmap aggragate column
Posted by GitBox <gi...@apache.org>.
morningman commented on issue #3832:
URL: https://github.com/apache/incubator-doris/issues/3832#issuecomment-642498594
I think this problem is fixed by this PR #3768
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] WingsGo edited a comment on issue #3832: [Bug]BE core due to invalid bitmap aggragate column
Posted by GitBox <gi...@apache.org>.
WingsGo edited a comment on issue #3832:
URL: https://github.com/apache/incubator-doris/issues/3832#issuecomment-642513744
> I think this problem is fixed by this PR #3768
It look great!!! Maybe I should look at the commit log first at next time. By the way, I think we should add more UT in BE and more annotation in FE to avoid incompatible code between FE and BE, just like schema change check, If we change the FE's contraint, we should also check the BE's logic if it is right.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org