You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "dutyu (via GitHub)" <gi...@apache.org> on 2023/05/26 04:08:37 UTC
[GitHub] [doris] dutyu opened a new issue, #20089: [Bug] BE crash when meet a query with a hms table which have some nested array type columns
dutyu opened a new issue, #20089:
URL: https://github.com/apache/doris/issues/20089
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Version
1.2.4
### What's Wrong?
SQL:
```sql
SELECT
count(1)
FROM
fin_dw.dwd_capcredit_youshu_enterprise_layer_a_multi_data_pdi
where
pday is not null
and data_list is not null
limit
10;
```
Table schema:
```sql
CREATE TABLE `dwd_capcredit_youshu_enterprise_layer_a_multi_data_pdi`(
`record_no_md5` string COMMENT 'xxx',
`rowkey_biz` string COMMENT 'xxx',
`query_time` string COMMENT 'xxx',
`data_list` array<struct<creditCode:string,entName:string,dataYear:string,levelA:string,levelB:string,regNo:string,index:string,queryState:string,updatedTime:string>> COMMENT 'xxx')
COMMENT 'xxx'
PARTITIONED BY (
`pday` `string`)
ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
LOCATION
'xxx'
TBLPROPERTIES (
'spark.sql.sources.schema.numPartCols'='1',
'spark.sql.sources.schema.part.0'='{"type":"struct","fields":[{"name":"record_no_md5","type":"string","nullable":true,"metadata":{"comment":"rowkey,md5(record_no)"}},{"name":"rowkey_biz","type":"string","nullable":true,"metadata":{"comment":"crNo"}},{"name":"query_time","type":"string","nullable":true,"metadata":{"comment":"查询时间"}},{"name":"data_list","type":{"type":"array","elementType":{"type":"struct","fields":[{"name":"creditCode","type":"string","nullable":true,"metadata":{}},{"name":"entName","type":"string","nullable":true,"metadata":{}},{"name":"dataYear","type":"string","nullable":true,"metadata":{}},{"name":"levelA","type":"string","nullable":true,"metadata":{}},{"name":"levelB","type":"string","nullable":true,"metadata":{}},{"name":"regNo","type":"string","nullable":true,"metadata":{}},{"name":"index","type":"string","nullable":true,"metadata":{}},{"name":"queryState","type":"string","nullable":true,"metadata":{}},{"name":"updatedTime","type":"string","nullabl
e":true,"metadata":{}}]},"containsNull":true},"nullable":true,"metadata":{"comment":"数据列表"}},{"name":"pday","type":"string","nullable":true,"metadata":{"comment":"天分区"}}]}',
'spark.sql.sources.schema.partCol.0'='pday',
'transient_lastDdlTime'='1683596827',
'creater'='xxx',
'bucketing_version'='2',
'spark.sql.sources.schema.numParts'='1',
'spark.sql.create.version'='2.3.2.3.1.4.0-315')
```
BE crash stack:
```c++
I0511 16:10:22.571741 228734 olap_server.cpp:440] compaction thread pool. type: CUMU, num_threads: 10, num_threads_pending_start: 0, num_active_threads: 0, max_threads: 10, min_threads: 10, num_total_queued_tasks: 0
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/doris/be/src/common/signal_handler.h:420
1# os::Linux::chained_handler(int, siginfo*, void*) in /usr/local/jdk1.8.0_301/jre/lib/amd64/server/libjvm.so
2# JVM_handle_linux_signal in /usr/local/jdk1.8.0_301/jre/lib/amd64/server/libjvm.so
3# signalHandler(int, siginfo*, void*) in /usr/local/jdk1.8.0_301/jre/lib/amd64/server/libjvm.so
4# 0x00007FCC52A00400 in /lib64/libc.so.6
5# doris::vectorized::DataTypeArray::create_column() const at /root/doris/be/src/vec/data_types/data_type_array.cpp:38
6# doris::vectorized::DataTypeNullable::create_column() const at /root/doris/be/src/vec/data_types/data_type_nullable.cpp:134
7# doris::vectorized::VectorizedFnCall::prepare(doris::RuntimeState*, doris::RowDescriptor const&, doris::vectorized::VExprContext*) at /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:57
8# doris::vectorized::VExpr::prepare(doris::RuntimeState*, doris::RowDescriptor const&, doris::vectorized::VExprContext*) at /root/doris/be/src/vec/exprs/vexpr.cpp:91
9# doris::vectorized::VectorizedFnCall::prepare(doris::RuntimeState*, doris::RowDescriptor const&, doris::vectorized::VExprContext*) at /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:49
10# doris::vectorized::VExprContext::prepare(doris::RuntimeState*, doris::RowDescriptor const&) at /root/doris/be/src/vec/exprs/vexpr_context.cpp:56
11# doris::ExecNode::prepare(doris::RuntimeState*) at /root/doris/be/src/exec/exec_node.cpp:227
12# doris::vectorized::VScanNode::prepare(doris::RuntimeState*) at /root/doris/be/src/vec/exec/scan/vscan_node.cpp:73
13# doris::vectorized::NewFileScanNode::prepare(doris::RuntimeState*) at /root/doris/be/src/vec/exec/scan/new_file_scan_node.cpp:39
14# doris::Exec...
```
### What You Expected?
BE should not crash
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dutyu commented on issue #20089: [Bug] BE crash when meet a query with a hms table which have some nested array type columns
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu commented on issue #20089:
URL: https://github.com/apache/doris/issues/20089#issuecomment-1563838307
> This bug is fixed in master, and will be released in 2.x. However, it only throws error into client, because Doris doesn't support array type nesting of struct&map type, and can only nest array type like `Array<Array<...>>`.
Got it, i will close this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] AshinGau commented on issue #20089: [Bug] BE crash when meet a query with a hms table which have some nested array type columns
Posted by "AshinGau (via GitHub)" <gi...@apache.org>.
AshinGau commented on issue #20089:
URL: https://github.com/apache/doris/issues/20089#issuecomment-1563823716
This bug is fixed in master, and will be released in 2.x. However, it only throws error into client, because Doris doesn't support array type nesting of struct&map type, and can only nest array type like `Array<Array<...>>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dutyu closed issue #20089: [Bug] BE crash when meet a query with a hms table which have some nested array type columns
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu closed issue #20089: [Bug] BE crash when meet a query with a hms table which have some nested array type columns
URL: https://github.com/apache/doris/issues/20089
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org