You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2021/05/09 08:18:00 UTC
[jira] [Commented] (IMPALA-9338) Impala crashing in
impala::RowDescriptor::TupleIsNullable(int)
[ https://issues.apache.org/jira/browse/IMPALA-9338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17341464#comment-17341464 ]
Quanlong Huang commented on IMPALA-9338:
----------------------------------------
I can still reproduce this issue. For query
{code:sql}
select * from tpch.customer c left outer join tpch.lineitem l ON c.c_custkey = l.l_orderkey and c.C_CUSTKEY = l.L_ORDERKEY
{code}
The failure is
{code:java}
I0509 16:11:23.493753 26733 status.cc:129] fe4fad9a77123159:2ab0896b00000000] invalid tuple_idx: Slot(id=0 type=BIGINT col_path=[0] offset=60 null=(offset=78 mask=20) slot_idx=5 field_idx=5)
parent=Tuple(id=0 size=79 slots=[Slot(id=0 type=BIGINT col_path=[0] offset=60 null=(offset=78 mask=20) slot_idx=5 field_idx=5), Slot(id=2 type=STRING col_path=[1] offset=0 null=(offset=78 mask=1) slot_idx=0 field_idx=0), Slot(id=3 type=STRING col_path=[2] offset=12 null=(offset=78 mask=2) slot_idx=1 field_idx=1), Slot(id=4 type=SMALLINT col_path=[3] offset=76 null=(offset=78 mask=80) slot_idx=7 field_idx=7), Slot(id=5 type=STRING col_path=[4] offset=24 null=(offset=78 mask=4) slot_idx=2 field_idx=2), Slot(id=6 type=DECIMAL(12,2) col_path=[5] offset=68 null=(offset=78 mask=40) slot_idx=6 field_idx=6), Slot(id=7 type=STRING col_path=[6] offset=36 null=(offset=78 mask=8) slot_idx=3 field_idx=3), Slot(id=8 type=STRING col_path=[7] offset=48 null=(offset=78 mask=10) slot_idx=4 field_idx=4)] tuple_path=[])
row=Tuple(id=1 size=158 slots=[Slot(id=1 type=BIGINT col_path=[0] offset=96 null=(offset=157 mask=1) slot_idx=8 field_idx=8), Slot(id=9 type=BIGINT col_path=[1] offset=104 null=(offset=157 mask=2) slot_idx=9 field_idx=9), Slot(id=10 type=BIGINT col_path=[2] offset=112 null=(offset=157 mask=4) slot_idx=10 field_idx=10), Slot(id=11 type=INT col_path=[3] offset=152 null=(offset=157 mask=80) slot_idx=15 field_idx=15), Slot(id=12 type=DECIMAL(12,2) col_path=[4] offset=120 null=(offset=157 mask=8) slot_idx=11 field_idx=11), Slot(id=13 type=DECIMAL(12,2) col_path=[5] offset=128 null=(offset=157 mask=10) slot_idx=12 field_idx=12), Slot(id=14 type=DECIMAL(12,2) col_path=[6] offset=136 null=(offset=157 mask=20) slot_idx=13 field_idx=13), Slot(id=15 type=DECIMAL(12,2) col_path=[7] offset=144 null=(offset=157 mask=40) slot_idx=14 field_idx=14), Slot(id=16 type=STRING col_path=[8] offset=0 null=(offset=156 mask=1) slot_idx=0 field_idx=0), Slot(id=17 type=STRING col_path=[9] offset=12 null=(offset=156 mask=2) slot_idx=1 field_idx=1), Slot(id=18 type=STRING col_path=[10] offset=24 null=(offset=156 mask=4) slot_idx=2 field_idx=2), Slot(id=19 type=STRING col_path=[11] offset=36 null=(offset=156 mask=8) slot_idx=3 field_idx=3), Slot(id=20 type=STRING col_path=[12] offset=48 null=(offset=156 mask=10) slot_idx=4 field_idx=4), Slot(id=21 type=STRING col_path=[13] offset=60 null=(offset=156 mask=20) slot_idx=5 field_idx=5), Slot(id=22 type=STRING col_path=[14] offset=72 null=(offset=156 mask=40) slot_idx=6 field_idx=6), Slot(id=23 type=STRING col_path=[15] offset=84 null=(offset=156 mask=80) slot_idx=7 field_idx=7)] tuple_path=[])
@ 0x1e6e7b1 impala::Status::Status()
@ 0x2cbb739 impala::SlotRef::Init()
@ 0x2cac7d9 impala::ScalarExpr::Create()
@ 0x2cac993 impala::ScalarExpr::Create()
@ 0x2cacbbd impala::ScalarExpr::Create()
@ 0x251ede4 impala::KrpcDataStreamSenderConfig::Init()
@ 0x2a23fbd impala::DataSinkConfig::CreateConfig()
@ 0x25069a1 impala::FragmentState::Init()
@ 0x2506703 impala::FragmentState::CreateFragmentStateMap()
@ 0x244f3df impala::QueryState::StartFInstances()
@ 0x243e574 impala::QueryExecMgr::ExecuteQueryHelper()
@ 0x244786c boost::_mfi::mf1<>::operator()()
@ 0x2447135 boost::_bi::list2<>::operator()<>()
@ 0x244673c boost::_bi::bind_t<>::operator()()
@ 0x2445b56 boost::detail::function::void_function_obj_invoker0<>::invoke()
@ 0x22fb1a1 boost::function0<>::operator()()
@ 0x297d2af impala::Thread::SuperviseThread()
@ 0x2985c10 boost::_bi::list5<>::operator()<>()
@ 0x2985b34 boost::_bi::bind_t<>::operator()()
@ 0x2985af5 boost::detail::thread_data<>::run()
@ 0x421a8d1 thread_proxy
@ 0x7f32807786b9 start_thread
@ 0x7f327d1e64dc clone
{code}
[~baggio000], do you have time on this? If not, I think we can remove 4.0 from the target version or reassign this to someone else.
> Impala crashing in impala::RowDescriptor::TupleIsNullable(int)
> --------------------------------------------------------------
>
> Key: IMPALA-9338
> URL: https://issues.apache.org/jira/browse/IMPALA-9338
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.3.0
> Reporter: Abhishek Rawat
> Assignee: Yida Wu
> Priority: Blocker
> Labels: crash
>
> Repro:
> {code:java}
> create database default;
> CREATE EXTERNAL TABLE default.dimension ( ssn_id INT, act_num CHAR(1), eff_dt CHAR(10), seq_num SMALLINT, entry_dt CHAR(10), map ARRAY<INT>, src CHAR(10), msg CHAR(1), msg_num CHAR(3), remarks CHAR(3), description CHAR(26), default_load_ts CHAR(26), map_cd VARCHAR(50) ) PARTITIONED BY ( year INT, ssn_hash INT ) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\u001C' WITH SERDEPROPERTIES ('colelction.delim'=',', 'field.delim'='\u001C', 'serialization.format'='\u001C') STORED AS PARQUET --LOCATION 'hdfs://prdnameservice/user/hive/warehouse/default.db/dimension' TBLPROPERTIES ('DO_NOT_UPDATE_STATS'='true', 'STATS_GENERATED'='TASK', 'STATS_GENERATED_VIA_STATS_TASK'='true', 'impala.lastComputeStatsTime'='1579246708', 'last_modified_by'='a00811p', 'last_modified_time'='1489791214', 'numRows'='7357715311', 'totalSize'='235136295799');
> CREATE EXTERNAL TABLE default.fact ( ssn_id_n INT, bor_act_sfx CHAR(1), start_dt CHAR(10), seq_num SMALLINT, msg_n CHAR(8), end_dt CHAR(10), reviews CHAR(50), description CHAR(50), detail CHAR(50), default_load_ts CHAR(26) ) PARTITIONED BY ( year INT, ssn_hash INT ) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\u0016' WITH SERDEPROPERTIES ('field.delim'='\u0016', 'serialization.format'='\u0016') STORED AS PARQUET --LOCATION 'hdfs://prdnameservice/user/hive/warehouse/default.db/fact' TBLPROPERTIES ('DO_NOT_UPDATE_STATS'='true', 'STATS_GENERATED'='TASK', 'STATS_GENERATED_VIA_STATS_TASK'='true', 'impala.lastComputeStatsTime'='1579242111', 'last_modified_by'='e32940', 'last_modified_time'='1484186332', 'numRows'='5142832439', 'totalSize'='105397898347');
> use default;
> select ssn_id_n, bor_act_sfx, amap.item, start_dt, reviews, concat(msg, msg_num) corr_code from dimension, dimension.map amap LEFT JOIN fact ON dimension.ssn_id = fact.ssn_id_n AND dimension.act_num = fact.bor_act_sfx AND dimension.eff_dt = fact.start_dt and dimension.year = fact.year --and dimension.month(cast(eff_dt as timestamp)) = fact.month(cast(start_dt as timestamp)) AND dimension.YEAR = fact.YEAR AND fact.year in (2018,2019) where dimension.msg like '%B295%' AND dimension.year in (2018,2019);{code}
> Stack Trace:
> {code:java}
> #0 0x0000000000f8b1b9 in impala::RowDescriptor::TupleIsNullable(int) const ()
> #1 0x000000000130911f in impala::SlotRef::Init(impala::RowDescriptor const&, impala::RuntimeState*) ()
> #2 0x000000000130748e in impala::ScalarExpr::Create(impala::TExpr const&, impala::RowDescriptor const&, impala::RuntimeState*, impala::ObjectPool*, impala::ScalarExpr**) ()
> #3 0x00000000013075e5 in impala::ScalarExpr::Create(std::vector<impala::TExpr, std::allocator<impala::TExpr> > const&, impala::RowDescriptor const&, impala::RuntimeState*, impala::ObjectPool*, std::vector<impala::ScalarExpr*, std::allocator<impala::ScalarExpr*> >*) ()
> #4 0x000000000130769f in impala::ScalarExpr::Create(std::vector<impala::TExpr, std::allocator<impala::TExpr> > const&, impala::RowDescriptor const&, impala::RuntimeState*, std::vector<impala::ScalarExpr*, std::allocator<impala::ScalarExpr*> >*) ()
> #5 0x000000000149c1aa in impala::KrpcDataStreamSender::Init(std::vector<impala::TExpr, std::allocator<impala::TExpr> > const&, impala::TDataSink const&, impala::RuntimeState*) ()
> #6 0x0000000001208ad3 in impala::DataSink::Create(impala::TPlanFragmentCtx const&, impala::TPlanFragmentInstanceCtx const&, impala::RowDescriptor const*, impala::RuntimeState*, impala::DataSink**) ()
> #7 0x0000000000fac9a4 in impala::FragmentInstanceState::Prepare() ()
> #8 0x0000000000fad3dd in impala::FragmentInstanceState::Exec() ()
> #9 0x0000000000f98e77 in impala::QueryState::ExecFInstance(impala::FragmentInstanceState*) ()
> #10 0x00000000011a1490 in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) ()
> #11 0x00000000011a203a in boost::detail::thread_data<boost::_bi::bind_t<void, void (std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > > >::run() ()
> #12 0x00000000017909ca in thread_proxy () #13 0x00007f8832fa6aa1 in __pthread_initialize_minimal_internal () from /lib64/libpthread.so.0 #14 0x0000000000000000 in ?? ()
> {code}
>
> The crash only happens when ROJ plan is selected. If, LOJ plan is selected the query runs successfully.
> Initial investigation indicates that the Scalar expression being contructed in the above stack trace is referencing an invalid tupleId in the row descriptor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org