You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Gabor Kaszab (Jira)" <ji...@apache.org> on 2023/01/18 08:36:00 UTC
[jira] [Commented] (IMPALA-2603) Incorrect results and plan for inline view referencing several collection types correlated with different ancestor blocks
[ https://issues.apache.org/jira/browse/IMPALA-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17678123#comment-17678123 ]
Gabor Kaszab commented on IMPALA-2603:
--------------------------------------
There have been a number of nested type related enhancements recently, and apparently this issue doesn't reproduce anymore. At least the query in the description runs fine for me, however, I still see the same query profile as above. Anyway, without crashing on a DCHECK we can resolve this in my opinion.
> Incorrect results and plan for inline view referencing several collection types correlated with different ancestor blocks
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-2603
> URL: https://issues.apache.org/jira/browse/IMPALA-2603
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 2.3.0
> Reporter: Taras Bobrovytsky
> Priority: Critical
> Labels: complextype, correctness, crash, downgraded, nested_types, query_generator, ramp-up
>
> *Problem*
> Queries with multiple nested inline views that have correlated references to nested collections (relative table references), can return incorrect results in RELEASE and hit a DCHECK in debug under the following condition:
> * There is an inline view that references multiple nested collections which come from different ancestor blocks at different levels of nesting.
> * In the example below, Impala fails to generate a correct plan for the "a" inline view because the references "t5" and "t6" reference different ancestor query blocks at different nesting levels.
> Query:
> {code}
> SELECT
> 1
> FROM
> customer t1
> INNER JOIN (
> SELECT
> 1
> FROM
> t1.c_orders t2
> INNER JOIN (
> SELECT
> 1
> FROM
> t2.o_lineitems t5
> INNER JOIN t1.c_orders t6
> ) as a
> ) as b;
> {code}
> Wrong Query Plan:
> {code}
> +------------------------------------------------------------------------------------+
> | Explain String |
> +------------------------------------------------------------------------------------+
> | Estimated Per-Host Requirements: Memory=176.00MB VCores=1 |
> | WARNING: The following tables are missing relevant table and/or column statistics. |
> | tpch_nested_parquet.customer |
> | |
> | 05:EXCHANGE [UNPARTITIONED] |
> | | |
> | 01:SUBPLAN |
> | | |
> | |--04:NESTED LOOP JOIN [CROSS JOIN] |
> | | | |
> | | |--02:SINGULAR ROW SRC |
> | | | |
> | | 03:UNNEST [t1.c_orders t2] |
> | | |
> | 00:SCAN HDFS [tpch_nested_parquet.customer t1] |
> | partitions=1/1 files=4 size=554.13MB |
> +------------------------------------------------------------------------------------+
> {code}
> Stack Trace:
> {code}
> #0 0x00007f5c10cf5cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
> #1 0x00007f5c10cf90d8 in __GI_abort () at abort.c:89
> #2 0x0000000002144d09 in google::DumpStackTraceAndExit () at src/utilities.cc:147
> #3 0x000000000213ddbd in google::LogMessage::Fail () at src/logging.cc:1315
> #4 0x000000000213fc45 in google::LogMessage::SendToLog (this=0x7f5b9ef08e00) at src/logging.cc:1269
> #5 0x000000000213d913 in google::LogMessage::Flush (this=this@entry=0x7f5b9ef08e00) at src/logging.cc:1138
> #6 0x000000000214059e in google::LogMessageFatal::~LogMessageFatal (this=0x7f5b9ef08e00, __in_chrg=<optimized out>) at src/logging.cc:1836
> #7 0x0000000001586657 in impala::Coordinator::ValidateCollectionSlots (this=0xc49ca00, batch=0xc3b3e00) at /home/dev/Impala/be/src/runtime/coordinator.cc:911
> #8 0x000000000158638d in impala::Coordinator::GetNext (this=0xc49ca00, batch=0x7dd3bd0, state=0xd934400) at /home/dev/Impala/be/src/runtime/coordinator.cc:890
> #9 0x00000000013710c3 in impala::ImpalaServer::QueryExecState::FetchNextBatch (this=0x7dd2000) at /home/dev/Impala/be/src/service/query-exec-state.cc:877
> #10 0x000000000136f169 in impala::ImpalaServer::QueryExecState::FetchRowsInternal (this=0x7dd2000, max_rows=1024, fetched_rows=0x7f5b9ef09370) at /home/dev/Impala/be/src/service/query-exec-state.cc:706
> #11 0x000000000136e80d in impala::ImpalaServer::QueryExecState::FetchRows (this=0x7dd2000, max_rows=1024, fetched_rows=0x7f5b9ef09370) at /home/dev/Impala/be/src/service/query-exec-state.cc:616
> #12 0x0000000001364418 in impala::ImpalaServer::FetchInternal (this=0x910ac00, query_id=..., start_over=false, fetch_size=1024, query_results=0x7f5b9ef09668) at /home/dev/Impala/be/src/service/impala-beeswax-server.cc:613
> #13 0x000000000135fd34 in impala::ImpalaServer::fetch (this=0x910ac00, query_results=..., query_handle=..., start_over=false, fetch_size=1024) at /home/dev/Impala/be/src/service/impala-beeswax-server.cc:288
> #14 0x00000000014fded4 in beeswax::BeeswaxServiceProcessor::process_fetch (this=0xc9fdf40, seqid=0, iprot=0x7d789c0, oprot=0x8fe8340, callContext=0x7efe9c0) at /home/dev/Impala/be/generated-sources/gen-cpp/BeeswaxService.cpp:3150
> #15 0x00000000014fc312 in beeswax::BeeswaxServiceProcessor::dispatchCall (this=0xc9fdf40, iprot=0x7d789c0, oprot=0x8fe8340, fname=..., seqid=0, callContext=0x7efe9c0) at /home/dev/Impala/be/generated-sources/gen-cpp/BeeswaxService.cpp:2952
> #16 0x00000000014e577b in impala::ImpalaServiceProcessor::dispatchCall (this=0xc9fdf40, iprot=0x7d789c0, oprot=0x8fe8340, fname=..., seqid=0, callContext=0x7efe9c0) at /home/dev/Impala/be/generated-sources/gen-cpp/ImpalaService.cpp:1673
> #17 0x00000000012fac15 in apache::thrift::TDispatchProcessor::process (this=0xc9fdf40, in=..., out=..., connectionContext=0x7efe9c0) at /home/dev/Impala/thirdparty/thrift-0.9.0/build/include/thrift/TDispatchProcessor.h:121
> #18 0x00000000020e39ea in apache::thrift::server::TThreadPoolServer::Task::run (this=0xcd9bbc0) at src/thrift/server/TThreadPoolServer.cpp:70
> #19 0x00000000020d012f in apache::thrift::concurrency::ThreadManager::Task::run (this=0x8fe8300) at src/thrift/concurrency/ThreadManager.cpp:187
> #20 0x00000000020d2d9a in apache::thrift::concurrency::ThreadManager::Worker::run (this=0xce50f90) at src/thrift/concurrency/ThreadManager.cpp:316
> #21 0x00000000012241a5 in impala::ThriftThread::RunRunnable (this=0x90d5a80, runnable=..., promise=0x7fffce7e4120) at /home/dev/Impala/be/src/rpc/thrift-thread.cc:61
> #22 0x000000000122596d in boost::_mfi::mf2<void, impala::ThriftThread, boost::shared_ptr<apache::thrift::concurrency::Runnable>, impala::Promise<unsigned long>*>::operator() (this=0x7f00780, p=0x90d5a80, a1=..., a2=0x7fffce7e4120) at /usr/include/boost/bind/mem_fn_template.hpp:280
> #23 0x00000000012257c8 in boost::_bi::list3<boost::_bi::value<impala::ThriftThread*>, boost::_bi::value<boost::shared_ptr<apache::thrift::concurrency::Runnable> >, boost::_bi::value<impala::Promise<unsigned long>*> >::operator()<boost::_mfi::mf2<void, impala::ThriftThread, boost::shared_ptr<apache::thrift::concurrency::Runnable>, impala::Promise<unsigned long>*>, boost::_bi::list0> (this=0x7f00790, f=..., a=...) at /usr/include/boost/bind/bind.hpp:392
> #24 0x0000000001225559 in boost::_bi::bind_t<void, boost::_mfi::mf2<void, impala::ThriftThread, boost::shared_ptr<apache::thrift::concurrency::Runnable>, impala::Promise<unsigned long>*>, boost::_bi::list3<boost::_bi::value<impala::ThriftThread*>, boost::_bi::value<boost::shared_ptr<apache::thrift::concurrency::Runnable> >, boost::_bi::value<impala::Promise<unsigned long>*> > >::operator() (this=0x7f00780) at /usr/include/boost/bind/bind_template.hpp:20
> #25 0x000000000122547a in boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf2<void, impala::ThriftThread, boost::shared_ptr<apache::thrift::concurrency::Runnable>, impala::Promise<unsigned long>*>, boost::_bi::list3<boost::_bi::value<impala::ThriftThread*>, boost::_bi::value<boost::shared_ptr<apache::thrift::concurrency::Runnable> >, boost::_bi::value<impala::Promise<unsigned long>*> > >, void>::invoke (function_obj_ptr=...) at /usr/include/boost/function/function_template.hpp:153
> #26 0x000000000125b4a8 in boost::function0<void>::operator() (this=0x7f5b9ef09e00) at /usr/include/boost/function/function_template.hpp:767
> #27 0x000000000146e091 in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*) (name=..., category=..., functor=..., thread_started=0x7fffce7e3f60) at /home/dev/Impala/be/src/util/thread.cc:314
> #28 0x00000000014772c7 in boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list0&, int) (this=0x90d85c0, f=@0x90d85b8: 0x146dd88 <impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*)>, a=...) at /usr/include/boost/bind/bind.hpp:457
> #29 0x0000000001477211 in boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > >::operator()() (this=0x90d85b8) at /usr/include/boost/bind/bind_template.hpp:20
> #30 0x00000000014771d6 in boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > > >::run() (this=0x90d8400) at /usr/include/boost/thread/detail/thread.hpp:117
> #31 0x00007f5c13cbfa4a in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0
> #32 0x00007f5c1316c182 in start_thread (arg=0x7f5b9ef0a700) at pthread_create.c:312
> #33 0x00007f5c10db947d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
> {code}
> impalad.FATAL:
> {code}
> Check failed: tuple->IsNull(slot_desc->null_indicator_offset())
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org