You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tianyi Wang (JIRA)" <ji...@apache.org> on 2017/09/08 03:43:02 UTC
[jira] [Resolved] (IMPALA-5867) Out-of-range yy month format can
crash Impala
[ https://issues.apache.org/jira/browse/IMPALA-5867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tianyi Wang resolved IMPALA-5867.
---------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.11.0
IMPALA-5867: Fix bugs parsing 2-digit year
This patch fixes several bugs parsing 1 or 2-digit year formats.
Existing code is broken in several ways:
1. With 1 or 2-digit year format and month/day missing, ParseDateTime()
throws an uncaught exception.
2. If now() is 02/29 in a leap year but (now() - 80 years) isn't,
DateTimeFormatContext::SetCenturyBreak() throws an uncaught
exception.
3. If the year parsed is 02/29 in a leap year but it isn't a leap year
100 years ago, TimestampParser::Parse() will consider the date as
invalid though it isn't.
This patch fixes above bugs and adds a few test cases in
be/src/runtime/timestamp-test.cc
The behaviors after change is:
1. A date without month or day is considered invalid. This is a
pre-existing difference from Hive, which defaults missing month/day
to 01/01.
2. Century break would be set to 02/28 80 years ago.
3. If parsed date is 00/02/29 but 1900/02/29 does not exist, treat
it as 03/01 when comparing to century break.
Change-Id: Ia4f430caea88b6c33f8050a1984ee0ee32ecb0a1
Reviewed-on: http://gerrit.cloudera.org:8080/7910
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins
> Out-of-range yy month format can crash Impala
> ---------------------------------------------
>
> Key: IMPALA-5867
> URL: https://issues.apache.org/jira/browse/IMPALA-5867
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.10.0
> Reporter: Tim Armstrong
> Assignee: Tianyi Wang
> Priority: Critical
> Labels: bugbash-2017-08-30
> Fix For: Impala 2.11.0
>
>
> This happens on master - commit c1c4815049ca3deab7465070b1207446cb3a1645
> {code}
> [localhost:21000] > select from_unixtime(unix_timestamp("85-12-01", "yy-mm-dd"));
> Query: select from_unixtime(unix_timestamp("85-12-01", "yy-mm-dd"))
> Query submitted at: 2017-08-30 12:23:57 (Coordinator: http://tarmstrong-box:25000)
> Error communicating with impalad: TSocket read 0 bytes
> {code}
> {code}
> terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::gregorian::bad_month> >'
> what(): Month number is out of range 1..12
> {code}
> {code}
> (gdb) bt
> #0 0x00007f8d126d1428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
> #1 0x00007f8d126d302a in __GI_abort () at abort.c:89
> #2 0x00007f8d1323184d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> #3 0x00007f8d1322f6b6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> #4 0x00007f8d1322f701 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> #5 0x00007f8d1322f919 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> #6 0x00000000012645da in boost::throw_exception<boost::gregorian::bad_month> (e=...) at toolchain/boost-1.57.0-p3/include/boost/throw_exception.hpp:69
> #7 0x00000000012632e0 in boost::CV::simple_exception_policy<unsigned short, (unsigned short)1, (unsigned short)12, boost::gregorian::bad_month>::on_error ()
> at toolchain/boost-1.57.0-p3/include/boost/date_time/constrained_value.hpp:110
> #8 0x0000000001261748 in boost::CV::constrained_value<boost::CV::simple_exception_policy<unsigned short, (unsigned short)1, (unsigned short)12, boost::gregorian::bad_month> >::assign (this=0x7f8cb345d600,
> value=0) at toolchain/boost-1.57.0-p3/include/boost/date_time/constrained_value.hpp:69
> #9 0x0000000001260250 in boost::CV::constrained_value<boost::CV::simple_exception_policy<unsigned short, (unsigned short)1, (unsigned short)12, boost::gregorian::bad_month> >::constrained_value (
> this=0x7f8cb345d600, value=0) at toolchain/boost-1.57.0-p3/include/boost/date_time/constrained_value.hpp:48
> #10 0x000000000125f324 in boost::gregorian::greg_month::greg_month (this=0x7f8cb345d600, theMonth=0) at toolchain/boost-1.57.0-p3/include/boost/date_time/gregorian/greg_month.hpp:64
> #11 0x0000000001413324 in impala::TimestampParser::ParseDateTime (str=0x7dac030 "85-12-01", str_len=8, dt_ctx=..., dt_result=0x7f8cb345d700) at be/src/runtime/timestamp-parse-util.cc:556
> #12 0x000000000141196a in impala::TimestampParser::Parse (str=0x7dac030 "85-12-01", len=8, dt_ctx=..., d=0x7f8cb345d998, t=0x7f8cb345d990) at be/src/runtime/timestamp-parse-util.cc:312
> #13 0x000000000141828f in impala::TimestampValue::Parse (str=0x7dac030 "85-12-01", len=8, dt_ctx=...) at be/src/runtime/timestamp-value.cc:72
> #14 0x00000000019b7dc2 in impala::TimestampFunctions::ToTimestamp (context=0x55b3fa0, date=..., fmt=...) at be/src/exprs/timestamp-functions-ir.cc:167
> #15 0x00000000019b797e in impala::TimestampFunctions::Unix (context=0x55b3fa0, string_val=..., fmt=...) at be/src/exprs/timestamp-functions-ir.cc:101
> #16 0x00000000019eca56 in impala::ScalarFnCall::InterpretEval<impala_udf::BigIntVal> (this=0xa431c00, eval=0xb39bf80, row=0x0) at be/src/exprs/scalar-fn-call.cc:489
> #17 0x00000000019dd27f in impala::ScalarFnCall::GetBigIntVal (this=0xa431c00, eval=0xb39bf80, row=0x0) at be/src/exprs/scalar-fn-call.cc:576
> #18 0x000000000199d82f in impala::ScalarExprEvaluator::GetValue (this=0xb39bf80, expr=..., row=0x0) at be/src/exprs/scalar-expr-evaluator.cc:298
> #19 0x000000000199d611 in impala::ScalarExprEvaluator::GetValue (this=0xb39bf80, row=0x0) at be/src/exprs/scalar-expr-evaluator.cc:268
> #20 0x00000000015094a1 in Java_org_apache_impala_service_FeSupport_NativeEvalExprsWithoutRow (env=0x9f469e0, caller_class=0x7f8cb345ea10, thrift_expr_batch=0x7f8cb345ea28, thrift_query_ctx_bytes=0x7f8cb345ea20)
> at be/src/service/fe-support.cc:231
> {code}
> {code}
> (gdb) p *dt_result
> $3 = {
> year = 1985,
> month = 0,
> day = 1,
> hour = 0,
> minute = 12,
> second = 0,
> fraction = 0,
> tz_offset = {
> <boost::date_time::time_duration<boost::posix_time::time_duration, boost::date_time::time_resolution_traits<boost::date_time::time_resolution_traits_adapted64_impl, (boost::date_time::time_resolutions)6, 1000000000l, 9u, long> >> = {
> <boost::less_than_comparable<boost::posix_time::time_duration, boost::equality_comparable<boost::posix_time::time_duration, boost::posix_time::time_duration, boost::detail::empty_base<boost::posix_time::time_duration>, boost::detail::false_t>, boost::detail::empty_base<boost::posix_time::time_duration>, boost::detail::true_t>> = {
> <boost::less_than_comparable1<boost::posix_time::time_duration, boost::equality_comparable<boost::posix_time::time_duration, boost::posix_time::time_duration, boost::detail::empty_base<boost::posix_time::time_duration>, boost::detail::false_t> >> = {
> <boost::equality_comparable<boost::posix_time::time_duration, boost::posix_time::time_duration, boost::detail::empty_base<boost::posix_time::time_duration>, boost::detail::false_t>> = {
> <boost::equality_comparable1<boost::posix_time::time_duration, boost::detail::empty_base<boost::posix_time::time_duration> >> = {
> <boost::detail::empty_base<boost::posix_time::time_duration>> = {<No data fields>}, <No data fields>}, <No data fields>}, <No data fields>}, <No data fields>},
> members of boost::date_time::time_duration<boost::posix_time::time_duration, boost::date_time::time_resolution_traits<boost::date_time::time_resolution_traits_adapted64_impl, (boost::date_time::time_resolutions)6, 1000000000l, 9u, long> >:
> ticks_ = {
> value_ = 0
> }
> }, <No data fields>}
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)