[jira] [Created] (IMPALA-7704) ASAN tests failing in HdfsParquetTableWriter

["Joe McDonnell (JIRA)" <ji...@apache.org>]

Joe McDonnell created IMPALA-7704:
-------------------------------------

             Summary: ASAN tests failing in HdfsParquetTableWriter
                 Key: IMPALA-7704
                 URL: https://issues.apache.org/jira/browse/IMPALA-7704
             Project: IMPALA
          Issue Type: Bug
          Components: Backend
    Affects Versions: Impala 3.1.0
            Reporter: Joe McDonnell


ASAN tests have been failing for the last few runs. Here is the output:
{noformat}
==117268==ERROR: AddressSanitizer: use-after-poison on address 0x7ef9312e0ec0 at pc 0x0000016a0c98 bp 0x7ef91e502540 sp 0x7ef91e501cf0
READ of size 32681 at 0x7ef9312e0ec0 thread T82364
#0 0x16a0c97 in __interceptor_memcpy /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:738
#1 0x7f02e43c2dca in jni_SetByteArrayRegion (/usr/java/jdk1.8.0_144/jre/lib/amd64/server/libjvm.so+0x6d3dca)
#2 0x4ae29b7 in hdfsWrite /container.redhat6/build/cdh/hadoop/3.0.0-cdh6.x-SNAPSHOT/rpm/BUILD/hadoop-3.0.0-cdh6.x-SNAPSHOT/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/hdfs.c:1626
#3 0x332ca67 in impala::HdfsTableWriter::Write(unsigned char const*, int) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-table-writer.cc:46:13
#4 0x25ac037 in impala::HdfsParquetTableWriter::BaseColumnWriter::Flush(long*, long*, long*) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-parquet-table-writer.cc:736:5
#5 0x25b0e57 in impala::HdfsParquetTableWriter::FlushCurrentRowGroup() /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-parquet-table-writer.cc:1195:5
#6 0x25b4147 in impala::HdfsParquetTableWriter::Finalize() /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-parquet-table-writer.cc:1161:3
#7 0x251e424 in impala::HdfsTableSink::FinalizePartitionFile(impala::RuntimeState*, impala::OutputPartition*) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-table-sink.cc:620:5
#8 0x2523d1b in impala::HdfsTableSink::FlushFinal(impala::RuntimeState*) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-table-sink.cc:660:5
#9 0x1fc09f0 in impala::FragmentInstanceState::ExecInternal() /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/fragment-instance-state.cc:324:3
#10 0x1fbbd5c in impala::FragmentInstanceState::Exec() /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/fragment-instance-state.cc:95:14
#11 0x1fd5f94 in impala::QueryState::ExecFInstance(impala::FragmentInstanceState*) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/query-state.cc:478:24
#12 0x1cdef96 in boost::function0<void>::operator()() const /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14
#13 0x23ac27e in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/util/thread.cc:359:3
#14 0x23b7708 in void boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int) /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9
#15 0x23b755b in boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >::operator()() /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
#16 0x39a11e9 in thread_proxy (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/impalad+0x39a11e9)
#17 0x7f02e19f4e24 in start_thread (/lib64/libpthread.so.0+0x7e24)
#18 0x7f02e150b34c in __clone (/lib64/libc.so.6+0xf834c)

0x7ef9312e0ec0 is located 161472 bytes inside of 524288-byte region [0x7ef9312b9800,0x7ef931339800)
allocated by thread T82364 here:
#0 0x16fcd88 in malloc /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:67
#1 0x1f2a5ec in impala::MemPool::FindChunk(long, bool) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/mem-pool.cc:140:45
#2 0x1f44c37 in unsigned char* impala::MemPool::Allocate<false>(long, int) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/mem-pool.h:284:9
#3 0x1f44c37 in impala::MemPool::Allocate(long) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/mem-pool.h:107
#4 0x25ad32c in impala::HdfsParquetTableWriter::BaseColumnWriter::FinalizeCurrentPage() /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-parquet-table-writer.cc:797:61
#5 0x25bbdea in impala::HdfsParquetTableWriter::BaseColumnWriter::AppendRow(impala::TupleRow*) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-parquet-table-writer.cc:590:5
#6 0x25b36cc in impala::HdfsParquetTableWriter::AppendRows(impala::RowBatch*, std::vector<int, std::allocator<int> > const&, bool*) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-parquet-table-writer.cc:1124:7
#7 0x251ddec in impala::HdfsTableSink::WriteRowsToPartition(impala::RuntimeState*, impala::RowBatch*, std::pair<std::unique_ptr<impala::OutputPartition, std::default_delete<impala::OutputPartition> >, std::vector<int, std::allocator<int> > >*) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-table-sink.cc:256:5
#8 0x2522f01 in impala::HdfsTableSink::Send(impala::RuntimeState*, impala::RowBatch*) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/hdfs-table-sink.cc:590:5
#9 0x1fc07ad in impala::FragmentInstanceState::ExecInternal() /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/fragment-instance-state.cc:316:5
#10 0x1fbbd5c in impala::FragmentInstanceState::Exec() /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/fragment-instance-state.cc:95:14
#11 0x1fd5f94 in impala::QueryState::ExecFInstance(impala::FragmentInstanceState*) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/query-state.cc:478:24
#12 0x1cdef96 in boost::function0<void>::operator()() const /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14
#13 0x23ac27e in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/util/thread.cc:359:3
#14 0x23b7708 in void boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int) /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9
#15 0x23b755b in boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >::operator()() /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27
#16 0x39a11e9 in thread_proxy (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/impalad+0x39a11e9)

Thread T82364 created by T82363 here:
#0 0x164420d in __interceptor_pthread_create /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/asan_interceptors.cc:317
#1 0x39a05c9 in boost::thread::start_thread_noexcept() (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/impalad+0x39a05c9)
#2 0x45e0360d (<unknown module>)

Thread T82363 created by T207 here:
#0 0x164420d in __interceptor_pthread_create /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/asan_interceptors.cc:317
#1 0x39a05c9 in boost::thread::start_thread_noexcept() (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/impalad+0x39a05c9)
#2 0x45e0360d (<unknown module>)

Thread T207 created by T199 here:
#0 0x164420d in __interceptor_pthread_create /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/asan_interceptors.cc:317
#1 0x39a05c9 in boost::thread::start_thread_noexcept() (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/impalad+0x39a05c9)
#2 0x45e0360d (<unknown module>)

Thread T199 created by T198 here:
#0 0x164420d in __interceptor_pthread_create /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/asan_interceptors.cc:317
#1 0x39a05c9 in boost::thread::start_thread_noexcept() (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/impalad+0x39a05c9)
#2 0x45e0360d (<unknown module>)

Thread T198 created by T0 here:
#0 0x164420d in __interceptor_pthread_create /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/asan_interceptors.cc:317
#1 0x39a05c9 in boost::thread::start_thread_noexcept() (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/impalad+0x39a05c9)
#2 0x45e0360d (<unknown module>)

SUMMARY: AddressSanitizer: use-after-poison /data/jenkins/workspace/impala-toolchain-package-build/label/impala-toolchnbld-cent70-ec2-c3-4xl-ondem/toolchain/source/llvm/llvm-5.0.1.src-p1/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:738 in __interceptor_memcpy
Shadow bytes around the buggy address:
0x0fdfa6254180: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
0x0fdfa6254190: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
0x0fdfa62541a0: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
0x0fdfa62541b0: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
0x0fdfa62541c0: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
=>0x0fdfa62541d0: f7 f7 f7 f7 f7 f7 f7 f7[f7]f7 f7 f7 f7 f7 f7 f7
0x0fdfa62541e0: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
0x0fdfa62541f0: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
0x0fdfa6254200: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
0x0fdfa6254210: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
0x0fdfa6254220: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07 
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==117268==ABORTING{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org