You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2018/10/29 20:29:00 UTC
[jira] [Resolved] (IMPALA-2680) Align un-aligned memory copies
[ https://issues.apache.org/jira/browse/IMPALA-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong resolved IMPALA-2680.
-----------------------------------
Resolution: Won't Fix
I don't think this actually makes sense, we've generally see that it's better to have unpadded, denser rows than to try to align things.
> Align un-aligned memory copies
> ------------------------------
>
> Key: IMPALA-2680
> URL: https://issues.apache.org/jira/browse/IMPALA-2680
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.2
> Reporter: Mostafa Mokhtar
> Assignee: Youwei Wang
> Priority: Minor
> Labels: performance
>
> Several operators do un-aligned memory copies which are significantly slower than aligned copies.
> These are the top call stacks
> {code}
> 1 of 31: 35.2% (1.000s of 2.844s)
> libc.so.6!__memcpy_sse2_unaligned - memcpy-sse2-unaligned.S
> impalad!memcpy+0x1d - string3.h:51
> impalad!impala::BufferedTupleStream::DeepCopyInternal<(bool)0>+0x80 - buffered-tuple-stream-ir.cc:83
> libc.so.6![Unknown stack frame(s)] - [Unknown]:[Unknown]
> impalad!impala::PartitionedHashJoinNode::ConstructBuildSide+0xe1 - partitioned-hash-join-node.cc:580
> impalad!impala::BlockingJoinNode::BuildSideThread+0x7c - blocking-join-node.cc:143
> {code}
> {code}
> Data Of Interest (CPU Metrics)
> 2 of 31: 14.3% (0.408s of 2.844s)
> libc.so.6!__memcpy_sse2_unaligned - memcpy-sse2-unaligned.S
> impalad!memcpy+0xa - string3.h:51
> impalad!impala::Tuple::DeepCopy+0x1d - tuple.cc:116
> impalad!impala::RowBatch::SerializeInternal+0x142 - row-batch.cc:285
> impalad!impala::RowBatch::Serialize+0x1c4 - row-batch.cc:195
> impalad!impala::RowBatch::Serialize+0x25 - row-batch.cc:168
> impalad!impala::DataStreamSender::SerializeBatch+0x12c - data-stream-sender.cc:463
> impalad!impala::DataStreamSender::Send+0x266 - data-stream-sender.cc:411
> impalad!impala::PlanFragmentExecutor::OpenInternal+0x327 - plan-fragment-executor.cc:355
> impalad!impala::PlanFragmentExecutor::Open+0x26b - plan-fragment-executor.cc:320
> impalad!impala::FragmentMgr::FragmentExecState::Exec+0x19 - fragment-exec-state.cc:50
> impalad!impala::FragmentMgr::FragmentExecThread+0x50 - fragment-mgr.cc:83
> impalad!boost::function0<void>::operator()+0x18 - function_template.hpp:767
> {code}
> {code}
> Data Of Interest (CPU Metrics)
> 3 of 31: 7.7% (0.220s of 2.844s)
> libc.so.6!__memcpy_sse2_unaligned - memcpy-sse2-unaligned.S
> impalad!memcpy+0x12 - string3.h:51
> impalad!impala::RowBatch::CopyRow+0 - row-batch.h:182
> impalad!impala::ExchangeNode::GetNext+0x148 - exchange-node.cc:135
> impalad!impala::PartitionedHashJoinNode::ProcessBuildInput+0x3bb - partitioned-hash-join-node.cc:628
> impalad!impala::PartitionedHashJoinNode::ConstructBuildSide+0xe1 - partitioned-hash-join-node.cc:580
> impalad!impala::BlockingJoinNode::BuildSideThread+0x7c - blocking-join-node.cc:143
> impalad!boost::function0<void>::operator()+0x18 - function_template.hpp:767
> {code}
> {code}
> Data Of Interest (CPU Metrics)
> 4 of 31: 7.5% (0.212s of 2.844s)
> libc.so.6!__memcpy_sse2_unaligned - memcpy-sse2-unaligned.S
> impalad!memcpy+0x7 - string3.h:51
> impalad!apache::thrift::transport::TBufferBase::write+0x18 - TBufferTransports.h:97
> impalad!apache::thrift::transport::TVirtualTransport<apache::thrift::transport::TBufferedTransport, apache::thrift::transport::TBufferBase>::write_virt+0x1 - TVirtualTransport.h:103
> impalad!apache::thrift::transport::TTransport::write+0xf - TTransport.h:158
> impalad!apache::thrift::protocol::TBinaryProtocolT<apache::thrift::transport::TTransport>::writeI32+0x6 - TBinaryProtocol.tcc:155
> impalad!apache::thrift::protocol::TVirtualProtocol<apache::thrift::protocol::TBinaryProtocolT<apache::thrift::transport::TTransport>, apache::thrift::protocol::TProtocolDefaults>::writeI32_virt+0x8 - TVirtualProtocol.h:405
> impalad!apache::thrift::protocol::TProtocol::writeI32+0x12 - TProtocol.h:448
> impalad!impala::TRowBatch::write+0x138 - Results_types.cpp:161
> impalad!impala::TTransmitDataParams::write+0x128 - ImpalaInternalService_types.cpp:2914
> impalad!impala::ImpalaInternalService_TransmitData_pargs::write+0x4a - ImpalaInternalService.cpp:571
> impalad!impala::ImpalaInternalServiceClient::send_TransmitData+0x6e - ImpalaInternalService.cpp:862
> impalad!impala::ImpalaInternalServiceClient::TransmitData+0x13 - ImpalaInternalService.cpp:851
> impalad!impala::ClientConnection<impala::ImpalaInternalServiceClient>::DoRpc<void (impala::TTransmitDataResult&, impala::TTransmitDataParams const&) impala::ImpalaInternalServiceClient::*, impala::TTransmitDataParams, impala::TTransmitDataResult>+0x4b - client-cache.h:229
> impalad!impala::DataStreamSender::Channel::TransmitDataHelper+0x58f - data-stream-sender.cc:205
> impalad!impala::DataStreamSender::Channel::TransmitData+0x10 - data-stream-sender.cc:177
> impalad!boost::function2<void, int, impala::TRowBatch*& const>::operator()+0x26d - function_template.hpp:767
> {code}
> {code}
> Data Of Interest (CPU Metrics)
> 5 of 31: 7.3% (0.208s of 2.844s)
> libc.so.6!__memcpy_sse2_unaligned - memcpy-sse2-unaligned.S
> impalad!memcpy+0x4 - string3.h:51
> impalad!apache::thrift::transport::TBufferBase::readAll+0x17 - TBufferTransports.h:78
> impalad!apache::thrift::transport::TBufferedTransport::readAll+0 - TBufferTransports.h:279
> impalad!apache::thrift::transport::TVirtualTransport<apache::thrift::transport::TBufferedTransport, apache::thrift::transport::TBufferBase>::readAll_virt+0x5 - TVirtualTransport.h:99
> impalad!apache::thrift::transport::TTransport::readAll+0x28 - TTransport.h:126
> impalad!apache::thrift::protocol::TBinaryProtocolT<apache::thrift::transport::TTransport>::readI32+0 - TBinaryProtocol.tcc:375
> impalad!apache::thrift::protocol::TVirtualProtocol<apache::thrift::protocol::TBinaryProtocolT<apache::thrift::transport::TTransport>, apache::thrift::protocol::TProtocolDefaults>::readI32_virt+0x1 - TVirtualProtocol.h:503
> impalad!apache::thrift::protocol::TProtocol::readI32+0x11 - TProtocol.h:606
> impalad!impala::TRowBatch::read+0x2d8 - Results_types.cpp:78
> impalad!impala::TTransmitDataParams::read+0x1cc - ImpalaInternalService_types.cpp:2855
> impalad!impala::ImpalaInternalService_TransmitData_args::read+0xa4 - ImpalaInternalService.cpp:531
> impalad!impala::ImpalaInternalServiceProcessor::process_TransmitData+0x1c3 - ImpalaInternalService.cpp:1101
> impalad!impala::ImpalaInternalServiceProcessor::dispatchCall+0xdd - ImpalaInternalService.cpp:922
> impalad!apache::thrift::TDispatchProcessor::process+0x70 - TDispatchProcessor.h:121
> impalad!apache::thrift::server::TThreadedServer::Task::run+0x212 - TThreadedServer.cpp:70
> impalad!impala::ThriftThread::RunRunnable+0x68 - thrift-thread.cc:61
> {code}
> {code}
> Data Of Interest (CPU Metrics)
> 8 of 31: 2.8% (0.080s of 2.844s)
> libc.so.6!__memcpy_sse2_unaligned - memcpy-sse2-unaligned.S
> libstdc++.so.6!std::string::_M_replace_safe+0x3a - [Unknown]:[Unknown]
> impalad!apache::thrift::protocol::TBinaryProtocolT<apache::thrift::transport::TTransport>::readStringBody<std::string>+0x9e - TBinaryProtocol.tcc:459
> impalad!readString<std::basic_string<char> >+0x3c - TBinaryProtocol.tcc:412
> impalad!apache::thrift::protocol::TVirtualProtocol<apache::thrift::protocol::TBinaryProtocolT<apache::thrift::transport::TTransport>, apache::thrift::protocol::TProtocolDefaults>::readString_virt+0x1 - TVirtualProtocol.h:515
> impalad!apache::thrift::protocol::TProtocol::readString+0x12 - TProtocol.h:621
> impalad!impala::TRowBatch::read+0x153 - Results_types.cpp:89
> impalad!impala::TTransmitDataParams::read+0x1cc - ImpalaInternalService_types.cpp:2855
> impalad!impala::ImpalaInternalService_TransmitData_args::read+0xa4 - ImpalaInternalService.cpp:531
> impalad!impala::ImpalaInternalServiceProcessor::process_TransmitData+0x1c3 - ImpalaInternalService.cpp:1101
> impalad!impala::ImpalaInternalServiceProcessor::dispatchCall+0xdd - ImpalaInternalService.cpp:922
> impalad!apache::thrift::TDispatchProcessor::process+0x70 - TDispatchProcessor.h:121
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org