[jira] [Commented] (IMPALA-5031) UBSAN clean and method for testing UBSAN cleanliness

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16675800#comment-16675800 ] 

ASF subversion and git services commented on IMPALA-5031:
---------------------------------------------------------

Commit a03e22011dfc6c93818e923d0dc29e6f61fefbe9 in impala's branch refs/heads/master from [~jbapple]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=a03e220 ]

IMPALA-5031: memcpy cannot take null arguments

This patch fixes UBSAN "null pointer passed as argument" errors in
data loading. These are undefined behavior according to "7.1.4 Use of
library functions" in the C99 standard (which is included in C++14 in
section [intro.refs]):

    If an argument to a function has an invalid value (such as a value
    outside the domain of the function, or a pointer outside the
    address space of the program, or a null pointer, or a pointer to
    non-modifiable storage when the corresponding parameter is not
    const-qualified) or a type (after promotion) not expected by a
    function with variable number of arguments, the behavior is
    undefined.

The interesting parts of the backtraces for the errors fixed in this
patch are below:

    runtime/string-buffer.h:54:12: runtime error: null pointer passed as argument 1, which is declared to never be null
    /usr/include/string.h:43:45: note: nonnull attribute specified here
    StringBuffer::Append(char const*, long) runtime/string-buffer.h:54:5
    ColumnStatsBase::CopyToBuffer(StringBuffer*, StringValue*) exec/parquet-column-stats.cc:151:51
    ColumnStats<StringValue>::MaterializeStringValuesToInternalBuffers() exec/parquet-column-stats.inline.h:237:70
    HdfsParquetTableWriter::BaseColumnWriter::MaterializeStatsValues() exec/hdfs-parquet-table-writer.cc:149:63
    HdfsParquetTableWriter::AppendRows(RowBatch*, vector<int> const&, bool*) exec/hdfs-parquet-table-writer.cc:1129:53
    HdfsTableSink::WriteRowsToPartition(RuntimeState*, RowBatch*, pair<unique_ptr<OutputPartition, default_delete<OutputPartition> >, vector<int, all> > >*) exec/hdfs-table-sink.cc:256:71
    HdfsTableSink::Send(RuntimeState*, RowBatch*) exec/hdfs-table-sink.cc:591:45

    util/streaming-sampler.h:111:22: runtime error: null pointer passed as argument 2, which is declared to never be null
    /usr/include/string.h:43:45: note: nonnull attribute specified here
    StreamingSampler<long, 64>::SetSamples(int, vector<long> const&) util/streaming-sampler.h:111:5
    RuntimeProfile::Update(vector<TRuntimeProfileNode> const&, int*) util/runtime-profile.cc:313:30
    RuntimeProfile::Update(TRuntimeProfileTree const&) util/runtime-profile.cc:246:3
    Coordinator::BackendState::InstanceStats::Update(TFragmentInstanceExecStatus const&, Coordinator::ExecSummary*, ProgressUpdater*) runtime/coordinator-backend-state.cc:474:13
    Coordinator::BackendState::ApplyExecStatusReport(TReportExecStatusParams const&, Coordinator::ExecSummary*, ProgressUpdater*) runtime/coordinator-backend-state.cc:287:21
    Coordinator::UpdateBackendExecStatus(TReportExecStatusParams const&) runtime/coordinator.cc:679:22
    ClientRequestState::UpdateBackendExecStatus(TReportExecStatusParams const&) service/client-request-state.cc:1254:18
    ImpalaServer::ReportExecStatus(TReportExecStatusResult&, TReportExecStatusParams const&) service/impala-server.cc:1343:18
    ImpalaInternalService::ReportExecStatus(TReportExecStatusResult&, TReportExecStatusParams const&) service/impala-internal-service.cc:87:19

Change-Id: Ib9acc8c32409e67253a987eb3d1fd7d921efcb51
Reviewed-on: http://gerrit.cloudera.org:8080/11812
Reviewed-by: Jim Apple <jb...@apache.org>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> UBSAN clean and method for testing UBSAN cleanliness
> ----------------------------------------------------
>
>                 Key: IMPALA-5031
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5031
>             Project: IMPALA
>          Issue Type: Task
>          Components: Backend, Infrastructure
>    Affects Versions: Impala 2.9.0
>            Reporter: Jim Apple
>            Assignee: Jim Apple
>            Priority: Minor
>
> http://releases.llvm.org/3.8.0/tools/clang/docs/UndefinedBehaviorSanitizer.html builds are supported after https://gerrit.cloudera.org/#/c/6186/, but Impala's test suite triggers many errors under UBSAN. Those errors should be fixed and then there should be a way to run the test suite under UBSAN and fail if there were any errors detected.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org