You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/03/10 16:46:00 UTC
[jira] [Commented] (IMPALA-10555) Hit DCHECK in
TmpFileGroup::RecoverWriteError
[ https://issues.apache.org/jira/browse/IMPALA-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17298961#comment-17298961 ]
ASF subversion and git services commented on IMPALA-10555:
----------------------------------------------------------
Commit b2088b30d7a4bc46878ca4815a89561551c05b15 in impala's branch refs/heads/master from Yida Wu
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b2088b3 ]
IMPALA-10555: Fix Hit DCHECK in TmpFileGroup::RecoverWriteError
The DCHECK error happens when there is an IO error during the spilling
process if the scratch directory is in a remote filesystem and doing
an error recovery(rewrite). Because currently the DCHECK only consider
the file number of local scratch files, it leads to a file number
requirement mismatch in the DCHECK.
Because the implementation of spilling to the local fs and the remote fs
are quite different, for simplify, we don't recover write error
for spilling to a remote fs in the current version. Instead, the errors
generated during spilling to remote would be returned directly to the
upper layer. So, we avoid the DCHECK logic for spilling to remote.
Tests:
* Added a unit test: TmpFileMgrTest::TestRemoteRemoveBuffer.
* Ran Unit Tests:
$IMPALA_HOME/be/build/latest/runtime/tmp-file-mgr-test
Change-Id: Ifd9aea4bf2fff634ea9a30bf6e87987be4e1c611
Reviewed-on: http://gerrit.cloudera.org:8080/17140
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Hit DCHECK in TmpFileGroup::RecoverWriteError
> ---------------------------------------------
>
> Key: IMPALA-10555
> URL: https://issues.apache.org/jira/browse/IMPALA-10555
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.0
> Reporter: Yida Wu
> Assignee: Yida Wu
> Priority: Major
>
> Hit DCHECK in TmpFileGroup::RecoverWriteError() when an IO error happens during spilling to a remote path.
> Stacks:
> #0 0x00007f46fad58fb7 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x00007f46fad5a921 in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2 0x0000000005d1f4b4 in google::DumpStackTraceAndExit() ()
> #3 0x0000000005d148ad in google::LogMessage::Fail() ()
> #4 0x0000000005d1619d in google::LogMessage::SendToLog() ()
> #5 0x0000000005d1420b in google::LogMessage::Flush() ()
> #6 0x0000000005d17e09 in google::LogMessageFatal::~LogMessageFatal() ()
> #7 0x00000000036ebd84 in impala::TmpFileGroup::RecoverWriteError (this=0x7ffcdb4a9610, handle=0xd622d00, write_status=...)
> at /home/yida/Impala/be/src/runtime/tmp-file-mgr.cc:1383
> #8 0x00000000036eb88f in impala::TmpFileGroup::WriteComplete (this=0x7ffcdb4a9610, handle=0xd622d00, write_status=...)
> at /home/yida/Impala/be/src/runtime/tmp-file-mgr.cc:1355
> #9 0x00000000036e9054 in impala::TmpFileGroup::<lambda(const impala::Status&)>::operator()(const impala::Status &) const (__closure=0x7f46cb8d64e0, write_status=...)
> at /home/yida/Impala/be/src/runtime/tmp-file-mgr.cc:1188
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org