You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Sahil Takiar (JIRA)" <ji...@apache.org> on 2019/07/15 18:05:00 UTC
[jira] [Created] (IMPALA-8764) Kudu data load failures due to
"Clock considered unsynchronized"
Sahil Takiar created IMPALA-8764:
------------------------------------
Summary: Kudu data load failures due to "Clock considered unsynchronized"
Key: IMPALA-8764
URL: https://issues.apache.org/jira/browse/IMPALA-8764
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.3.0
Reporter: Sahil Takiar
Dataload error:
{code}
03:08:38 03:08:38 Error executing impala SQL: Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql See: Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
{code}
Digging through the mini-cluster logs, I see that the Kudu tservers crashed with this error:
{code}
F0715 02:58:43.202059 649 hybrid_clock.cc:339] Check failed: _s.ok() unable to get current time with error bound: Service unavailable: could not read system time source: Error reading clock. Clock considered unsynchronized
*** Check failure stack trace: ***
Wrote minidump to Impala/testdata/cluster/cdh6/node-3/var/log/kudu/ts/minidumps/kudu-tserver/395e6bb9-9b2f-468e-4d37d898-74b96d61.dmp
Wrote minidump to Impala/testdata/cluster/cdh6/node-3/var/log/kudu/ts/minidumps/kudu-tserver/395e6bb9-9b2f-468e-4d37d898-74b96d61.dmp
*** Aborted at 1563184723 (unix time) try "date -d @1563184723" if you are using GNU date ***
PC: @ 0x7ff75ed631f7 __GI_raise
*** SIGABRT (@0x7d100000232) received by PID 562 (TID 0x7ff756c1e700) from PID 562; stack trace: ***
@ 0x7ff760b545e0 (unknown)
@ 0x7ff75ed631f7 __GI_raise
@ 0x7ff75ed648e8 __GI_abort
@ 0x1fb7309 kudu::AbortFailureFunction()
@ 0x9c054d google::LogMessage::Fail()
@ 0x9c240d google::LogMessage::SendToLog()
@ 0x9c0089 google::LogMessage::Flush()
@ 0x9c2eaf google::LogMessageFatal::~LogMessageFatal()
@ 0xc0c60e kudu::clock::HybridClock::WalltimeWithErrorOrDie()
@ 0xc0c67e kudu::clock::HybridClock::NowWithError()
@ 0xc0d4aa kudu::clock::HybridClock::NowForMetrics()
@ 0x9a29c0 kudu::FunctionGauge<>::WriteValue()
@ 0x1fb0dc0 kudu::Gauge::WriteAsJson()
@ 0x1fb3212 kudu::MetricEntity::WriteAsJson()
@ 0x1fb390e kudu::MetricRegistry::WriteAsJson()
@ 0xa856a3 kudu::server::DiagnosticsLog::LogMetrics()
@ 0xa8789a kudu::server::DiagnosticsLog::RunThread()
@ 0x1ff44d7 kudu::Thread::SuperviseThread()
@ 0x7ff760b4ce25 start_thread
@ 0x7ff75ee2634d __clone
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)