You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@impala.apache.org by 烟圈儿 <12...@qq.com> on 2019/03/04 06:28:05 UTC
回复：[jira] [Created] (IMPALA-8274) Missing update to index intoprofiles vector in Coordinator::BackendState::ApplyExecStatusReport()
修补这个bug 需要修改底层源码么？
------------------ 原始邮件 ------------------
发件人: "Michael Ho (JIRA)"<ji...@apache.org>;
发送时间: 2019年3月4日(星期一) 上午8:41
收件人: "issues"<is...@impala.apache.org>;
主题: [jira] [Created] (IMPALA-8274) Missing update to index intoprofiles vector in Coordinator::BackendState::ApplyExecStatusReport()
Michael Ho created IMPALA-8274:
----------------------------------
Summary: Missing update to index into profiles vector in Coordinator::BackendState::ApplyExecStatusReport()
Key: IMPALA-8274
URL: https://issues.apache.org/jira/browse/IMPALA-8274
Project: IMPALA
Issue Type: Bug
Components: Distributed Exec
Reporter: Michael Ho
Assignee: Michael Ho
{{idx}} isn't updated in case we skip a duplicated or stale duplicated update of a fragment instance. As a result, we may end up passing the wrong profile to {{instance_stats->Update()}}. This may lead to random crashes in {{Coordinator::BackendState::InstanceStats::Update}}.
{noformat}
int idx = 0;
const bool has_profile = thrift_profiles.profile_trees.size() > 0;
TRuntimeProfileTree empty_profile;
for (const FragmentInstanceExecStatusPB& instance_exec_status :
backend_exec_status.instance_exec_status()) {
int64_t report_seq_no = instance_exec_status.report_seq_no();
int instance_idx = GetInstanceIdx(instance_exec_status.fragment_instance_id());
DCHECK_EQ(instance_stats_map_.count(instance_idx), 1);
InstanceStats* instance_stats = instance_stats_map_[instance_idx];
int64_t last_report_seq_no = instance_stats->last_report_seq_no_;
DCHECK(instance_stats->exec_params_.instance_id ==
ProtoToQueryId(instance_exec_status.fragment_instance_id()));
// Ignore duplicate or out-of-order messages.
if (report_seq_no <= last_report_seq_no) {
VLOG_QUERY << Substitute("Ignoring stale update for query instance $0 with "
"seq no $1", PrintId(instance_stats->exec_params_.instance_id), report_seq_no);
continue; <<--- // XXX bad
}
DCHECK(!instance_stats->done_);
DCHECK(!has_profile || idx < thrift_profiles.profile_trees.size());
const TRuntimeProfileTree& profile =
has_profile ? thrift_profiles.profile_trees[idx++] : empty_profile;
instance_stats->Update(instance_exec_status, profile, exec_summary,
scan_range_progress);
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: [jira] [Created] (IMPALA-8274) Missing update to index
intoprofiles vector in Coordinator::BackendState::ApplyExecStatusReport()
Posted by Quanlong Huang <hu...@gmail.com>.
Hi,
Welcome to the Impala community! The tickets you mentioned are not easy for
the beginning.
欢迎来到Impala社区！你列的这两个ticket有点难，并不适合上手。
If you are looking for a task to start, you can pick one from this list:
https://issues.apache.org/jira/browse/IMPALA-7686?jql=project%20%3D%20IMPALA%20AND%20status%20%3D%20Open%20AND%20labels%20%3D%20newbie%20AND%20assignee%20in%20(EMPTY)
如果你想找一个任务来入门的话，可以在上面这个列表里找。
Besides, please use English in the community. Most of the people here can't
read Chinese. If you have any troubles, feel free to contact me:
huangquanlong@gmail.com
另外，在社区里请用英语。这里很多人看不懂中文。如果你觉得很麻烦，也可以用上面的邮箱联系我。
Thanks,
Quanlong
On Mon, Mar 4, 2019 at 2:38 PM 烟圈儿 <12...@qq.com> wrote:
> 修补这个bug 需要修改底层源码么？
>
>
>
>
> ------------------ 原始邮件 ------------------
> 发件人: "Michael Ho (JIRA)"<ji...@apache.org>;
> 发送时间: 2019年3月4日(星期一) 上午8:41
> 收件人: "issues"<is...@impala.apache.org>;
>
> 主题: [jira] [Created] (IMPALA-8274) Missing update to index intoprofiles
> vector in Coordinator::BackendState::ApplyExecStatusReport()
>
>
>
> Michael Ho created IMPALA-8274:
> ----------------------------------
>
> Summary: Missing update to index into profiles vector in
> Coordinator::BackendState::ApplyExecStatusReport()
> Key: IMPALA-8274
> URL: https://issues.apache.org/jira/browse/IMPALA-8274
> Project: IMPALA
> Issue Type: Bug
> Components: Distributed Exec
> Reporter: Michael Ho
> Assignee: Michael Ho
>
>
> {{idx}} isn't updated in case we skip a duplicated or stale duplicated
> update of a fragment instance. As a result, we may end up passing the wrong
> profile to {{instance_stats->Update()}}. This may lead to random crashes in
> {{Coordinator::BackendState::InstanceStats::Update}}.
>
> {noformat}
> int idx = 0;
> const bool has_profile = thrift_profiles.profile_trees.size() > 0;
> TRuntimeProfileTree empty_profile;
> for (const FragmentInstanceExecStatusPB& instance_exec_status :
> backend_exec_status.instance_exec_status()) {
> int64_t report_seq_no = instance_exec_status.report_seq_no();
> int instance_idx =
> GetInstanceIdx(instance_exec_status.fragment_instance_id());
> DCHECK_EQ(instance_stats_map_.count(instance_idx), 1);
> InstanceStats* instance_stats = instance_stats_map_[instance_idx];
> int64_t last_report_seq_no = instance_stats->last_report_seq_no_;
> DCHECK(instance_stats->exec_params_.instance_id ==
> ProtoToQueryId(instance_exec_status.fragment_instance_id()));
> // Ignore duplicate or out-of-order messages.
> if (report_seq_no <= last_report_seq_no) {
> VLOG_QUERY << Substitute("Ignoring stale update for query instance
> $0 with "
> "seq no $1", PrintId(instance_stats->exec_params_.instance_id),
> report_seq_no);
> continue; <<--- // XXX bad
> }
>
> DCHECK(!instance_stats->done_);
> DCHECK(!has_profile || idx < thrift_profiles.profile_trees.size());
> const TRuntimeProfileTree& profile =
> has_profile ? thrift_profiles.profile_trees[idx++] : empty_profile;
> instance_stats->Update(instance_exec_status, profile, exec_summary,
> scan_range_progress);
> {noformat}
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v7.6.3#76005)