You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@impala.apache.org by "Matthew Jacobs (Code Review)" <ge...@cloudera.org> on 2016/06/17 01:35:01 UTC
[Impala-CR](cdh5-trunk) IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
Matthew Jacobs has uploaded a new change for review.
http://gerrit.cloudera.org:8080/3397
Change subject: IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
......................................................................
IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
We've seen a number of issues that appear to be a race
around the RuntimeProfile handling on the coordinator. We
don't have a consistent repro and haven't been able to find
the root cause of the crashes/hangs, but we have seen a
number of stacks from crashes that include the following
frames:
...
RuntimeProfile::ComputeTimeInProfile
RuntimeProfile::UpdateAverage
Coordinator::UpdateAverageProfile
Coordinator::UpdateFragmentExecStatus
...
This patch fixes a case where we iterate over children_
without holding the lock. I can't prove that this causes the
hangs but we certainly should fix this regardless.
Change-Id: Ibb40ebfa2e58ecaa8668d30ad92003fc75483517
---
M be/src/util/runtime-profile.cc
1 file changed, 5 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/97/3397/1
--
To view, visit http://gerrit.cloudera.org:8080/3397
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibb40ebfa2e58ecaa8668d30ad92003fc75483517
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Matthew Jacobs <mj...@cloudera.com>
[Impala-CR](cdh5-trunk) IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
Posted by "Matthew Jacobs (Code Review)" <ge...@cloudera.org>.
Matthew Jacobs has posted comments on this change.
Change subject: IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
......................................................................
Patch Set 3: Code-Review+2
rebase, carrying Dan's +2
--
To view, visit http://gerrit.cloudera.org:8080/3397
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibb40ebfa2e58ecaa8668d30ad92003fc75483517
Gerrit-PatchSet: 3
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-HasComments: No
[Impala-CR](cdh5-trunk) IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
Posted by "Internal Jenkins (Code Review)" <ge...@cloudera.org>.
Internal Jenkins has posted comments on this change.
Change subject: IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
......................................................................
Patch Set 3: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/3397
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibb40ebfa2e58ecaa8668d30ad92003fc75483517
Gerrit-PatchSet: 3
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-HasComments: No
[Impala-CR](cdh5-trunk) IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change.
Change subject: IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
......................................................................
Patch Set 2: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/3397
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibb40ebfa2e58ecaa8668d30ad92003fc75483517
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-HasComments: No
[Impala-CR](cdh5-trunk) IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
Posted by "Matthew Jacobs (Code Review)" <ge...@cloudera.org>.
Matthew Jacobs has uploaded a new patch set (#2).
Change subject: IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
......................................................................
IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
We've seen a number of issues that appear to be a race
around the RuntimeProfile handling on the coordinator. We
don't have a consistent repro and haven't been able to find
the root cause of the crashes/hangs, but we have seen a
number of stacks from crashes that include the following
frames:
...
RuntimeProfile::ComputeTimeInProfile
RuntimeProfile::UpdateAverage
Coordinator::UpdateAverageProfile
Coordinator::UpdateFragmentExecStatus
...
This patch fixes a case where we iterate over children_
without holding the lock. I can't prove that this causes the
hangs but we certainly should fix this regardless.
Testing: Passed several exhaustive test runs successfully.
Change-Id: Ibb40ebfa2e58ecaa8668d30ad92003fc75483517
---
M be/src/util/runtime-profile.cc
1 file changed, 5 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/97/3397/2
--
To view, visit http://gerrit.cloudera.org:8080/3397
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ibb40ebfa2e58ecaa8668d30ad92003fc75483517
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Matthew Jacobs <mj...@cloudera.com>
[Impala-CR](cdh5-trunk) IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
Posted by "Internal Jenkins (Code Review)" <ge...@cloudera.org>.
Internal Jenkins has posted comments on this change.
Change subject: IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
......................................................................
Patch Set 3: Verified-1
Build failed: http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge/2591/
--
To view, visit http://gerrit.cloudera.org:8080/3397
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibb40ebfa2e58ecaa8668d30ad92003fc75483517
Gerrit-PatchSet: 3
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-HasComments: No
[Impala-CR](cdh5-trunk) IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
Posted by "Internal Jenkins (Code Review)" <ge...@cloudera.org>.
Internal Jenkins has submitted this change and it was merged.
Change subject: IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
......................................................................
IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
We've seen a number of issues that appear to be a race
around the RuntimeProfile handling on the coordinator. We
don't have a consistent repro and haven't been able to find
the root cause of the crashes/hangs, but we have seen a
number of stacks from crashes that include the following
frames:
...
RuntimeProfile::ComputeTimeInProfile
RuntimeProfile::UpdateAverage
Coordinator::UpdateAverageProfile
Coordinator::UpdateFragmentExecStatus
...
This patch fixes a case where we iterate over children_
without holding the lock. I can't prove that this causes the
hangs but we certainly should fix this regardless.
Testing: Passed several exhaustive test runs successfully.
Change-Id: Ibb40ebfa2e58ecaa8668d30ad92003fc75483517
Reviewed-on: http://gerrit.cloudera.org:8080/3397
Reviewed-by: Matthew Jacobs <mj...@cloudera.com>
Tested-by: Internal Jenkins
---
M be/src/util/runtime-profile.cc
1 file changed, 5 insertions(+), 2 deletions(-)
Approvals:
Matthew Jacobs: Looks good to me, approved
Internal Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/3397
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ibb40ebfa2e58ecaa8668d30ad92003fc75483517
Gerrit-PatchSet: 4
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-CR](cdh5-trunk) IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
Posted by "Dan Hecht (Code Review)" <ge...@cloudera.org>.
Dan Hecht has posted comments on this change.
Change subject: IMPALA-3757: Add missing lock in RuntimeProfile::ComputeTimeInProfile
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/3397
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibb40ebfa2e58ecaa8668d30ad92003fc75483517
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Matthew Jacobs <mj...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-HasComments: No