You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Tim Armstrong (Code Review)" <ge...@cloudera.org> on 2020/12/05 01:01:44 UTC
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16821
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
IMPALA-9865: part 1: basic profile log parser
This adds a utility that consumes the Impala profile log format from
stdin and pretty-prints the profiles.
It supports some basic filters - --query_id, --min_timestamp and
--max_timestamp.
If --gen_experimental_profile=true is set, it dumps the aggregated
part of the profile with the full output for the new experimental
profiles. In a future change, we should detect this based on
the profile version set.
This utility will be extended in future with more options, but
is already useful in that it can handle the new experimental
profile format and produce pretty-printed output consistent
with the Impala web UI and impala-shell.
Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
---
M be/src/util/CMakeLists.txt
A be/src/util/impala-profile-tool.cc
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
4 files changed, 114 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16821/1
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/7789/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 07 Dec 2020 22:29:44 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/16821/2/be/src/util/impala-profile-tool.cc
File be/src/util/impala-profile-tool.cc:
http://gerrit.cloudera.org:8080/#/c/16821/2/be/src/util/impala-profile-tool.cc@31
PS2, Line 31: // is pretty-printed to standard output.
> Add simple usage example in the doc maybe? like
Done
http://gerrit.cloudera.org:8080/#/c/16821/2/be/src/util/impala-profile-tool.cc@59
PS2, Line 59: getline(cin, line);
> Tried to run the parser against my local runtime profile log. It seems It a
Thanks, this seems to be a better way to use getline. I haven't done a lot of parsing using C++ streams so kinda learning as I go.
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 07 Dec 2020 22:04:44 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 6:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6764/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 11 Dec 2020 04:59:26 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 5: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 10 Dec 2020 19:47:19 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 6: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6764/
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 11 Dec 2020 10:37:31 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/16821/1/be/src/util/impala-profile-tool.cc
File be/src/util/impala-profile-tool.cc:
http://gerrit.cloudera.org:8080/#/c/16821/1/be/src/util/impala-profile-tool.cc@34
PS1, Line 34: // --query_id=<query id>: given an impala query ID, only process profiles with this query id
line too long (92 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Sat, 05 Dec 2020 01:02:34 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/7788/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 07 Dec 2020 22:27:08 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Riza Suminto, David Rorke, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16821
to look at the new patch set (#3).
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
IMPALA-9865: part 1: basic profile log parser
This adds a utility that consumes the Impala profile log format from
stdin and pretty-prints the profiles.
It supports some basic filters - --query_id, --min_timestamp and
--max_timestamp.
If --gen_experimental_profile=true is set, it dumps the aggregated
part of the profile with the full output for the new experimental
profiles. In a future change, we should detect this based on
the profile version set.
This utility will be extended in future with more options, but
is already useful in that it can handle the new experimental
profile format and produce pretty-printed output consistent
with the Impala web UI and impala-shell.
Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
---
M be/src/util/CMakeLists.txt
A be/src/util/impala-profile-tool.cc
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
4 files changed, 117 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16821/3
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 5:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6760/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 10 Dec 2020 19:47:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has removed a vote on this change.
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Removed Verified-1 by Impala Public Jenkins <im...@cloudera.com>
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 7: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 7
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 11 Dec 2020 23:17:41 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 4: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 07 Dec 2020 22:39:57 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 7:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6769/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 7
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 11 Dec 2020 17:39:27 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 5:
Hit IMPALA-10066
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 11 Dec 2020 04:58:49 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
IMPALA-9865: part 1: basic profile log parser
This adds a utility that consumes the Impala profile log format from
stdin and pretty-prints the profiles.
It supports some basic filters - --query_id, --min_timestamp and
--max_timestamp.
If --gen_experimental_profile=true is set, it dumps the aggregated
part of the profile with the full output for the new experimental
profiles. In a future change, we should detect this based on
the profile version set.
This utility will be extended in future with more options, but
is already useful in that it can handle the new experimental
profile format and produce pretty-printed output consistent
with the Impala web UI and impala-shell.
Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Reviewed-on: http://gerrit.cloudera.org:8080/16821
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/util/CMakeLists.txt
A be/src/util/impala-profile-tool.cc
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
4 files changed, 116 insertions(+), 0 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 8
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 5:
THanks, yeah, there's lots to do. I just added a JSON output here - https://gerrit.cloudera.org/#/c/16855/ and cleaned up some of the --help usage stuff.
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 10 Dec 2020 20:10:11 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 7:
PRecommit failure unrelated, need to triage to see what happened but restarting merge
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 7
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 11 Dec 2020 17:40:24 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/7780/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 1
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Sat, 05 Dec 2020 01:23:16 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Riza Suminto, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16821
to look at the new patch set (#2).
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
IMPALA-9865: part 1: basic profile log parser
This adds a utility that consumes the Impala profile log format from
stdin and pretty-prints the profiles.
It supports some basic filters - --query_id, --min_timestamp and
--max_timestamp.
If --gen_experimental_profile=true is set, it dumps the aggregated
part of the profile with the full output for the new experimental
profiles. In a future change, we should detect this based on
the profile version set.
This utility will be extended in future with more options, but
is already useful in that it can handle the new experimental
profile format and produce pretty-printed output consistent
with the Impala web UI and impala-shell.
Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
---
M be/src/util/CMakeLists.txt
A be/src/util/impala-profile-tool.cc
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
4 files changed, 115 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16821/2
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Riza Suminto, David Rorke, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16821
to look at the new patch set (#4).
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
IMPALA-9865: part 1: basic profile log parser
This adds a utility that consumes the Impala profile log format from
stdin and pretty-prints the profiles.
It supports some basic filters - --query_id, --min_timestamp and
--max_timestamp.
If --gen_experimental_profile=true is set, it dumps the aggregated
part of the profile with the full output for the new experimental
profiles. In a future change, we should detect this based on
the profile version set.
This utility will be extended in future with more options, but
is already useful in that it can handle the new experimental
profile format and produce pretty-printed output consistent
with the Impala web UI and impala-shell.
Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
---
M be/src/util/CMakeLists.txt
A be/src/util/impala-profile-tool.cc
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
4 files changed, 116 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16821/4
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/7781/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Sat, 05 Dec 2020 01:24:24 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 7: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 7
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 11 Dec 2020 17:39:26 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 2:
(2 comments)
> Uploaded patch set 2.
Hi Tim,
Thanks for writing this parser. This is very useful.
I just have 2 comment.
http://gerrit.cloudera.org:8080/#/c/16821/2/be/src/util/impala-profile-tool.cc
File be/src/util/impala-profile-tool.cc:
http://gerrit.cloudera.org:8080/#/c/16821/2/be/src/util/impala-profile-tool.cc@31
PS2, Line 31: // is pretty-printed to standard output.
Add simple usage example in the doc maybe? like
impala-profile-tool < impala_profile_log_1.1-1607057366897
http://gerrit.cloudera.org:8080/#/c/16821/2/be/src/util/impala-profile-tool.cc@59
PS2, Line 59: getline(cin, line);
Tried to run the parser against my local runtime profile log. It seems It always hit "Error reading line" when it reach EOF.
What if we move this getline as the loop condition? say
for (std::string line; std::getline(cin, line); ) {
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 2
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Comment-Date: Sat, 05 Dec 2020 02:48:40 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 4: Code-Review+2
This is really cool!
I'm already thinking about extensions to the behavior, and I think this will be very useful. Great to get a basic version in first.
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 10 Dec 2020 19:45:37 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 6: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 11 Dec 2020 04:59:25 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9865: part 1: basic profile log parser
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16821 )
Change subject: IMPALA-9865: part 1: basic profile log parser
......................................................................
Patch Set 5: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6760/
--
To view, visit http://gerrit.cloudera.org:8080/16821
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6178399addddc96e176f7067cc47347e51cda2f3
Gerrit-Change-Number: 16821
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: David Rorke <dr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Fri, 11 Dec 2020 01:19:08 +0000
Gerrit-HasComments: No