You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/01/15 07:29:01 UTC
[jira] [Commented] (IMPALA-9865) Utility to pretty-print thrift
profiles at various levels
[ https://issues.apache.org/jira/browse/IMPALA-9865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17265776#comment-17265776 ]
ASF subversion and git services commented on IMPALA-9865:
---------------------------------------------------------
Commit 5fddbb569a68bd438d9f4847481c8db1fce5b67c in impala's branch refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=5fddbb5 ]
IMPALA-9865: part 2/2: add verbosity to profile tool
Adds a --profile_verbosity option for impala-profile-tool with
the following levels:
* 0: minimal
* 1: legacy - matches old output, this is the default still
* 2: default - basic descriptive stats, used for V2 profile.
* 3: extended
* 4: full
This will help with transition to the V2 profile because we
can have a nice, high-level, readable text profile by default
with the option to produce more detailed profiles and alternate
views of the profile from the thrift profile.
Use the profile version in impala-profile-tool to dump the
more verbose output for the V2 profile while preserving the
same output for the legacy profile.
Reduce verbosity of v2 profile output - only include mean/min/max
by default. I intend to refine the output at the different
verbosity levels for the v2 profiles further as part of IMPALA-9382,
it is still fairly noisy.
Fix output with/without gen_experimental_profile - there
was a small difference in that the summary stats were not
output in the averaged profile.
Testing:
* Add an end-to-end test that generates output for a small
profile log and compares against expected files.
* Tweak other profile tests to reflect changes to output.
Change-Id: I82618a813e29af7996dfaed78873b2a73bc0231d
Reviewed-on: http://gerrit.cloudera.org:8080/16881
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Utility to pretty-print thrift profiles at various levels
> ---------------------------------------------------------
>
> Key: IMPALA-9865
> URL: https://issues.apache.org/jira/browse/IMPALA-9865
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Fix For: Impala 4.0
>
> Attachments: image-2020-11-20-09-25-08-082.png
>
>
> The prototyping work in IMPALA-9382 revealed some hard trade-offs between having a full-fidelity text profile and readability.
> We want to have a text profile with less information by default so that it is more readable, and rely on the thrift profile for more detailed debugging. This would be easier if we provided a utility that can pretty-print a thrift profile at different levels of detail.
> This JIRA is to reduce the default level of pretty-printing for aggregated profiles, but provide a utility that can dump both the basic and full versions. My thought is to start off with the same 4 levels as explain, but maybe only implement 2 levels to start off with - basic and extended.
> The utility should be able to handle the same cases as bin/parse-thrift-profile.py (profile log and single profile in a file) and maybe print only a specified query from a profile log. We can use the DeserializeFromArchiveString() method that was removed in IMPALA-9381, then pretty-print the deserialised profile.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org