[jira] (IMPALA-15097) Show runtime filter effectiveness in plan-graph.py

["Aleksandr Efimov (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-15097 ]


    Aleksandr Efimov deleted comment on IMPALA-15097:
    -------------------------------------------

was (Author: JIRAUSER313444):
I’ve been working with Impala profile parsing/diagnostic tooling recently, and runtime filters are an area where we’ve hit a few practical edge cases. I’d like to make the first patch a bit more useful than only drawing dashed lines.

Proposed initial scope:
- mark runtime-filter consumer edges as dashed when the profile shows that the filter processed input but rejected nothing;
- show compact effectiveness counters on the graph when available, e.g. rows/files/row-groups rejected out of processed/total;
- read the counters from the scan-node runtime filter sections, while keeping the graph unchanged when the profile does not contain enough information;
- make sure the parser handles the current runtime filter table format, including the effective-targets column;
- add a focused regression test with a small profile snippet, and verify the output on the attached profile.

I’ll keep broader profile-graph improvements as follow-ups so this issue stays focused on runtime filter effectiveness.

> Show runtime filter effectiveness in plan-graph.py
> --------------------------------------------------
>
>                 Key: IMPALA-15097
>                 URL: https://issues.apache.org/jira/browse/IMPALA-15097
>             Project: IMPALA
>          Issue Type: Improvement
>            Reporter: Quanlong Huang
>            Assignee: Aleksandr Efimov
>            Priority: Major
>         Attachments: query-plan1.svg, rf_profile.txt
>
>
> When verbosity is larger than 0, plan-graph.py generates nodes and lines for runtime filters. It'd be helpful to show ineffective runtime filters, e.g., by using dash lines.
> It'd be more helpful if we should how many files/rows, etc. are skipped by the filter.
> Uploaded an example profile and the generated svg file. The command is
> {code:java}
> bin/diagnostics/experimental/plan-graph.py -v1 rf_profile.txt | dot -T svg -o query-plan1.svg {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org