You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2021/03/06 02:02:08 UTC
[GitHub] [incubator-doris] morningman opened a new issue #5474: [Proposal] Visualize the query plan and query profile
morningman opened a new issue #5474:
URL: https://github.com/apache/incubator-doris/issues/5474
**Is your feature request related to a problem? Please describe.**
Currently, Doris query plan and query profile are only be viewed in plain text format, which is not easy
to view and analyze the relationship of each node in plan or profile tree.
So I decide to visualize them to help user analyze them more easily.
**Describe the solution you'd like**
**Query Plan**
Query plan is a relative simple structure. Each plan fragment in the query plan is already a tree structure.
What we do is to connect all fragments together (DataSink -> ExchangeNode), and then draw a tree graph
for it.
```
┌──────────────┐
│[2: DATA SINK]│
│Fragment: 2] │
│RESULT SINK │
└──────────────┘
│
│
┌─────────────────────────────────┐
│[2: HASH JOIN] │
│Fragment: 2] │
│join op: INNER JOIN (PARTITIONED)│
└─────────────────────────────────┘
┌───────────────┴──────────────┐
│ │
┌─────────────┐ ┌─────────────┐
│[3: EXCHANGE]│ │[4: EXCHANGE]│
│Fragment: 2] │ │Fragment: 2] │
└─────────────┘ └─────────────┘
└┐ └┐
│ │
┌────────────────────────────┐ ┌────────────────────────────┐
│[3: DATA SINK] │ │[4: DATA SINK] │
│Fragment: 0] │ │Fragment: 1] │
│STREAM DATA SINK │ │STREAM DATA SINK │
│ EXCHANGE ID: 03 │ │ EXCHANGE ID: 04 │
│ HASH_PARTITIONED: `a`.`k1`│ │ HASH_PARTITIONED: `b`.`k1`│
└────────────────────────────┘ └────────────────────────────┘
┌┘ ┌┘
│ │
┌─────────────────┐ ┌─────────────────┐
│[0: OlapScanNode]│ │[1: OlapScanNode]│
│Fragment: 0] │ │Fragment: 1] │
│TABLE: tbl1 │ │TABLE: tbl1 │
└─────────────────┘ └─────────────────┘
```
**Query Profile**
Query profile is more complicate because it does not only show the relationship of each plan fragments.
But it also shows all instances in each plan fragments.
So I plan to divide the presentation of the profile into several steps:
1. Fragment Tree
The fragment tree is a tree used to display the overall query execution structure. They show the connection relationship
between each fragment, and the connection relationship of the execution nodes within each fragment.
And at the same time, the execution time of the instance that takes the longest in each Fragment will be displayed.
Through this figure, the user can understand the overall execution structure, and can further expand a fragment to
continue analyzing the profile according to the maximum execution time.
```
┌──────────────────────┐
│[-1: DataBufferSender]│
│Fragment: 0 │
│MaxActiveTime: 7.850ms│
└──────────────────────┘
│
│
┌───────────────────┐
│[2: HASH_JOIN_NODE]│
│Fragment: 0 │
└───────────────────┘
┌────────────┴─────────────┐
│ │
┌──────────────────┐ ┌──────────────────┐
│[3: EXCHANGE_NODE]│ │[4: EXCHANGE_NODE]│
│Fragment: 0 │ │Fragment: 0 │
└──────────────────┘ └──────────────────┘
│ │
│ │
┌────────────────────────┐ ┌────────────────────────┐
│[3: DataStreamSender] │ │[4: DataStreamSender] │
│Fragment: 2 │ │Fragment: 1 │
│MaxActiveTime: 218.157us│ │MaxActiveTime: 562.931us│
└────────────────────────┘ └────────────────────────┘
┌┘ ┌┘
│ │
┌───────────────────┐ ┌───────────────────┐
│[0: OLAP_SCAN_NODE]│ │[1: OLAP_SCAN_NODE]│
│Fragment: 2 │ │Fragment: 1 │
└───────────────────┘ └───────────────────┘
│ │
│ │
┌─────────────┐ ┌─────────────┐
│[OlapScanner]│ │[OlapScanner]│
│Fragment: 2 │ │Fragment: 1 │
└─────────────┘ └─────────────┘
│ │
│ │
┌─────────────────┐ ┌─────────────────┐
│[SegmentIterator]│ │[SegmentIterator]│
│Fragment: 2 │ │Fragment: 1 │
└─────────────────┘ └─────────────────┘
```
2. Instance List
The user can continue to expand a specified fragment, and then get a list of all instances of this fragment.
The list contains instance id, instance execution host and instance execution time.
Further, the user can continue to expand an instance for further analysis.
```
+----------------------------------+------------------+------------+
| Instances | Host | ActiveTime |
+----------------------------------+------------------+------------+
| f8db008ccdf4ed8-807102c30d795eaa | 10.81.85.89:9062 | 7.850ms |
+----------------------------------+------------------+------------+
```
3. Instance Tree
Instance Tree will display the detailed information of each execution node in an instance.
Here is the most fine-grained information of the profile
```
┌──────────────────────────────┐
│[-1: DataBufferSender] │
│(Active: 0ns, non-child: 0.00)│
│ - Counters: │
│ - AppendBatchTime: 0ns │
│ - NumSentRows: 0 │
└──────────────────────────────┘
│
│
┌───────────────────────────────────┐
│[2: HASH_JOIN_NODE] │
│(Active: 7.842ms, non-child: 0.00) │
│ - Counters: │
│ - BuildBuckets: 1.024K (1024)│
│ - BuildRows: 0 │
│ - BuildTime: 6.989us │
│ - LoadFactor: 0.00 │
│ - PeakMemoryUsage: 48.00 KB │
│ - ProbeRows: 0 │
│ - ProbeTime: 0ns │
│ - PushDownComputeTime: 0ns │
│ - PushDownTime: 0ns │
│ - RowsReturned: 0 │
│ - RowsReturnedRate: 0 │
└───────────────────────────────────┘
┌─────────────────────┴──────────────────────┐
│ │
┌──────────────────────────────────────────┐ ┌──────────────────────────────────────────┐
│[3: EXCHANGE_NODE] │ │[4: EXCHANGE_NODE] │
│(Active: 7.779ms, non-child: 23.54) │ │(Active: 6.440ms, non-child: 19.49) │
│ - Counters: │ │ - Counters: │
│ - BytesReceived: 0.00 │ │ - BytesReceived: 0.00 │
│ - ConvertRowBatchTime: 572ns │ │ - ConvertRowBatchTime: 623ns │
│ - DataArrivalWaitTime: 7.771ms │ │ - DataArrivalWaitTime: 6.432ms │
│ - DeserializeRowBatchTimer: 0ns │ │ - DeserializeRowBatchTimer: 0ns │
│ - FirstBatchArrivalWaitTime: 7.771ms│ │ - FirstBatchArrivalWaitTime: 6.431ms│
│ - PeakMemoryUsage: 0.00 │ │ - PeakMemoryUsage: 0.00 │
│ - RowsReturned: 0 │ │ - RowsReturned: 0 │
│ - RowsReturnedRate: 0 │ │ - RowsReturnedRate: 0 │
│ - SendersBlockedTotalTimer(*): 0ns │ │ - SendersBlockedTotalTimer(*): 0ns │
└──────────────────────────────────────────┘ └──────────────────────────────────────────┘
```
**Describe alternatives you've considered**
First, I will only implement this in text graph.
Next, It can be show in web frontend.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #5474: [Proposal] Visualize the query plan and query profile
Posted by GitBox <gi...@apache.org>.
morningman closed issue #5474:
URL: https://github.com/apache/incubator-doris/issues/5474
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org