You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by st...@apache.org on 2023/07/11 02:09:09 UTC
[impala] branch master updated: IMPALA-12182: Add CPU utilization chart for RuntimeProfile's sampled metrics
This is an automated email from the ASF dual-hosted git repository.
stigahuang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
The following commit(s) were added to refs/heads/master by this push:
new 36a63d0a3 IMPALA-12182: Add CPU utilization chart for RuntimeProfile's sampled metrics
36a63d0a3 is described below
commit 36a63d0a3309e714862b0cdc05f3d4dc7464a7eb
Author: Surya Hebbar <sh...@cloudera.com>
AuthorDate: Fri Jun 2 03:39:33 2023 +0530
IMPALA-12182: Add CPU utilization chart for RuntimeProfile's sampled metrics
This change adds support for a stacked area chart for CPU utilization
to the query timeline display, while also providing the ability to scale
timetick values, precision, and the ability to horizontally scale
the fragment timing diagram along with the utilization chart.
Rendering of different components within the diagram has been decoupled
to isolate scaling of timeticks, also to improve the overall
efficiency by making the rendering functions asynchronous
for better performance during resize events. Additionally,
re-rendering of fragment diagram is only triggered during new
fragment events.
The following are the associated key bindings to scale the timeline
with mouse wheel events.
- shift + wheel events on #fragment_diagram
- shift + wheel events on #timeticks_footer
- alt + shift + wheel events on #timeticks_footer for precision control
Note:
Ctrl + mouse wheel events and ctrl + '+'/'-' events can be used to
resize the timeline through the browser.
Mouse wheel events have been associated with respective components
for better efficiency and maintainability.
Constraints have been added to above attributes to limit scaling/zooming
for appropriate display and rendering across all DOM elements.
RESOURCE_TRACE_RATIO query option provides the utilization values to be
traced within the RuntimeProfile. It contains samples of CPU utilization
metrics for user, sys and iowait. These time series counters are available
within the profile having the following names.
Per Node Profiles -
- HostCpuIoWaitPercentage
- HostCpuSysPercentage
- HostCpuUserPercentage
The samples are updated based on 'periodic_counter_update_period_ms'
providing the 'period' within profile's 'Per Node Profiles'.
These are retrieved from the ChunkedTimeSeriesCounter in the
RuntimeProfile. Currently, JSON profiles and webUI summary pages
contain the downsampled values.
Utilization samples are aligned with the fragment diagram by
associating the number of samples and the period.
Aggregate CPU usage for each node is being calculated
after accumulating the basis point values for user, sys
and iowait. These are being displayed after grouping the
associated counters for each node as a stacked line chart.
c3.js charting library based on d3.v5 is being used to plot
the utilization.
The license associated with d3 v5 during the related time frame
has been included along with the charting library's.
Support for experimental profile V2 is currently not included.
Scaling a large number of values to support profile V2 would be
possible with appropriate down-sampling in the back-end.
Testing: Manual testing with TPC-DS and TPC-H queries
Change-Id: Idea2a6db217dbfaa7a0695aeabb6d9c1ecf62158
Reviewed-on: http://gerrit.cloudera.org:8080/20008
Reviewed-by: Riza Suminto <ri...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
.gitattributes | 3 +
LICENSE.txt | 57 ++++++
bin/rat_exclude_files.txt | 2 +
www/c3/c3.v7.min.css | 1 +
www/c3/c3.v7.min.js | 2 +
www/d3.v5.min.js | 2 +
www/query_timeline.tmpl | 451 ++++++++++++++++++++++++++++++++++++----------
7 files changed, 423 insertions(+), 95 deletions(-)
diff --git a/.gitattributes b/.gitattributes
index cd119a647..69f8eb682 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -15,7 +15,10 @@ www/bootstrap/css/bootstrap.min.css.map binary
www/bootstrap/js/bootstrap.js binary
www/bootstrap/js/bootstrap.min.js
www/bootstrap/js/bootstrap.min.js binary
+www/c3/c3.v7.min.js binary
+www/c3/c3.v7.min.css binary
www/d3.v3.min.js binary
+www/d3.v5.min.js binary
www/dagre-d3.min.js binary
www/DataTables-1.10.18/js/dataTables.bootstrap.min.js binary
www/DataTables-1.10.18/js/dataTables.bootstrap4.min.js biinary
diff --git a/LICENSE.txt b/LICENSE.txt
index 62598a490..977c1abce 100644
--- a/LICENSE.txt
+++ b/LICENSE.txt
@@ -1115,3 +1115,60 @@ www/Chart*: MIT license
OTHER DEALINGS IN THE SOFTWARE.
--------------------------------------------------------------------------------
+
+www/d3.v5.min.js: BSD 3-clause license
+
+Copyright 2010-2017 Mike Bostock
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+ list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice,
+ this list of conditions and the following disclaimer in the documentation
+ and/or other materials provided with the distribution.
+
+* Neither the name of the author nor the names of contributors may be used to
+ endorse or promote products derived from this software without specific prior
+ written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+--------------------------------------------------------------------------------
+
+www/c3/c3*: MIT license
+
+The MIT License (MIT)
+
+Copyright (c) 2013 Masayuki Tanaka
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+the Software, and to permit persons to whom the Software is furnished to do so,
+subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+--------------------------------------------------------------------------------
diff --git a/bin/rat_exclude_files.txt b/bin/rat_exclude_files.txt
index ffa059080..3cef6ca20 100644
--- a/bin/rat_exclude_files.txt
+++ b/bin/rat_exclude_files.txt
@@ -54,7 +54,9 @@ shell/ext-py/six-1.14.0/*
shell/ext-py/sqlparse-0.3.1/*
shell/ext-py/thrift-0.16.0/*
shell/ext-py/thrift_sasl-0.4.3/*
+www/c3/*
www/d3.v3.min.js
+www/d3.v5.min.js
www/jquery/jquery-3.5.1.min.js
tests/comparison/leopard/static/css/hljs.css
tests/comparison/leopard/static/js/highlight.pack.js
diff --git a/www/c3/c3.v7.min.css b/www/c3/c3.v7.min.css
new file mode 100644
index 000000000..86778ebea
--- /dev/null
+++ b/www/c3/c3.v7.min.css
@@ -0,0 +1 @@
+.c3 svg{font:10px sans-serif;-webkit-tap-highlight-color:transparent}.c3 line,.c3 path{fill:none;stroke:#000}.c3 text{-webkit-user-select:none;-moz-user-select:none;user-select:none}.c3-bars path,.c3-event-rect,.c3-legend-item-tile,.c3-xgrid-focus,.c3-ygrid{shape-rendering:crispEdges}.c3-chart-arc path{stroke:#fff}.c3-chart-arc rect{stroke:#fff;stroke-width:1}.c3-chart-arc text{fill:#fff;font-size:13px}.c3-grid line{stroke:#aaa}.c3-grid text{fill:#aaa}.c3-xgrid,.c3-ygrid{stroke-dasharray [...]
\ No newline at end of file
diff --git a/www/c3/c3.v7.min.js b/www/c3/c3.v7.min.js
new file mode 100644
index 000000000..dc3066981
--- /dev/null
+++ b/www/c3/c3.v7.min.js
@@ -0,0 +1,2 @@
+/* @license C3.js v0.7.20 | (c) C3 Team and other contributors | http://c3js.org/ */
+!function(t,e){"object"==typeof exports&&"undefined"!=typeof module?module.exports=e():"function"==typeof define&&define.amd?define(e):(t=t||self).c3=e()}(this,function(){"use strict";function l(t){var e=this;e.d3=window.d3?window.d3:"undefined"!=typeof require?require("d3"):void 0,e.api=t,e.config=e.getDefaultConfig(),e.data={},e.cache={},e.axes={}}function n(t){this.internal=new l(this),this.internal.loadConfig(t),this.internal.beforeInit(t),this.internal.init(),this.internal.afterInit [...]
\ No newline at end of file
diff --git a/www/d3.v5.min.js b/www/d3.v5.min.js
new file mode 100644
index 000000000..1309c949e
--- /dev/null
+++ b/www/d3.v5.min.js
@@ -0,0 +1,2 @@
+// https://d3js.org v5.9.7 Copyright 2019 Mike Bostock
+!function(t,n){"object"==typeof exports&&"undefined"!=typeof module?n(exports):"function"==typeof define&&define.amd?define(["exports"],n):n(t.d3=t.d3||{})}(this,function(t){"use strict";function n(t,n){return t<n?-1:t>n?1:t>=n?0:NaN}function e(t){var e;return 1===t.length&&(e=t,t=function(t,r){return n(e(t),r)}),{left:function(n,e,r,i){for(null==r&&(r=0),null==i&&(i=n.length);r<i;){var o=r+i>>>1;t(n[o],e)<0?r=o+1:i=o}return r},right:function(n,e,r,i){for(null==r&&(r=0),null==i&&(i=n.len [...]
diff --git a/www/query_timeline.tmpl b/www/query_timeline.tmpl
index 0f00e3d81..1347b7308 100644
--- a/www/query_timeline.tmpl
+++ b/www/query_timeline.tmpl
@@ -21,6 +21,10 @@ under the License.
</div>
+<script src="{{ __common__.host-url }}/www/d3.v5.min.js"></script>
+<script src="{{ __common__.host-url }}/www/c3/c3.v7.min.js"></script>
+<link href="{{ __common__.host-url }}/www/c3/c3.v7.min.css" rel="stylesheet">
+
<div class="container">
<style id="css">
@@ -48,7 +52,7 @@ under the License.
</label>
</div>
<label>
- <input type="checkbox" id="plan_order" onClick="renderTiming()"/>
+ <input type="checkbox" id="plan_order" onClick="renderFragmentDiagram()"/>
Print tree in plan order (if unchecked, print in fragment order)
</label>
@@ -79,16 +83,19 @@ under the License.
</div>
<div id="timing_diagram" style="border:1px solid #c3c3c3;">
- <div style="margin-top:5px; border:1px solid #c3c3c3; overflow:hidden;">
- <svg id="phases_header" height="15px"></svg>
+ <div style="border:1px solid #c3c3c3;">
+ <svg id="phases_header"></svg>
</div>
- <div style="margin-top:5px; border:1px solid #c3c3c3; overflow-y:auto; overflow-x:hidden;">
+ <div style="border:1px solid #c3c3c3; overflow-y:scroll; overflow-x:hidden;">
<svg id="fragment_diagram"></svg>
</div>
- <div style="margin-top:5px; border:1px solid #c3c3c3; overflow:hidden;">
- <svg id="timeticks_footer" height="15px"></svg>
+ <div style="border:1px solid #c3c3c3;">
+ <svg id="timeticks_footer"></svg>
</div>
</div>
+<div id="utilization_diagram">
+ <!--Utilization metrics is not available. Please make sure to set query option RESOURCE_TRACE_RATIO=true.-->
+</div>
{{/plan_metadata_unavailable}}
@@ -99,6 +106,18 @@ under the License.
$("#plan-timing-tab").addClass("active");
var profile = {};
+
+var stroke_fill_colors = { black : "#000000", dark_grey : "#505050",
+ light_grey : "#F0F0F0", transperent : "rgba(0, 0, 0, 0)" };
+
+var row_height = 15;
+var char_width = 6;
+var margin_header_footer = 5;
+var margin_chart_end = 60;
+var border_stroke_width = 2;
+var diagram_width = window.innerWidth - border_stroke_width; // border width
+
+// #phases_header
var phases = [
{ color: "#C0C0FF", label: "Prepare" }
, { color: "#E0E0E0", label: "Open" }
@@ -107,27 +126,59 @@ var phases = [
, { color: "#C0FFC0", label: "Process Remaining Batches" }
, { color: "#FFC0C0", label: "Close" }
];
+
+// #fragment_diagram
var fragment_colors = ["#A9A9A9", "#FF8C00", "#8A2BE2", "#A52A2A", "#00008B", "#006400",
"#228B22", "#4B0082", "#DAA520", "#008B8B", "#000000", "#DC143C"];
var fragments = [];
-var rownum = 0;
-var char_width = 6;
-var max_namelen = 0;
var all_nodes = [];
var receiver_nodes = [];
-var profile_available = false;
+var max_namelen = 0;
+var frag_name_width;
+var name_width;
+var chart_width;
+
+// #timeticks_footer
+var ntics = 10;
+var integer_part_estimate = 4;
+var decimals = 2;
var maxts = 0;
+var last_maxts = -1;
+
+// #utilization_diagram
+var utilization_counter_names = ["avg io wait", "avg sys", "avg user"];
+var cpu_utilization_chart;
+var cpu_nodes_usage_aggregate;
+var sampled_timeseries;
+var timeaxis_name = 'timeticks';
+var prev_num_samples = 0;
+var max_samples = 64;
+var max_samples_available = 0;
+var max_samples_collected = 0;
+var max_samples_period;
+var utilization_style;
+
+var fragment_events_parse_successful = false;
+var utilization_metrics_parse_successful = false;
+
+// binding DOM elements
+var timing_diagram = document.getElementById("timing_diagram");
+var phases_header = document.getElementById("phases_header");
+var fragment_diagram = document.getElementById("fragment_diagram");
+var timeticks_footer = document.getElementById("timeticks_footer");
+var utilization_diagram = document.getElementById("utilization_diagram");
var export_filename = document.getElementById("export_filename");
var export_format = document.getElementById("export_format");
+
export_filename.value = export_filename.value.replace(/\W/g,'_');
function get_svg_rect(fill_color, x, y, width, height, dash, stroke_color) {
var rect = document.createElementNS("http://www.w3.org/2000/svg", "rect");
- rect.setAttribute("x", x + "px");
- rect.setAttribute("y", y + "px");
- rect.setAttribute("width", width + "px");
- rect.setAttribute("height", height + "px");
+ rect.setAttribute("x", `${x}px`);
+ rect.setAttribute("y", `${y}px`);
+ rect.setAttribute("width", `${width}px`);
+ rect.setAttribute("height", `${height}px`);
rect.setAttribute("fill", fill_color);
if (dash) {
rect.setAttribute("stroke", stroke_color);
@@ -139,9 +190,9 @@ function get_svg_rect(fill_color, x, y, width, height, dash, stroke_color) {
function get_svg_text(text, fill_color, x, y, height, container_center, max_width = 0) {
var text_el = document.createElementNS("http://www.w3.org/2000/svg", "text");
text_el.appendChild(document.createTextNode(text));
- text_el.setAttribute("x", x + "px");
- text_el.setAttribute("y", y + "px");
- text_el.style.fontSize = height / 1.5 + "px";
+ text_el.setAttribute("x", `${x}px`);
+ text_el.setAttribute("y", `${y}px`);
+ text_el.style.fontSize = `${height / 1.5}px`;
if (container_center) {
text_el.setAttribute("dominant-baseline", "middle");
text_el.setAttribute("text-anchor", "middle");
@@ -156,10 +207,10 @@ function get_svg_text(text, fill_color, x, y, height, container_center, max_widt
function get_svg_line(stroke_color, x1, y1, x2, y2, dash) {
var line = document.createElementNS("http://www.w3.org/2000/svg", "line");
- line.setAttribute("x1", x1 + "px");
- line.setAttribute("y1", y1 + "px");
- line.setAttribute("x2", x2 + "px");
- line.setAttribute("y2", y2 + "px");
+ line.setAttribute("x1", `${x1}px`);
+ line.setAttribute("y1", `${y1}px`);
+ line.setAttribute("x2", `${x2}px`);
+ line.setAttribute("y2", `${y2}px`);
line.setAttribute("stroke", stroke_color);
if (dash) {
line.setAttribute("stroke-dasharray", "2 2");
@@ -182,7 +233,7 @@ function DrawBars(svg, rownum, row_height, events, xoffset, px_per_ns) {
last_end = x + width;
// Block phase outline
- svg.appendChild(get_svg_rect("#000000", x, y, width, bar_height, false));
+ svg.appendChild(get_svg_rect(stroke_fill_colors.black, x, y, width, bar_height, false));
if (width > 2) {
svg.appendChild(get_svg_rect(phases[color_idx].color, x + 1, y + 1,
width - 2, bar_height - 2, false));
@@ -195,7 +246,7 @@ function DrawBars(svg, rownum, row_height, events, xoffset, px_per_ns) {
var dx = (endts - ts) * px_per_ns;
var ignore_px = 2; // Don't print tiny skews
if (Math.abs(dx) > ignore_px) {
- svg.appendChild(get_svg_line("#505050", last_end - dx, y , last_end - dx,
+ svg.appendChild(get_svg_line(stroke_fill_colors.dark_grey, last_end - dx, y , last_end - dx,
y + bar_height, false));
}
});
@@ -203,14 +254,13 @@ function DrawBars(svg, rownum, row_height, events, xoffset, px_per_ns) {
});
}
-function collectFromProfile(ignored_arg) {
+function collectFragmentEventsFromProfile(ignored_arg) {
rownum = 0;
- maxts = 0;
max_namelen = 0;
fragments = [];
- var color_idx = 0;
all_nodes = [];
receiver_nodes = [];
+ var color_idx = 0;
try {
// First pass: compute sizes
profile.child_profiles[2].child_profiles.forEach(function(fp) {
@@ -257,7 +307,7 @@ function collectFromProfile(ignored_arg) {
var table_name = pp.info_strings.find(({ key }) => key === "Table Name")
.value.split(/[.]/);
node_name = node_name.replace("SCAN",
- "SCAN [" + table_name[table_name.length - 1] + "]");
+ `SCAN [${table_name[table_name.length - 1]}]`);
}
var is_receiver = node_type == "EXCHANGE_NODE" ||
@@ -343,7 +393,7 @@ function collectFromProfile(ignored_arg) {
cp = cp.child_profiles[node.path[pi]];
}
console.assert(cp.node_metadata.data_sink_id == undefined ||
- cp.profile_name.indexOf("(dst_id=" + node.node_id + ")"));
+ cp.profile_name.indexOf(`(dst_id=${node.node_id})`));
console.assert(cp.node_metadata.plan_node_id == undefined ||
cp.node_metadata.plan_node_id == node.node_id);
@@ -359,59 +409,68 @@ function collectFromProfile(ignored_arg) {
fragments.push(fragment);
}
});
- profile_available = true;
+ frag_name_width = (Math.max(2, (fragments.length - 1).toString().length) + 3) * char_width;
+ name_width = max_namelen * char_width + (frag_name_width + 2);
+ fragment_events_parse_successful = true;
} catch(e) {
+ fragment_events_parse_successful = false;
console.log(e);
- profile_available = false;
}
}
-function renderTiming(ignored_arg) {
- var plan_order = document.getElementById('plan_order').checked;
-
- var timing_diagram = document.getElementById('timing_diagram');
- var phases_header = document.getElementById('phases_header');
- var fragment_diagram = document.getElementById('fragment_diagram');
- var timeticks_footer = document.getElementById('timeticks_footer');
+function setDimensions(ignored_arg) {
+ var display_height = Math.min(window.innerHeight - timing_diagram.offsetTop - 50
+ - (utilization_metrics_parse_successful? getUtilizationHeight() : 0), rownum * row_height);
- phases_header.innerHTML = "";
- fragment_diagram.innerHTML = "";
- timeticks_footer.innerHTML = "";
+ chart_width = diagram_width - name_width - margin_chart_end - border_stroke_width;
- var row_height = 15;
- var display_height = Math.min(window.innerHeight - timing_diagram.offsetTop - 70,
- rownum * row_height);
- fragment_diagram.parentNode.style.height = display_height * 1 + "px";
- fragment_diagram.setAttribute("height", rownum * row_height + "px");
+ phases_header.style.height = `${row_height}px`;
+ fragment_diagram.parentElement.style.height = `${display_height}px`;
+ fragment_diagram.style.height = `${rownum * row_height}px`;
+ timeticks_footer.style.height = `${row_height}px`;
- fragment_diagram.setAttribute("width", timing_diagram.clientWidth - 50);
- phases_header.setAttribute("width", fragment_diagram.clientWidth);
- timeticks_footer.setAttribute("width", fragment_diagram.clientWidth);
+ fragment_diagram.parentElement.style.width = `${diagram_width}px`;
+ phases_header.parentElement.style.width = `${diagram_width}px`;
+ timeticks_footer.parentElement.style.width = `${diagram_width}px`;
+ timing_diagram.parentElement.style.width = `${diagram_width}px`;
- var frag_name_width = (Math.max(2, (fragments.length - 1).toString().length) + 3) * char_width;
- var name_width = max_namelen * char_width + (frag_name_width + 2);
- var chart_width = fragment_diagram.clientWidth - name_width;
+ fragment_diagram.style.width = `${diagram_width}px`;
+ phases_header.style.width = `${diagram_width}px`;
+ timeticks_footer.style.width = `${diagram_width}px`;
+}
- var px_per_ns = chart_width / maxts;
+function clearDOMChildren(element) {
+ while (element.firstChild) {
+ element.removeChild(element.firstChild);
+ }
+}
- var text_y = row_height - 4;
+async function renderPhases() {
+ clearDOMChildren(phases_header);
var color_idx = 0;
var width = Math.ceil(chart_width / phases.length);
phases.forEach(function(p) {
var x = name_width + Math.ceil(chart_width * color_idx / phases.length);
- x = Math.min(x, phases_header.clientWidth - width);
+ x = Math.min(x, diagram_width - width);
- phases_header.appendChild(get_svg_rect("#000000", x, 0, width, row_height, false));
+ phases_header.appendChild(get_svg_rect(stroke_fill_colors.black, x, 0, width, row_height, false));
phases_header.appendChild(get_svg_rect(phases[color_idx++].color, x + 1, 1,
width - 2, row_height - 2, false));
- phases_header.appendChild(get_svg_text(p.label, "#000000", x + width / 2,
- text_y - 2, row_height, true, Math.min(p.label.length * char_width, width / 1.5)));
+ phases_header.appendChild(get_svg_text(p.label, stroke_fill_colors.black, x + width / 2,
+ (row_height + 4) / 2 , row_height, true, Math.min(p.label.length * char_width, width / 1.5)));
});
+}
+async function renderFragmentDiagram() {
+ clearDOMChildren(fragment_diagram);
+ last_maxts = maxts;
+ var plan_order = document.getElementById('plan_order').checked;
+ var px_per_ns = chart_width / maxts;
var rownum_l = 0;
var max_indent = 0;
var pending_children = 0;
var pending_senders = 0;
+ var text_y = row_height - 4;
fragments.forEach(function printFragment(fragment) {
if (!fragment.printed) {
fragment.printed = true;
@@ -434,10 +493,10 @@ function renderTiming(ignored_arg) {
// Plan node timing row
DrawBars(fragment_diagram, rownum_l, row_height, node.events, name_width, px_per_ns);
if (node.type == "HASH_JOIN_NODE") {
- fragment_diagram.appendChild(get_svg_text("X", "#000000",
+ fragment_diagram.appendChild(get_svg_text("X", stroke_fill_colors.black,
name_width + Math.min.apply(null, node.events[2].tslist) * px_per_ns,
text_y, row_height, false));
- fragment_diagram.appendChild(get_svg_text("O", "#000000",
+ fragment_diagram.appendChild(get_svg_text("O", stroke_fill_colors.black,
name_width + Math.min.apply(null, node.events[2].tslist) * px_per_ns,
text_y, row_height, false));
}
@@ -468,8 +527,8 @@ function renderTiming(ignored_arg) {
// Dotted rectangle for received rows
var x2 = name_width + Math.max.apply(null,fevents[4].tslist) * px_per_ns;
- fragment_diagram.appendChild(get_svg_rect("rgba(0,0,0,0)", x, y + 4, x2 - x,
- row_height - 10, true, fragment.color));
+ fragment_diagram.appendChild(get_svg_rect(stroke_fill_colors.transperent,
+ x, y + 4, x2 - x, row_height - 10, true, fragment.color));
}
if (node.is_sender && node.parent_node.rendering.rownum != rownum_l - 1) {
@@ -516,43 +575,217 @@ function renderTiming(ignored_arg) {
fragments.forEach(function(fragment) {
fragment.printed = false;
});
- rownum_l = 0;
- var text_y = (rownum_l + 1) * row_height - 4;
+}
- // Time scale below timing diagram
- var ntics = 10;
- var sec_per_tic = maxts / ntics / 1000000000;
+async function renderTimeticks() {
+ clearDOMChildren(timeticks_footer);
+ var sec_per_tic = maxts / ntics / 1e9;
var px_per_tic = chart_width / ntics;
var x = name_width;
- var decimals = 2;
+ var y = 0;
+ var text_y = row_height - 4;
var timetick_label;
for (var i = 1; i <= ntics; ++i) {
- var y = rownum_l * row_height;
- timeticks_footer.appendChild(get_svg_rect("#000000", x, y, px_per_tic,
+ timeticks_footer.appendChild(get_svg_rect(stroke_fill_colors.black, x, y, px_per_tic,
row_height, false));
- timeticks_footer.appendChild(get_svg_rect("#F0F0F0", x + 1, y + 1, px_per_tic - 2,
+ timeticks_footer.appendChild(get_svg_rect(stroke_fill_colors.light_grey, x + 1, y + 1, px_per_tic - 2,
row_height - 2, false));
timetick_label = (i * sec_per_tic).toFixed(decimals);
- timeticks_footer.appendChild(get_svg_text(timetick_label, "#000000",
+ timeticks_footer.appendChild(get_svg_text(timetick_label, stroke_fill_colors.black,
x + px_per_tic - timetick_label.length * char_width + 2, text_y, row_height, false));
x += px_per_tic;
}
}
+function getUtilizationHeight() {
+ return window.innerHeight * 0.3;
+}
+
+function toogleUtilizationVisibility() {
+ if (utilization_metrics_parse_successful) {
+ utilization_diagram.style.display = 'flex';
+ } else {
+ utilization_diagram.style.display = 'none';
+ }
+}
+
+function initializeUtilizationChart() {
+ try {
+ cpu_utilization_chart.destroy();
+ } catch (e) {}
+ cpu_utilization_chart = c3.generate({
+ bindto : "#utilization_diagram",
+ data : {
+ columns : [[timeaxis_name, 0]],
+ type : 'area',
+ groups : [ utilization_counter_names ],
+ order : 'asc',
+ x : timeaxis_name
+ }, size : {
+ height : getUtilizationHeight(),
+ width : diagram_width
+ }, padding : {
+ left : name_width,
+ right : margin_chart_end
+ }, axis : {
+ x :
+ {
+ padding : {
+ left : 0,
+ right : 0
+ },
+ tick : {
+ format : function (x) { return x.toFixed(decimals); }
+ }
+ }
+ }, legend : {
+ show : false
+ }, tooltip : {
+ format : {
+ value : function (value, ratio, id, index) { return value.toFixed(decimals) + '%'; }
+ }
+ }
+ });
+ cpu_utilization_chart.load({
+ unload : true
+ });
+}
+
+function initializeUtilizationMetrics() {
+ // user, sys, io and sampled timeticks
+ cpu_nodes_usage_aggregate = new Array(3);
+ for (var i = 0; i < 3; ++i) {
+ cpu_nodes_usage_aggregate[i] = new Array(max_samples + 2).fill(0);
+ cpu_nodes_usage_aggregate[i][0] = utilization_counter_names[i];
+ }
+ sampled_timeseries = new Array(max_samples + 2).fill(null);
+ sampled_timeseries[0] = timeaxis_name;
+}
+
+function accumulateUtilization(utilization_array, time_series_counter) {
+ var utilization_samples = time_series_counter.data.split(",").map(el => parseInt(el) / 100);
+ if (utilization_samples.length <= 1) return;
+ for (var j = 0; j < utilization_samples.length; ++j) {
+ utilization_array[j + 1] += utilization_samples[j];
+ }
+ if (time_series_counter.num > max_samples_collected) {
+ max_samples_available = utilization_samples.length;
+ max_samples_period = time_series_counter.period;
+ max_samples_collected = time_series_counter.num;
+ }
+}
+
+function clearTimesamples(timesamples_array) {
+ for (var j = 1; j <= max_samples_available + 1; ++j) {
+ timesamples_array[j] = null;
+ }
+}
+
+function clearUtilization(utilization_array) {
+ for (var j = 1; j <= max_samples_available + 1; ++j) {
+ utilization_array[j] = 0;
+ }
+}
+
+function collectUtilizationFromProfile(ignored_arg) {
+ try {
+ var per_node_profiles = profile.child_profiles[2].child_profiles[0];
+ if (cpu_utilization_chart == undefined) {
+ initializeUtilizationMetrics();
+ initializeUtilizationChart();
+ }
+ // Update the plot, only when number of samples in SummaryStatsCounter is updated
+ if (profile.child_profiles[1].summary_stats_counters[0].num_of_samples != prev_num_samples) {
+ // For each node's profile
+ per_node_profiles.child_profiles.forEach(function (cpu_node, cpu_node_i) {
+ for (var i = 0; i < 3; ++i) {// usr, sys and iowait
+ accumulateUtilization(cpu_nodes_usage_aggregate[i],
+ cpu_node.time_series_counters[i]);
+ }
+ });
+ if (max_samples_available <= 2) {
+ cpu_utilization_chart.destroy();
+ cpu_utilization_chart = undefined;
+ clearDOMChildren(utilization_diagram);
+ utilization_diagram.style = "";
+ utilization_diagram.className = "";
+ utilization_diagram.style.height = `${getUtilizationHeight()}px`;
+ utilization_diagram.innerHTML = `<span style="color:#ff0f0f;font-size:20px;">
+ Warning: not enough samples for CPU utilization plot. Please decrease the
+ value of starting flag variable <b></i>periodic_counter_update_period_ms
+ </b></i> to increase the granularity of CPU utilization plot.</span>`;
+ utilization_diagram.style.justifyContent = "center";
+ utilization_diagram.style.alignItems = "center";
+ utilization_metrics_parse_successful = true;
+ return;
+ }
+ cpu_nodes_usage_aggregate.forEach(function (acc_usage) {
+ for (var i = 1; i <= max_samples_available; ++i) {
+ acc_usage[i] = acc_usage[i] / per_node_profiles.child_profiles.length;
+ }
+ });
+ var avg_period = max_samples_collected * max_samples_period / max_samples_available;
+ avg_period = avg_period / 1000;
+ for (var k = 0; k <= max_samples_available; k++) {
+ sampled_timeseries[k + 1] = (k * avg_period);
+ }
+ if (maxts / 1e9 > sampled_timeseries[max_samples_available + 1]) {
+ sampled_timeseries[max_samples_available + 1] = maxts / 1e9;
+ } else {
+ maxts = sampled_timeseries[max_samples_available + 1] * 1e9;
+ }
+ cpu_utilization_chart.load({
+ columns : [...cpu_nodes_usage_aggregate, sampled_timeseries]
+ });
+ cpu_nodes_usage_aggregate.forEach(function (acc_usage) {
+ clearUtilization(acc_usage);
+ });
+ clearTimesamples(sampled_timeseries);
+ prev_num_samples = profile.child_profiles[1].summary_stats_counters[0].num_of_samples;
+ }
+ utilization_metrics_parse_successful = true;
+ } catch (e) {
+ utilization_metrics_parse_successful = false;
+ console.log(e);
+ }
+}
+
+async function renderUtilization() {
+ cpu_utilization_chart.resize({
+ height: getUtilizationHeight(),
+ width: diagram_width
+ });
+ utilization_diagram.style.width = `${diagram_width}px`;
+}
+
+function renderAll() {
+ if (fragment_events_parse_successful) {
+ setDimensions();
+ renderPhases();
+ renderFragmentDiagram();
+ renderTimeticks();
+ }
+ if (utilization_metrics_parse_successful) {
+ renderUtilization();
+ }
+}
+
function refresh() {
var req = new XMLHttpRequest();
req.onload = function() {
if (req.status == 200) {
profile = JSON.parse(req.responseText)["profile_json"];
- collectFromProfile();
- if (!profile_available) {
- setTimeout(refresh, 1000);
+ collectFragmentEventsFromProfile();
+ collectUtilizationFromProfile();
+ toogleUtilizationVisibility();
+ if (maxts == last_maxts) {
+ renderUtilization();
} else {
- if (profile.child_profiles[0].info_strings.find(({key}) => key === "End Time").value
- == "") {
- setTimeout(refresh, 1000);
- }
- renderTiming();
+ renderAll();
+ }
+ if (profile.child_profiles[0].info_strings.find(({key}) =>
+ key === "End Time").value == "") {
+ setTimeout(refresh, 1000);
}
}
}
@@ -563,37 +796,65 @@ function refresh() {
// Attaches a SVG blob of the complete timeline to the associated link
export_link.addEventListener('click', function(event) {
if (export_format.value == ".html") {
- var phases_header = document.getElementById('phases_header');
- var fragment_diagram = document.getElementById('fragment_diagram');
- var timeticks_footer = document.getElementById('timeticks_footer');
var export_style = document.getElementById("css");
// Deep clone 'parentNode's as wrappers to SVG components
- var phases_header_wrapper = phases_header.parentNode.cloneNode(true);
var fragment_diagram_wrapper = fragment_diagram.parentNode.cloneNode(true);
- var timeticks_footer_wrapper = timeticks_footer.parentNode.cloneNode(true);
// Set dimensions for fragment diagram's wrapper
fragment_diagram_wrapper.style.height = fragment_diagram.style.height;
- fragment_diagram_wrapper.style.width = fragment_diagram.style.width;
- var html_blob = new Blob([`<!DOCTYPE html><body>`,
- `<h1 style="font-family:monospace;">Query {{query_id}}</h1>`,
- export_style.outerHTML, phases_header_wrapper.outerHTML,
- fragment_diagram_wrapper.outerHTML, timeticks_footer_wrapper.outerHTML,
- `</body></html>`], {type: "text/html;charset=utf-8"});
+ var html_blob = new Blob(['<!DOCTYPE html><body>',
+ '<h1 style=\"font-family:monospace;\">Query {{query_id}}</h1>',
+ `<style>${export_style.innerHTML} ${utilization_style}</style>`,
+ phases_header.parentNode.outerHTML, fragment_diagram_wrapper.outerHTML,
+ timeticks_footer.parentElement.outerHTML, utilization_diagram.outerHTML,
+ '</body></html>'], {type: 'text/html;charset=utf-8'});
export_link.href = URL.createObjectURL(html_blob);
}
export_link.download = `${export_filename.value}${export_format.value}`;
export_link.click();
});
-window.addEventListener('resize', function(event) {
- if (profile_available) {
- renderTiming();
+timeticks_footer.addEventListener('wheel', function(e) {
+ var rendering_constraint = char_width * (decimals + integer_part_estimate) >= chart_width / ntics;
+ if (e.shiftKey) {
+ if (e.altKey) {
+ if (decimals <= 1 && e.wheelDelta <= 0) return;
+ if (e.wheelDelta >= 0) {
+ if (rendering_constraint) return;
+ ++decimals;
+ } else {
+ --decimals;
+ }
+ } else {
+ if (ntics <= 10 && e.wheelDelta <= 0) return;
+ if (rendering_constraint && e.wheelDelta >= 0) return;
+ ntics += Math.round(e.wheelDelta / 200);
+ }
+ renderTimeticks();
}
-}, true);
+});
-window.onload = refresh;
+fragment_diagram.addEventListener('wheel', function(e) {
+ var rendering_constraint = char_width * (decimals + integer_part_estimate) >= chart_width / ntics;
+ if (e.shiftKey) {
+ if (diagram_width <= window.innerWidth && e.wheelDelta <= 0) return;
+ if (rendering_constraint && e.wheelDelta <= 0) return;
+ diagram_width += Math.round(e.wheelDelta * 6 / 10);
+ renderAll();
+ }
+});
+window.addEventListener('resize', function(event) {
+ diagram_width = Math.max(window.innerWidth, diagram_width - border_stroke_width);
+ renderAll();
+}, true);
+
+window.onload = function() {
+ refresh();
+ fetch(make_url("/www/c3/c3.v7.min.css"))
+ .then(res => res.text())
+ .then(style => utilization_style = style);
+}
</script>