You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by bo...@apache.org on 2025/04/11 07:21:14 UTC
(impala) branch master updated (f222574f0 -> 0ed4e869d)
This is an automated email from the ASF dual-hosted git repository.
boroknagyz pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
from f222574f0 IMPALA-13660: Support caching broadcast hash joins
new 6e4613442 IMPALA-13933: run-iceberg-rest-server.sh should use IMPALA_MAVEN_OPTIONS
new 2849708b8 IMPALA-13932: (addendum) Adds e2e test for IMPALA-13932
new 37d4a4eb5 IMPALA-13389: Refactor webUI scripts to use ES6 syntax
new 0ed4e869d IMPALA-13930: ImpylaHS2Connection should only open cursor as needed
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/src/service/impala-server.cc | 2 +-
testdata/bin/run-iceberg-rest-server.sh | 2 +-
.../QueryTest/iceberg-merge-duplicate-check.test | 20 +
.../QueryTest/local-timestamp-functions.test | 13 +-
.../queries/QueryTest/utc-timestamp-functions.test | 96 -----
tests/common/impala_connection.py | 112 +++---
tests/custom_cluster/test_local_tz_conversion.py | 17 +-
tests/custom_cluster/test_session_expiration.py | 8 +-
tests/query_test/test_exprs.py | 34 +-
tests/query_test/test_iceberg.py | 6 +
tests/query_test/test_observability.py | 2 +-
www/admission_controller.tmpl | 78 ++--
www/backends.tmpl | 6 +-
www/catalog.tmpl | 10 +-
www/catalog_ha_info.tmpl | 2 +-
www/catalog_operations.tmpl | 2 +-
www/common-header.tmpl | 4 +-
www/flags.tmpl | 2 +-
www/hadoop-varz.tmpl | 2 +-
www/jvm-threadz.tmpl | 2 +-
www/log_level.tmpl | 4 +-
www/metric_group.tmpl | 2 +-
www/profile_docs.tmpl | 2 +-
www/queries.tmpl | 58 +--
www/query_backends.tmpl | 18 +-
www/query_detail_tabs.tmpl | 80 ++--
www/query_finstances.tmpl | 28 +-
www/query_plan.tmpl | 84 ++---
www/query_plan_text.tmpl | 22 +-
www/query_profile.tmpl | 24 +-
www/query_stmt.tmpl | 28 +-
www/query_summary.tmpl | 17 +-
www/query_timeline.tmpl | 85 +++--
www/rpcz.tmpl | 98 ++---
www/scripts/common_util.js | 14 +-
www/scripts/queries/profileParseWorker.js | 6 +-
www/scripts/query_timeline/chart_commons.js | 43 ++-
www/scripts/query_timeline/fragment_diagram.js | 420 +++++++++++----------
.../query_timeline/fragment_metrics_diagram.js | 126 ++++---
www/scripts/query_timeline/global_members.js | 22 +-
.../query_timeline/host_utilization_diagram.js | 145 ++++---
.../tests/queries/profileParseWorker.test.js | 4 +-
.../tests/query_timeline/chart_commons.test.js | 102 ++---
.../tests/query_timeline/fragment_diagram.test.js | 4 +-
.../fragment_metrics_diagram.test.js | 16 +-
.../host_utilization_diagram.test.js | 18 +-
www/sessions.tmpl | 8 +-
www/thread-group.tmpl | 2 +-
48 files changed, 930 insertions(+), 970 deletions(-)
create mode 100644 testdata/workloads/functional-query/queries/QueryTest/iceberg-merge-duplicate-check.test
(impala) 01/04: IMPALA-13933: run-iceberg-rest-server.sh should use IMPALA_MAVEN_OPTIONS
Posted by bo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
boroknagyz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 6e4613442e8a1ae6dc2d4d2cef0831b36d8ccb30
Author: Zoltan Borok-Nagy <bo...@cloudera.com>
AuthorDate: Fri Apr 4 18:31:15 2025 +0200
IMPALA-13933: run-iceberg-rest-server.sh should use IMPALA_MAVEN_OPTIONS
run-iceberg-rest-server.sh should use IMPALA_MAVEN_OPTIONS.
The easiest way to achieve this is to invoke maven via bin/mvn-quiet.sh
The missing options can cause maven failures.
Change-Id: Ib01ed2ef420b9e965dd8d0e495c370a1ddf323a8
Reviewed-on: http://gerrit.cloudera.org:8080/22736
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
testdata/bin/run-iceberg-rest-server.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/testdata/bin/run-iceberg-rest-server.sh b/testdata/bin/run-iceberg-rest-server.sh
index 1de003508..3dae9d5ba 100755
--- a/testdata/bin/run-iceberg-rest-server.sh
+++ b/testdata/bin/run-iceberg-rest-server.sh
@@ -24,5 +24,5 @@ PATH=${PATH}:/usr/local/bin
cd $IMPALA_HOME
. bin/impala-config.sh
-mvn -f "java/iceberg-rest-catalog-test/pom.xml" exec:java \
+bin/mvn-quiet.sh -f "java/iceberg-rest-catalog-test/pom.xml" exec:java \
-Dexec.mainClass="org.apache.iceberg.rest.IcebergRestCatalogTest"
(impala) 04/04: IMPALA-13930: ImpylaHS2Connection should only open cursor as needed
Posted by bo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
boroknagyz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 0ed4e869de43532c72dab514850944c3a6036bd1
Author: Riza Suminto <ri...@cloudera.com>
AuthorDate: Thu Apr 3 15:42:47 2025 -0700
IMPALA-13930: ImpylaHS2Connection should only open cursor as needed
Before this patch, ImpylaHS2Connection unconditionally opened a
cursor (and HS2 session) as it connected, followed by running a "SET
ALL" query to populate the default query options.
This patch changes the behavior of ImpylaHS2Connection to open the
default cursor only when querying is needed for the first time. This
helps preserve assertions for a test that is sensitive about client
connection, like IMPALA-13925. Default query options are now parsed from
newly instantiated TQueryOptions object rather than issuing a "SET ALL"
query or making BeeswaxService.get_default_configuration() RPC.
Fix test_query_profile_contains_query_compilation_metadata_cached_event
slightly by setting the 'sync_ddl' option because the test is flaky
without it.
Tweak test_max_hs2_sessions_per_user to run queries so that sessions
will open.
Deduplicate test cases between utc-timestamp-functions.test and
local-timestamp-functions.test. Rename TestUtcTimestampFunctions to
TestTimestampFunctions, and expand it to also tests
local-timestamp-functions.test and
file-formats-with-local-tz-conversion.test. The table_format is now
contrained to 'test/none' because it is unnecessary to permute other
table_format.
Deprecate 'use_local_tz_for_unix_timestamp_conversions' in favor of
query option with the same name. Filed IMPALA-13953 to update the
documentation of 'use_local_tz_for_unix_timestamp_conversions'
flag/option.
Testing:
Run and pass a few pytests such as:
test_admission_controller.py
test_observability.py
test_runtime_filters.py
test_session_expiration.py.
test_set.py
Change-Id: I9d5e3e5c11ad386b7202431201d1a4cff46cbff5
Reviewed-on: http://gerrit.cloudera.org:8080/22731
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
be/src/service/impala-server.cc | 2 +-
.../QueryTest/local-timestamp-functions.test | 13 ++-
.../queries/QueryTest/utc-timestamp-functions.test | 96 ------------------
tests/common/impala_connection.py | 112 ++++++++++-----------
tests/custom_cluster/test_local_tz_conversion.py | 17 ++--
tests/custom_cluster/test_session_expiration.py | 8 +-
tests/query_test/test_exprs.py | 34 +++++--
tests/query_test/test_observability.py | 2 +-
8 files changed, 105 insertions(+), 179 deletions(-)
diff --git a/be/src/service/impala-server.cc b/be/src/service/impala-server.cc
index ffee7dfc3..3364dc54c 100644
--- a/be/src/service/impala-server.cc
+++ b/be/src/service/impala-server.cc
@@ -370,7 +370,7 @@ DEFINE_bool(use_local_tz_for_unix_timestamp_conversions, false,
"When true, TIMESTAMPs are interpreted in the local time zone when converting to "
"and from Unix times. When false, TIMESTAMPs are interpreted in the UTC time zone. "
"Set to true for Hive compatibility. "
- "Can be overriden with the query option with the same name.");
+ "DEPRECATED: use query option with the same name instead.");
// Provide a workaround for IMPALA-1658.
DEFINE_bool(convert_legacy_hive_parquet_utc_timestamps, false,
diff --git a/testdata/workloads/functional-query/queries/QueryTest/local-timestamp-functions.test b/testdata/workloads/functional-query/queries/QueryTest/local-timestamp-functions.test
index fb706bfae..bc533088e 100644
--- a/testdata/workloads/functional-query/queries/QueryTest/local-timestamp-functions.test
+++ b/testdata/workloads/functional-query/queries/QueryTest/local-timestamp-functions.test
@@ -1,4 +1,4 @@
-# This test should be run with -use_local_tz_for_unix_timestamp_conversions=true
+# This test should be run with use_local_tz_for_unix_timestamp_conversions=true
====
---- QUERY
SET timezone=CET;
@@ -60,3 +60,14 @@ TIMESTAMP
---- RESULTS
1969-12-31 16:00:00
====
+---- QUERY
+# Regression test for IMPALA-5121. AVG used to contain a back and forth timezone
+# conversion (if use_local_tz_for_unix_timestamp_conversions is true) that could affect
+# the results if there were values from different DST rules.
+SET timezone=CET;
+select avg(timestamp_col) from functional.alltypestiny;
+---- TYPES
+TIMESTAMP
+---- RESULTS
+2009-02-15 00:00:30
+====
\ No newline at end of file
diff --git a/testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test b/testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test
index 1e60a56fb..f2b740ec6 100644
--- a/testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test
+++ b/testdata/workloads/functional-query/queries/QueryTest/utc-timestamp-functions.test
@@ -15,100 +15,4 @@ unix_micros_to_utc_timestamp(-17987443200 * 1000000 - 1);
TIMESTAMP,TIMESTAMP,TIMESTAMP,TIMESTAMP,TIMESTAMP,TIMESTAMP,TIMESTAMP
---- RESULTS
NULL,1970-01-01 00:00:00,1970-01-01 00:00:00.000001000,1400-01-01 00:00:00,9999-12-31 23:59:59,9999-12-31 23:59:59.000001000,NULL
-====
----- QUERY
-# This following test is copied from local-timestamp-functions.test as from IMPALA-10171
-# flag use_local_tz_for_unix_timestamp_conversions can be overriden with a query option.
-# TODO: remove/reduce this test duplication
-SET timezone=CET;
-SET use_local_tz_for_unix_timestamp_conversions=1;
-SELECT
-from_unixtime(0),
-from_unixtime((40 * 365) * 24 * 60 * 60),
-from_unixtime((180 + 40 * 365) * 24 * 60 * 60);
----- TYPES
-STRING,STRING,STRING
----- RESULTS
-'1970-01-01 01:00:00','2009-12-22 01:00:00','2010-06-20 02:00:00'
-====
----- QUERY
-# This following test is copied from local-timestamp-functions.test as from IMPALA-10171
-# flag use_local_tz_for_unix_timestamp_conversions can be overriden with a query option.
-# TODO: remove/reduce this test duplication
-set timezone="America/Los_Angeles";
-SET use_local_tz_for_unix_timestamp_conversions=1;
-SELECT
-from_unixtime(0),
-from_unixtime((40 * 365) * 24 * 60 * 60),
-from_unixtime((180 + 40 * 365) * 24 * 60 * 60);
----- TYPES
-STRING,STRING,STRING
----- RESULTS
-'1969-12-31 16:00:00','2009-12-21 16:00:00','2010-06-19 17:00:00'
-====
----- QUERY
-# This following test is copied from local-timestamp-functions.test as from IMPALA-10171
-# flag use_local_tz_for_unix_timestamp_conversions can be overriden with a query option.
-# TODO: remove/reduce this test duplication
-SET timezone=CET;
-SET use_local_tz_for_unix_timestamp_conversions=1;
-SELECT
-unix_timestamp('1970-01-01 01:00:00'),
-unix_timestamp('2009-12-22 01:00:00'),
-unix_timestamp('2010-06-20 02:00:00');
----- TYPES
-BIGINT,BIGINT,BIGINT
----- RESULTS
-0,1261440000,1276992000
-====
----- QUERY
-# This following test is copied from local-timestamp-functions.test as from IMPALA-10171
-# flag use_local_tz_for_unix_timestamp_conversions can be overriden with a query option.
-# TODO: remove/reduce this test duplication
-set timezone="America/Los_Angeles";
-SET use_local_tz_for_unix_timestamp_conversions=1;
-SELECT
-unix_timestamp('1969-12-31 16:00:00'),
-unix_timestamp('2009-12-21 16:00:00'),
-unix_timestamp('2010-06-19 17:00:00');
----- TYPES
-BIGINT,BIGINT,BIGINT
----- RESULTS
-0,1261440000,1276992000
-====
----- QUERY
-# This following test is copied from local-timestamp-functions.test as from IMPALA-10171
-# flag use_local_tz_for_unix_timestamp_conversions can be overriden with a query option.
-# TODO: remove/reduce this test duplication
-SET timezone=CET;
-SET use_local_tz_for_unix_timestamp_conversions=1;
-select cast(0 as timestamp);
----- TYPES
-TIMESTAMP
----- RESULTS
-1970-01-01 01:00:00
-====
----- QUERY
-# This following test is copied from local-timestamp-functions.test as from IMPALA-10171
-# flag use_local_tz_for_unix_timestamp_conversions can be overriden with a query option.
-# TODO: remove/reduce this test duplication
-SET timezone="America/Los_Angeles";
-SET use_local_tz_for_unix_timestamp_conversions=1;
-select cast(0 as timestamp);
----- TYPES
-TIMESTAMP
----- RESULTS
-1969-12-31 16:00:00
-====
----- QUERY
-# Regression test for IMPALA-5121. AVG used to contain a back and forth timezone
-# conversion (if use_local_tz_for_unix_timestamp_conversions is true) that could affect
-# the results if there were values from different DST rules.
-SET timezone=CET;
-SET use_local_tz_for_unix_timestamp_conversions=1;
-select avg(timestamp_col) from functional.alltypestiny;
----- TYPES
-TIMESTAMP
----- RESULTS
-2009-02-15 00:00:30
====
\ No newline at end of file
diff --git a/tests/common/impala_connection.py b/tests/common/impala_connection.py
index 323d2a2b8..50cddfe4d 100644
--- a/tests/common/impala_connection.py
+++ b/tests/common/impala_connection.py
@@ -32,6 +32,7 @@ import impala.dbapi as impyla
import impala.error as impyla_error
import impala.hiveserver2 as hs2
import tests.common
+from Query.ttypes import TQueryOptions
from RuntimeProfile.ttypes import TRuntimeProfileFormat
from tests.beeswax.impala_beeswax import (
DEFAULT_SLEEP_INTERVAL,
@@ -126,20 +127,46 @@ def build_summary_table_from_thrift(thrift_exec_summary):
return output
-def collect_default_query_options(options, name, val, kind):
- if kind == 'REMOVED':
- return
+def collect_default_query_options(options, name, val):
name = name.lower()
- val = str(val).strip('"')
- if ',' in val or '/' in val:
- # Value is a list or a timezone name containing a slash. Wrap it with double quotes.
- val = '"{}"'.format(val)
+ if val is not None:
+ val = str(val).strip('"')
+ if ',' in val or '/' in val:
+ # Value is a list or a timezone name containing a slash. Wrap it with double quotes.
+ val = '"{}"'.format(val)
if not val:
- # Value is optional with None as default.
+ # Value is optional with None as default or just turned into an empty string.
val = '""'
options[name] = val
+def parse_query_options_from_thrift():
+ """Populate 'options' map with default query options parsed from TQueryOptions
+ attributes."""
+ result = dict()
+ tquery_opts = TQueryOptions()
+ for key in dir(tquery_opts):
+ non_opts_attrs = ['read', 'write', 'validate', 'thrift_spec']
+ if not key.startswith('_') and key not in non_opts_attrs:
+ value = getattr(tquery_opts, key)
+ if isinstance(value, set):
+ # The default value of some query options, e.g.,
+ # enabled_runtime_filter_types, can be a set of integer.
+ # Turn the set into comma separated values.
+ value = ','.join([str(v) for v in value])
+ # No need to supply 'kind' since TQueryOptions already exclude
+ # removed query options.
+ collect_default_query_options(result, key, value)
+ return result
+
+
+# A map of default query option obtained from TQueryOptions.
+# Query option names (the keys) are in lower case string for consistency.
+# Values are all strings and might be double-quoted, making it legal for both setting
+# through 'SET' query or ImpalaConnection.set_configuration_option().
+DEFAULT_QUERY_OPTIONS = parse_query_options_from_thrift()
+
+
# Common wrapper around the internal types of HS2/Beeswax operation/query handles.
class OperationHandle(object):
def __init__(self, handle, sql_stmt):
@@ -202,12 +229,11 @@ class ImpalaConnection(with_metaclass(abc.ABCMeta, object)):
"""Clears all existing configuration."""
pass
- @abc.abstractmethod
def get_default_configuration(self):
"""Return the default configuration for the connection, before any modifications are
made to the session state. Returns a map with the config variable as the key and a
string representation of the default value as the value."""
- pass
+ return DEFAULT_QUERY_OPTIONS.copy()
@abc.abstractmethod
def connect(self):
@@ -421,9 +447,6 @@ class BeeswaxConnection(ImpalaConnection):
self.__beeswax_client = ImpalaBeeswaxClient(host_port, use_kerberos, user=user,
password=password, use_ssl=use_ssl)
self.__host_port = host_port
- # Default query option, obtained at first call to get_default_configuration().
- # Query option names are in lower case for consistency.
- self.__default_query_options = None
self.QUERY_STATES = self.__beeswax_client.query_states
def get_test_protocol(self):
@@ -443,19 +466,6 @@ class BeeswaxConnection(ImpalaConnection):
return True
return False
- def __collect_default_options(self):
- options = {}
- for item in self.__beeswax_client.get_default_configuration():
- collect_default_query_options(
- options, item.key, item.value, str(item.level))
- LOG.debug("Default query options: {0}".format(options))
- return options
-
- def get_default_configuration(self):
- if self.__default_query_options is None:
- self.__default_query_options = self.__collect_default_options()
- return self.__default_query_options.copy()
-
def clear_configuration(self):
self.__beeswax_client.clear_query_options()
# A hook in conftest sets tests.common.current_node.
@@ -597,9 +607,6 @@ class ImpylaHS2Connection(ImpalaConnection):
# that they will not share the same session with self.__cursor.
self.__impyla_conn = None
self.__cursor = None
- # Default query option obtained from initial connect.
- # Query option names are in lower case for consistency.
- self.__default_query_options = {}
# List of all cursors that created through execute_async.
self.__async_cursors = list()
# Query options to send along with each query.
@@ -631,9 +638,6 @@ class ImpylaHS2Connection(ImpalaConnection):
return True
return False
- def get_default_configuration(self):
- return self.__default_query_options.copy()
-
def clear_configuration(self):
self.__query_options.clear()
if hasattr(tests.common, "current_node") and not self._is_hive:
@@ -651,6 +655,11 @@ class ImpylaHS2Connection(ImpalaConnection):
# The session may no longer be valid if the impalad was restarted during the test.
pass
+ def default_cursor(self):
+ if self.__cursor is None:
+ self.__cursor = self.__open_single_cursor(user=self.__user)
+ return self.__cursor
+
def connect(self):
host, port = self.__host_port.split(":")
conn_kwargs = {}
@@ -660,18 +669,8 @@ class ImpylaHS2Connection(ImpalaConnection):
self.__impyla_conn = impyla.connect(
host=host, port=int(port), use_http_transport=self.__use_http_transport,
http_path=self.__http_path, use_ssl=self.__use_ssl, **conn_kwargs)
- self.__cursor = self.__open_single_cursor(user=self.__user)
- # Get the default query options for the session before any modifications are made.
- self.__default_query_options = {}
- if not self._is_hive:
- self.__cursor.execute("set all")
- for name, val, kind in self.__cursor:
- collect_default_query_options(self.__default_query_options, name, val, kind)
- self.__cursor.close_operation()
- LOG.debug("Default query options: {0}".format(self.__default_query_options))
- self.log_client("connected to {0} with impyla {1} session {2}".format(
- self.__host_port, self.get_test_protocol(), self.__get_session_id(self.__cursor)
- ))
+ self.log_client("connected to {0} with impyla {1}".format(
+ self.__host_port, self.get_test_protocol()))
except Exception as e:
self.log_client("failed connecting to {0} with impyla {1}".format(
self.__host_port, self.get_test_protocol()
@@ -681,7 +680,8 @@ class ImpylaHS2Connection(ImpalaConnection):
def close(self):
self.log_client("closing 1 sync and {0} async {1} connections to: {2}".format(
len(self.__async_cursors), self.get_test_protocol(), self.__host_port))
- self.__close_single_cursor(self.__cursor)
+ if self.__cursor is not None:
+ self.__close_single_cursor(self.__cursor)
for async_cursor in self.__async_cursors:
self.__close_single_cursor(async_cursor)
# Remove all async cursors.
@@ -699,8 +699,8 @@ class ImpylaHS2Connection(ImpalaConnection):
Returns a list of (catalogName, dbName, tableName, tableType, tableComment).
"""
self.log_client("getting tables for database: {0}".format(database))
- self.__cursor.get_tables(database_name=database)
- return self.__cursor.fetchall()
+ self.default_cursor().get_tables(database_name=database)
+ return self.default_cursor().fetchall()
def close_query(self, operation_handle, fetch_profile_after_close=False):
self.log_handle(operation_handle, 'closing query for operation')
@@ -719,29 +719,28 @@ class ImpylaHS2Connection(ImpalaConnection):
(u"executing against {0} at {1}. session: {2} main_cursor: {3} "
u"user: {4}\n{5}").format(
(self._is_hive and 'Hive' or 'Impala'), self.__host_port,
- self.__get_session_id(cursor), (cursor == self.__cursor), user,
+ self.__get_session_id(cursor), (cursor == self.default_cursor()), user,
format_sql_for_logging(sql_stmt))
)
def execute(self, sql_stmt, user=None, fetch_profile_after_close=False,
fetch_exec_summary=False, profile_format=TRuntimeProfileFormat.STRING):
- cursor = self.__cursor
+ same_user = (user == self.__user)
+ cursor = (self.default_cursor() if same_user
+ # Must create a new cursor to supply 'user'.
+ else self.__open_single_cursor(user=user))
result = None
try:
- if user != self.__user:
- # Must create a new cursor to supply 'user'.
- cursor = self.__open_single_cursor(user=user)
self.__log_execute(cursor, user, sql_stmt)
cursor.execute(sql_stmt, configuration=self.__query_options)
handle = OperationHandle(cursor, sql_stmt)
- self.log_handle(handle, "started query in session {0}".format(
- self.__get_session_id(cursor)))
+ self.log_handle(handle, "query started")
result = self.__fetch_results_and_profile(
handle, fetch_profile_after_close=fetch_profile_after_close,
fetch_exec_summary=fetch_exec_summary, profile_format=profile_format)
finally:
cursor.close_operation()
- if cursor != self.__cursor:
+ if not same_user:
self.__close_single_cursor(cursor)
return result
@@ -1103,9 +1102,6 @@ class MinimalHS2Connection(ImpalaConnection):
def clear_configuration(self):
raise NotImplementedError()
- def get_default_configuration(self):
- raise NotImplementedError()
-
def get_host_port(self):
return self.__host_port
diff --git a/tests/custom_cluster/test_local_tz_conversion.py b/tests/custom_cluster/test_local_tz_conversion.py
index 0543f92c8..46cc10b4e 100644
--- a/tests/custom_cluster/test_local_tz_conversion.py
+++ b/tests/custom_cluster/test_local_tz_conversion.py
@@ -18,9 +18,12 @@
from __future__ import absolute_import, division, print_function
import pytest
from tests.common.custom_cluster_test_suite import CustomClusterTestSuite
-from tests.common.test_vector import ImpalaTestDimension
from tests.common.skip import SkipIfFS
-from tests.common.test_dimensions import create_exec_option_dimension
+from tests.common.test_dimensions import (
+ add_exec_option_dimension,
+ create_exec_option_dimension,
+)
+
class TestLocalTzConversion(CustomClusterTestSuite):
"""Tests for --use_local_tz_for_unix_timestamp_conversions"""
@@ -32,16 +35,15 @@ class TestLocalTzConversion(CustomClusterTestSuite):
cluster_sizes=[0], disable_codegen_options=[False, True], batch_sizes=[0]))
# Test with and without expr rewrites to cover regular expr evaluations
# as well as constant folding, in particular, timestamp literals.
- cls.ImpalaTestMatrix.add_dimension(
- ImpalaTestDimension('enable_expr_rewrites', *[0,1]))
+ add_exec_option_dimension(cls, 'enable_expr_rewrites', [0, 1])
@classmethod
def add_custom_cluster_constraints(cls):
# Do not call the super() implementation because this class needs to relax the
# set of constraints.
cls.ImpalaTestMatrix.add_constraint(lambda v:
- v.get_value('table_format').file_format == 'text' and
- v.get_value('table_format').compression_codec == 'none')
+ v.get_value('table_format').file_format == 'text'
+ and v.get_value('table_format').compression_codec == 'none')
@SkipIfFS.hbase
@pytest.mark.execute_serially
@@ -51,9 +53,6 @@ class TestLocalTzConversion(CustomClusterTestSuite):
TODO: this test can be probably removed as a query option is created for
use_local_tz_for_unix_timestamp_conversions in IMPALA-10171
"""
- vector.get_value('exec_option')['enable_expr_rewrites'] = \
- vector.get_value('enable_expr_rewrites')
-
# Tests for UTC timestamp functions, i.e. functions that do not depend on the
# behavior of the flag --use_local_tz_for_unix_timestamp_conversions. These tests
# are also executed in test_exprs.py to ensure the same behavior when running
diff --git a/tests/custom_cluster/test_session_expiration.py b/tests/custom_cluster/test_session_expiration.py
index cf468891e..6694abd90 100644
--- a/tests/custom_cluster/test_session_expiration.py
+++ b/tests/custom_cluster/test_session_expiration.py
@@ -193,12 +193,16 @@ class TestSessionExpiration(CustomClusterTestSuite):
be seen in the webui."""
impalad = self.cluster.get_first_impalad()
self.close_impala_clients()
- # Create 2 sessions.
+ # Create 2 clients.
client1 = impalad.service.create_hs2_client()
client2 = impalad.service.create_hs2_client()
+ # Run query to open session.
+ client1.execute('select "client1 should succeed"')
+ client2.execute('select "client2 should succeed"')
try:
# Trying to open a third session should fail.
- impalad.service.create_hs2_client()
+ client3 = impalad.service.create_hs2_client()
+ client3.execute('select "client3 should fail"')
assert False, "should have failed"
except Exception as e:
assert re.match(r".*Number of sessions for user \S+ exceeds coordinator limit 2",
diff --git a/tests/query_test/test_exprs.py b/tests/query_test/test_exprs.py
index 55aace3c5..a4d4aaa7e 100644
--- a/tests/query_test/test_exprs.py
+++ b/tests/query_test/test_exprs.py
@@ -27,6 +27,7 @@ from tests.common.test_dimensions import (
add_exec_option_dimension,
create_exec_option_dimension,
create_uncompressed_text_dimension)
+from tests.common.skip import SkipIfFS
from tests.util.test_file_parser import QueryTestSectionReader
LOG = logging.getLogger('test_exprs')
@@ -288,27 +289,38 @@ class TestExprLimits(ImpalaTestSuite):
assert False, "Failed to execute query %s: %s" % (sql_str, e)
-class TestUtcTimestampFunctions(ImpalaTestSuite):
- """Tests for UTC timestamp functions, i.e. functions that do not depend on the behavior
- of the flag --use_local_tz_for_unix_timestamp_conversions. Tests added here should
- also be run in the custom cluster test test_local_tz_conversion.py to ensure they
- have the same behavior when the conversion flag is set to true."""
+class TestTimestampFunctions(ImpalaTestSuite):
+ """Tests for UTC timestamp functions and local timestamp functions."""
@classmethod
def add_test_dimensions(cls):
- super(TestUtcTimestampFunctions, cls).add_test_dimensions()
+ super(TestTimestampFunctions, cls).add_test_dimensions()
# Test with and without expr rewrites to cover regular expr evaluations
# as well as constant folding, in particular, timestamp literals.
add_exec_option_dimension(cls, 'enable_expr_rewrites', EXPR_REWRITE_OPTIONS)
- if cls.exploration_strategy() == 'core':
- # Test with file format that supports codegen
- cls.ImpalaTestMatrix.add_constraint(lambda v:
- v.get_value('table_format').file_format == 'text'
- and v.get_value('table_format').compression_codec == 'none')
+ add_exec_option_dimension(cls, 'use_local_tz_for_unix_timestamp_conversions',
+ [0, 1])
+ # No need to permute different file format.
+ cls.ImpalaTestMatrix.add_constraint(lambda v:
+ v.get_value('table_format').file_format == 'text'
+ and v.get_value('table_format').compression_codec == 'none')
def test_utc_functions(self, vector):
+ """Tests for UTC timestamp functions, i.e. functions that do not depend on the
+ behavior of the use_local_tz_for_unix_timestamp_conversions option."""
self.run_test_case('QueryTest/utc-timestamp-functions', vector)
+ @SkipIfFS.hbase
+ def test_timestamp_functions(self, vector):
+ if vector.get_exec_option('use_local_tz_for_unix_timestamp_conversions') == 1:
+ # Tests for local timestamp functions, i.e. functions that depend on the
+ # behavior of the use_local_tz_for_unix_timestamp_conversions option.
+ self.run_test_case('QueryTest/local-timestamp-functions', vector)
+
+ # Test that scanning of different file formats is not affected by
+ # use_local_tz_for_unix_timestamp_conversions option.
+ self.run_test_case('QueryTest/file-formats-with-local-tz-conversion', vector)
+
class TestConstantFoldingNoTypeLoss(ImpalaTestSuite):
""""Regression tests for IMPALA-11462."""
diff --git a/tests/query_test/test_observability.py b/tests/query_test/test_observability.py
index 0a6fac365..247124c65 100644
--- a/tests/query_test/test_observability.py
+++ b/tests/query_test/test_observability.py
@@ -380,7 +380,7 @@ class TestObservability(ImpalaTestSuite):
the table is cached."""
refresh_query = "refresh functional.alltypes"
select_query = "select * from functional.alltypes"
- self.execute_query(refresh_query).runtime_profile
+ self.execute_query(refresh_query, {'sync_ddl': '1'})
runtime_profile = self.execute_query(select_query).runtime_profile
event_regexes = [r'Query Compilation:',
r'Metadata of all .* tables cached:',
(impala) 03/04: IMPALA-13389: Refactor webUI scripts to use ES6 syntax
Posted by bo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
boroknagyz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 37d4a4eb56f56a392208eac4178ee2b739f90d52
Author: Surya Hebbar <sh...@cloudera.com>
AuthorDate: Wed Sep 25 14:35:37 2024 +0530
IMPALA-13389: Refactor webUI scripts to use ES6 syntax
Currently, the scripts in the entire webUI contain variable and function
declarations using the ES5 standard.
In such declarations, all the variables are attached to the browser's
window object, polluting the global scope. Additionally, many unnamed
functions can be represented with cleaner and concise syntax.
This patch refactors such declarations, using the ES6 syntax.
-> Replacing 'var' declarations with 'let' and 'const'
- To improve browser's memory utilization
- For better scoping, immutability and readability
-> Replacing unnamed function declarations with arrow functions
- Better scoping by binding 'this' object to the surrounding context
These improve maintainability and browser's memory utilization.
Across many instances within the webUI scripts, no particular naming
scheme is being followed for variable and function identifiers.
Hence, they have been revised with the following naming scheme.
-> All function names have been declared using camel case.
-> All constant primitive values and strings have been declared
in uppercase.
-> All other types of variables have been declared using snake case.
This naming scheme allows easier distinction between functions,
constants and other variables, based on the identifiers.
These changes to code style are further enforced during code review
through IMPALA-13473.
Change-Id: Ie38f2c642ede14956a2c6d551a58e42538204768
Reviewed-on: http://gerrit.cloudera.org:8080/21851
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
www/admission_controller.tmpl | 78 ++--
www/backends.tmpl | 6 +-
www/catalog.tmpl | 10 +-
www/catalog_ha_info.tmpl | 2 +-
www/catalog_operations.tmpl | 2 +-
www/common-header.tmpl | 4 +-
www/flags.tmpl | 2 +-
www/hadoop-varz.tmpl | 2 +-
www/jvm-threadz.tmpl | 2 +-
www/log_level.tmpl | 4 +-
www/metric_group.tmpl | 2 +-
www/profile_docs.tmpl | 2 +-
www/queries.tmpl | 58 +--
www/query_backends.tmpl | 18 +-
www/query_detail_tabs.tmpl | 80 ++--
www/query_finstances.tmpl | 28 +-
www/query_plan.tmpl | 84 ++---
www/query_plan_text.tmpl | 22 +-
www/query_profile.tmpl | 24 +-
www/query_stmt.tmpl | 28 +-
www/query_summary.tmpl | 17 +-
www/query_timeline.tmpl | 85 +++--
www/rpcz.tmpl | 98 ++---
www/scripts/common_util.js | 14 +-
www/scripts/queries/profileParseWorker.js | 6 +-
www/scripts/query_timeline/chart_commons.js | 43 ++-
www/scripts/query_timeline/fragment_diagram.js | 420 +++++++++++----------
.../query_timeline/fragment_metrics_diagram.js | 126 ++++---
www/scripts/query_timeline/global_members.js | 22 +-
.../query_timeline/host_utilization_diagram.js | 145 ++++---
.../tests/queries/profileParseWorker.test.js | 4 +-
.../tests/query_timeline/chart_commons.test.js | 102 ++---
.../tests/query_timeline/fragment_diagram.test.js | 4 +-
.../fragment_metrics_diagram.test.js | 16 +-
.../host_utilization_diagram.test.js | 18 +-
www/sessions.tmpl | 8 +-
www/thread-group.tmpl | 2 +-
37 files changed, 798 insertions(+), 790 deletions(-)
diff --git a/www/admission_controller.tmpl b/www/admission_controller.tmpl
index 7dd6e6d89..ae2d6b2e9 100644
--- a/www/admission_controller.tmpl
+++ b/www/admission_controller.tmpl
@@ -101,7 +101,7 @@ Example of json received from the impala server
{{/statestore_update_staleness_detail}}
<script src='{{ __common__.host-url }}/www/Chart-2.9.4.min.js'></script>
<script type="text/javascript">
-window.onload = function() {
+window.onload = () => {
renderGraph();
formatMemoryColumns();
};
@@ -110,30 +110,30 @@ window.onload = function() {
// label is always omitted. This function bakes in a lot of assumptions for our chart.
function afterFit(axis) {
const ticks = axis.getTicks();
- const widest = axis._labelSizes.widest.width;
- const chartWidth = axis.width;
+ const WIDEST_WIDTH = axis._labelSizes.widest.width;
+ const CHART_WIDTH = axis.width;
// Apply a rough heuristic for rotation.
- const maxFit = Math.trunc(3 * chartWidth / (2 * widest));
+ const MAX_FIT_COUNT = Math.trunc(3 * CHART_WIDTH / (2 * WIDEST_WIDTH));
// Limit to 20 labels.
- const willFit = Math.min(maxFit, 20);
+ const WILL_FIT_COUNT = Math.min(MAX_FIT_COUNT, 20);
// Ensure first and last are always shown.
- const validLabelIndices = new Set();
- validLabelIndices.add(0);
- validLabelIndices.add(ticks.length - 1);
+ const valid_label_indices = new Set();
+ valid_label_indices.add(0);
+ valid_label_indices.add(ticks.length - 1);
- const numLabels = ticks.length % 2 === 0 ? willFit : willFit - 1;
+ const NUM_LABELS = ticks.length % 2 === 0 ? WILL_FIT_COUNT : WILL_FIT_COUNT - 1;
- const interval = ticks.length / (numLabels - 1);
- for (let i = 1; i < willFit - 1; i += 1) {
- validLabelIndices.add(Math.floor(interval * i));
+ const INTERVAL = ticks.length / (NUM_LABELS - 1);
+ for (let i = 1; i < WILL_FIT_COUNT - 1; i += 1) {
+ valid_label_indices.add(Math.floor(INTERVAL * i));
}
ticks.forEach((tick, index) => {
Object.assign(
tick,
- { label: validLabelIndices.has(index) ? tick.label : null },
+ { label: valid_label_indices.has(index) ? tick.label : null },
);
});
}
@@ -141,24 +141,24 @@ function afterFit(axis) {
// Picks up all the canvas elements associated with each resource pool and renders its
// peak memory usage histogram.
function renderGraph() {
- var plainJson = document.getElementById("resource_pools_plain_json");
- var json = JSON.parse(plainJson.innerText);
- var canvases = document.getElementsByTagName("canvas");
+ const plain_json = document.getElementById("resource_pools_plain_json");
+ const json = JSON.parse(plain_json.innerText);
+ const canvases = document.getElementsByTagName("canvas");
- for (var pool_idx = 0; pool_idx < json.length; pool_idx++){
- var histogram = json[pool_idx]["peak_mem_usage_histogram"]
- var pool_name = json[pool_idx]['pool_name'];
- var hist_labels = new Array();
- var hist_values = new Array();
- for (var i = 0; i < histogram.length; i++) {
- var hist_elem = histogram[i];
+ for (let pool_idx = 0; pool_idx < json.length; pool_idx++){
+ const POOL_NAME = json[pool_idx]['pool_name'];
+ const histogram = json[pool_idx]["peak_mem_usage_histogram"];
+ const hist_labels = new Array();
+ const hist_values = new Array();
+ for (let i = 0; i < histogram.length; i++) {
+ const hist_elem = histogram[i];
//hist_labels.push((hist_elem[0]-1) + " - " + hist_elem[0] + " GB");
- hist_labels.push((hist_elem[0]+1) + " GB");
+ hist_labels.push((hist_elem[0] + 1) + " GB");
hist_values.push(hist_elem[1]);
}
hist_labels[hist_labels.length-1] += " and above"
// Render the bar chart now.
- var chart_Data = {
+ const chart_data = {
labels: hist_labels,
datasets: [{
label: 'Number of Queries',
@@ -167,10 +167,10 @@ function renderGraph() {
}],
};
- var ctx = canvases[pool_name].getContext('2d');
+ const ctx = canvases[POOL_NAME].getContext('2d');
window.myBar = new Chart(ctx, {
type: 'bar',
- data: chart_Data,
+ data: chart_data,
options: {
responsive: true,
legend: {
@@ -196,13 +196,13 @@ function renderGraph() {
// Picks up all the elements classified as memory and replaces it with pretty printed
// value.
function formatMemoryColumns() {
- var cols = document.getElementsByClassName("memory");
- for (var idx = 0; idx < cols.length; idx++) {
+ const cols = document.getElementsByClassName("memory");
+ for (let idx = 0; idx < cols.length; idx++) {
cols[idx].innerText = formatMemory(cols[idx].innerText);
}
}
-var memory_key = [
+const memory_key = [
{'unit': 'B', 'val': 1},
{'unit': 'KB', 'val': 1024},
{'unit': 'MB', 'val': 1048576},
@@ -211,19 +211,19 @@ var memory_key = [
// Helper method that takes in a value (in bytes) and outputs its pretty printed value.
function formatMemory(val) {
- var mem_bytes = parseInt(val);
- for (var idx = memory_key.length - 1; idx >= 0; idx--) {
- var result = parseFloat(mem_bytes / memory_key[idx].val);
- if (result < 1) continue;
- return result.toFixed(2) + " " + memory_key[idx].unit;
+ const MEM_BYTES = parseInt(val);
+ for (let idx = memory_key.length - 1; idx >= 0; idx--) {
+ const RESULT_MEM = parseFloat(MEM_BYTES / memory_key[idx].val);
+ if (RESULT_MEM < 1) continue;
+ return RESULT_MEM.toFixed(2) + " " + memory_key[idx].unit;
}
return "0 B";
}
function reset_all() {
if (!confirm('Are you sure you want to reset stats for all resource pools ?')) return;
- var xhr = new XMLHttpRequest();
- xhr.onload = function(ignored_arg) {
+ const xhr = new XMLHttpRequest();
+ xhr.onload = () => {
window.location.reload(true);
}
xhr.open('GET', make_url("/resource_pool_reset"), true);
@@ -232,8 +232,8 @@ function reset_all() {
function reset_method(pool_name) {
if (!confirm('Are you sure you want to reset stats for ' + pool_name +' ?')) return;
- var xhr = new XMLHttpRequest();
- xhr.onload = function(ignored_arg) {
+ const xhr = new XMLHttpRequest();
+ xhr.onload = () => {
window.location.reload(true);
}
xhr.open('GET', make_url("/resource_pool_reset?pool_name=" + pool_name), true);
diff --git a/www/backends.tmpl b/www/backends.tmpl
index 199dce56e..0774d228d 100644
--- a/www/backends.tmpl
+++ b/www/backends.tmpl
@@ -162,19 +162,19 @@ under the License.
{{/num_blacklisted_backends}}
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#backends').DataTable({
"order": [[ 0, "desc" ]],
"pageLength": 100
});
});
- $(document).ready(function() {
+ $(document).ready(() => {
$('#quiescing_backends').DataTable({
"order": [[ 0, "desc" ]],
"pageLength": 100
});
});
- $(document).ready(function() {
+ $(document).ready(() => {
$('#blacklisted_backends').DataTable({
"order": [[ 0, "desc" ]],
"pageLength": 100
diff --git a/www/catalog.tmpl b/www/catalog.tmpl
index 32a8eaf49..9ba4d3d1f 100644
--- a/www/catalog.tmpl
+++ b/www/catalog.tmpl
@@ -75,7 +75,7 @@ under the License.
</div>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#large-tables').DataTable({
"columnDefs": [{"targets": 1, "render": renderSize}],
"order": [[ 1, "desc" ]],
@@ -120,7 +120,7 @@ under the License.
</div>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#frequent-tables').DataTable({
"order": [[ 0, "desc" ]],
"pageLength": 10
@@ -164,7 +164,7 @@ under the License.
</div>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#high-file-count-tables').DataTable({
"order": [[ 1, "desc" ]],
"pageLength": 10
@@ -218,7 +218,7 @@ under the License.
</div>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#long-metadata-loading-tables').DataTable({
"order": [[ 2, "desc" ]],
"pageLength": 10,
@@ -282,7 +282,7 @@ under the License.
</div>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#{{name}}-tables').DataTable({
"pageLength": 5
});
diff --git a/www/catalog_ha_info.tmpl b/www/catalog_ha_info.tmpl
index e2376b3ec..0df45c6b0 100644
--- a/www/catalog_ha_info.tmpl
+++ b/www/catalog_ha_info.tmpl
@@ -73,7 +73,7 @@ under the License.
statestored for the catalog's high availability information.</h5>
{{/is_active_statestored}}
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#notified-subscribers-tbl').DataTable({
"order": [[ 6, "asc" ]],
"pageLength": 100
diff --git a/www/catalog_operations.tmpl b/www/catalog_operations.tmpl
index 0198dd4c4..24e483ccb 100644
--- a/www/catalog_operations.tmpl
+++ b/www/catalog_operations.tmpl
@@ -67,7 +67,7 @@ under the License.
</div>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#summary').DataTable({
"order": [[ 1, "desc" ]],
"pageLength": 10
diff --git a/www/common-header.tmpl b/www/common-header.tmpl
index 605fb2502..59a8db5b5 100644
--- a/www/common-header.tmpl
+++ b/www/common-header.tmpl
@@ -77,8 +77,8 @@ common-footer.tmpl) }}
// For Apache Knox compatibility, all urls that are accessed by javascript should have
// their path wrapped with this.
function make_url(path) {
- var root_link = document.getElementById('root-link');
- var s = root_link.href.split("?");
+ const root_link = document.getElementById('root-link');
+ const s = root_link.href.split("?");
url = s[0] + path;
if (s.length > 1) {
if (path.includes("?")) {
diff --git a/www/flags.tmpl b/www/flags.tmpl
index 1102057b9..c9cbd774d 100644
--- a/www/flags.tmpl
+++ b/www/flags.tmpl
@@ -43,7 +43,7 @@ under the License.
</table>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#flags').DataTable({
"order": [[ 1, "desc" ]],
"pageLength": 100
diff --git a/www/hadoop-varz.tmpl b/www/hadoop-varz.tmpl
index b17e74c7b..9e451068c 100644
--- a/www/hadoop-varz.tmpl
+++ b/www/hadoop-varz.tmpl
@@ -37,7 +37,7 @@ under the License.
</table>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#configs').DataTable({
"aaSorting": [[ 0, "asc" ]],
"pageLength": 100
diff --git a/www/jvm-threadz.tmpl b/www/jvm-threadz.tmpl
index 6c9fc9dc4..d4bae84ce 100644
--- a/www/jvm-threadz.tmpl
+++ b/www/jvm-threadz.tmpl
@@ -52,7 +52,7 @@ under the License.
</table>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#java-threads-tbl').DataTable({
"order": [[ 1, "asc" ]],
"pageLength": 100
diff --git a/www/log_level.tmpl b/www/log_level.tmpl
index 24df65704..b693d67fd 100644
--- a/www/log_level.tmpl
+++ b/www/log_level.tmpl
@@ -85,9 +85,9 @@ under the License.
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('select[name=glog]').val({{glog_level}});
- var text=$('select[name=glog]').children("option:selected").text();
+ const text = $('select[name=glog]').children("option:selected").text();
document.getElementById("glog_text").innerHTML = text;
});
diff --git a/www/metric_group.tmpl b/www/metric_group.tmpl
index a8124376a..fb15f0a8b 100644
--- a/www/metric_group.tmpl
+++ b/www/metric_group.tmpl
@@ -51,7 +51,7 @@ under the License.
</table>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#{{name}}-tbl').DataTable({
"order": [[ 0, "asc" ]],
"pageLength": 100
diff --git a/www/profile_docs.tmpl b/www/profile_docs.tmpl
index eddd6b8d7..b802f8bcb 100644
--- a/www/profile_docs.tmpl
+++ b/www/profile_docs.tmpl
@@ -81,7 +81,7 @@ under the License.
</div>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#counters-table').DataTable({
"pageLength": 20
});
diff --git a/www/queries.tmpl b/www/queries.tmpl
index 49c0ad458..3d87342f7 100644
--- a/www/queries.tmpl
+++ b/www/queries.tmpl
@@ -294,12 +294,12 @@ command line parameter.</p>
{{/query_locations}}
</table>
<script>
- var dbOpenReq = indexedDB.open("imported_queries");
- var db;
+ const db_open_req = indexedDB.open("imported_queries");
+ let db;
- const profileParseWorker = new Worker(
+ const profile_parse_worker = new Worker(
"{{ __common__.host-url }}/www/scripts/queries/profileParseWorker.js");
- var query_processor_start_time, raw_total_size, compressed_total_size, upload_count;
+ let query_processor_start_time, raw_total_size, compressed_total_size, upload_count;
function insertRowVal(row, val) {
row.insertCell().innerHTML = val;
@@ -311,9 +311,9 @@ command line parameter.</p>
}
function deleteProfile(el) {
- var row = el.parentElement.parentElement;
- var profileStore = db.transaction("profiles", "readwrite").objectStore("profiles");
- profileStore.delete(row.cells[0].textContent).onsuccess = () => {
+ const row = el.parentElement.parentElement;
+ const profile_store = db.transaction("profiles", "readwrite").objectStore("profiles");
+ profile_store.delete(row.cells[0].textContent).onsuccess = () => {
setScrollReload();
};
}
@@ -342,22 +342,22 @@ command line parameter.</p>
function uploadProfile() {
json_profile_chooser.disabled = true;
- var fileReader = new FileReader();
- fileReader.readAsText(json_profile_chooser.files[upload_count]);
- fileReader.onload = (e) => {
- profileParseWorker.postMessage(e.target.result);
+ const file_reader = new FileReader();
+ file_reader.readAsText(json_profile_chooser.files[upload_count]);
+ file_reader.onload = (e) => {
+ profile_parse_worker.postMessage(e.target.result);
};
}
- profileParseWorker.onmessage = (e) => {
+ profile_parse_worker.onmessage = (e) => {
if (e.data.error) {
showImportedQueriesStatusMessage("Error parsing some JSON profiles");
setTimeout(setScrollReload, 1500);
console.log(e.data.error);
return;
}
- var profileStore = db.transaction("profiles", "readwrite").objectStore("profiles");
- profileStore.put(e.data).onsuccess = () => {
+ const profile_store = db.transaction("profiles", "readwrite").objectStore("profiles");
+ profile_store.put(e.data).onsuccess = () => {
raw_total_size += json_profile_chooser.files[upload_count].size;
compressed_total_size += e.data.profile.length;
upload_count++;
@@ -376,38 +376,38 @@ command line parameter.</p>
};
};
- dbOpenReq.onupgradeneeded = (e) => {
+ db_open_req.onupgradeneeded = (e) => {
db = e.target.result;
- var profileStore = db.createObjectStore("profiles", { keyPath : "id" });
+ const profile_store = db.createObjectStore("profiles", { keyPath : "id" });
- profileStore.createIndex("start_time", "start_time");
+ profile_store.createIndex("start_time", "start_time");
};
- dbOpenReq.onsuccess = (e) => {
+ db_open_req.onsuccess = (e) => {
db = e.target.result;
db.onerror = (e) => {
console.log("IndexedDB error");
console.log(e);
}
- var profileStore = db.transaction("profiles", "readonly").objectStore("profiles");
- var countReq = profileStore.count();
+ const profile_store = db.transaction("profiles", "readonly").objectStore("profiles");
+ const count_req = profile_store.count();
- countReq.onsuccess = () => {
- if (countReq.result) {
+ count_req.onsuccess = () => {
+ if (count_req.result) {
imported_queries_header.innerHTML = imported_queries_header.innerHTML
- .replace("0", String(countReq.result));
+ .replace("0", String(count_req.result));
} else {
clear_profiles_button.remove();
}
};
- profileStore.index("start_time").openCursor(null, "prev").onsuccess = (e) => {
- var cursor = e.target.result;
+ profile_store.index("start_time").openCursor(null, "prev").onsuccess = (e) => {
+ const cursor = e.target.result;
if (cursor) {
- var query = cursor.value;
- var row = imported_queries_table.insertRow();
- var query_link = document.createElement("a");
+ const query = cursor.value;
+ const row = imported_queries_table.insertRow();
+ const query_link = document.createElement("a");
query_link.href = `{{ __common__.host-url }}/query_timeline?query_id=${query.id}&imported=true`;
query_link.innerHTML = query.id;
insertRowVal(row, query_link.outerHTML);
@@ -424,7 +424,7 @@ command line parameter.</p>
insertRowVal(row, query.coordinator_slots);
insertRowVal(row, query.executor_slots);
insertRowVal(row, query.statement);
- var deleteButton = document.createElement("input");
+ const deleteButton = document.createElement("input");
deleteButton.type = "button";
deleteButton.value = "X";
deleteButton.className = "btn btn-primary";
diff --git a/www/query_backends.tmpl b/www/query_backends.tmpl
index 3c8136701..c547999aa 100644
--- a/www/query_backends.tmpl
+++ b/www/query_backends.tmpl
@@ -25,7 +25,7 @@ under the License.
<label>
<input type="checkbox" checked="true" id="toggle" onClick="toggleRefresh()"/>
<span id="refresh_on">Auto-refresh on</span>
- </label> Last updated: <span id="last-updated"></span>
+ </label> Last updated: <span id="last_updated"></span>
</div>
<br/>
@@ -49,14 +49,14 @@ under the License.
</table>
<script>
-var intervalId = 0;
-var table = null;
-var refresh = function () {
+let interval_id = 0;
+let table = null;
+const refresh = () => {
table.ajax.reload();
document.getElementById("last-updated").textContent = new Date();
};
-$(document).ready(function() {
+$(document).ready(() => {
table = $('#backends').DataTable({
ajax: { url: make_url("/query_backends?query_id={{query_id}}&json"),
dataSrc: "backend_states",
@@ -73,15 +73,15 @@ $(document).ready(function() {
"order": [[ 0, "desc" ]],
"pageLength": 100
});
- intervalId = setInterval( refresh, 1000 );
+ interval_id = setInterval(refresh, 1000 );
});
function toggleRefresh() {
- if (document.getElementById("toggle").checked === true) {
- intervalId = setInterval(refresh, 1000);
+ if (document.getElementById("toogle").checked === true) {
+ interval_id = setInterval(refresh, 1000);
document.getElementById("refresh_on").textContent = "Auto-refresh on";
} else {
- clearInterval(intervalId);
+ clearInterval(interval_id);
document.getElementById("refresh_on").textContent = "Auto-refresh off";
}
}
diff --git a/www/query_detail_tabs.tmpl b/www/query_detail_tabs.tmpl
index 9850d69fe..6df50035c 100644
--- a/www/query_detail_tabs.tmpl
+++ b/www/query_detail_tabs.tmpl
@@ -107,47 +107,45 @@ under the License.
</ul>
<script>
-if (typeof index === "undefined") {
- var index = {};
-
- // For inflight (executing or waiting) query
- index.action = 0;
- index.default_db = 1;
- index.stmt_type = 2;
- index.start_time = 3;
-
- // For not inflight query
- index.end_time = 3;
-
- // For executing query
- index.executing = {
- duration : 4,
- queued_duration : 5,
- mem_usage : 6,
- mem_est : 7,
- progress : 8,
- query_progress : 9,
- bytes_read : 10,
- bytes_sent : 11,
- last_event : 12,
- rows_fetched : 13
- };
-
- // For waiting query
- index.waiting = {
- waiting_time : 4,
- duration : 5,
- queued_duration : 6,
- mem_usage : 7,
- mem_est : 8,
- progress : 9,
- query_progress : 10,
- bytes_read : 11,
- bytes_sent : 12,
- last_event : 13,
- rows_fetched : 14
- };
-}
+const index = {};
+
+// For inflight (executing or waiting) query
+index.action = 0;
+index.default_db = 1;
+index.stmt_type = 2;
+index.start_time = 3;
+
+// For not inflight query
+index.end_time = 3;
+
+// For executing query
+index.executing = {
+ duration : 4,
+ queued_duration : 5,
+ mem_usage : 6,
+ mem_est : 7,
+ progress : 8,
+ query_progress : 9,
+ bytes_read : 10,
+ bytes_sent : 11,
+ last_event : 12,
+ rows_fetched : 13
+};
+
+// For waiting query
+index.waiting = {
+ waiting_time : 4,
+ duration : 5,
+ queued_duration : 6,
+ mem_usage : 7,
+ mem_est : 8,
+ progress : 9,
+ query_progress : 10,
+ bytes_read : 11,
+ bytes_sent : 12,
+ last_event : 13,
+ rows_fetched : 14
+};
function update_executing_record(data_row, record_json) {
for (name in index.executing) {
diff --git a/www/query_finstances.tmpl b/www/query_finstances.tmpl
index 75c5464e3..739f12d17 100644
--- a/www/query_finstances.tmpl
+++ b/www/query_finstances.tmpl
@@ -46,9 +46,9 @@ under the License.
</table>
<script>
-var intervalId = 0;
-var table = null;
-var refresh = function () {
+let interval_id = 0;
+let table = null;
+const refresh = () => {
table.ajax.reload();
document.getElementById("last-updated").textContent = new Date();
};
@@ -57,7 +57,7 @@ var refresh = function () {
// row. Also clears the last report timestamp field for instances that have not started or
// have already finished execution.
function unpackJson(json) {
- var result = new Array();
+ const result = new Array();
if (typeof json.backend_instances === "undefined") {
// Table will be empty, remove it.
table.table().destroy(true);
@@ -69,11 +69,11 @@ function unpackJson(json) {
toggleRefresh();
return json;
}
- for (var i = 0; i < json.backend_instances.length; ++i) {
- var backend_state = json.backend_instances[i];
- var instance_stats = backend_state.instance_stats;
- for (var j = 0; j < instance_stats.length; ++j) {
- var instance = instance_stats[j];
+ for (let i = 0; i < json.backend_instances.length; ++i) {
+ const backend_state = json.backend_instances[i];
+ const instance_stats = backend_state.instance_stats;
+ for (let j = 0; j < instance_stats.length; ++j) {
+ const instance = instance_stats[j];
instance.host = backend_state.host;
if (instance.done) instance.time_since_last_heard_from = "";
if (!instance.first_status_update_received) {
@@ -86,7 +86,7 @@ function unpackJson(json) {
return result;
}
-$(document).ready(function() {
+$(document).ready(() => {
table = $('#finstances').DataTable({
ajax: { url: make_url("/query_finstances?query_id={{query_id}}&json"),
dataSrc: unpackJson,
@@ -100,15 +100,15 @@ $(document).ready(function() {
"order": [[ 0, "desc" ]],
"pageLength": 100
});
- intervalId = setInterval(refresh, 1000);
+ interval_id = setInterval(refresh, 1000);
});
function toggleRefresh() {
- if (document.getElementById("toggle").checked === true) {
- intervalId = setInterval(refresh, 1000);
+ if (toggle.getElementById("toogle").checked === true) {
+ interval_id = setInterval(refresh, 1000);
document.getElementById("refresh_on").textContent = "Auto-refresh on";
} else {
- clearInterval(intervalId);
+ clearInterval(interval_id);
document.getElementById("refresh_on").textContent = "Auto-refresh off";
}
}
diff --git a/www/query_plan.tmpl b/www/query_plan.tmpl
index ef82aa851..a54682d01 100644
--- a/www/query_plan.tmpl
+++ b/www/query_plan.tmpl
@@ -23,7 +23,7 @@ under the License.
<div class="container" style="width:1200px;margin:0 auto;">
-<style id="css">
+<style id="page_export_css">
/* Text style for graph nodes */
.node {
color: white;
@@ -113,18 +113,18 @@ data_stream_target attribute. -->
<script>
$("#plan-tab").addClass("active");
-var g = new dagreD3.graphlib.Graph().setGraph({rankDir: "BT"});
+const g = new dagreD3.graphlib.Graph().setGraph({rankDir: "BT"});
-var svg = d3.select("svg");
-var inner = svg.select("g");
+const svg = d3.select("svg");
+const inner = svg.select("g");
-var export_link = document.getElementById("export_link");
-var export_filename = document.getElementById("export_filename");
-var export_format = document.getElementById("export_format");
+const export_link = document.getElementById("export_link");
+const export_filename = document.getElementById("export_filename");
+const export_format = document.getElementById("export_format");
export_filename.value = export_filename.value.replace(/\W/g,'_');
// Set up zoom support
-var zoom = d3.behavior.zoom().on("zoom", function() {
+const zoom = d3.behavior.zoom().on("zoom", () => {
inner.attr("transform", "translate(" + d3.event.translate + ")" +
"scale(" + d3.event.scale + ")");
});
@@ -132,11 +132,11 @@ svg.call(zoom);
// Set of colours to use, with the same colour used for every node in the same plan
// fragment.
-var colours = ["#A9A9A9", "#FF8C00", "#8A2BE2", "#A52A2A", "#00008B", "#006400",
+const colours = ["#A9A9A9", "#FF8C00", "#8A2BE2", "#A52A2A", "#00008B", "#006400",
"#228B22", "#4B0082", "#DAA520", "#008B8B", "#000000", "#DC143C"]
// Shades of red in order of intensity, used for colouring nodes by time taken
-var cols_by_time = ["#000000", "#1A0500", "#330A00", "#4C0F00", "#661400", "#801A00",
+const cols_by_time = ["#000000", "#1A0500", "#330A00", "#4C0F00", "#661400", "#801A00",
"#991F00", "#B22400", "#CC2900", "#E62E00", "#FF3300", "#FF4719"];
// Recursively build a list of edges and states that comprise the plan graph
@@ -151,9 +151,9 @@ function build(node, parent, edges, states, colour_idx, max_node_time) {
"max_time_val": node["max_time_val"],
"style": "fill: " + colours[colour_idx]});
if (parent != null) {
- var label_val = "" + node["output_card"].toLocaleString();
+ const LABEL_VAL = "" + node["output_card"].toLocaleString();
edges.push({ start: node["label"], end: parent,
- style: { label: label_val }});
+ style: { label: LABEL_VAL }});
}
// Add an inter-fragment edges
if (node["data_stream_target"]) {
@@ -171,35 +171,35 @@ function build(node, parent, edges, states, colour_idx, max_node_time) {
});
}
max_node_time = Math.max(node["max_time_val"], max_node_time)
- for (var i = 0; i < node["children"].length; ++i) {
+ for (let i = 0; i < node["children"].length; ++i) {
max_node_time = build(
node["children"][i], node["label"], edges, states, colour_idx, max_node_time);
}
return max_node_time;
}
-var is_first = true;
+let is_first = true;
function renderGraph(ignored_arg) {
if (req.status != 200) return;
- var json = JSON.parse(req.responseText);
+ const json = JSON.parse(req.responseText);
if (json.error) {
- clearInterval(intervalId);
+ clearInterval(interval_id);
return;
}
refresh_record(json.record_json);
- var plan = json["plan_json"];
- var inflight = json["inflight"];
+ const plan = json["plan_json"];
+ const inflight = json["inflight"];
if (!inflight) {
- clearInterval(intervalId);
+ clearInterval(interval_id);
}
- var states = []
- var edges = []
- var colour_idx = 0;
+ const states = []
+ const edges = []
+ let colour_idx = 0;
- var max_node_time = 0;
- plan["plan_nodes"].forEach(function(parent) {
+ let max_node_time = 0;
+ plan["plan_nodes"].forEach((parent) => {
max_node_time = Math.max(
build(parent, null, edges, states, colour_idx, max_node_time));
// Pick a new colour for each plan fragment
@@ -207,10 +207,10 @@ function renderGraph(ignored_arg) {
});
// Keep a map of names to states for use when processing edges.
- var states_by_name = { }
- states.forEach(function(state) {
+ const states_by_name = { }
+ states.forEach((state) => {
// Build the label for the node from the name and the detail
- var html = "<span>" + state.name + "</span><br/>";
+ let html = "<span>" + state.name + "</span><br/>";
html += "<span>" + state.detail + "</span><br/>";
html += "<span>" + state.num_instances + " instance";
if (state.num_instances > 1) {
@@ -219,13 +219,13 @@ function renderGraph(ignored_arg) {
html += "</span><br/>";
html += "<span>Max: " + state.max_time + ", avg: " + state.avg_time + "</span>";
- var style = state.style;
+ let style = state.style;
// If colouring nodes by total time taken, choose a shade in the cols_by_time list
// with idx proportional to the max time of the node divided by the max time over all
// nodes.
if (document.getElementById("colour_scheme").checked) {
- var idx = (cols_by_time.length - 1) * (state.max_time_val / (1.0 * max_node_time));
+ let idx = (cols_by_time.length - 1) * (state.max_time_val / (1.0 * max_node_time));
style = "fill: " + cols_by_time[Math.floor(idx)];
}
g.setNode(state.name, { "label": html,
@@ -234,7 +234,7 @@ function renderGraph(ignored_arg) {
states_by_name[state.name] = state;
});
- edges.forEach(function(edge) {
+ edges.forEach((edge) => {
// Impala marks 'broadcast' as a property of the receiver, not the sender. We use
// '(BCAST)' to denote that a node is duplicating its output to all receivers.
if (states_by_name[edge.end].is_broadcast) {
@@ -243,24 +243,24 @@ function renderGraph(ignored_arg) {
g.setEdge(edge.start, edge.end, edge.style);
});
- g.nodes().forEach(function(v) {
- var node = g.node(v);
+ g.nodes().forEach((v) => {
+ const node = g.node(v);
node.rx = node.ry = 5;
});
// Create the renderer
- var render = new dagreD3.render();
+ const render = new dagreD3.render();
// Run the renderer. This is what draws the final graph.
render(inner, g);
// Center the graph, but only the first time through (so as to not lose user zooms).
if (is_first) {
- var initialScale = 0.75;
- zoom.translate([(svg.attr("width") - g.graph().width * initialScale) / 2, 20])
- .scale(initialScale)
+ const initial_scale = 0.75;
+ zoom.translate([(svg.attr("width") - g.graph().width * initial_scale) / 2, 20])
+ .scale(initial_scale)
.event(svg);
- svg.attr('height', Math.max(g.graph().height * initialScale + 40, 600));
+ svg.attr('height', Math.max(g.graph().height * initial_scale + 40, 600));
is_first = false;
}
@@ -276,11 +276,11 @@ function refresh() {
}
// Attaches a blob of the current SVG viewport to the associated link
-export_link.addEventListener('click', function(event) {
+export_link.addEventListener('click', (event) => {
if (export_format.value === ".html") {
- var svg_viewport = document.querySelector("svg");
- var export_style = document.getElementById("css");
- var html_blob = new Blob([`<!DOCTYPE html><body>`,
+ const svg_viewport = document.querySelector("svg");
+ const export_style = document.getElementById("page_export_css");
+ const html_blob = new Blob([`<!DOCTYPE html><body>`,
`<h1 style="font-family:monospace;">Query {{query_id}}</h1>`,
export_style.outerHTML, svg_viewport.outerHTML, `</body></html>`],
{type: "text/html;charset=utf-8"});
@@ -293,6 +293,6 @@ export_link.addEventListener('click', function(event) {
// Force one refresh before starting the timer.
refresh();
-var intervalId = setInterval(refresh, 2000);
+const interval_id = setInterval(refresh, 2000);
</script>
diff --git a/www/query_plan_text.tmpl b/www/query_plan_text.tmpl
index 57b780aca..4797f876c 100644
--- a/www/query_plan_text.tmpl
+++ b/www/query_plan_text.tmpl
@@ -29,19 +29,19 @@ $("#plan-text-tab").addClass("active");
import {inflateParseJSON}
from "{{ __common__.host-url }}/www/scripts/compression_util.js";
-var dbOpenReq = indexedDB.open("imported_queries");
-var db;
+const db_open_req = indexedDB.open("imported_queries");
+let db;
-var supported_tabs = ["Query", "Timeline", "Text plan", "Profile"];
+const supported_tabs = ["Query", "Timeline", "Text plan", "Profile"];
if (window.location.search.includes("imported")) {
- var alertMessage = document.getElementsByClassName("alert alert-danger")[0];
- if (alertMessage) {
- alertMessage.remove();
+ const alert_message = document.getElementsByClassName("alert alert-danger")[0];
+ if (alert_message) {
+ alert_message.remove();
}
- var nav_links = document.getElementsByClassName("nav nav-tabs")[0];
+ const nav_links = document.getElementsByClassName("nav nav-tabs")[0];
nav_links = nav_links.getElementsByClassName("nav-link");
- for (var i = 0; i < nav_links.length;) {
+ for (let i = 0; i < nav_links.length;) {
if (supported_tabs.includes(nav_links[i].textContent)) {
nav_links[i].href = `${nav_links[i].href}&imported=true`;
i++;
@@ -50,14 +50,14 @@ if (window.location.search.includes("imported")) {
}
}
- dbOpenReq.onsuccess = (e) => {
+ db_open_req.onsuccess = (e) => {
db = e.target.result;
db.onerror = (e) => {
console.log("IndexedDB error");
console.log(e);
}
- var profileStore = db.transaction("profiles", "readonly").objectStore("profiles");
- profileStore.get(getQueryID()).onsuccess = (e) => {
+ const profile_store = db.transaction("profiles", "readonly").objectStore("profiles");
+ profile_store.get(getQueryID()).onsuccess = (e) => {
query_plan.textContent = inflateParseJSON(e.target.result.profile).contents
.child_profiles[0].info_strings.find(({key}) => key === "Plan").value;
};
diff --git a/www/query_profile.tmpl b/www/query_profile.tmpl
index 5fce17e62..d21ff5e68 100644
--- a/www/query_profile.tmpl
+++ b/www/query_profile.tmpl
@@ -49,10 +49,10 @@ $("#profile-tab").addClass("active");
import {inflateParseJSON}
from "{{ __common__.host-url }}/www/scripts/compression_util.js";
-var dbOpenReq = indexedDB.open("imported_queries");
-var db;
+const db_open_req = indexedDB.open("imported_queries");
+let db;
-var supported_tabs = ["Query", "Timeline", "Text plan", "Profile"];
+const supported_tabs = ["Query", "Timeline", "Text plan", "Profile"];
function profileToString(profile, indent="") {
let info_strings = "";
@@ -88,19 +88,19 @@ function profileToString(profile, indent="") {
return `${indent}${profile.profile_name}:\n${info_strings}${event_sequences}${child_profiles}${counters}`;
}
-document.querySelectorAll('.profile-download').forEach(function (profile_link) {
+document.querySelectorAll('.profile-download').forEach((profile_link) => {
profile_link.download = profile_link.download.replace(/\W/g,'_');
});
if (window.location.search.includes("imported")) {
profile_download_section.remove();
- var alertMessage = document.getElementsByClassName("alert alert-danger")[0];
- if (alertMessage) {
- alertMessage.remove();
+ const alert_message = document.getElementsByClassName("alert alert-danger")[0];
+ if (alert_message) {
+ alert_message.remove();
}
- var nav_links = document.getElementsByClassName("nav nav-tabs")[0];
+ const nav_links = document.getElementsByClassName("nav nav-tabs")[0];
nav_links = nav_links.getElementsByClassName("nav-link");
- for (var i = 0; i < nav_links.length;) {
+ for (let i = 0; i < nav_links.length;) {
if (supported_tabs.includes(nav_links[i].textContent)) {
nav_links[i].href = `${nav_links[i].href}&imported=true`;
i++;
@@ -109,14 +109,14 @@ if (window.location.search.includes("imported")) {
}
}
- dbOpenReq.onsuccess = (e) => {
+ db_open_req.onsuccess = (e) => {
db = e.target.result;
db.onerror = (e) => {
console.log("IndexedDB error");
console.log(e);
}
- var profileStore = db.transaction("profiles", "readonly").objectStore("profiles");
- profileStore.get(getQueryID()).onsuccess = (e) => {
+ const profile_store = db.transaction("profiles", "readonly").objectStore("profiles");
+ profile_store.get(getQueryID()).onsuccess = (e) => {
plain_text_profile_field.textContent = profileToString(
inflateParseJSON(e.target.result.profile).contents);
};
diff --git a/www/query_stmt.tmpl b/www/query_stmt.tmpl
index c2e08611c..5756a8dff 100644
--- a/www/query_stmt.tmpl
+++ b/www/query_stmt.tmpl
@@ -39,19 +39,19 @@ $("#stmt-tab").addClass("active");
import {inflateParseJSON}
from "{{ __common__.host-url }}/www/scripts/compression_util.js";
-var dbOpenReq = indexedDB.open("imported_queries");
-var db;
+const db_open_req = indexedDB.open("imported_queries");
+let db;
-var supported_tabs = ["Query", "Timeline", "Text plan", "Profile"];
+const supported_tabs = ["Query", "Timeline", "Text plan", "Profile"];
if (window.location.search.includes("imported")) {
- var alertMessage = document.getElementsByClassName("alert alert-danger")[0];
- if (alertMessage) {
- alertMessage.remove();
+ const alert_message = document.getElementsByClassName("alert alert-danger")[0];
+ if (alert_message) {
+ alert_message.remove();
}
- var nav_links = document.getElementsByClassName("nav nav-tabs")[0];
+ const nav_links = document.getElementsByClassName("nav nav-tabs")[0];
nav_links = nav_links.getElementsByClassName("nav-link");
- for (var i = 0; i < nav_links.length;) {
+ for (let i = 0; i < nav_links.length;) {
if (supported_tabs.includes(nav_links[i].textContent)) {
nav_links[i].href = `${nav_links[i].href}&imported=true`;
i++;
@@ -60,19 +60,19 @@ if (window.location.search.includes("imported")) {
}
}
- dbOpenReq.onsuccess = (e) => {
+ db_open_req.onsuccess = (e) => {
db = e.target.result;
db.onerror = (e) => {
console.log("IndexedDB error");
console.log(e);
}
- var profileStore = db.transaction("profiles", "readonly").objectStore("profiles");
- profileStore.get(getQueryID()).onsuccess = (e) => {
- var sql_query = inflateParseJSON(e.target.result.profile).contents
+ const profile_store = db.transaction("profiles", "readonly").objectStore("profiles");
+ profile_store.get(getQueryID()).onsuccess = (e) => {
+ const sql_query = inflateParseJSON(e.target.result.profile).contents
.child_profiles[0].info_strings.find(({key}) => key === "Sql Statement").value;
- var sql_stmt_body = document.createElement("pre");
+ const sql_stmt_body = document.createElement("pre");
sql_stmt_body.className = "code";
- var sql_code = sql_stmt_body.appendChild(document.createElement("code"))
+ const sql_code = sql_stmt_body.appendChild(document.createElement("code"))
sql_code.innerHTML = sql_query;
tab_body.innerHTML = "";
tab_body.appendChild(sql_stmt_body);
diff --git a/www/query_summary.tmpl b/www/query_summary.tmpl
index 74e60cf27..bfb3976e3 100644
--- a/www/query_summary.tmpl
+++ b/www/query_summary.tmpl
@@ -25,7 +25,7 @@ under the License.
<label>
<input type="checkbox" checked="true" id="toggle" onClick="toggleRefresh()"/>
<span id="refresh_on">Auto-refresh on</span>
- </label> Last updated: <span id="last-updated"></span>
+ </label> Last updated: <span id="last_updated"></span>
</div>
<h3>Timeline</h3>
@@ -38,17 +38,17 @@ $("#summary-tab").addClass("active");
// Periodically refreshes the summary details
function refresh() {
- var xhr = new XMLHttpRequest();
+ const xhr = new XMLHttpRequest();
xhr.responseType = 'text';
xhr.timeout = 60000;
- xhr.onload = function(ignored_arg) {
+ xhr.onload = (ignored_arg) => {
if (xhr.status != 200) {
return;
}
- var blob = xhr.response;
+ const blob = xhr.response;
json = JSON.parse(blob);
if (json.error) {
- clearInterval(intervalId);
+ clearInterval(interval_id);
document.getElementById("toggle").checked = false;
return;
}
@@ -58,20 +58,19 @@ function refresh() {
document.getElementById("status").textContent = json["status"];
document.getElementById("last-updated").textContent = new Date();
}
- xhr.ontimeout = function(){ }
xhr.open('GET', make_url("/query_summary?query_id={{query_id}}&json"), true);
xhr.send();
}
document.getElementById("last-updated").textContent = new Date();
-var intervalId = setInterval(refresh, 1000);
+let interval_id = setInterval(refresh, 1000);
function toggleRefresh() {
if (document.getElementById("toggle").checked === true) {
- intervalId = setInterval(refresh, 1000);
+ interval_id = setInterval(refresh, 1000);
document.getElementById("refresh_on").textContent = "Auto-refresh on";
} else {
- clearInterval(intervalId);
+ clearInterval(interval_id);
document.getElementById("refresh_on").textContent = "Auto-refresh off";
}
}
diff --git a/www/query_timeline.tmpl b/www/query_timeline.tmpl
index 7413a6d2b..3c39c9cc2 100644
--- a/www/query_timeline.tmpl
+++ b/www/query_timeline.tmpl
@@ -28,7 +28,7 @@ under the License.
<div class="container">
-<style id="page_export-css">
+<style id="page_export_css">
#fragment_diagram, #phases_header, #timeticks_footer {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue",
Arial, "Noto Sans", sans-serif;
@@ -171,17 +171,17 @@ import {collectUtilizationFromProfile, toogleUtilizationVisibility,
import {collectFragmentMetricsFromProfile, closeFragmentMetricsChart} from
"{{ __common__.host-url }}/www/scripts/query_timeline/fragment_metrics_diagram.js";
import {profile, set_profile, maxts, set_maxts, diagram_width, set_diagram_width,
- border_stroke_width, resizeHorizontalAll}
+ BORDER_STROKE_WIDTH, resizeHorizontalAll}
from "{{ __common__.host-url }}/www/scripts/query_timeline/global_members.js";
import {inflateParseJSON} from "{{ __common__.host-url }}/www/scripts/compression_util.js";
-var chart_export_style;
-var last_maxts;
+let chart_export_style;
+let last_maxts;
-var dbOpenReq = indexedDB.open("imported_queries");
-var db;
+const db_open_req = indexedDB.open("imported_queries");
+let db;
-var supported_tabs = ["Query", "Timeline", "Text plan", "Profile"];
+const supported_tabs = ["Query", "Timeline", "Text plan", "Profile"];
function refreshView() {
collectFragmentEventsFromProfile();
@@ -196,13 +196,13 @@ function refreshView() {
if (window.location.search.includes("imported")) {
- var alertMessage = document.getElementsByClassName("alert alert-danger")[0];
- if (alertMessage) {
- alertMessage.remove();
+ const alert_message = document.getElementsByClassName("alert alert-danger")[0];
+ if (alert_message) {
+ alert_message.remove();
}
- var nav_links = document.getElementsByClassName("nav nav-tabs")[0];
+ const nav_links = document.getElementsByClassName("nav nav-tabs")[0];
nav_links = nav_links.getElementsByClassName("nav-link");
- for (var i = 0; i < nav_links.length;) {
+ for (let i = 0; i < nav_links.length;) {
if (supported_tabs.includes(nav_links[i].textContent)) {
nav_links[i].href = `${nav_links[i].href}&imported=true`;
i++;
@@ -211,22 +211,22 @@ if (window.location.search.includes("imported")) {
}
}
- dbOpenReq.onsuccess = (e) => {
+ db_open_req.onsuccess = (e) => {
db = e.target.result;
db.onerror = (e) => {
console.log("IndexedDB error");
console.log(e);
}
- var profileStore = db.transaction("profiles", "readonly").objectStore("profiles");
- profileStore.get(getQueryID()).onsuccess = (e) => {
+ const profile_store = db.transaction("profiles", "readonly").objectStore("profiles");
+ profile_store.get(getQueryID()).onsuccess = (e) => {
set_profile(inflateParseJSON(e.target.result.profile).contents);
refreshView();
};
};
} else {
window.onload = function refreshProfile() {
- var req = new XMLHttpRequest();
- req.onload = function() {
+ const req = new XMLHttpRequest();
+ req.onload = () => {
if (req.status === 200) {
set_profile(JSON.parse(req.responseText)["profile_json"]);
refreshView();
@@ -241,20 +241,20 @@ if (window.location.search.includes("imported")) {
}
}
-scrollable_screen.addEventListener('mousemove', function(e) {
- var scroll_margin = window.innerWidth / 10;
- var scroll_amount = window.innerWidth / 3;
- if (e.clientX <= scroll_margin) {
- scrollable_screen.scrollBy({left : - scroll_amount, behavior : "smooth"});
- } else if (e.clientX >= window.innerWidth - scroll_margin) {
- scrollable_screen.scrollBy({left : + scroll_amount, behavior : "smooth"});
+scrollable_screen.addEventListener('mousemove', (e) => {
+ const SCROLL_MARGIN = window.innerWidth / 10;
+ const SCROLL_AMOUNT = window.innerWidth / 3;
+ if (e.clientX <= SCROLL_MARGIN) {
+ scrollable_screen.scrollBy({left : - SCROLL_AMOUNT, behavior : "smooth"});
+ } else if (e.clientX >= window.innerWidth - SCROLL_MARGIN) {
+ scrollable_screen.scrollBy({left : + SCROLL_AMOUNT, behavior : "smooth"});
}
});
-window.addEventListener('resize', function(event) {
- var window_diagram_width = window.innerWidth - border_stroke_width;
- if (diagram_width <= window_diagram_width) {
- set_diagram_width(window_diagram_width);
+window.addEventListener('resize', (event) => {
+ const WINDOW_DIAGRAM_WIDTH = window.innerWidth - BORDER_STROKE_WIDTH;
+ if (diagram_width <= WINDOW_DIAGRAM_WIDTH) {
+ set_diagram_width(WINDOW_DIAGRAM_WIDTH);
hor_zoomout.disabled = true;
} else {
hor_zoomout.disabled = false;
@@ -262,40 +262,39 @@ window.addEventListener('resize', function(event) {
resizeHorizontalAll();
}, true);
-hor_zoomin.addEventListener('click', function(event) {
+hor_zoomin.addEventListener('click', (event) => {
set_diagram_width(diagram_width + window.innerWidth * 0.2)
set_ntics(ntics + 2);
resizeHorizontalAll();
hor_zoomout.disabled = false;
});
-hor_zoomout.addEventListener('click', function(event) {
- var prev_diagram_width = diagram_width;
- var next_width = diagram_width - window.innerWidth * 0.2;
- var window_diagram_width = window.innerWidth - border_stroke_width;
- if (next_width <= window_diagram_width) {
- next_width = window_diagram_width;
+hor_zoomout.addEventListener('click', (event) => {
+ const PREV_DIAGRAM_WIDTH = diagram_width;
+ const WINDOW_DIAGRAM_WIDTH = window.innerWidth - BORDER_STROKE_WIDTH;
+ let next_width = diagram_width - window.innerWidth * 0.2;
+ if (next_width <= WINDOW_DIAGRAM_WIDTH) {
+ next_width = WINDOW_DIAGRAM_WIDTH;
hor_zoomout.disabled = true;
}
set_diagram_width(next_width);
- var next_ntics = ntics + (diagram_width - prev_diagram_width) * 10 / window.innerWidth;
- next_ntics = Math.round(next_ntics);
- set_ntics(next_ntics);
+ const NEXT_NTICS = ntics + (diagram_width - PREV_DIAGRAM_WIDTH) * 10 / window.innerWidth;
+ NEXT_NTICS = Math.round(NEXT_NTICS);
+ set_ntics(NEXT_NTICS);
resizeHorizontalAll();
});
// Attaches a SVG blob of the complete timeline to the associated link
-export_link.addEventListener('click', function (event) {
+export_link.addEventListener('click', (event) => {
if (export_format.value === ".html") {
- var export_style = document.getElementById("page_export-css");
-
+ const export_style = document.getElementById("page_export_css");
// Deep clone 'parentNode's as wrappers to SVG components
- var fragment_diagram_wrapper = fragment_diagram.parentNode.cloneNode(true);
+ const fragment_diagram_wrapper = fragment_diagram.parentNode.cloneNode(true);
// Set dimensions for fragment diagram's wrapper
fragment_diagram_wrapper.style.height = fragment_diagram.style.height;
- var html_blob = new Blob(['<!DOCTYPE html><body>',
+ const html_blob = new Blob(['<!DOCTYPE html><body>',
'<h1 style=\"font-family:monospace;\">Query {{query_id}}</h1>',
`<style>${export_style.innerHTML} ${chart_export_style}</style>`,
phases_header.parentNode.outerHTML, fragment_diagram_wrapper.outerHTML,
diff --git a/www/rpcz.tmpl b/www/rpcz.tmpl
index 40eced089..780dcc2b4 100644
--- a/www/rpcz.tmpl
+++ b/www/rpcz.tmpl
@@ -234,33 +234,33 @@ under the License.
<script>
function reset_all() {
- var xhr = new XMLHttpRequest();
+ const xhr = new XMLHttpRequest();
xhr.open('GET', make_url("/rpcz_reset"), true);
xhr.send();
}
function reset_method(server, method) {
- var xhr = new XMLHttpRequest();
+ const xhr = new XMLHttpRequest();
xhr.open('GET', make_url("/rpcz_reset?server=" + server + "&method=" + method), true);
xhr.send();
}
function reset_server(server) {
- var xhr = new XMLHttpRequest();
+ const xhr = new XMLHttpRequest();
xhr.open('GET', make_url("/rpcz_reset?server=" + server), true);
xhr.send();
}
// Update all metrics for services in "servers", which use Impala's old RPC layer.
function update_impala_services(json) {
- for (var i = 0; i < json["servers"].length; ++i) {
- var tbl_json = json["servers"][i];
- var table = document.getElementById(tbl_json["name"]);
+ for (let i = 0; i < json["servers"].length; ++i) {
+ const tbl_json = json["servers"][i];
+ const table = document.getElementById(tbl_json["name"]);
if (!table) continue;
// Delete all existing rows, stopping at 1 to save the header
- for (var j = table.rows.length - 1; j >= 1; --j) table.deleteRow(j);
- tbl_json["methods"].forEach(function(method) {
- var row = table.insertRow();
+ for (let j = table.rows.length - 1; j >= 1; --j) table.deleteRow(j);
+ tbl_json["methods"].forEach((method) => {
+ const row = table.insertRow();
row.insertCell().innerHTML = "<samp>" + method.name + "</samp>";
row.insertCell().innerHTML = method.summary;
cell = row.insertCell()
@@ -272,12 +272,12 @@ function update_impala_services(json) {
cell.style.display= getRWDisplay();
cell.innerHTML = method.write;
row.insertCell().innerHTML = method.in_flight;
- var reset_cell = row.insertCell();
+ const reset_cell = row.insertCell();
reset_cell.align = "center";
- var button = document.createElement("button");
+ const button = document.createElement("button");
button.className = "btn btn-warning btn-xs";
button.appendChild(document.createTextNode("Reset"));
- button.onclick = function() { reset_method(method.server_name, method.name); }
+ button.onclick = () => { reset_method(method.server_name, method.name); }
reset_cell.appendChild(button);
});
}
@@ -287,47 +287,48 @@ function update_impala_services(json) {
function update_krpc_services(json) {
if (!json["services"]) return;
// Update each service
- for (var i = 0; i < json["services"].length; ++i) {
- var svc_json = json["services"][i];
- var svc_name = svc_json["service_name"];
- var table = document.getElementById(svc_name + "_metrics");
+ for (let i = 0; i < json["services"].length; ++i) {
+ const svc_json = json["services"][i];
+ const SVC_NAME = svc_json["service_name"];
+ const table = document.getElementById(SVC_NAME + "_metrics");
// Skip updates for unknown services.
if (!table) continue;
// Update service metrics
- var keys = ["queue_size", "idle_threads", "mem_usage", "mem_peak",
+ const keys = ["queue_size", "idle_threads", "mem_usage", "mem_peak",
"rpcs_queue_overflow", "incoming_queue_time"];
- for (var j = 0; j < keys.length; ++j) {
- var key = keys[j];
- var cell = document.getElementById(svc_name + "_" + key);
+ for (let j = 0; j < keys.length; ++j) {
+ const KEY = keys[j];
+ const cell = document.getElementById(SVC_NAME + "_" + KEY);
// Skip update for unknown values.
if (!cell) continue;
- cell.innerHTML = svc_json[key];
+ cell.innerHTML = svc_json[KEY];
}
// Update metrics for individual methods.
- var num_methods = svc_json["rpc_method_metrics"].length;
- for (var j = 0; j < num_methods; ++j) {
- var method_json = svc_json["rpc_method_metrics"][j];
- var method_name = method_json["method_name"];
+ const num_methods = svc_json["rpc_method_metrics"].length;
+ for (let j = 0; j < num_methods; ++j) {
+ const method_json = svc_json["rpc_method_metrics"][j];
+ const METHOD_NAME = method_json["method_name"];
// Update all metrics for this method.
- var keys = ["handler_latency", "payload_size"];
- for (var l = 0; l < keys.length; ++l) {
- var key = keys[l];
- var cell = $(table).find("." + method_name + "_" + key)[0];
+ const keys = ["handler_latency", "payload_size"];
+ for (let l = 0; l < keys.length; ++l) {
+ const KEY = keys[l];
+ const cell = $(table).find("." + METHOD_NAME + "_" + KEY)[0];
// Skip update for unknown values.
if (!cell) continue;
- cell.innerHTML = method_json[key];
+ cell.innerHTML = method_json[KEY];
}
}
}
}
function update_krpc_conn_metrics_datatable(json) {
- var table = $('#per_conn_metrics').DataTable();
+ const table = $('#per_conn_metrics').DataTable();
+ let rows;
if (!json["rpc_use_unix_domain_socket"]) {
- var rows = $.map(json["per_conn_metrics"], function(row) {
+ rows = $.map(json["per_conn_metrics"], (row) => {
return [[row["remote_addr"], row["num_calls_in_flight"], row["outbound_queue_size"],
row["socket_stats"]["rtt"] ?? '',
row["socket_stats"]["rttvar"] ?? '',
@@ -343,7 +344,7 @@ function update_krpc_conn_metrics_datatable(json) {
row["socket_stats"]["receive_queue_bytes"] ?? '']];
});
} else {
- var rows = $.map(json["per_conn_metrics"], function(row) {
+ rows = $.map(json["per_conn_metrics"], (row) => {
return [[row["remote_addr"], row["num_calls_in_flight"], row["outbound_queue_size"],
'N/A', 'N/A', 'N/A', 'N/A', 'N/A', 'N/A',
'N/A', 'N/A', 'N/A', 'N/A', 'N/A', 'N/A']];
@@ -354,9 +355,10 @@ function update_krpc_conn_metrics_datatable(json) {
}
function update_krpc_inbound_conn_metrics_datatable(json) {
- var table = $('#inbound_per_conn_metrics').DataTable();
+ const table = $('#inbound_per_conn_metrics').DataTable();
+ let rows;
if (!json["rpc_use_unix_domain_socket"]) {
- var rows = $.map(json["inbound_per_conn_metrics"], function(row) {
+ rows = $.map(json["inbound_per_conn_metrics"], (row) => {
return [[row["remote_addr"], row["num_calls_in_flight"],
row["socket_stats"]["rtt"] ?? '',
row["socket_stats"]["rttvar"] ?? '',
@@ -372,7 +374,7 @@ function update_krpc_inbound_conn_metrics_datatable(json) {
row["socket_stats"]["receive_queue_bytes"] ?? '']];
});
} else {
- var rows = $.map(json["inbound_per_conn_metrics"], function(row) {
+ rows = $.map(json["inbound_per_conn_metrics"], (row) => {
return [[row["remote_addr"], row["num_calls_in_flight"],
'N/A', 'N/A', 'N/A', 'N/A', 'N/A', 'N/A',
'N/A', 'N/A', 'N/A', 'N/A', 'N/A', 'N/A']];
@@ -382,7 +384,7 @@ function update_krpc_inbound_conn_metrics_datatable(json) {
table.clear().rows.add(rows).draw();
}
-$(document).ready(function() {
+$(document).ready(() => {
table = $('#per_conn_metrics').DataTable({
"order": [[ 0, "asc" ]],
"pageLength": 50
@@ -390,12 +392,12 @@ $(document).ready(function() {
});
function refresh() {
- var xhr = new XMLHttpRequest();
+ const xhr = new XMLHttpRequest();
xhr.responseType = 'text';
xhr.timeout = 60000;
- xhr.onload = function(ignored_arg) {
+ xhr.onload = () => {
if (xhr.status != 200) return;
- var blob = xhr.response;
+ const blob = xhr.response;
json = JSON.parse(blob);
update_impala_services(json);
update_krpc_services(json);
@@ -404,35 +406,35 @@ function refresh() {
document.getElementById("last-updated").textContent = new Date();
}
- xhr.ontimeout = function() {}
+ xhr.ontimeout = () => {}
xhr.open('GET', make_url("/rpcz?json"), true);
xhr.send();
}
document.getElementById("last-updated").textContent = new Date();
-var intervalId = setInterval(refresh, 1000);
+let interval_id = setInterval(refresh, 1000);
function toggleRefresh() {
- if (document.getElementById("toggle").checked === true) {
- intervalId = setInterval(refresh, 1000);
+ if (toggle.checked === true) {
+ interval_id = setInterval(refresh, 1000);
document.getElementById("refresh_on").textContent = "Auto-refresh on";
} else {
- clearInterval(intervalId);
+ clearInterval(interval_id);
document.getElementById("refresh_on").textContent = "Auto-refresh off";
}
}
function getRWDisplay() {
- if (document.getElementById('show_rw_stats').checked) {
+ if (document.getElementById("show_rw_stats").checked) {
return "";
}
return "none";
}
function toggleRWStats() {
- var rwDisplay = getRWDisplay();
+ const rw_display = getRWDisplay();
for (item of document.getElementsByClassName('rwstats')) {
- item.style.display = rwDisplay;
+ item.style.display = rw_display;
}
}
diff --git a/www/scripts/common_util.js b/www/scripts/common_util.js
index ff123d5d9..50f625b13 100644
--- a/www/scripts/common_util.js
+++ b/www/scripts/common_util.js
@@ -16,8 +16,8 @@
// under the License.
function getReadableSize(value, precision = 2) {
- var units = ['B', 'KB', 'MB', 'GB', 'TB', 'PB'];
- var current_unit = 0;
+ const units = ['B', 'KB', 'MB', 'GB', 'TB', 'PB'];
+ let current_unit = 0;
while( value >= 1e3 ) {
value = value / 1e3;
++current_unit;
@@ -38,10 +38,10 @@ function getReadableTimeNS(value) {
}
function getReadableTimeMS(value) {
- var hour = false;
- var minute = false;
- var second = false;
- var re = "";
+ let hour = false;
+ let minute = false;
+ let second = false;
+ let re = "";
if (value >= 3600000) {
re += (Math.floor(value / 3600000) + "h");
value = value % 3600000;
@@ -102,7 +102,7 @@ function renderTime(data, type, row) {
* Query ID always starts from index 10 in above format
*/
function getQueryID() {
- let query_id = window.location.search.split("query_id=")[1];
+ const query_id = window.location.search.split("query_id=")[1];
if (query_id) {
return query_id.substring(0, query_id.indexOf("&"));
}
diff --git a/www/scripts/queries/profileParseWorker.js b/www/scripts/queries/profileParseWorker.js
index 957c68de8..38a848396 100644
--- a/www/scripts/queries/profileParseWorker.js
+++ b/www/scripts/queries/profileParseWorker.js
@@ -19,10 +19,10 @@ importScripts("../../pako.min.js");
importScripts("../common_util.js");
self.onmessage = (e) => {
- var query = {};
+ const query = {};
try {
- var profile = JSON.parse(e.data).contents;
- var val = profile.profile_name;
+ const profile = JSON.parse(e.data).contents;
+ const val = profile.profile_name;
query.id = val.substring(val.indexOf("=") + 1, val.length - 1);
query.user = profile.child_profiles[0].info_strings
.find(({key}) => key === "User").value;
diff --git a/www/scripts/query_timeline/chart_commons.js b/www/scripts/query_timeline/chart_commons.js
index 760dd7718..c59b27594 100644
--- a/www/scripts/query_timeline/chart_commons.js
+++ b/www/scripts/query_timeline/chart_commons.js
@@ -17,19 +17,20 @@
import {maxts, set_maxts, clearDOMChildren} from "./global_members.js";
-export var exportedForTest;
+export let exportedForTest;
// In the data array provided for c3's line chart,
// 0th element is name of dataset
// 1st element is zero in all datasets(to compensate missing value for line chart)
// Hence the values index start at 2
-export var array_values_start_index = 2;
+export const ARRAY_VALUES_START_INDEX = 2;
function accumulateTimeseriesValues(values_array, time_series_counter, max_samples) {
- var samples = time_series_counter.data.split(",").map(el => parseInt(el));
- var max_traverse_len = Math.min(samples.length, values_array.length - 2);
+ const samples = time_series_counter.data.split(",").map(el => parseInt(el));
+ const MAX_TRAVERSE_LEN = Math.min(samples.length, values_array.length
+ - ARRAY_VALUES_START_INDEX);
if (isNaN(samples[0])) return;
- for (var j = 0; j < max_traverse_len; ++j) {
- values_array[j + 2] += samples[j];
+ for (let j = 0; j < MAX_TRAVERSE_LEN; ++j) {
+ values_array[j + ARRAY_VALUES_START_INDEX] += samples[j];
}
if (time_series_counter.num > max_samples.collected || time_series_counter.period
> max_samples.period) {
@@ -40,17 +41,18 @@ function accumulateTimeseriesValues(values_array, time_series_counter, max_sampl
}
export function generateTimesamples(timesamples_array, max_samples, extend) {
- var avg_period = max_samples.collected * max_samples.period / max_samples.available;
- avg_period = avg_period / 1000;
- var max_traverse_len = Math.min(max_samples.available, timesamples_array.length - 2);
- for (var k = 0; k <= max_traverse_len; ++k) {
- timesamples_array[k + 1] = (k * avg_period);
+ const AVG_PERIOD = max_samples.collected * max_samples.period /
+ (max_samples.available * 1000);
+ const MAX_TRAVERSE_LEN = Math.min(max_samples.available, timesamples_array.length
+ - ARRAY_VALUES_START_INDEX);
+ for (let k = 0; k <= MAX_TRAVERSE_LEN; ++k) {
+ timesamples_array[k + 1] = (k * AVG_PERIOD);
}
if (maxts / 1e9 > timesamples_array[max_samples.available + 1]) {
// extend by one additional point or correct the final point
if (extend) {
// extend
- timesamples_array[max_samples.available + 2] = maxts / 1e9;
+ timesamples_array[max_samples.available + ARRAY_VALUES_START_INDEX] = maxts / 1e9;
} else {
// correct
timesamples_array[max_samples.available + 1] = maxts / 1e9;
@@ -58,16 +60,17 @@ export function generateTimesamples(timesamples_array, max_samples, extend) {
} else {
set_maxts(timesamples_array[max_samples.available + 1] * 1e9);
}
- var j = max_samples.available + (extend ? 3 : 2);
+ let j = max_samples.available + (extend ? 3 : 2);
for (; j < timesamples_array.length && timesamples_array[j] !== null; ++j) {
timesamples_array[j] = null;
}
}
export function mapTimeseriesCounters(time_series_counters, counters) {
- for (var i = 0; i < counters.length; i++) {
- var no_change = true;
- for (var j = 0; j < time_series_counters.length; j++) {
+ let no_change;
+ for (let i = 0; i < counters.length; i++) {
+ no_change = true;
+ for (let j = 0; j < time_series_counters.length; j++) {
if (time_series_counters[j].counter_name === counters[i][0]) {
counters[i][2] = j;
no_change = false;
@@ -80,16 +83,16 @@ export function mapTimeseriesCounters(time_series_counters, counters) {
}
export function clearTimeseriesValues(values_array, max_samples) {
- var max_traverse_len = Math.min(max_samples.available + 1, values_array.length - 1);
- for (var j = array_values_start_index; j <= max_traverse_len; ++j) {
+ const MAX_TRAVERSE_LEN = Math.min(max_samples.available + 1, values_array.length - 1);
+ for (let j = ARRAY_VALUES_START_INDEX; j <= MAX_TRAVERSE_LEN; ++j) {
values_array[j] = null;
}
}
export function aggregateProfileTimeseries(parent_profile, aggregate_array,
counters, max_samples) {
- parent_profile.child_profiles.forEach(function (time_series_profiles) {
- for (var i = 0; i < aggregate_array.length; ++i) {
+ parent_profile.child_profiles.forEach((time_series_profiles) => {
+ for (let i = 0; i < aggregate_array.length; ++i) {
accumulateTimeseriesValues(aggregate_array[i],
time_series_profiles.time_series_counters[counters[i][2]],
max_samples);
diff --git a/www/scripts/query_timeline/fragment_diagram.js b/www/scripts/query_timeline/fragment_diagram.js
index dd0920ea7..74ab0123b 100644
--- a/www/scripts/query_timeline/fragment_diagram.js
+++ b/www/scripts/query_timeline/fragment_diagram.js
@@ -16,8 +16,8 @@
// under the License.
import {profile, set_maxts, maxts, decimals, set_decimals, diagram_width,
- set_diagram_width, diagram_min_height,
- margin_header_footer, border_stroke_width, margin_chart_end, clearDOMChildren,
+ set_diagram_width, DIAGRAM_MIN_HEIGHT,
+ MARGIN_HEADER_FOOTER, BORDER_STROKE_WIDTH, MARGIN_CHART_END, clearDOMChildren,
resizeHorizontalAll} from "./global_members.js";
import {host_utilization_chart, getUtilizationWrapperHeight}
from "./host_utilization_diagram.js";
@@ -26,23 +26,24 @@ import {fragment_metrics_chart, getFragmentMetricsWrapperHeight,
import {showTooltip, hideTooltip} from "./chart_commons.js"
import "./global_dom.js";
-export var exportedForTest;
+export let exportedForTest;
-export var name_width;
-export var page_additional_height;
+export let name_width;
+export let page_additional_height;
-var stroke_fill_colors = { black : "#000000", dark_grey : "#505050",
+const stroke_fill_colors = { black : "#000000", dark_grey : "#505050",
light_grey : "#F0F0F0", transperent : "rgba(0, 0, 0, 0)" };
-var rownum;
-var row_height = 15;
-var integer_part_estimate = 4;
-var char_width = 6;
-var bucket_size = 5;
-var fragment_events_parse_successful = false;
-var decimals_multiplier = Math.pow(10, decimals);
+const ROW_HEIGHT = 15;
+const INTEGER_PART_ESTIMATE = 4;
+const CHAR_WIDTH = 6;
+const BUCKET_SIZE = 5;
+
+let rownum;
+let fragment_events_parse_successful = false;
+let decimals_multiplier = Math.pow(10, 2);
// #phases_header
-var phases = [
+const phases = [
{ color: "#C0C0FF", label: "Prepare" },
{ color: "#E0E0E0", label: "Open" },
{ color: "#FFFFC0", label: "Produce First Batch" },
@@ -52,19 +53,17 @@ var phases = [
];
// #fragment_diagram
-var fragment_colors = ["#A9A9A9", "#FF8C00", "#8A2BE2", "#A52A2A", "#00008B", "#006400",
+const fragment_colors = ["#A9A9A9", "#FF8C00", "#8A2BE2", "#A52A2A", "#00008B", "#006400",
"#228B22", "#4B0082", "#DAA520", "#008B8B", "#000000", "#DC143C"];
-var fragments = [];
-var fragment_diagram_title = getSvgTitle("");
-var all_nodes = [];
-var receiver_nodes = [];
-var max_namelen = 0;
-var frag_name_width;
-var chart_width;
-var timestamp_gridline;
+const fragment_diagram_title = getSvgTitle("");
+
+let frag_name_width;
+let chart_width;
+let timestamp_gridline;
+let fragments = [];
// #timeticks_footer
-export var ntics = 10;
+export let ntics = 10;
export function set_ntics(val) {
ntics = val;
@@ -83,7 +82,7 @@ function rtd(num) { // round to decimals
// 'dasharray' and 'stroke_color' are optional parameters
function getSvgRect(fill_color, x, y, width, height, dasharray, stroke_color) {
- var rect = document.createElementNS("http://www.w3.org/2000/svg", "rect");
+ const rect = document.createElementNS("http://www.w3.org/2000/svg", "rect");
rect.setAttribute("x", rtd(x));
rect.setAttribute("y", rtd(y));
rect.setAttribute("width", rtd(width));
@@ -101,7 +100,7 @@ function getSvgRect(fill_color, x, y, width, height, dasharray, stroke_color) {
// 'max_width' is an optional parameters
function getSvgText(text, fill_color, x, y, height, container_center, max_width) {
- var text_el = document.createElementNS("http://www.w3.org/2000/svg", "text");
+ const text_el = document.createElementNS("http://www.w3.org/2000/svg", "text");
text_el.appendChild(document.createTextNode(text));
text_el.setAttribute("x", x);
text_el.setAttribute("y", y);
@@ -119,7 +118,7 @@ function getSvgText(text, fill_color, x, y, height, container_center, max_width)
}
function getSvgLine(stroke_color, x1, y1, x2, y2, dash) {
- var line = document.createElementNS("http://www.w3.org/2000/svg", "line");
+ const line = document.createElementNS("http://www.w3.org/2000/svg", "line");
line.setAttribute("x1", rtd(x1));
line.setAttribute("y1", rtd(y1));
line.setAttribute("x2", rtd(x2));
@@ -132,42 +131,42 @@ function getSvgLine(stroke_color, x1, y1, x2, y2, dash) {
}
function getSvgTitle(text) {
- var title = document.createElementNS("http://www.w3.org/2000/svg", "title");
+ const title = document.createElementNS("http://www.w3.org/2000/svg", "title");
title.textContent = text;
return title;
}
function getSvgGroup() {
- var group = document.createElementNS("http://www.w3.org/2000/svg", "g");
+ const group = document.createElementNS("http://www.w3.org/2000/svg", "g");
return group;
}
function markMinMaxAvg(event) {
if (event.no_bar !== undefined) return;
- var min = event.ts_list[0];
- var max = min;
- var i = 1;
+ let min = event.ts_list[0];
+ let max = min;
+ let i = 1;
for (; i < event.ts_list.length; i++) {
if (min > event.ts_list[i]) min = event.ts_list[i];
if (max < event.ts_list[i]) max = event.ts_list[i];
}
- var event_span_t = max - min;
- event.parts = Array(bucket_size).fill(null).map(
+ const EVENT_SPAN_T = max - min;
+ event.parts = Array(BUCKET_SIZE).fill(null).map(
() => ({count : 0, min : Infinity, max : -Infinity, avg : 0}));
- var k;
- var ts;
- var part;
+ let k;
+ let ts;
+ let part;
for (i = 0; i < event.ts_list.length; i++) {
ts = event.ts_list[i];
- k = ((ts - min) * bucket_size / event_span_t) | 0;
- if (k >= bucket_size) k = bucket_size - 1;
+ k = ((ts - min) * BUCKET_SIZE / EVENT_SPAN_T) | 0;
+ if (k >= BUCKET_SIZE) k = BUCKET_SIZE - 1;
part = event.parts[k];
if (ts < part.min) part.min = ts;
if (ts > part.max) part.max = ts;
part.avg += ts;
part.count += 1;
}
- for (k = 0; k < bucket_size; k++) {
+ for (k = 0; k < BUCKET_SIZE; k++) {
event.parts[k].avg /= event.parts[k].count;
}
}
@@ -197,18 +196,18 @@ function DrawBars(svg, rownum, row_height, events, xoffset, px_per_ns) {
// max : < Maximum timestamp in the current bucket >
// }, < ... 4 other objects of the same format > ]
// }
- var bar_height = row_height - 2;
- var last_e_index = events.length - 1;
- var last_ts_index = events[last_e_index].ts_list.length - 1;
- var plan_node = getSvgGroup();
+ const BAR_HEIGHT = row_height - 2;
+ const LAST_E_INDEX = events.length - 1;
+ const LAST_TS_INDEX = events[last_e_index].ts_list.length - 1;
+ let plan_node = getSvgGroup();
plan_node.classList.add("plan_node");
// coordinates start with (0,0) in the top-left corner
- var y = rownum * row_height; // the y-position for the current phase rectangle
- var cp_y = y; // copy of y, used to bring y-position back to the top(initial value)
- var dx, dy; // dimensions of the current phase rectangle
- var bucketed = false;
+ let y = rownum * row_height; // the y-position for the current phase rectangle
+ const cp_y = y; // copy of y, used to bring y-position back to the top(initial value)
+ let dx, dy; // dimensions of the current phase rectangle
+ let bucketed = false;
- if (events[0].ts_list.length > bucket_size) {
+ if (events[0].ts_list.length > BUCKET_SIZE) {
events.map(markMinMaxAvg);
bucketed = true;
} else {
@@ -217,25 +216,25 @@ function DrawBars(svg, rownum, row_height, events, xoffset, px_per_ns) {
});
}
- var i;
- var color_idx = last_e_index;
- for (i = 0; i <= last_e_index; ++i) {
+ let i;
+ let color_idx = LAST_E_INDEX;
+ for (i = 0; i <= LAST_E_INDEX; ++i) {
if (events[i].no_bar) --color_idx;
}
- var j;
- var dasharray; // current stroke-dasharray
- for (var i = last_e_index; i >= 0; --i) {
+ let j;
+ let dasharray; // current stroke-dasharray
+ for (let i = LAST_E_INDEX; i >= 0; --i) {
y = cp_y; // reuse calculation, returing to the initial y-position
if (events[i].no_bar === undefined) {
- var l_dx = 0; // previous phase rectangle's width (x-dimension)
+ let l_dx = 0; // previous phase rectangle's width (x-dimension)
if (bucketed) {
// Represent the aggregate distribution of instances for each phase
- var part;
+ let part;
for (j = 0; j < events[i].parts.length; j++) {
part = events[i].parts[j]; // mantain a reference for reuse
if (part.count > 0) {
- dy = (part.count / events[i].ts_list.length) * bar_height;
+ dy = (part.count / events[i].ts_list.length) * BAR_HEIGHT;
} else {
// skip to the next iteration, if the current bucket has no timestamps
continue;
@@ -255,7 +254,7 @@ function DrawBars(svg, rownum, row_height, events, xoffset, px_per_ns) {
}
} else {
// Represent phases of each instance seperately, according to the timestamps
- dy = bar_height / events[i].ts_list.length;
+ dy = BAR_HEIGHT / events[i].ts_list.length;
for (j = 0; j < events[i].ts_list.length; j++) {
dx = events[i].ts_list[j] * px_per_ns;
// Calculations for the 'stroke-dasharray' have been done same as above
@@ -271,15 +270,15 @@ function DrawBars(svg, rownum, row_height, events, xoffset, px_per_ns) {
}
y = rownum * row_height;
- if (events[last_e_index].no_bar === undefined) {
+ if (events[LAST_E_INDEX].no_bar === undefined) {
// Plan node's top and bottom outlines
- var top_edge_ts, bottom_edge_ts;
+ let top_edge_ts, bottom_edge_ts;
if (bucketed) {
- top_edge_ts = events[last_e_index].parts[0].max;
- bottom_edge_ts = events[last_e_index].parts[bucket_size - 1].max;
+ top_edge_ts = events[LAST_E_INDEX].parts[0].max;
+ bottom_edge_ts = events[LAST_E_INDEX].parts[bucket_size - 1].max;
} else {
- top_edge_ts = events[last_e_index].ts_list[0];
- bottom_edge_ts = events[last_e_index].ts_list[last_ts_index];
+ top_edge_ts = events[LAST_E_INDEX].ts_list[0];
+ bottom_edge_ts = events[LAST_E_INDEX].ts_list[LAST_TS_INDEX];
}
plan_node.appendChild(getSvgLine(stroke_fill_colors.black, xoffset, y,
xoffset + top_edge_ts * px_per_ns, y, false));
@@ -293,61 +292,61 @@ function DrawBars(svg, rownum, row_height, events, xoffset, px_per_ns) {
async function renderPhases() {
clearDOMChildren(phases_header);
- var color_idx = 0;
- var width = Math.ceil(chart_width / phases.length);
- phases.forEach(function(p) {
- var x = name_width + Math.ceil(chart_width * color_idx / phases.length);
- x = Math.min(x, diagram_width - width);
-
- phases_header.appendChild(getSvgRect(stroke_fill_colors.black, x, 0, width,
- row_height));
+ let color_idx = 0;
+ const PHASE_WIDTH = Math.ceil(chart_width / phases.length);
+ phases.forEach((p) => {
+ let x = name_width + Math.ceil(chart_width * color_idx / phases.length);
+ x = Math.min(x, diagram_width - PHASE_WIDTH);
+
+ phases_header.appendChild(getSvgRect(stroke_fill_colors.black, x, 0, PHASE_WIDTH,
+ ROW_HEIGHT));
phases_header.appendChild(getSvgRect(phases[color_idx++].color, x + 1, 1,
- width - 2, row_height - 2));
- phases_header.appendChild(getSvgText(p.label, stroke_fill_colors.black, x + width
- / 2, (row_height + 4) / 2 , row_height, true,
- Math.min(p.label.length * char_width, width / 1.5)));
+ PHASE_WIDTH - 2, ROW_HEIGHT - 2));
+ phases_header.appendChild(getSvgText(p.label, stroke_fill_colors.black, x + PHASE_WIDTH
+ / 2, (ROW_HEIGHT + 4) / 2 , ROW_HEIGHT, true,
+ Math.min(p.label.length * CHAR_WIDTH, PHASE_WIDTH / 1.5)));
});
}
async function renderFragmentDiagram() {
clearDOMChildren(fragment_diagram);
- var px_per_ns = chart_width / maxts;
- var rownum_l = 0;
- var pending_children = 0;
- var pending_senders = 0;
- var text_y = row_height - 4;
+ const PX_PER_NS = chart_width / maxts;
+ let rownum_l = 0;
+ let pending_children = 0;
+ let pending_senders = 0;
+ let text_y = ROW_HEIGHT - 4;
fragment_diagram.appendChild(fragment_diagram_title);
fragments.forEach(function printFragment(fragment) {
if (!fragment.printed) {
fragment.printed = true;
- var pending_fragments = [];
- var fevents = fragment.events;
+ let pending_fragments = [];
+ let fevents = fragment.events;
- var frag_name = fragment.name.replace("Coordinator ", "").replace("Fragment ", "");
+ let frag_name = fragment.name.replace("Coordinator ", "").replace("Fragment ", "");
fragment_diagram.appendChild(getSvgText(frag_name, fragment.color, 1, text_y,
- row_height, false));
+ ROW_HEIGHT, false));
// Fragment/sender timing row
- var fragment_svg_group = getSvgGroup();
- DrawBars(fragment_svg_group, rownum_l, row_height, fevents, name_width, px_per_ns);
+ let fragment_svg_group = getSvgGroup();
+ DrawBars(fragment_svg_group, rownum_l, ROW_HEIGHT, fevents, name_width, PX_PER_NS);
- for (var i = 0; i < fragment.nodes.length; ++i) {
- var node = fragment.nodes[i];
+ for (let i = 0; i < fragment.nodes.length; ++i) {
+ let node = fragment.nodes[i];
if (node.events !== undefined) {
// Plan node timing row
- DrawBars(fragment_svg_group, rownum_l, row_height, node.events, name_width,
- px_per_ns);
+ DrawBars(fragment_svg_group, rownum_l, ROW_HEIGHT, node.events, name_width,
+ PX_PER_NS);
if (node.type === "HASH_JOIN_NODE") {
fragment_diagram.appendChild(getSvgText("X", stroke_fill_colors.black,
- name_width + Math.min.apply(null, node.events[2].ts_list) * px_per_ns,
- text_y, row_height, false));
+ name_width + Math.min.apply(null, node.events[2].ts_list) * PX_PER_NS,
+ text_y, ROW_HEIGHT, false));
fragment_diagram.appendChild(getSvgText("O", stroke_fill_colors.black,
- name_width + Math.min.apply(null, node.events[2].ts_list) * px_per_ns,
- text_y, row_height, false));
+ name_width + Math.min.apply(null, node.events[2].ts_list) * PX_PER_NS,
+ text_y, ROW_HEIGHT, false));
}
}
@@ -360,50 +359,52 @@ async function renderFragmentDiagram() {
pending_children--;
}
- var label_x = frag_name_width + char_width * pending_children;
- var label_width = Math.min(char_width * node.name.length,
+ let label_x = frag_name_width + CHAR_WIDTH * pending_children;
+ let label_width = Math.min(CHAR_WIDTH * node.name.length,
name_width - label_x - 2);
fragment_diagram.appendChild(getSvgText(node.name, fragment.color,
- label_x, text_y, row_height, false));
+ label_x, text_y, ROW_HEIGHT, false));
if (node.parent_node !== undefined) {
- var y = row_height * node.parent_node.rendering.rownum;
+ let y = ROW_HEIGHT * node.parent_node.rendering.rownum;
if (node.is_sender) {
- var x = name_width + Math.min.apply(null, fevents[3].ts_list) * px_per_ns;
+ let x = name_width + Math.min.apply(null, fevents[3].ts_list) * PX_PER_NS;
// Dotted horizontal connector to received rows
fragment_diagram.appendChild(getSvgLine(fragment.color, name_width,
- y + row_height / 2 - 1, x, y + row_height / 2 - 1, true));
+ y + ROW_HEIGHT / 2 - 1, x, y + ROW_HEIGHT / 2 - 1, true));
// Dotted rectangle for received rows
- var x2 = name_width + Math.max.apply(null, fevents[4].ts_list) * px_per_ns;
+ let x2 = name_width + Math.max.apply(null, fevents[4].ts_list) * PX_PER_NS;
fragment_diagram.appendChild(getSvgRect(stroke_fill_colors.transperent,
- x, y + 4, x2 - x, row_height - 10, "2 2", fragment.color));
+ x, y + 4, x2 - x, ROW_HEIGHT - 10, "2 2", fragment.color));
}
if (node.is_sender && node.parent_node.rendering.rownum !== rownum_l - 1) {
// DAG edge on right side to distant sender
- var x = name_width - (pending_senders) * char_width - char_width / 2;
+ let x = name_width - (pending_senders) * CHAR_WIDTH - CHAR_WIDTH / 2;
fragment_diagram.appendChild(getSvgLine(fragment.color,
node.parent_node.rendering.label_end,
- y + row_height / 2, x , y + row_height / 2, false));
+ y + ROW_HEIGHT / 2, x , y + ROW_HEIGHT / 2, false));
fragment_diagram.appendChild(getSvgLine(fragment.color, x,
- y + row_height / 2, x, text_y - row_height / 2 + 3, false));
+ y + ROW_HEIGHT / 2, x, text_y - ROW_HEIGHT / 2 + 3, false));
fragment_diagram.appendChild(getSvgLine(fragment.color, x,
- text_y - row_height / 2 + 3, label_x + label_width,
- text_y - row_height / 2 + 3, false));
+ text_y - ROW_HEIGHT / 2 + 3, label_x + label_width,
+ text_y - ROW_HEIGHT / 2 + 3, false));
} else {
// DAG edge from parent to immediate child
- var x = frag_name_width + (pending_children + 1) * char_width -
- char_width / 2;
+ let x = frag_name_width + (pending_children + 1) * CHAR_WIDTH -
+ CHAR_WIDTH / 2;
fragment_diagram.appendChild(getSvgLine(fragment.color, x, y +
- row_height - 3, x, text_y - row_height + 6, false));
+ ROW_HEIGHT - 3, x, text_y - ROW_HEIGHT + 6, false));
}
}
node.rendering = { rownum: rownum_l, label_end: label_x + label_width };
- if (node.num_children) // Scan (leaf) node
+ if (node.num_children) {
+ // Scan (leaf) node
pending_children += (node.num_children - node.is_receiver);
- text_y += row_height;
+ }
+ text_y += ROW_HEIGHT;
rownum_l++;
if (node.is_receiver) {
@@ -421,27 +422,27 @@ async function renderFragmentDiagram() {
pending_fragments.reverse().forEach(printFragment);
}
});
- fragments.forEach(function(fragment) {
+ fragments.forEach((fragment) => {
fragment.printed = false;
});
}
async function renderTimeticks() {
clearDOMChildren(timeticks_footer);
- var sec_per_tic = maxts / ntics / 1e9;
- var px_per_tic = chart_width / ntics;
- var x = name_width;
- var y = 0;
- var text_y = row_height - 4;
- var timetick_label;
- for (var i = 1; i <= ntics; ++i) {
+ let sec_per_tic = maxts / ntics / 1e9;
+ let px_per_tic = chart_width / ntics;
+ let x = name_width;
+ let y = 0;
+ let text_y = ROW_HEIGHT - 4;
+ let timetick_label;
+ for (let i = 1; i <= ntics; ++i) {
timeticks_footer.appendChild(getSvgRect(stroke_fill_colors.black, x, y, px_per_tic,
- row_height));
+ ROW_HEIGHT));
timeticks_footer.appendChild(getSvgRect(stroke_fill_colors.light_grey, x + 1,
- y + 1, px_per_tic - 2, row_height - 2));
+ y + 1, px_per_tic - 2, ROW_HEIGHT - 2));
timetick_label = (i * sec_per_tic).toFixed(decimals);
timeticks_footer.appendChild(getSvgText(timetick_label, stroke_fill_colors.black,
- x + px_per_tic - timetick_label.length * char_width + 2, text_y, row_height,
+ x + px_per_tic - timetick_label.length * CHAR_WIDTH + 2, text_y, ROW_HEIGHT,
false));
x += px_per_tic;
}
@@ -449,43 +450,43 @@ async function renderTimeticks() {
export function collectFragmentEventsFromProfile() {
rownum = 0;
- max_namelen = 0;
+ let max_namelen = 0;
fragments = [];
- all_nodes = [];
- receiver_nodes = [];
- var color_idx = 0;
+ const all_nodes = [];
+ const receiver_nodes = [];
+ let color_idx = 0;
try {
// First pass: compute sizes
- var execution_profile = profile.child_profiles[2];
- var execution_profile_name = execution_profile.profile_name.split(" ").slice(0,2)
+ const execution_profile = profile.child_profiles[2];
+ const EXECUTION_PROFILE_NAME = execution_profile.profile_name.split(" ").slice(0,2)
.join(" ");
- console.assert(execution_profile_name === "Execution Profile");
- execution_profile.child_profiles.forEach(function(fp) {
+ console.assert(EXECUTION_PROFILE_NAME === "Execution Profile");
+ execution_profile.child_profiles.forEach((fp) => {
if (fp.child_profiles !== undefined &&
fp.child_profiles[0].event_sequences !== undefined) {
- var cp = fp.child_profiles[0];
- var fevents = fp.child_profiles[0].event_sequences[0].events;
+ let cp = fp.child_profiles[0];
+ const fevents = fp.child_profiles[0].event_sequences[0].events;
// Build list of timestamps that spans instances for each event
- for (var en = 0; en < fevents.length; ++en) {
+ for (let en = 0; en < fevents.length; ++en) {
if (fevents[en].label.includes("AsyncCodegen")) {
fevents[en].no_bar = true;
continue;
}
fevents[en].ts_list = [ fevents[en].timestamp ];
}
- for (var instance = 1; instance < fp.child_profiles.length; ++instance) {
+ for (let instance = 1; instance < fp.child_profiles.length; ++instance) {
if (fp.child_profiles[instance].event_sequences !== undefined) {
if (fp.child_profiles[instance].event_sequences[0].events.length ==
fevents.length) {
- for (var en = 0; en < fevents.length; ++en) {
+ for (let en = 0; en < fevents.length; ++en) {
if (fevents[en].no_bar) continue;
fevents[en].ts_list.push(
fp.child_profiles[instance].event_sequences[0].events[en].timestamp);
}
} else {
- var en = 0, i = 0;
+ let en = 0, i = 0;
while(i < fevents.length && en < fp.child_profiles[instance]
.event_sequences[0].events.length) {
if (fevents[i].no_bar) {
@@ -503,7 +504,7 @@ export function collectFragmentEventsFromProfile() {
}
}
- var fragment = {
+ const fragment = {
name: fp.profile_name,
events: fevents,
nodes: [ ],
@@ -513,18 +514,18 @@ export function collectFragmentEventsFromProfile() {
color_idx = (color_idx + 1) % fragment_colors.length;
set_maxts(Math.max(maxts, fevents[fevents.length - 1].timestamp));
max_namelen = Math.max(max_namelen, fp.profile_name.length);
- var node_path = [];
- var node_stack = [];
- var node_name;
+ const node_path = [];
+ const node_stack = [];
+ let node_name;
cp.child_profiles.forEach(function get_plan_nodes(pp, index) {
if (pp.node_metadata === undefined) return;
node_path.push(index);
// pp.profile_name : "AGGREGATION_NODE (id=52)"
- var name_flds = pp.profile_name.split(/[()]/);
+ let name_flds = pp.profile_name.split(/[()]/);
// name_flds : ["AGGREGATION_NODE ", "id=52", ""]
- var node_type = name_flds[0].trim();
+ let node_type = name_flds[0].trim();
// node_type: "AGGREGATION_NODE"
- var node_id = name_flds.length > 1 ?
+ let node_id = name_flds.length > 1 ?
parseInt(name_flds[1].split(/[=]/)[1]) : 0;
// node_id: 52
node_name = pp.profile_name.replace("_NODE", "").replace("_", " ")
@@ -532,18 +533,18 @@ export function collectFragmentEventsFromProfile() {
.replace("Hash Join Builder", "JOIN BUILD")
.replace("join node_", "");
if (node_type.indexOf("SCAN_NODE") >= 0) {
- var table_name = pp.info_strings.find(({ key }) => key === "Table Name")
+ let table_name = pp.info_strings.find(({ key }) => key === "Table Name")
.value.split(/[.]/);
node_name = node_name.replace("SCAN",
`SCAN [${table_name[table_name.length - 1]}]`);
}
- var is_receiver = node_type === "EXCHANGE_NODE" ||
+ const IS_RECEIVER = node_type === "EXCHANGE_NODE" ||
(node_type === "HASH_JOIN_NODE" && pp.num_children < 3);
- var is_sender = (node_type === "Hash Join Builder" ||
+ const IS_SENDER = (node_type === "Hash Join Builder" ||
node_type === "KrpcDataStreamSender");
- var parent_node;
+ let parent_node;
if (node_type === "PLAN_ROOT_SINK") {}
else if (pp.node_metadata.data_sink_id !== undefined) {
parent_node = receiver_nodes[node_id]; // Exchange sender dst
@@ -557,12 +558,13 @@ export function collectFragmentEventsFromProfile() {
max_namelen = Math.max(max_namelen, node_name.length + node_stack.length + 1);
+ let node_events;
if (pp.event_sequences !== undefined) {
- var node_events = pp.event_sequences[0].events;
+ node_events = pp.event_sequences[0].events;
// Start the instance event list for each event with timestamps
// from this instance
- for (var en = 0; en < node_events.length; ++en) {
+ for (let en = 0; en < node_events.length; ++en) {
node_events[en].ts_list = [ node_events[en].timestamp ];
if ((node_type === "HASH_JOIN_NODE" ||
node_type === "NESTED_LOOP_JOIN_NODE") && (en === 1 || en === 2)) {
@@ -570,7 +572,7 @@ export function collectFragmentEventsFromProfile() {
}
}
}
- var node = {
+ let node = {
name: node_name,
type: node_type,
node_id: node_id,
@@ -580,15 +582,15 @@ export function collectFragmentEventsFromProfile() {
parent_node: parent_node,
events: node_events,
path: node_path.slice(0),
- is_receiver: is_receiver,
- is_sender: is_sender
+ is_receiver: IS_RECEIVER,
+ is_sender: IS_SENDER
}
- if (is_sender) {
+ if (IS_SENDER) {
node.parent_node.sender_frag_index = fragments.length;
}
- if (is_receiver) {
+ if (IS_RECEIVER) {
receiver_nodes[node_id] = node;
}
@@ -610,13 +612,13 @@ export function collectFragmentEventsFromProfile() {
});
// For each node, retrieve the instance timestamps for the remaining instances
- for (var ni = 0; ni < fragment.nodes.length; ++ni) {
- var node = fragment.nodes[ni];
- for (var cpn = 1; cpn < fp.child_profiles.length; ++cpn) {
- var cp = fp.child_profiles[cpn];
+ for (let ni = 0; ni < fragment.nodes.length; ++ni) {
+ const node = fragment.nodes[ni];
+ for (let cpn = 1; cpn < fp.child_profiles.length; ++cpn) {
+ cp = fp.child_profiles[cpn];
// Use the saved node path to traverse to the same position in this instance
- for (var pi = 0; pi < node.path.length; ++pi) {
+ for (let pi = 0; pi < node.path.length; ++pi) {
cp = cp.child_profiles[node.path[pi]];
}
console.assert(cp.node_metadata.data_sink_id === undefined ||
@@ -626,11 +628,11 @@ export function collectFragmentEventsFromProfile() {
// Add instance events to this node
if (cp.event_sequences !== undefined) {
if (node.events.length === cp.event_sequences[0].events.length) {
- for (var en = 0; en < node.events.length; ++en) {
+ for (let en = 0; en < node.events.length; ++en) {
node.events[en].ts_list.push(cp.event_sequences[0].events[en].timestamp);
}
} else {
- var i = 0, en = 0;
+ let i = 0, en = 0;
while(i < node.events.length && en < cp.event_sequences[0].events.length) {
if (node.events[i].label === cp.event_sequences[0].events[en].label) {
node.events[i].ts_list.push(cp.event_sequences[0].events[en].timestamp);
@@ -647,8 +649,8 @@ export function collectFragmentEventsFromProfile() {
}
});
frag_name_width = (Math.max(2, (fragments.length - 1).toString().length) + 3)
- * char_width;
- name_width = max_namelen * char_width + (frag_name_width + 2);
+ * CHAR_WIDTH;
+ name_width = max_namelen * CHAR_WIDTH + (frag_name_width + 2);
fragment_events_parse_successful = true;
} catch(e) {
fragment_events_parse_successful = false;
@@ -658,20 +660,20 @@ export function collectFragmentEventsFromProfile() {
export function setTimingDiagramDimensions(ignored_arg) {
// phashes header and timeticks footer with margin and border by considering the offset
- page_additional_height = timing_diagram.offsetTop + (row_height + margin_header_footer
- + border_stroke_width * 4) * 2;
- var fragment_diagram_initial_height = window.innerHeight - page_additional_height;
- var remaining_height = fragment_diagram_initial_height - getUtilizationWrapperHeight()
+ page_additional_height = timing_diagram.offsetTop + (ROW_HEIGHT + MARGIN_HEADER_FOOTER
+ + BORDER_STROKE_WIDTH * 4) * 2;
+ const FRAGMENT_DIAGRAM_INITIAL_HEIGHT = window.innerHeight - page_additional_height;
+ const REMAINING_HEIGHT = FRAGMENT_DIAGRAM_INITIAL_HEIGHT - getUtilizationWrapperHeight()
- getFragmentMetricsWrapperHeight();
- var display_height = Math.max(diagram_min_height, Math.min(remaining_height,
- rownum * row_height));
- chart_width = diagram_width - name_width - margin_chart_end - border_stroke_width;
+ const DISPLAY_HEIGHT = Math.max(DIAGRAM_MIN_HEIGHT, Math.min(REMAINING_HEIGHT,
+ rownum * ROW_HEIGHT));
+ chart_width = diagram_width - name_width - MARGIN_CHART_END - BORDER_STROKE_WIDTH;
- phases_header.style.height = `${row_height}px`;
- fragment_diagram.parentElement.style.height = `${display_height}px`;
- fragment_diagram.style.height = `${Math.max(diagram_min_height,
- rownum * row_height)}px`;
- timeticks_footer.style.height = `${row_height}px`;
+ phases_header.style.height = `${ROW_HEIGHT}px`;
+ fragment_diagram.parentElement.style.height = `${DISPLAY_HEIGHT}px`;
+ fragment_diagram.style.height = `${Math.max(DIAGRAM_MIN_HEIGHT,
+ rownum * ROW_HEIGHT)}px`;
+ timeticks_footer.style.height = `${ROW_HEIGHT}px`;
fragment_diagram.parentElement.style.width = `${diagram_width}px`;
phases_header.parentElement.style.width = `${diagram_width}px`;
@@ -692,22 +694,22 @@ export function renderTimingDiagram() {
}
}
-fragment_diagram.addEventListener('mouseout', function(e) {
+fragment_diagram.addEventListener('mouseout', (e) => {
hideTooltip(host_utilization_chart);
hideTooltip(fragment_metrics_chart);
removeChildIfExists(fragment_diagram, timestamp_gridline);
});
-fragment_diagram.addEventListener('mousemove', function(e) {
+fragment_diagram.addEventListener('mousemove', (e) => {
if (e.clientX + scrollable_screen.scrollLeft >= name_width && e.clientX + scrollable_screen.scrollLeft <= name_width + chart_width){
removeChildIfExists(fragment_diagram, timestamp_gridline);
timestamp_gridline = getSvgLine(stroke_fill_colors.black, e.clientX + scrollable_screen.scrollLeft, 0, e.clientX + scrollable_screen.scrollLeft,
parseInt(fragment_diagram.style.height), false);
fragment_diagram.appendChild(timestamp_gridline);
- var gridline_time = ((maxts * (e.clientX + scrollable_screen.scrollLeft - name_width) / chart_width) / 1e9);
- showTooltip(host_utilization_chart, gridline_time);
- showTooltip(fragment_metrics_chart, gridline_time);
- fragment_diagram_title.textContent = gridline_time.toFixed(decimals) + " s";
+ const GRIDLINE_TIME = ((maxts * (e.clientX + scrollable_screen.scrollLeft - name_width) / chart_width) / 1e9);
+ showTooltip(host_utilization_chart, GRIDLINE_TIME);
+ showTooltip(fragment_metrics_chart, GRIDLINE_TIME);
+ fragment_diagram_title.textContent = GRIDLINE_TIME.toFixed(decimals) + " s";
} else {
try {
host_utilization_chart.tooltip.hide();
@@ -718,31 +720,31 @@ fragment_diagram.addEventListener('mousemove', function(e) {
}
});
-fragment_diagram.addEventListener('wheel', function(e) {
+fragment_diagram.addEventListener('wheel', (e) => {
if (e.shiftKey) {
- var window_diagram_width = window.innerWidth - border_stroke_width;
- if (e.wheelDelta <= 0 && diagram_width <= window_diagram_width) return;
- var next_diagram_width = diagram_width + Math.round(e.wheelDelta);
+ const WINDOW_DIAGRAM_WIDTH = window.innerWidth - BORDER_STROKE_WIDTH;
+ if (e.wheelDelta <= 0 && diagram_width <= WINDOW_DIAGRAM_WIDTH) return;
+ const NEXT_DIAGRAM_WIDTH = diagram_width + Math.round(e.wheelDelta);
hor_zoomout.disabled = false;
- if (next_diagram_width <= window_diagram_width) {
- next_diagram_width = window_diagram_width;
+ if (NEXT_DIAGRAM_WIDTH <= WINDOW_DIAGRAM_WIDTH) {
+ NEXT_DIAGRAM_WIDTH = WINDOW_DIAGRAM_WIDTH;
hor_zoomout.disabled = true;
}
- var next_chart_width = next_diagram_width - name_width - margin_chart_end
- - border_stroke_width;
- var next_ntics = (next_diagram_width - diagram_width) * 10 / window.innerWidth;
- next_ntics = ntics + Math.round(next_ntics);
- if (next_ntics < 10) next_ntics = 10;
- var rendering_constraint = char_width * (decimals + integer_part_estimate) >=
- next_chart_width / next_ntics;
- if (rendering_constraint) return;
- ntics = next_ntics;
- set_diagram_width(next_diagram_width);
+ const NEXT_CHART_WIDTH = NEXT_DIAGRAM_WIDTH - name_width - MARGIN_CHART_END
+ - BORDER_STROKE_WIDTH;
+ const NEXT_NTICS = (NEXT_DIAGRAM_WIDTH - diagram_width) * 10 / window.innerWidth;
+ NEXT_NTICS = ntics + Math.round(NEXT_NTICS);
+ if (NEXT_NTICS < 10) NEXT_NTICS = 10;
+ const RENDERING_CONSTRAINT = CHAR_WIDTH * (decimals + INTEGER_PART_ESTIMATE) >=
+ NEXT_CHART_WIDTH / NEXT_NTICS;
+ if (RENDERING_CONSTRAINT) return;
+ ntics = NEXT_NTICS;
+ set_diagram_width(NEXT_DIAGRAM_WIDTH);
resizeHorizontalAll();
}
});
-timeticks_footer.addEventListener('wheel', function(e) {
+timeticks_footer.addEventListener('wheel', (e) => {
if (e.shiftKey) {
if (e.altKey) {
if (e.wheelDelta <= 0) {
@@ -750,20 +752,20 @@ timeticks_footer.addEventListener('wheel', function(e) {
set_decimals(decimals - 1);
decimals_multiplier /= 10;
} else {
- var rendering_constraint = char_width * ((decimals + 1) + integer_part_estimate)
+ const RENDERING_CONSTRAINT = CHAR_WIDTH * ((decimals + 1) + INTEGER_PART_ESTIMATE)
>= chart_width / ntics;
- if (rendering_constraint) return;
+ if (RENDERING_CONSTRAINT) return;
set_decimals(decimals + 1);
decimals_multiplier *= 10;
}
} else {
if (e.wheelDelta <= 0 && ntics <= 10) return;
- var next_ntics = ntics + Math.round(e.wheelDelta / 200);
- if (next_ntics <= 10) return;
- var rendering_constraint = char_width * (decimals + integer_part_estimate)
- >= chart_width / next_ntics;
- if (rendering_constraint) return;
- ntics = next_ntics;
+ const NEXT_NTICS = ntics + Math.round(e.wheelDelta / 200);
+ if (NEXT_NTICS <= 10) return;
+ const RENDERING_CONSTRAINT = CHAR_WIDTH * (decimals + INTEGER_PART_ESTIMATE)
+ >= chart_width / NEXT_NTICS;
+ if (RENDERING_CONSTRAINT) return;
+ ntics = NEXT_NTICS;
}
renderTimeticks();
}
diff --git a/www/scripts/query_timeline/fragment_metrics_diagram.js b/www/scripts/query_timeline/fragment_metrics_diagram.js
index f329acaf3..9751db426 100644
--- a/www/scripts/query_timeline/fragment_metrics_diagram.js
+++ b/www/scripts/query_timeline/fragment_metrics_diagram.js
@@ -17,53 +17,57 @@
import {name_width, page_additional_height, setTimingDiagramDimensions}
from "./fragment_diagram.js"
-import {profile, diagram_width, diagram_controls_height, diagram_min_height,
- resizeVerticalAll, margin_chart_end, decimals} from "./global_members.js";
+import {profile, diagram_width, DIAGRAM_MIN_HEIGHT, MARGIN_CHART_END, resizeVerticalAll,
+ decimals} from "./global_members.js";
import {aggregateProfileTimeseries, generateTimesamples, clearTimeseriesValues,
mapTimeseriesCounters, displayWarning, destroyChart} from "./chart_commons.js";
import {resetUtilizationHeight, resizeUtilizationChart, getUtilizationWrapperHeight}
from "./host_utilization_diagram.js";
import "./global_dom.js";
-export var exportedForTest;
+export let exportedForTest;
-export var fragment_metrics_parse_successful = false;
-export var fragment_metrics_chart = null;
+export let fragment_metrics_parse_successful = false;
+export let fragment_metrics_chart = null;
-var fragment_id_selections = new Map();
-var fragment_metrics_aggregate;
-var fragment_metrics_counters = [
+const fragment_id_selections = new Map();
+const fragment_metrics_counters = [
["MemoryUsage", "memory usage", 0],
["ThreadUsage", "thread usage", 0]
];
-var sampled_fragment_metrics_timeseries;
-var fragment_metrics_timeaxis_name = "fragment metrics timeticks";
-var max_samples_fragment_metrics = {
+const max_samples_fragment_metrics = {
allocated : 1000,
available : 0,
collected : 0,
period : null
};
-var fragment_metrics_resize_factor = 0.2;
+const FRAGMENT_METRICS_TIMEAXIS_NAME = "fragment metrics timeticks";
+const DIAGRAM_CONTROLS_HEIGHT = Math.max(28, window.innerHeight * 0.025);
-var fragment_metrics_close_btn = document.getElementById("fragment_metrics_close_btn");
-var host_utilization_resize_bar = document.getElementById("fragment_metrics_resize_bar");
+let fragment_metrics_aggregate;
+let sampled_fragment_metrics_timeseries;
+
+let fragment_metrics_resize_factor = 0.2;
+
+const fragment_metrics_close_btn = document.getElementById("fragment_metrics_close_btn");
+const fragment_metrics_resize_bar = document.getElementById(
+ "fragment_metrics_resize_bar");
function initializeFragmentMetricsChart() {
fragment_metrics_diagram.style = null;
fragment_metrics_chart = c3.generate({
bindto : "#fragment_metrics_diagram",
data : {
- columns : [[fragment_metrics_timeaxis_name, 0]],
+ columns : [[FRAGMENT_METRICS_TIMEAXIS_NAME, 0]],
type : "area",
order : "asc",
- x : fragment_metrics_timeaxis_name
+ x : FRAGMENT_METRICS_TIMEAXIS_NAME
}, size : {
height : getFragmentMetricsHeight(),
width : diagram_width
}, padding : {
left : name_width,
- right : margin_chart_end
+ right : MARGIN_CHART_END
}, axis : {
x :
{
@@ -72,19 +76,19 @@ function initializeFragmentMetricsChart() {
right : 0
},
tick : {
- format : function (x) { return x.toFixed(decimals); }
+ format : (x) => x.toFixed(decimals)
}
},
y :
{
tick : {
- format : function (y) { return getReadableSize(y, 1); }
+ format : (y) => getReadableSize(y, 1)
}
},
y2 :
{
tick : {
- format : function (y2) { return (y2 === Math.floor(y2) ? y2 : ""); }
+ format : (y2) => (y2 === Math.floor(y2) ? y2 : "")
},
show : true
}
@@ -92,36 +96,34 @@ function initializeFragmentMetricsChart() {
show : false
}, tooltip : {
format : {
- value : function (value, ratio, id, index) {
+ value : (value, ratio, id, index) => {
if (id.includes("memory usage")){
return getReadableSize(value, decimals);
} else {
return value + " Thread(s)";
}
},
- title : function (x, index) {
- return x.toFixed(decimals) + "s";
- }
+ title : (x, index) => x.toFixed(decimals)
}
}
});
fragment_metrics_chart.load({
unload : true
});
- var chart_width = diagram_width - margin_chart_end - name_width;
- fragment_metrics_resize_bar.style.marginLeft = `${name_width + chart_width / 4}px`;
- fragment_metrics_resize_bar.style.width = `${chart_width / 2}px`;
- fragment_metrics_resize_bar.style.marginRight = `${chart_width / 4}px`;
+ const CHART_WIDTH = diagram_width - MARGIN_CHART_END - name_width;
+ fragment_metrics_resize_bar.style.marginLeft = `${name_width + CHART_WIDTH / 4}px`;
+ fragment_metrics_resize_bar.style.width = `${CHART_WIDTH / 2}px`;
+ fragment_metrics_resize_bar.style.marginRight = `${CHART_WIDTH / 4}px`;
}
function dragResizeBar(mousemove_e) {
if (mousemove_e.target.classList[0] === "c3-event-rect") return;
- var next_height = getFragmentMetricsHeight() + (fragment_metrics_resize_bar.offsetTop -
+ const NEXT_HEIGHT = getFragmentMetricsHeight() + (fragment_metrics_resize_bar.offsetTop -
mousemove_e.clientY);
- if (next_height >= diagram_min_height &&
- window.innerHeight - next_height - diagram_controls_height >=
- page_additional_height + getUtilizationWrapperHeight() + diagram_min_height) {
- fragment_metrics_resize_factor = next_height / window.innerHeight;
+ if (NEXT_HEIGHT >= DIAGRAM_MIN_HEIGHT &&
+ window.innerHeight - NEXT_HEIGHT - DIAGRAM_CONTROLS_HEIGHT >=
+ page_additional_height + getUtilizationWrapperHeight() + DIAGRAM_MIN_HEIGHT) {
+ fragment_metrics_resize_factor = NEXT_HEIGHT / window.innerHeight;
}
resizeVerticalAll();
}
@@ -130,13 +132,13 @@ function initializeFragmentMetrics(parent_profile, counters, max_samples, timeax
// user, sys, io and sampled timeticks
max_samples.available = 0;
max_samples.period = 0;
- var fragment_metrics_aggregate = new Array(counters.length);
- for (var i = 0; i < counters.length; ++i) {
+ const fragment_metrics_aggregate = new Array(counters.length);
+ for (let i = 0; i < counters.length; ++i) {
fragment_metrics_aggregate[i] = new Array(max_samples.allocated + 3)
.fill(null);
fragment_metrics_aggregate[i][1] = 0;
}
- var sampled_fragment_metrics_timeseries = new Array(max_samples.allocated + 3)
+ const sampled_fragment_metrics_timeseries = new Array(max_samples.allocated + 3)
.fill(null);
sampled_fragment_metrics_timeseries[0] = timeaxis_name;
mapTimeseriesCounters(parent_profile.child_profiles[0].time_series_counters, counters);
@@ -144,13 +146,13 @@ function initializeFragmentMetrics(parent_profile, counters, max_samples, timeax
}
function getFragmentMetricsHeight() {
- return Math.max(diagram_min_height, window.innerHeight *
+ return Math.max(DIAGRAM_MIN_HEIGHT, window.innerHeight *
fragment_metrics_resize_factor);
}
export function getFragmentMetricsWrapperHeight() {
return (fragment_metrics_parse_successful && fragment_id_selections.size > 0)
- * (getFragmentMetricsHeight() + diagram_controls_height);
+ * (getFragmentMetricsHeight() + DIAGRAM_CONTROLS_HEIGHT);
}
export function toogleFragmentMetricsVisibility() {
@@ -172,23 +174,23 @@ export function collectFragmentMetricsFromProfile() {
try {
// "Execution Profile" contains the different fragment's counters and
// there averaged metrics
- var profile_fragments = profile.child_profiles[2].child_profiles;
+ const profile_fragments = profile.child_profiles[2].child_profiles;
// for all the fragments selected in the execution profile
- profile_fragments.every(function (fragment_profile) {
+ profile_fragments.every((fragment_profile) => {
// initialize the collection arrays for subsequent collections
// arrays are overwritten without further re-allocation to reduce memory usage
if(fragment_id_selections.has(fragment_profile.profile_name)) {
if (fragment_metrics_chart === null) {
({fragment_metrics_aggregate, sampled_fragment_metrics_timeseries} =
initializeFragmentMetrics(fragment_profile, fragment_metrics_counters,
- max_samples_fragment_metrics, fragment_metrics_timeaxis_name));
+ max_samples_fragment_metrics, FRAGMENT_METRICS_TIMEAXIS_NAME));
initializeFragmentMetricsChart();
}
// map counters to axes
- var axes_mappings = {};
- for (var i = 0; i < fragment_metrics_counters.length; ++i) {
+ const axes_mappings = {};
+ for (let i = 0; i < fragment_metrics_counters.length; ++i) {
fragment_metrics_aggregate[i][0] =
`${fragment_profile.profile_name} - ${fragment_metrics_counters[i][1]}`;
}
@@ -201,12 +203,12 @@ export function collectFragmentMetricsFromProfile() {
// display warnings in case less samples are available without plotting
if (max_samples_fragment_metrics.available <= 0) {
- var fragment_metrics_samples_message =
+ const FRAGMENT_METRICS_SAMPLES_MESSAGE =
`Warning: Not enough samples for fragment metrics plot.`;
fragment_metrics_chart = destroyChart(fragment_metrics_chart,
fragment_metrics_diagram);
- displayWarning(fragment_metrics_diagram, fragment_metrics_samples_message,
- diagram_width, name_width, margin_chart_end);
+ displayWarning(fragment_metrics_diagram, FRAGMENT_METRICS_SAMPLES_MESSAGE,
+ diagram_width, name_width, MARGIN_CHART_END);
fragment_metrics_parse_successful = true;
return true;
}
@@ -221,7 +223,7 @@ export function collectFragmentMetricsFromProfile() {
axes : axes_mappings
});
// clear utilization value and timestamp samples arrays
- fragment_metrics_aggregate.forEach(function (acc_usage) {
+ fragment_metrics_aggregate.forEach((acc_usage) => {
clearTimeseriesValues(acc_usage, max_samples_fragment_metrics);
});
}
@@ -238,10 +240,10 @@ export function collectFragmentMetricsFromProfile() {
export async function resizeFragmentMetricsChart() {
if (fragment_metrics_chart === null) return;
- var chart_width = diagram_width - margin_chart_end - name_width;
- fragment_metrics_resize_bar.style.marginLeft = `${name_width + chart_width / 4}px`;
- fragment_metrics_resize_bar.style.width = `${chart_width / 2}px`;
- fragment_metrics_resize_bar.style.marginRight = `${chart_width / 4}px`;
+ const CHART_WIDTH = diagram_width - MARGIN_CHART_END - name_width;
+ fragment_metrics_resize_bar.style.marginLeft = `${name_width + CHART_WIDTH / 4}px`;
+ fragment_metrics_resize_bar.style.width = `${CHART_WIDTH / 2}px`;
+ fragment_metrics_resize_bar.style.marginRight = `${CHART_WIDTH / 4}px`;
fragment_metrics_chart.resize({
height : getFragmentMetricsHeight(),
width : diagram_width
@@ -250,15 +252,15 @@ export async function resizeFragmentMetricsChart() {
export function updateFragmentMetricsChartOnClick(click_event) {
if (click_event.target.tagName !== "rect") return;
- var selected_fragment_id = click_event.target.parentElement.parentElement.id;
- fragment_id_selections.delete(selected_fragment_id);
+ const SELECTED_FRAGMENT_ID = click_event.target.parentElement.parentElement.id;
+ fragment_id_selections.delete(SELECTED_FRAGMENT_ID);
if (fragment_metrics_chart !== null) {
- var unloaded = fragment_metrics_chart.internal.getTargets().every(function (target) {
- if (target.id.includes(selected_fragment_id)) {
- var unload_ids = new Array(fragment_metrics_counters.length);
- for (var i = 0; i < fragment_metrics_counters.length; i++) {
+ const DID_UNLOAD = fragment_metrics_chart.internal.getTargets().every((target) => {
+ if (target.id.includes(SELECTED_FRAGMENT_ID)) {
+ const unload_ids = new Array(fragment_metrics_counters.length);
+ for (let i = 0; i < fragment_metrics_counters.length; i++) {
unload_ids[i] =
- `${selected_fragment_id} - ${fragment_metrics_counters[i][1]}`;
+ `${SELECTED_FRAGMENT_ID} - ${fragment_metrics_counters[i][1]}`;
}
fragment_metrics_chart.unload({ids : unload_ids});
return false;
@@ -272,12 +274,12 @@ export function updateFragmentMetricsChartOnClick(click_event) {
setTimingDiagramDimensions();
return;
}
- if (!unloaded) return;
+ if (!DID_UNLOAD) return;
}
if(fragment_id_selections.size === 0) {
resetUtilizationHeight();
}
- fragment_id_selections.set(selected_fragment_id, 1);
+ fragment_id_selections.set(SELECTED_FRAGMENT_ID, 1);
collectFragmentMetricsFromProfile();
resizeUtilizationChart();
setTimingDiagramDimensions();
@@ -304,8 +306,8 @@ fragment_metrics_resize_bar.addEventListener('mousedown',
fragment_metrics_close_btn.addEventListener('click', closeFragmentMetricsChart);
-fragment_metrics_close_btn.style.height = `${diagram_controls_height}px`;
-fragment_metrics_close_btn.style.fontSize = `${diagram_controls_height / 2}px`;
+fragment_metrics_close_btn.style.height = `${DIAGRAM_CONTROLS_HEIGHT}px`;
+fragment_metrics_close_btn.style.fontSize = `${DIAGRAM_CONTROLS_HEIGHT / 2}px`;
if (typeof process !== "undefined" && process.env.NODE_ENV === 'test') {
exportedForTest = {initializeFragmentMetrics};
diff --git a/www/scripts/query_timeline/global_members.js b/www/scripts/query_timeline/global_members.js
index 7d6896ae7..481890893 100644
--- a/www/scripts/query_timeline/global_members.js
+++ b/www/scripts/query_timeline/global_members.js
@@ -19,15 +19,19 @@ import {renderTimingDiagram, setTimingDiagramDimensions} from "./fragment_diagra
import {resizeFragmentMetricsChart} from "./fragment_metrics_diagram.js";
import {resizeUtilizationChart} from "./host_utilization_diagram.js";
-export var profile = {};
-export var maxts = 0;
-export var decimals = 2;
-export var border_stroke_width = 2;
-export var diagram_width = window.innerWidth - border_stroke_width; // border width
-export var margin_header_footer = 5;
-export var margin_chart_end = 60;
-export var diagram_min_height = 200;
-export var diagram_controls_height = Math.max(28, window.innerHeight * 0.025);
+// Stroke width of fragment diagrams borders
+export const BORDER_STROKE_WIDTH = 2;
+// Default margin provided for fragment diagram's header and footer
+export const MARGIN_HEADER_FOOTER = 5;
+// Default margin provided on the right(at the end) of all charts
+export const MARGIN_CHART_END = 60;
+// The minimum height of diagrams / charts
+export const DIAGRAM_MIN_HEIGHT = 200;
+
+export let diagram_width = window.innerWidth - BORDER_STROKE_WIDTH; // border width
+export let profile = {};
+export let maxts = 0;
+export let decimals = 2;
export function set_profile(val) {
profile = val;
diff --git a/www/scripts/query_timeline/host_utilization_diagram.js b/www/scripts/query_timeline/host_utilization_diagram.js
index 1db8a247c..bf210490b 100644
--- a/www/scripts/query_timeline/host_utilization_diagram.js
+++ b/www/scripts/query_timeline/host_utilization_diagram.js
@@ -15,80 +15,81 @@
// specific language governing permissions and limitations
// under the License.
-import {profile, set_maxts, maxts, diagram_width, diagram_controls_height,
- resizeVerticalAll, diagram_min_height, margin_chart_end, decimals}
- from "./global_members.js";
+import {profile, diagram_width, resizeVerticalAll, DIAGRAM_MIN_HEIGHT, MARGIN_CHART_END,
+ decimals} from "./global_members.js";
import {name_width, page_additional_height, setTimingDiagramDimensions}
from "./fragment_diagram.js";
-import {array_values_start_index, aggregateProfileTimeseries, generateTimesamples,
+import {ARRAY_VALUES_START_INDEX, aggregateProfileTimeseries, generateTimesamples,
clearTimeseriesValues, mapTimeseriesCounters, displayWarning, destroyChart}
from "./chart_commons.js";
import {getFragmentMetricsWrapperHeight} from "./fragment_metrics_diagram.js";
import "./global_dom.js";
-export var exportedForTest;
+export let exportedForTest;
// #host_utilization_diagram
-export var host_utilization_visible = true;
-export var host_utilization_chart = null;
-export var utilization_metrics_parse_successful = false;
+export let host_utilization_visible = true;
+export let host_utilization_chart = null;
+export let utilization_metrics_parse_successful = false;
-var cpu_utilization_counters = [
+const cpu_utilization_counters = [
["HostCpuIoWaitPercentage", "avg io wait", 0],
["HostCpuSysPercentage", "avg sys", 0],
["HostCpuUserPercentage", "avg user", 0]
];
-var read_write_metrics_counters = [
+const read_write_metrics_counters = [
["HostDiskReadThroughput", "avg disk read", 0],
["HostDiskWriteThroughput", "avg disk write", 0],
["HostNetworkRx", "avg network rx", 0],
["HostNetworkTx", "avg network tx", 0]
];
-var cpu_nodes_usage_aggregate;
-var read_write_metrics_aggregate;
-var sampled_utilization_timeseries;
-var utilization_timeaxis_name = "utilization timeticks";
-var prev_utilization_num_samples = 0;
-var max_samples_utilization = {
+const UTILIZATION_TIMEAXIS_NAME = "utilization timeticks";
+const MIN_NUM_SAMPLES = 3;
+const DIAGRAM_CONTROLS_HEIGHT = Math.max(28, window.innerHeight * 0.025);
+
+let cpu_nodes_usage_aggregate;
+let read_write_metrics_aggregate;
+let sampled_utilization_timeseries;
+let prev_utilization_num_samples = 0;
+const max_samples_utilization = {
allocated : 64,
available : 0,
collected : 0,
period : null
};
-var host_utilization_resize_factor = 0.2;
-var min_num_samples = 3;
+let host_utilization_resize_factor = 0.2;
-var host_utilization_close_btn = document.getElementById("host_utilization_close_btn");
-var host_utilization_resize_bar = document.getElementById("host_utilization_resize_bar");
+const host_utilization_close_btn = document.getElementById("host_utilization_close_btn");
+const host_utilization_resize_bar = document.getElementById("host_utilization_resize_bar");
function initializeUtilizationChart() {
- var axis_mappings = {};
- for(var i = 0; i < cpu_utilization_counters.length; ++i) {
+ const axis_mappings = {};
+ for(let i = 0; i < cpu_utilization_counters.length; ++i) {
axis_mappings[cpu_utilization_counters[i][1]] = "y";
}
- for(var i = 0; i < read_write_metrics_counters.length; ++i) {
+ for(let i = 0; i < read_write_metrics_counters.length; ++i) {
axis_mappings[read_write_metrics_counters[i][1]] = "y2";
}
- var cpu_utilization_counter_group = new Array(cpu_utilization_counters.length);
- for (var i = 0; i < cpu_utilization_counters.length; i++){
+ const cpu_utilization_counter_group = new Array(cpu_utilization_counters.length);
+ for (let i = 0; i < cpu_utilization_counters.length; i++){
cpu_utilization_counter_group[i] = cpu_utilization_counters[i][1];
}
host_utilization_diagram.style = null;
host_utilization_chart = c3.generate({
bindto : "#host_utilization_diagram",
data : {
- columns : [[utilization_timeaxis_name, 0]],
+ columns : [[UTILIZATION_TIMEAXIS_NAME, 0]],
axes: axis_mappings,
type : "area",
groups : [ cpu_utilization_counter_group ],
order : "asc",
- x : utilization_timeaxis_name
+ x : UTILIZATION_TIMEAXIS_NAME
}, size : {
height : getUtilizationHeight(),
width : diagram_width
}, padding : {
left : name_width,
- right : margin_chart_end
+ right : MARGIN_CHART_END
}, axis : {
x :
{
@@ -97,19 +98,19 @@ function initializeUtilizationChart() {
right : 0
},
tick : {
- format : function (x) { return x.toFixed(decimals); }
+ format : (x) => x.toFixed(decimals)
}
},
y :
{
tick : {
- format : function (y) { return y + '%'; }
+ format : (y) => y + '%'
}
},
y2 :
{
tick : {
- format : function (y2) { return `${getReadableSize(y2, 1)}/s`; }
+ format : (y2) => `${getReadableSize(y2, 1)}/s`
},
show : true
}
@@ -117,37 +118,35 @@ function initializeUtilizationChart() {
show : false
}, tooltip : {
format : {
- value : function (value, ratio, id, index) {
+ value : (value, ratio, id, index) => {
if (cpu_utilization_counter_group.includes(id)){
return value.toFixed(decimals) + '%';
} else {
return `${getReadableSize(value, decimals)}/s`;
}
},
- title : function (x, index) {
- return x.toFixed(decimals) + "s";
- }
+ title : (x, index) => x.toFixed(decimals) + "s"
}
}
});
host_utilization_chart.load({
unload : true
});
- var chart_width = diagram_width - margin_chart_end - name_width;
+ const CHART_WIDTH = diagram_width - MARGIN_CHART_END - name_width;
prev_utilization_num_samples = 0;
- host_utilization_resize_bar.style.marginLeft = `${name_width + chart_width / 4}px`;
- host_utilization_resize_bar.style.width = `${chart_width / 2}px`;
- host_utilization_resize_bar.style.marginRight = `${chart_width / 4}px`;
+ host_utilization_resize_bar.style.marginLeft = `${name_width + CHART_WIDTH / 4}px`;
+ host_utilization_resize_bar.style.width = `${CHART_WIDTH / 2}px`;
+ host_utilization_resize_bar.style.marginRight = `${CHART_WIDTH / 4}px`;
}
function dragResizeBar(mousemove_e) {
if (mousemove_e.target.classList[0] === "c3-event-rect") return;
- var next_height = getUtilizationHeight() + (host_utilization_resize_bar.offsetTop -
+ const NEXT_HEIGHT = getUtilizationHeight() + (host_utilization_resize_bar.offsetTop -
mousemove_e.clientY);
- if (next_height >= diagram_min_height && window.innerHeight - next_height
- - diagram_controls_height >= page_additional_height
- + getFragmentMetricsWrapperHeight() + diagram_min_height) {
- host_utilization_resize_factor = next_height / window.innerHeight;
+ if (NEXT_HEIGHT >= DIAGRAM_MIN_HEIGHT && window.innerHeight - NEXT_HEIGHT
+ - DIAGRAM_CONTROLS_HEIGHT >= page_additional_height
+ + getFragmentMetricsWrapperHeight() + DIAGRAM_MIN_HEIGHT) {
+ host_utilization_resize_factor = NEXT_HEIGHT / window.innerHeight;
resizeVerticalAll();
}
}
@@ -156,23 +155,23 @@ function initializeUtilizationMetrics(parent_profile, counters_y1, counters_y2,
max_samples, timeaxis_name) {
console.assert(parent_profile.profile_name === "Per Node Profiles");
// user, sys, io and sampled timeticks
- var cpu_nodes_usage_aggregate = new Array(counters_y1.length);
+ const cpu_nodes_usage_aggregate = new Array(counters_y1.length);
max_samples.available = 0;
max_samples.period = 0;
- for (var i = 0; i < counters_y1.length; ++i) {
+ for (let i = 0; i < counters_y1.length; ++i) {
cpu_nodes_usage_aggregate[i] = new Array(max_samples.allocated + 2)
.fill(null);
cpu_nodes_usage_aggregate[i][0] = counters_y1[i][1];
cpu_nodes_usage_aggregate[i][1] = 0;
}
- var read_write_metrics_aggregate = new Array(counters_y2.length);
- for (var i = 0; i < counters_y2.length; ++i) {
+ const read_write_metrics_aggregate = new Array(counters_y2.length);
+ for (let i = 0; i < counters_y2.length; ++i) {
read_write_metrics_aggregate[i] = new Array(max_samples.allocated + 2)
.fill(null);
read_write_metrics_aggregate[i][0] = counters_y2[i][1];
read_write_metrics_aggregate[i][1] = 0;
}
- var sampled_utilization_timeseries = new Array(max_samples.allocated + 2)
+ const sampled_utilization_timeseries = new Array(max_samples.allocated + 2)
.fill(null);
sampled_utilization_timeseries[0] = timeaxis_name;
mapTimeseriesCounters(parent_profile.child_profiles[0].time_series_counters, counters_y1);
@@ -182,13 +181,13 @@ function initializeUtilizationMetrics(parent_profile, counters_y1, counters_y2,
}
function getUtilizationHeight() {
- return Math.max(diagram_min_height, window.innerHeight *
+ return Math.max(DIAGRAM_MIN_HEIGHT, window.innerHeight *
host_utilization_resize_factor);
}
export function getUtilizationWrapperHeight() {
return (utilization_metrics_parse_successful && host_utilization_visible) *
- (getUtilizationHeight() + diagram_controls_height);
+ (getUtilizationHeight() + DIAGRAM_CONTROLS_HEIGHT);
}
export function resetUtilizationHeight() {
@@ -205,10 +204,10 @@ export function toogleUtilizationVisibility() {
export async function resizeUtilizationChart() {
if (host_utilization_chart === null) return;
- var chart_width = diagram_width - margin_chart_end - name_width;
- host_utilization_resize_bar.style.marginLeft = `${name_width + chart_width / 4}px`;
- host_utilization_resize_bar.style.width = `${chart_width / 2}px`;
- host_utilization_resize_bar.style.marginRight = `${chart_width / 4}px`;
+ const CHART_WIDTH = diagram_width - MARGIN_CHART_END - name_width;
+ host_utilization_resize_bar.style.marginLeft = `${name_width + CHART_WIDTH / 4}px`;
+ host_utilization_resize_bar.style.width = `${CHART_WIDTH / 2}px`;
+ host_utilization_resize_bar.style.marginRight = `${CHART_WIDTH / 4}px`;
host_utilization_chart.resize({
height : getUtilizationHeight(),
width : diagram_width
@@ -227,16 +226,16 @@ export function collectUtilizationFromProfile() {
try {
// initialize the collection arrays for subsequent collections
// arrays are overwritten without further re-allocation to reduce memory usage
- var per_node_profiles = profile.child_profiles[2].child_profiles[0];
+ const per_node_profiles = profile.child_profiles[2].child_profiles[0];
console.assert(per_node_profiles.profile_name === "Per Node Profiles");
if (host_utilization_chart === null) {
({cpu_nodes_usage_aggregate, read_write_metrics_aggregate,
sampled_utilization_timeseries} = initializeUtilizationMetrics(
per_node_profiles, cpu_utilization_counters, read_write_metrics_counters,
- max_samples_utilization, utilization_timeaxis_name));
+ max_samples_utilization, UTILIZATION_TIMEAXIS_NAME));
initializeUtilizationChart();
}
- var impala_server_profile = profile.child_profiles[1];
+ const impala_server_profile = profile.child_profiles[1];
console.assert(impala_server_profile.profile_name === "ImpalaServer");
// Update the plot, only when number of samples in SummaryStatsCounter is updated
if (impala_server_profile.summary_stats_counters[0].num_of_samples ==
@@ -252,28 +251,28 @@ export function collectUtilizationFromProfile() {
read_write_metrics_counters, max_samples_utilization);
// display warnings in case less samples are available without plotting
- if (max_samples_utilization.available < min_num_samples) {
- var utilization_samples_message = `Warning: Not enough samples for
+ if (max_samples_utilization.available < MIN_NUM_SAMPLES) {
+ const UTILIZATION_SAMPLES_MESSAGE = `Warning: Not enough samples for
CPU utilization plot. Please decrease the value of starting flag
variable <b><i>periodic_counter_update_period_ms </b></i> to
increase the granularity of CPU utilization plot.`;
host_utilization_chart = destroyChart(host_utilization_chart,
host_utilization_diagram);
- displayWarning(host_utilization_diagram, utilization_samples_message,
- diagram_width, name_width, margin_chart_end);
+ displayWarning(host_utilization_diagram, UTILIZATION_SAMPLES_MESSAGE,
+ diagram_width, name_width, MARGIN_CHART_END);
utilization_metrics_parse_successful = true;
return;
}
// average the aggregated metrics
- cpu_nodes_usage_aggregate.forEach(function (acc_usage) {
- for (var i = array_values_start_index; i < max_samples_utilization.available
- + array_values_start_index; ++i) {
+ cpu_nodes_usage_aggregate.forEach((acc_usage) => {
+ for (let i = ARRAY_VALUES_START_INDEX; i < max_samples_utilization.available
+ + ARRAY_VALUES_START_INDEX; ++i) {
acc_usage[i] = acc_usage[i] / (100 * per_node_profiles.child_profiles.length);
}
});
- read_write_metrics_aggregate.forEach(function (acc_usage) {
- for (var i = array_values_start_index; i < max_samples_utilization.available
- + array_values_start_index; ++i) {
+ read_write_metrics_aggregate.forEach((acc_usage) => {
+ for (let i = ARRAY_VALUES_START_INDEX; i < max_samples_utilization.available
+ + ARRAY_VALUES_START_INDEX; ++i) {
acc_usage[i] = acc_usage[i] / per_node_profiles.child_profiles.length;
}
});
@@ -286,10 +285,10 @@ export function collectUtilizationFromProfile() {
sampled_utilization_timeseries]
});
// clear utilization value and timestamp samples arrays
- cpu_nodes_usage_aggregate.forEach(function (acc_usage) {
+ cpu_nodes_usage_aggregate.forEach((acc_usage) => {
clearTimeseriesValues(acc_usage, max_samples_utilization);
});
- read_write_metrics_aggregate.forEach(function (acc_usage) {
+ read_write_metrics_aggregate.forEach((acc_usage) => {
clearTimeseriesValues(acc_usage, max_samples_utilization);
});
prev_utilization_num_samples = profile.child_profiles[1].summary_stats_counters[0]
@@ -321,13 +320,13 @@ host_utilization_resize_bar.addEventListener('mousedown',
});
-host_utilization_close_btn.addEventListener('click', function(e) {
+host_utilization_close_btn.addEventListener('click', (e) => {
host_utilization_visible = false;
destroyUtilizationChart();
});
-host_utilization_close_btn.style.height = `${diagram_controls_height}px`;
-host_utilization_close_btn.style.fontSize = `${diagram_controls_height / 2}px`;
+host_utilization_close_btn.style.height = `${DIAGRAM_CONTROLS_HEIGHT}px`;
+host_utilization_close_btn.style.fontSize = `${DIAGRAM_CONTROLS_HEIGHT / 2}px`;
if (typeof process !== "undefined" && process.env.NODE_ENV === 'test') {
exportedForTest = {initializeUtilizationMetrics};
diff --git a/www/scripts/tests/queries/profileParseWorker.test.js b/www/scripts/tests/queries/profileParseWorker.test.js
index 5bbfbe744..2c10b26e7 100644
--- a/www/scripts/tests/queries/profileParseWorker.test.js
+++ b/www/scripts/tests/queries/profileParseWorker.test.js
@@ -23,7 +23,7 @@ describe("Test Compression Library", () => {
// Test whether the compression library imported by the worker script
// properly utilizes the pako library's compression methods
test("Basic Test", () => {
- var exampleJSONProfileText = readFileSync("../../../testdata/impala-profiles/"
+ const exampleJSONProfileText = readFileSync("../../../testdata/impala-profiles/"
+ "impala_profile_log_tpcds_compute_stats_extended.expected.pretty.json",
{encoding : "utf-8"});
import("../../../pako.min.js").then((pako) => {
@@ -32,4 +32,4 @@ describe("Test Compression Library", () => {
.toBe(exampleJSONProfileText);
});
});
-});
\ No newline at end of file
+});
diff --git a/www/scripts/tests/query_timeline/chart_commons.test.js b/www/scripts/tests/query_timeline/chart_commons.test.js
index 62c688a29..a44bb13ae 100644
--- a/www/scripts/tests/query_timeline/chart_commons.test.js
+++ b/www/scripts/tests/query_timeline/chart_commons.test.js
@@ -24,7 +24,7 @@ describe("Test mapTimeseriesCounters", () => {
// Test whether the method correctly searches and maps indexes of counters based
// on counter_name
test("Basic Test (Serial Order)", () => {
- var parent_profile =
+ const parent_profile =
{
"profile_name": "Per Node Profiles",
"num_children": 3,
@@ -53,20 +53,20 @@ describe("Test mapTimeseriesCounters", () => {
}
]
};
- var counters = [
+ const counters = [
["HostCpuIoWaitPercentage", "avg io wait", 0],
["HostCpuSysPercentage", "avg sys", 0],
["HostCpuUserPercentage", "avg user", 0]
];
expect(mapTimeseriesCounters(parent_profile.child_profiles[0].time_series_counters,
counters)).toBe(undefined);
- for (var i = 0; i < counters.length; i++) {
+ for (let i = 0; i < counters.length; i++) {
expect(counters[i][2]).toBe(i);
}
});
test("Basic Test (Reverse Order)", () => {
- var parent_profile =
+ const parent_profile =
{
"profile_name": "Per Node Profiles",
"num_children": 3,
@@ -95,20 +95,20 @@ describe("Test mapTimeseriesCounters", () => {
}
]
};
- var counters = [
+ const counters = [
["HostCpuIoWaitPercentage", "avg io wait", 0],
["HostCpuSysPercentage", "avg sys", 0],
["HostCpuUserPercentage", "avg user", 0]
];
expect(mapTimeseriesCounters(parent_profile.child_profiles[0].time_series_counters,
counters)).toBe(undefined);
- for (var i = 0; i < counters.length; i++) {
+ for (let i = 0; i < counters.length; i++) {
expect(counters[i][2]).toBe(counters.length - i - 1);
}
});
test("Edge Case (No such 'counter_name' within profile)", () => {
- var parent_profile =
+ const parent_profile =
{
"profile_name": "Per Node Profiles",
"num_children": 3,
@@ -137,7 +137,7 @@ describe("Test mapTimeseriesCounters", () => {
}
]
};
- var counters = [
+ const counters = [
["HostPercentage", "avg io wait", 0],
["HostSysPercenage", "avg sys", 0],
["HostUserPercntage", "avg user", 0]
@@ -154,17 +154,17 @@ describe("Test mapTimeseriesCounters", () => {
describe("Test accumulateTimeseriesValues", () => {
// Test whether the method correctly accumlates values after parsing values from 'data'
// in 'time_series_counters' and correctly updates 'max_samples' even in corner cases
- var {accumulateTimeseriesValues} = exportedForTest;
- var data_type = "value type";
+ const {accumulateTimeseriesValues} = exportedForTest;
+ const DATA_TYPE = "value type";
test("Basic Case (time_series_counter.num > max_samples.collected)", () => {
- var max_samples = {
+ const max_samples = {
allocated : 7,
period : 0,
available : 0,
collected : 0
};
- var values_array = [data_type, 0, 60, 100, 40, 38, 49, 61, 27];
- var time_series_counter = {
+ const values_array = [DATA_TYPE, 0, 60, 100, 40, 38, 49, 61, 27];
+ const time_series_counter = {
period: 100,
num: 2000,
data: "30, 100, 40"
@@ -173,7 +173,7 @@ describe("Test accumulateTimeseriesValues", () => {
expect(accumulateTimeseriesValues(values_array, time_series_counter, max_samples))
.toBe(undefined);
- expect(values_array).toEqual([data_type, 0, 90, 200, 80, 38, 49, 61, 27]);
+ expect(values_array).toEqual([DATA_TYPE, 0, 90, 200, 80, 38, 49, 61, 27]);
expect(max_samples).toEqual({
allocated : 7,
@@ -184,14 +184,14 @@ describe("Test accumulateTimeseriesValues", () => {
});
test("Basic Case (time_series_counter.period > max_samples.period", () => {
- var max_samples = {
+ const max_samples = {
allocated : 7,
period : 100,
available : 1000,
collected : 1000
};
- var values_array = [data_type, 0, 60, 100, 40, 38, 49, 61, 27];
- var time_series_counter = {
+ const values_array = [DATA_TYPE, 0, 60, 100, 40, 38, 49, 61, 27];
+ const time_series_counter = {
period: 200,
num: 300,
data: "30, 100, 40"
@@ -200,7 +200,7 @@ describe("Test accumulateTimeseriesValues", () => {
expect(accumulateTimeseriesValues(values_array, time_series_counter, max_samples))
.toBe(undefined);
- expect(values_array).toEqual([data_type, 0, 90, 200, 80, 38, 49, 61, 27]);
+ expect(values_array).toEqual([DATA_TYPE, 0, 90, 200, 80, 38, 49, 61, 27]);
expect(max_samples).toEqual({
allocated : 7,
@@ -212,14 +212,14 @@ describe("Test accumulateTimeseriesValues", () => {
test(`Basic Case (time_series_counter.period <= max_samples.period
&& time_series_counter.num <= max_samples.collected)`, () => {
- var max_samples = {
+ const max_samples = {
allocated : 7,
period : 100,
available : 1000,
collected : 1000
};
- var values_array = [data_type, 0, 60, 100, 40, 38, 49, 61, 27];
- var time_series_counter = {
+ const values_array = [DATA_TYPE, 0, 60, 100, 40, 38, 49, 61, 27];
+ const time_series_counter = {
period: 100,
num: 300,
data: "30, 100, 40"
@@ -228,7 +228,7 @@ describe("Test accumulateTimeseriesValues", () => {
expect(accumulateTimeseriesValues(values_array, time_series_counter, max_samples))
.toBe(undefined);
- expect(values_array).toEqual([data_type, 0, 90, 200, 80, 38, 49, 61, 27]);
+ expect(values_array).toEqual([DATA_TYPE, 0, 90, 200, 80, 38, 49, 61, 27]);
expect(max_samples).toEqual({
allocated : 7,
@@ -239,14 +239,14 @@ describe("Test accumulateTimeseriesValues", () => {
});
test(`Edge Case (values_array length is smaller than collected samples)`, () => {
- var max_samples = {
+ const max_samples = {
allocated : 2,
period : 100,
available : 2,
collected : 1000
};
- var values_array = [data_type, 0, 60, 100];
- var time_series_counter = {
+ const values_array = [DATA_TYPE, 0, 60, 100];
+ const time_series_counter = {
period: 100,
num: 300,
data: "30, 100, 40"
@@ -255,7 +255,7 @@ describe("Test accumulateTimeseriesValues", () => {
expect(accumulateTimeseriesValues(values_array, time_series_counter, max_samples))
.toBe(undefined);
- expect(values_array).toEqual([data_type, 0, 90, 200]);
+ expect(values_array).toEqual([DATA_TYPE, 0, 90, 200]);
expect(max_samples).toEqual({
allocated : 2,
@@ -269,91 +269,91 @@ describe("Test accumulateTimeseriesValues", () => {
describe("Test generateTimesamples", () => {
// Test whether time sample values generated based on 'max_samples' are correct,
// even in corner cases, with different 'max_samples' scenarios
- var data_type = "timesample type";
+ const DATA_TYPE = "timesample type";
test("Basic Case (max_samples.allocated > max_samples.available)", () => {
- var max_samples = {
+ const max_samples = {
allocated : 10,
period : 1000,
available : 4,
collected : 10
};
- var timesamples_array = new Array(max_samples.allocated + 2).fill(null);
- timesamples_array[0] = data_type;
+ const timesamples_array = new Array(max_samples.allocated + 2).fill(null);
+ timesamples_array[0] = DATA_TYPE;
expect(generateTimesamples(timesamples_array, max_samples)).toBe(undefined);
- expect(timesamples_array).toEqual([data_type, 0, 2.5, 5, 7.5, 10, null, null,
+ expect(timesamples_array).toEqual([DATA_TYPE, 0, 2.5, 5, 7.5, 10, null, null,
null, null, null, null]);
});
test("Edge Case (max_samples.allocated < max_samples.available)", () => {
- var max_samples = {
+ const max_samples = {
allocated : 10,
period : 1000,
available : 20,
collected : 10
};
- var timesamples_array = new Array(max_samples.allocated + 2);
- timesamples_array[0] = data_type;
+ const timesamples_array = new Array(max_samples.allocated + 2);
+ timesamples_array[0] = DATA_TYPE;
expect(generateTimesamples(timesamples_array, max_samples)).toBe(undefined);
- expect(timesamples_array).toEqual([data_type, 0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4,
+ expect(timesamples_array).toEqual([DATA_TYPE, 0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4,
4.5, 5]);
});
test("Edge Case (max_samples.allocated = max_samples.available)", () => {
- var max_samples = {
+ const max_samples = {
allocated : 10,
period : 1000,
available : 10,
collected : 10
};
- var timesamples_array = new Array(max_samples.allocated + 2);
- timesamples_array[0] = data_type;
+ const timesamples_array = new Array(max_samples.allocated + 2);
+ timesamples_array[0] = DATA_TYPE;
expect(generateTimesamples(timesamples_array, max_samples)).toBe(undefined);
- expect(timesamples_array).toEqual([data_type, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]);
+ expect(timesamples_array).toEqual([DATA_TYPE, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]);
});
});
describe("Test clearTimeseriesValues", () => {
// Test whether Timeseries arrays are being properly truncated in the correct range
- var data_type = "value type";
+ const DATA_TYPE = "value type";
test("Basic Case (max_samples.available < max_samples.allocated)", () => {
- var max_samples = {
+ const max_samples = {
allocated : 7,
period : 1000,
available : 3,
collected : 10
};
- var values_array = [data_type, 0, 4, 3, 20, 10, 100, 10];
+ const values_array = [DATA_TYPE, 0, 4, 3, 20, 10, 100, 10];
expect(clearTimeseriesValues(values_array, max_samples)).toBe(undefined);
- expect(values_array).toEqual([data_type, 0, null, null, null, 10, 100, 10]);
+ expect(values_array).toEqual([DATA_TYPE, 0, null, null, null, 10, 100, 10]);
});
test("Edge Case (max_samples.available >= max_samples.allocated)", () => {
- var max_samples = {
+ const max_samples = {
allocated : 7,
period : 1000,
available : 30,
collected : 10
};
- var values_array = [data_type, 0, 3, 4, 3, 20, 10, 100, 10];
+ const values_array = [DATA_TYPE, 0, 3, 4, 3, 20, 10, 100, 10];
expect(clearTimeseriesValues(values_array, max_samples)).toBe(undefined);
- expect(values_array).toEqual([data_type, 0, null, null, null, null, null, null, null]);
+ expect(values_array).toEqual([DATA_TYPE, 0, null, null, null, null, null, null, null]);
});
});
describe("Test aggregateProfileTimeseries", () => {
// Test correctness of values being aggregated from parsing the profile
test("Basic Case", () => {
- var parent_profile =
+ const parent_profile =
{
"profile_name": "Per Node Profiles",
"num_children": 3,
@@ -426,19 +426,19 @@ describe("Test aggregateProfileTimeseries", () => {
}
]
};
- var max_samples = {
+ const max_samples = {
allocated : 10,
period : 0,
available : 0,
collected : 0
};
- var counters = [
+ const counters = [
["HostCpuIoWaitPercentage", "avg io wait", 0],
["HostCpuSysPercentage", "avg sys", 0],
["HostCpuUserPercentage", "avg user", 0]
];
- var aggregate_array = new Array(counters.length);
- for (var i = 0; i < counters.length; ++i) {
+ const aggregate_array = new Array(counters.length);
+ for (let i = 0; i < counters.length; ++i) {
aggregate_array[i] = new Array(max_samples.allocated + 2).fill(0);
aggregate_array[i][0] = counters[i][1];
}
diff --git a/www/scripts/tests/query_timeline/fragment_diagram.test.js b/www/scripts/tests/query_timeline/fragment_diagram.test.js
index e54a8bc66..372152ed8 100644
--- a/www/scripts/tests/query_timeline/fragment_diagram.test.js
+++ b/www/scripts/tests/query_timeline/fragment_diagram.test.js
@@ -20,8 +20,8 @@ import {exportedForTest} from "../../query_timeline/fragment_diagram.js";
describe("Test getSvg*", () => {
// Test whether getSvg* methods correctly set attributes and return expected elements
- var {getSvgRect, getSvgLine, getSvgText, getSvgTitle, getSvgGroup} = exportedForTest;
- var stroke_fill_colors = { black : "#000000", dark_grey : "#505050",
+ const {getSvgRect, getSvgLine, getSvgText, getSvgTitle, getSvgGroup} = exportedForTest;
+ const stroke_fill_colors = { black : "#000000", dark_grey : "#505050",
light_grey : "#F0F0F0", transperent : "rgba(0, 0, 0, 0)" };
test("Test getSvgRect", () => {
diff --git a/www/scripts/tests/query_timeline/fragment_metrics_diagram.test.js b/www/scripts/tests/query_timeline/fragment_metrics_diagram.test.js
index 5d3b21b51..080e2fd25 100644
--- a/www/scripts/tests/query_timeline/fragment_metrics_diagram.test.js
+++ b/www/scripts/tests/query_timeline/fragment_metrics_diagram.test.js
@@ -20,9 +20,9 @@ import {exportedForTest} from "../../query_timeline/fragment_metrics_diagram.js"
describe("Test initializeFragmentMetrics", () => {
// Test whether aggregate arrays and time sample arrays are correctly allocated
// based on counters and max_samples
- var {initializeFragmentMetrics} = exportedForTest;
+ const {initializeFragmentMetrics} = exportedForTest;
test("Basic Test", () => {
- var parent_profile =
+ const parent_profile =
{
"profile_name": "Coordinator Fragment F31",
"num_children": 1,
@@ -45,25 +45,25 @@ describe("Test initializeFragmentMetrics", () => {
}
]
};
- var max_samples = {
+ const max_samples = {
allocated : 4,
period : 0,
available : 0,
collected : 0
};
- var counters = [
+ const counters = [
["MemoryUsage", "memory usage", 0],
["ThreadUsage", "thread usage", 0]
];
- var timeaxis_name = "fragment metrics timeticks";
- var {fragment_metrics_aggregate, sampled_fragment_metrics_timeseries}
- = initializeFragmentMetrics(parent_profile, counters, max_samples, timeaxis_name);
+ const TIMEAXIS_NAME = "fragment metrics timeticks";
+ const {fragment_metrics_aggregate, sampled_fragment_metrics_timeseries}
+ = initializeFragmentMetrics(parent_profile, counters, max_samples, TIMEAXIS_NAME);
expect(fragment_metrics_aggregate).toEqual([
[null, 0, null, null, null, null, null],
[null, 0, null, null, null, null, null]
]);
expect(sampled_fragment_metrics_timeseries).toEqual(
- [timeaxis_name, null, null, null, null, null, null]
+ [TIMEAXIS_NAME, null, null, null, null, null, null]
);
});
});
diff --git a/www/scripts/tests/query_timeline/host_utilization_diagram.test.js b/www/scripts/tests/query_timeline/host_utilization_diagram.test.js
index 5872298f2..547d69cb8 100644
--- a/www/scripts/tests/query_timeline/host_utilization_diagram.test.js
+++ b/www/scripts/tests/query_timeline/host_utilization_diagram.test.js
@@ -21,9 +21,9 @@ import {exportedForTest} from "../../query_timeline/host_utilization_diagram.js"
describe("Test initializeUtilizationMetrics", () => {
// Test whether aggregate arrays and time sample arrays are correctly allocated
// based on counters and max_samples
- var {initializeUtilizationMetrics} = exportedForTest;
+ const {initializeUtilizationMetrics} = exportedForTest;
test("Basic Test", () => {
- var parent_profile =
+ const parent_profile =
{
"profile_name": "Per Node Profiles",
"num_children": 3,
@@ -52,24 +52,24 @@ describe("Test initializeUtilizationMetrics", () => {
}
]
};
- var max_samples = {
+ const max_samples = {
allocated : 3,
period : 0,
available : 0,
collected : 0
};
- var counters_y1 = [
+ const counters_y1 = [
["HostCpuUserPercentage", "avg io wait", 0],
["HostCpuSysPercentage", "avg sys", 0]
];
- var counters_y2 = [
+ const counters_y2 = [
["HostNetworkRx", "avg network rx", 0]
];
- var timeaxis_name = "utilization timeticks";
- var {cpu_nodes_usage_aggregate, read_write_metrics_aggregate,
+ const TIMEAXIS_NAME = "utilization timeticks";
+ const {cpu_nodes_usage_aggregate, read_write_metrics_aggregate,
sampled_utilization_timeseries} = initializeUtilizationMetrics(
parent_profile, counters_y1, counters_y2,
- max_samples, timeaxis_name);
+ max_samples, TIMEAXIS_NAME);
expect(cpu_nodes_usage_aggregate).toEqual([
[counters_y1[0][1], 0, null, null, null],
[counters_y1[1][1], 0, null, null, null]
@@ -78,7 +78,7 @@ describe("Test initializeUtilizationMetrics", () => {
[counters_y2[0][1], 0, null, null, null]
]);
expect(sampled_utilization_timeseries).toEqual(
- [timeaxis_name, null, null, null, null]
+ [TIMEAXIS_NAME, null, null, null, null]
);
expect(counters_y1[0][2]).toBe(0);
expect(counters_y1[1][2]).toBe(1);
diff --git a/www/sessions.tmpl b/www/sessions.tmpl
index 2e0280d4d..be8b92419 100644
--- a/www/sessions.tmpl
+++ b/www/sessions.tmpl
@@ -160,7 +160,7 @@ This data is only available if --max_hs2_sessions_per_user is set to a positive
</table>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#client-hosts-tbl').DataTable({
"order": [[ 1, "desc" ]],
"pageLength": 100
@@ -169,7 +169,7 @@ This data is only available if --max_hs2_sessions_per_user is set to a positive
</script>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#connections-tbl').DataTable({
"order": [[ 4, "desc" ]],
"pageLength": 100
@@ -178,7 +178,7 @@ This data is only available if --max_hs2_sessions_per_user is set to a positive
</script>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#sessions-tbl').DataTable({
"order": [[ 1, "desc" ]],
"pageLength": 100
@@ -187,7 +187,7 @@ This data is only available if --max_hs2_sessions_per_user is set to a positive
</script>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#users-tbl').DataTable({
"order": [[ 1, "desc" ]],
"pageLength": 100
diff --git a/www/thread-group.tmpl b/www/thread-group.tmpl
index 2c32af61f..e02c04a60 100644
--- a/www/thread-group.tmpl
+++ b/www/thread-group.tmpl
@@ -45,7 +45,7 @@ under the License.
</table>
<script>
- $(document).ready(function() {
+ $(document).ready(() => {
$('#threads-tbl').DataTable({
"order": [[ 2, "desc" ]],
"pageLength": 100
(impala) 02/04: IMPALA-13932: (addendum) Adds e2e test for IMPALA-13932
Posted by bo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
boroknagyz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 2849708b8701485637478ab6201b0780e80570ca
Author: Zoltan Borok-Nagy <bo...@cloudera.com>
AuthorDate: Thu Apr 10 14:29:55 2025 +0200
IMPALA-13932: (addendum) Adds e2e test for IMPALA-13932
This patch adds e2e test for IMPALA-13932.
Change-Id: I07537b31717d568422ea042ad2aeef906b3cab2e
Reviewed-on: http://gerrit.cloudera.org:8080/22767
Reviewed-by: Peter Rozsa <pr...@cloudera.com>
Reviewed-by: Daniel Becker <da...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
.../QueryTest/iceberg-merge-duplicate-check.test | 20 ++++++++++++++++++++
tests/query_test/test_iceberg.py | 6 ++++++
2 files changed, 26 insertions(+)
diff --git a/testdata/workloads/functional-query/queries/QueryTest/iceberg-merge-duplicate-check.test b/testdata/workloads/functional-query/queries/QueryTest/iceberg-merge-duplicate-check.test
new file mode 100644
index 000000000..17675e226
--- /dev/null
+++ b/testdata/workloads/functional-query/queries/QueryTest/iceberg-merge-duplicate-check.test
@@ -0,0 +1,20 @@
+====
+---- QUERY
+create table merge_duplicate_check
+partitioned by spec(id)
+stored by iceberg
+tblproperties('format-version'='2') as
+ select id, bigint_col, string_col
+ from functional_parquet.alltypes
+ where id < 1000;
+
+set num_nodes=1;
+set mt_dop=1;
+merge into merge_duplicate_check target using functional.alltypes source on (target.id = source.id)
+when matched then update set target.bigint_col = target.bigint_col + 1;
+select sum(bigint_col) from merge_duplicate_check;
+---- RESULTS
+46000
+---- TYPES
+BIGINT
+====
diff --git a/tests/query_test/test_iceberg.py b/tests/query_test/test_iceberg.py
index c41e5162d..ad13b1dc0 100644
--- a/tests/query_test/test_iceberg.py
+++ b/tests/query_test/test_iceberg.py
@@ -2078,6 +2078,12 @@ class TestIcebergV2Table(IcebergTestSuite):
warehouse_prefix=os.getenv("FILESYSTEM_PREFIX"))
self.run_test_case('QueryTest/iceberg-merge-equality-insert', vector, unique_database)
+ def test_merge_duplicate_check(self, vector, unique_database):
+ """Regression test for IMPALA-13932"""
+ # Remove 'num_nodes' option so we can set it at .test file.
+ vector.unset_exec_option('num_nodes')
+ self.run_test_case('QueryTest/iceberg-merge-duplicate-check', vector, unique_database)
+
def test_cleanup(self, unique_database):
"""Test that all uncommitted files written by Impala are removed from the file
system when a DML commit to an Iceberg table fails, and that the effects of the