You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by as...@apache.org on 2022/01/26 00:19:58 UTC
[impala] branch master updated (1211866 -> b96439f)
This is an automated email from the ASF dual-hosted git repository.
asherman pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git.
from 1211866 IMPALA-10910, IMPALA-5509: Runtime filter: dictionary filter support
new 2e91696 IMPALA-11062: "now_string" option not working properly
new b96439f IMPALA-11078 Add simple CSP header to webui.
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
LICENSE.txt | 23 ++++++++++++++++++
be/src/runtime/query-driver.cc | 2 ++
be/src/util/webserver.cc | 12 ++++++++++
bin/rat_exclude_files.txt | 1 +
tests/custom_cluster/test_web_pages.py | 16 +++++++++++++
tests/webserver/test_web_pages.py | 2 +-
www/Chart-2.7.3.min.js | 10 ++++++++
www/admission_controller.tmpl | 2 +-
www/rpcz.tmpl | 44 +++++++++++++++++-----------------
9 files changed, 88 insertions(+), 24 deletions(-)
create mode 100644 www/Chart-2.7.3.min.js
[impala] 02/02: IMPALA-11078 Add simple CSP header to webui.
Posted by as...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
asherman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit b96439f6806731d2b3cac84d0252f2b4c73aca7f
Author: Andrew Sherman <as...@cloudera.com>
AuthorDate: Tue Jan 11 09:57:36 2022 -0800
IMPALA-11078 Add simple CSP header to webui.
Content Security Policy (CSP) is a computer security standard designed
to prevent cross-site scripting, clickjacking and other code injection
attacks. CSP provides a method for websites to declare approved origins
of content that browsers should be allowed to load on that website.
A good resource is https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP
If a page breaks the rules then the included script or css will
typically not be run by the browser.
In the Impala webui we use a CSP header to declare that all web content
comes from the impalad, with some 'unsafe' inline code.
A new server flag "--disable_content_security_policy_header=true" can be
set to disable the emission of this header in case of any compatibility
issues.
A few small changes were needed to make this CSP header work. Chart.js
was previously included via http, this was changed to being bundled
like other javascript and css we use. Some dodgy array code that
handles connection metrics was also fixed.
TESTING:
The main webui tests all now validate the CSP header is present.
A test for the new flag is also added.
Change-Id: Idc335d65b117661da0b420ddb7c9ccd80d8d76ab
Reviewed-on: http://gerrit.cloudera.org:8080/18168
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
LICENSE.txt | 23 ++++++++++++++++++
be/src/util/webserver.cc | 12 ++++++++++
bin/rat_exclude_files.txt | 1 +
tests/custom_cluster/test_web_pages.py | 16 +++++++++++++
tests/webserver/test_web_pages.py | 2 +-
www/Chart-2.7.3.min.js | 10 ++++++++
www/admission_controller.tmpl | 2 +-
www/rpcz.tmpl | 44 +++++++++++++++++-----------------
8 files changed, 86 insertions(+), 24 deletions(-)
diff --git a/LICENSE.txt b/LICENSE.txt
index 20b17d4..f4c62b9 100644
--- a/LICENSE.txt
+++ b/LICENSE.txt
@@ -1125,3 +1125,26 @@ You can contact the author at:
- xxHash source repository: https://github.com/Cyan4973/xxHash
--------------------------------------------------------------------------------
+
+www/Chart*: MIT license
+
+ Copyright (c) 2021 Chart.js Contributors
+
+ Permission is hereby granted, free of charge, to any person obtaining a copy of this
+ software and associated documentation files (the "Software"), to deal in the Software
+ without restriction, including without limitation the rights to use, copy, modify,
+ merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
+ permit persons to whom the Software is furnished to do so, subject to the following
+ conditions:
+
+ The above copyright notice and this permission notice shall be included in all copies or
+ substantial portions of the Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
+ INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
+ PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT
+ OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ OTHER DEALINGS IN THE SOFTWARE.
+
+--------------------------------------------------------------------------------
diff --git a/be/src/util/webserver.cc b/be/src/util/webserver.cc
index 5fbf9df..323ebaf 100644
--- a/be/src/util/webserver.cc
+++ b/be/src/util/webserver.cc
@@ -143,6 +143,10 @@ DEFINE_bool(webserver_ldap_passwords_in_clear_ok, false,
"(Advanced) If true, allows the webserver to start with LDAP authentication even if "
"SSL is not enabled, a potentially insecure configuration.");
+DEFINE_bool(disable_content_security_policy_header, false,
+ "If true then the webserver will not add the Content-Security-Policy "
+ "HTTP header to HTTP responses");
+
DECLARE_bool(enable_ldap_auth);
DECLARE_string(hostname);
DECLARE_bool(is_coordinator);
@@ -169,6 +173,11 @@ static const char* ERROR_KEY = "__error_msg__";
static const char* CRLF = "\r\n";
+// The value to be returned in the Content-Security-Policy header.
+static const char* CSP_HEADER = "default-src 'self'; style-src 'self' 'unsafe-inline'; "
+ "script-src 'self' 'unsafe-inline'; "
+ "img-src 'self' data:;";
+
// Returns $IMPALA_HOME if set, otherwise /tmp/impala_www
const char* GetDefaultDocumentRoot() {
stringstream ss;
@@ -226,6 +235,9 @@ void SendResponse(struct sq_connection* connection, const string& response_code_
oss << h << CRLF;
}
oss << "X-Frame-Options: " << FLAGS_webserver_x_frame_options << CRLF;
+ if (!FLAGS_disable_content_security_policy_header) {
+ oss << "Content-Security-Policy: " << CSP_HEADER << CRLF;
+ }
oss << "Content-Type: " << content_type << CRLF;
oss << "Content-Length: " << content.size() << CRLF;
oss << CRLF;
diff --git a/bin/rat_exclude_files.txt b/bin/rat_exclude_files.txt
index 32c12fd..0255ab4 100644
--- a/bin/rat_exclude_files.txt
+++ b/bin/rat_exclude_files.txt
@@ -41,6 +41,7 @@ www/datatables-1.10.18.*
www/bootstrap/css/bootstrap*
www/bootstrap/js/bootstrap*
www/favicon.ico
+www/Chart*
tests/comparison/leopard/static/css/bootstrap*
tests/comparison/leopard/static/fonts/glyphicons-halflings*
tests/comparison/leopard/static/js/bootstrap*
diff --git a/tests/custom_cluster/test_web_pages.py b/tests/custom_cluster/test_web_pages.py
index 2d6a4bf..687f6d0 100644
--- a/tests/custom_cluster/test_web_pages.py
+++ b/tests/custom_cluster/test_web_pages.py
@@ -237,3 +237,19 @@ class TestWebPage(CustomClusterTestSuite):
assert found_links == expected_statestore_links, msg
elif port == "25020":
assert found_links == expected_catalog_links, msg
+
+ @pytest.mark.execute_serially
+ @CustomClusterTestSuite.with_args(
+ impalad_args="--disable_content_security_policy_header=true",
+ statestored_args="--disable_content_security_policy_header=true",
+ catalogd_args="--disable_content_security_policy_header=true"
+ )
+ def test_cdp_header_disabled(self):
+ """Test that if servers are started with the flag
+ --disable_content_security_policy_header=true then the emission of the CDP header is
+ disabled."""
+ ports = ["25000", "25010", "25020"] # Respectively the impalad, statestore, catalog.
+ for port in ports:
+ response = requests.get("http://localhost:%s" % port)
+ assert 'Content-Security-Policy' not in response.headers, \
+ "CSP header present despite being disabled (port %s)" % port
diff --git a/tests/webserver/test_web_pages.py b/tests/webserver/test_web_pages.py
index d870ec4..8defb25 100644
--- a/tests/webserver/test_web_pages.py
+++ b/tests/webserver/test_web_pages.py
@@ -29,7 +29,6 @@ import requests
class TestWebPage(ImpalaTestSuite):
-
ROOT_URL = "http://localhost:{0}/"
SET_JAVA_LOGLEVEL_URL = "http://localhost:{0}/set_java_loglevel"
RESET_JAVA_LOGLEVEL_URL = "http://localhost:{0}/reset_java_loglevel"
@@ -180,6 +179,7 @@ class TestWebPage(ImpalaTestSuite):
assert string_to_search in response.text, "URL: {0} Str:'{1}'\nResp:{2}".format(
input_url, string_to_search, response.text)
responses.append(response)
+ assert 'Content-Security-Policy' in response.headers, "CSP header missing"
return responses
def get_debug_page(self, page_url, port=25000):
diff --git a/www/Chart-2.7.3.min.js b/www/Chart-2.7.3.min.js
new file mode 100644
index 0000000..653e7cf
--- /dev/null
+++ b/www/Chart-2.7.3.min.js
@@ -0,0 +1,10 @@
+/*!
+ * Chart.js
+ * http://chartjs.org/
+ * Version: 2.7.3
+ *
+ * Copyright 2018 Chart.js Contributors
+ * Released under the MIT license
+ * https://github.com/chartjs/Chart.js/blob/master/LICENSE.md
+ */
+!function(t){if("object"==typeof exports&&"undefined"!=typeof module)module.exports=t();else if("function"==typeof define&&define.amd)define([],t);else{("undefined"!=typeof window?window:"undefined"!=typeof global?global:"undefined"!=typeof self?self:this).Chart=t()}}(function(){return function o(r,s,l){function u(e,t){if(!s[e]){if(!r[e]){var i="function"==typeof require&&require;if(!t&&i)return i(e,!0);if(d)return d(e,!0);var n=new Error("Cannot find module '"+e+"'");throw n.code="MODUL [...]
\ No newline at end of file
diff --git a/www/admission_controller.tmpl b/www/admission_controller.tmpl
index 46c73d5..b51c3da 100644
--- a/www/admission_controller.tmpl
+++ b/www/admission_controller.tmpl
@@ -97,7 +97,7 @@ Example of json received from the impala server
<strong>{{statestore_update_staleness_detail}}</strong>
</div>
{{/statestore_update_staleness_detail}}
-<script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.7.3/Chart.min.js" type="text/javascript"></script>
+<script src='{{ __common__.host-url }}/www/Chart-2.7.3.min.js'></script>
<script type="text/javascript">
window.onload = function() {
renderGraph();
diff --git a/www/rpcz.tmpl b/www/rpcz.tmpl
index a9d1b62..802ccf9 100644
--- a/www/rpcz.tmpl
+++ b/www/rpcz.tmpl
@@ -298,17 +298,17 @@ function update_krpc_conn_metrics_datatable(json) {
var table = $('#per_conn_metrics').DataTable();
var rows = $.map(json["per_conn_metrics"], function(row) {
return [[row["remote_ip"], row["num_calls_in_flight"], row["outbound_queue_size"],
- row["socket_stats"]["rtt"]],
- row["socket_stats"]["rttvar"]],
- row["socket_stats"]["cnd_cwnd"]],
- row["socket_stats"]["total_retrans"]],
- row["socket_stats"]["pacing_rate"]],
- row["socket_stats"]["max_pacing_rate"]],
- row["socket_stats"]["bytes_acked"]],
- row["socket_stats"]["bytes_received"]],
- row["socket_stats"]["segs_out"]],
- row["socket_stats"]["segs_in"]],
- row["socket_stats"]["send_queue_bytes"]],
+ row["socket_stats"]["rtt"],
+ row["socket_stats"]["rttvar"],
+ row["socket_stats"]["cnd_cwnd"],
+ row["socket_stats"]["total_retrans"],
+ row["socket_stats"]["pacing_rate"],
+ row["socket_stats"]["max_pacing_rate"],
+ row["socket_stats"]["bytes_acked"],
+ row["socket_stats"]["bytes_received"],
+ row["socket_stats"]["segs_out"],
+ row["socket_stats"]["segs_in"],
+ row["socket_stats"]["send_queue_bytes"],
row["socket_stats"]["receive_queue_bytes"]]];
});
@@ -319,17 +319,17 @@ function update_krpc_inbound_conn_metrics_datatable(json) {
var table = $('#inbound_per_conn_metrics').DataTable();
var rows = $.map(json["inbound_per_conn_metrics"], function(row) {
return [[row["remote_ip"], row["num_calls_in_flight"],
- row["socket_stats"]["rtt"]],
- row["socket_stats"]["rttvar"]],
- row["socket_stats"]["cnd_cwnd"]],
- row["socket_stats"]["total_retrans"]],
- row["socket_stats"]["pacing_rate"]],
- row["socket_stats"]["max_pacing_rate"]],
- row["socket_stats"]["bytes_acked"]],
- row["socket_stats"]["bytes_received"]],
- row["socket_stats"]["segs_out"]],
- row["socket_stats"]["segs_in"]],
- row["socket_stats"]["send_queue_bytes"]],
+ row["socket_stats"]["rtt"],
+ row["socket_stats"]["rttvar"],
+ row["socket_stats"]["cnd_cwnd"],
+ row["socket_stats"]["total_retrans"],
+ row["socket_stats"]["pacing_rate"],
+ row["socket_stats"]["max_pacing_rate"],
+ row["socket_stats"]["bytes_acked"],
+ row["socket_stats"]["bytes_received"],
+ row["socket_stats"]["segs_out"],
+ row["socket_stats"]["segs_in"],
+ row["socket_stats"]["send_queue_bytes"],
row["socket_stats"]["receive_queue_bytes"]]];
});
[impala] 01/02: IMPALA-11062: "now_string" option not working properly
Posted by as...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
asherman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 2e916968992321bf013bc229c89b070d47bb33fa
Author: Steve Carlin <sc...@cloudera.com>
AuthorDate: Sat Dec 18 07:18:48 2021 -0800
IMPALA-11062: "now_string" option not working properly
The "now_string" option was not working when queries are
being sent through the "ExecutePlannedStatement" API. The
string needed to be copied over to the TQueryCtx object
used in RuntimeState.
Change-Id: I977b3a53553aaa40f8d82e7b5b6883b1a6a23065
Reviewed-on: http://gerrit.cloudera.org:8080/18108
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Aman Sinha <am...@cloudera.com>
---
be/src/runtime/query-driver.cc | 2 ++
1 file changed, 2 insertions(+)
diff --git a/be/src/runtime/query-driver.cc b/be/src/runtime/query-driver.cc
index c5d979c..3200ad7 100644
--- a/be/src/runtime/query-driver.cc
+++ b/be/src/runtime/query-driver.cc
@@ -89,6 +89,8 @@ Status QueryDriver::SetExternalPlan(
// Update local_time_zone in the external request
exec_request_->query_exec_request.query_ctx.__set_local_time_zone(
query_ctx.local_time_zone);
+ exec_request_->query_exec_request.query_ctx.__set_now_string(
+ query_ctx.now_string);
return Status::OK();
}