You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by cs...@apache.org on 2026/09/16 09:18:27 UTC
(impala) 03/03: IMPALA-15007: Set verify_cert=True by default in impala-shell
This is an automated email from the ASF dual-hosted git repository.
asf-gitbox-commits pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 099d6319a26d6613eedfade39d54aacddd0e9a1d
Author: Csaba Ringhofer <cs...@cloudera.com>
AuthorDate: Fri Sep 11 17:00:03 2026 +0200
IMPALA-15007: Set verify_cert=True by default in impala-shell
If --ssl is set, but --ca_cert is not, verify the server by
default using system openssl certificates. Can be turned off
with --no_verify_cert.
This is an intentional breaking change for Impala(shell) 5.0.
Assisted-By: Claude Opus 4.8
Change-Id: I8bd07c841fa4a78b9d05a07f73456dd08bdce724
Reviewed-on: http://gerrit.cloudera.org:8080/24823
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
SECURITY.md | 20 +++++++++----
docs/topics/impala_shell_options.xml | 24 ++++++++++++---
docs/topics/impala_ssl.xml | 18 ++++++++----
shell/README.md | 8 +++--
shell/impala_shell/impala_shell.py | 3 +-
shell/impala_shell/impala_shell_config_defaults.py | 2 +-
shell/impala_shell/option_parser.py | 34 +++++++++++++++++-----
tests/custom_cluster/test_client_keepalive.py | 1 +
tests/custom_cluster/test_client_ssl.py | 10 +++++--
tests/custom_cluster/test_thrift_socket.py | 2 +-
10 files changed, 90 insertions(+), 32 deletions(-)
diff --git a/SECURITY.md b/SECURITY.md
index f8948d824..cc3e0a279 100644
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -262,7 +262,7 @@ subset:
| `--authorized_proxy_user_config` / `--authorized_proxy_group_config` | unset *(documented: `docs/topics/impala_delegation.xml`)* | required for Hue-style impersonation; whitelists which authenticated principals may `doas` to which users | breaks B1 if mis-scoped |
| `--cookie_secret_file` | empty *(documented: `be/src/rpc/authentication.cc` line 98)* | when unset, HS2-HTTP cookies fall back to per-process random — sessions do not survive cluster restarts but are not forgeable *(maintainer)* | shared cluster-wide secret for cookie HMAC |
| `--abort_on_config_error` | `true` *(maintainer)* | when off, security misconfigurations may not prevent startup | |
-| `impala-shell --ssl` | `false` | when true, must also configure `ca_cert` or `verify_cert` to validate server certificate | required for TLS configured endpoints |
+| `impala-shell --ssl` | `false` | when enabled, the server certificate is verified by default since Impala 5.0 (against system CA certs, or `--ca_cert` if set), `false` before 5.0; `--no_verify_cert` (or `verify_cert=false`) opts out *(documented: `docs/topics/impala_ssl.xml`)* | required for TLS configured endpoints |
**The insecure-default case.** A number of these flags ship in the "off, must
be turned on for production" posture. The maintainer ruling on whether the
@@ -560,8 +560,10 @@ The operator deploying Impala in production **must**:
15. Secure the underlying storage (HDFS, S3, ADLS, Ozone) with native ACLs;
Impala enforces only what it can see *(documented:
`docs/topics/impala_security_files.xml`)*.
-16. Provide `.impalarc` for `impala-shell` users configuring `ssl` and either
- `ca_cert` or `verify_cert` *(maintainer)*.
+16. For `impala-shell` users connecting with `ssl`, the server certificate is
+ verified by default; provide `.impalarc` with `ca_cert` when the server
+ uses a self-signed or private-CA certificate, and reserve
+ `verify_cert=false` for testing *(maintainer)*.
## §11 Known misuse patterns
@@ -591,9 +593,15 @@ The operator deploying Impala in production **must**:
cluster.** Impala 2.0+ accepts both Kerberos and LDAP on the same port; an
operator who *also* leaves a single coordinator unauthenticated produces a
bypass *(documented: `docs/topics/impala_mixed_security.xml`)*.
-- **`impala-shell` failure to verify server certificate.** Invoking
- `impala-shell --ssl` without specifying `--ca_cert` or `--verify_cert` is a
- known insecure default that will be addressed in a future release.
+- **`impala-shell` connecting with certificate verification disabled.**
+ `impala-shell --ssl` verifies the server certificate by default; passing
+ `--no_verify_cert` (or setting `verify_cert=false`) disables that check and
+ leaves the connection open to man-in-the-middle attacks. Reserve it for
+ testing, and use `--ca_cert` for self-signed or private-CA servers. This is
+ the default since Impala 5.0 (IMPALA-15007); before 5.0, `verify_cert` was
+ `false` by default and `impala-shell --ssl` did **not** verify the server
+ certificate unless `--ca_cert` or `--verify_cert` was set
+ *(documented: `docs/topics/impala_ssl.xml`)*.
## §11a Known non-findings (recurring false positives)
diff --git a/docs/topics/impala_shell_options.xml b/docs/topics/impala_shell_options.xml
index 2ceaf9d9a..57d076d60 100644
--- a/docs/topics/impala_shell_options.xml
+++ b/docs/topics/impala_shell_options.xml
@@ -488,10 +488,26 @@ under the License.
</entry>
<entry>
The local pathname pointing to the third-party CA certificate, or to a copy of the server
- certificate for self-signed server certificates. If <codeph>--ca_cert</codeph> is not set,
- <cmdname>impala-shell</cmdname> enables TLS/SSL, but does not validate the server certificate. This is
- useful for connecting to a known-good Impala that is only running over TLS/SSL, when a copy of the
- certificate is not available (such as when debugging customer installations).
+ certificate for self-signed server certificates. If <codeph>--ca_cert</codeph> is set, the server
+ certificate is always verified against it. If <codeph>--ca_cert</codeph> is not set,
+ <cmdname>impala-shell</cmdname> verifies the server certificate against the system's default CA
+ certificates (see <codeph>--verify_cert</codeph> / <codeph>--no_verify_cert</codeph> below).
+ </entry>
+ </row>
+ <row>
+ <entry>
+ --verify_cert / --no_verify_cert
+ </entry>
+ <entry>
+ verify_cert=true|false
+ </entry>
+ <entry>
+ Controls whether <cmdname>impala-shell</cmdname> verifies the Impala server's TLS certificate when
+ TLS/SSL is enabled and <codeph>--ca_cert</codeph> is not set. Verification against the system's
+ default CA certificates is enabled by default. Pass <codeph>--no_verify_cert</codeph> (or set
+ <codeph>verify_cert=false</codeph> in a config file) to skip verification, for example when
+ connecting to an Impala that uses a self-signed certificate and a copy of that certificate is not
+ available. This option has no effect if <codeph>--ca_cert</codeph> is set.
</entry>
</row>
<row rev="1.2.2">
diff --git a/docs/topics/impala_ssl.xml b/docs/topics/impala_ssl.xml
index 4e261a8ea..acced799e 100644
--- a/docs/topics/impala_ssl.xml
+++ b/docs/topics/impala_ssl.xml
@@ -103,14 +103,20 @@ under the License.
<li>
<codeph>--ca_cert</codeph>: The local pathname pointing to the
third-party CA certificate, or to a copy of the server certificate
- for self-signed server certificates. </li>
+ for self-signed server certificates. When set, the server
+ certificate is always verified against it. </li>
+ <li>
+ <codeph>--no_verify_cert</codeph>: Disables verification of the
+ server certificate (see below). </li>
</ul>
<p> If <codeph>--ca_cert</codeph> is not set,
- <cmdname>impala-shell</cmdname> enables TLS/SSL, but does not
- validate the server certificate. This is useful for connecting to a
- known-good Impala that is only running over TLS/SSL, when a copy of
- the certificate is not available (such as when debugging customer
- installations). </p>
+ <cmdname>impala-shell</cmdname> verifies the server certificate
+ against the system's default CA certificates. To skip verification,
+ pass <codeph>--no_verify_cert</codeph> (or set
+ <codeph>verify_cert=false</codeph> in a configuration file). This is
+ useful for connecting to a known-good Impala that is only running over
+ TLS/SSL, when a copy of the certificate is not available (such as when
+ debugging customer installations).</p>
<p>For <cmdname>impala-shell</cmdname> to successfully connect to an
Impala cluster that has the minimum allowed TLS/SSL version set to 1.2
(<codeph>--ssl_minimum_version=tlsv1.2</codeph>), the Python version
diff --git a/shell/README.md b/shell/README.md
index 76645ef7b..64f58e9bf 100644
--- a/shell/README.md
+++ b/shell/README.md
@@ -65,11 +65,13 @@ Impala.
To connect to a secure host using kerberos and SSL:
```
-$ impala-shell -k --ssl --verify_cert -i impalad-secure-host.domain.com
+$ impala-shell -k --ssl -i impalad-secure-host.domain.com
```
-The command above will try to verify the server with default OpenSSL CA certificates.
-Set --ca_cert to use a custom certificate or remove --verify_cert to skip verification.
+The command above verifies the server with the default OpenSSL CA certificates. Set
+--ca_cert to use a custom certificate (for example a copy of a self-signed server
+certificate), or pass --no_verify_cert to skip verification entirely (insecure). The
+same can be configured from an .impalarc file with ca_cert=... or verify_cert=false.
### Disconnecting
diff --git a/shell/impala_shell/impala_shell.py b/shell/impala_shell/impala_shell.py
index 1ed8c2314..ac9535ad9 100644
--- a/shell/impala_shell/impala_shell.py
+++ b/shell/impala_shell/impala_shell.py
@@ -2454,7 +2454,8 @@ def impala_shell_main():
if options.ca_cert is None and not options.verify_cert:
if options.verbose:
print("SSL is enabled. Impala server certificates will NOT be verified "
- "(set --ca_cert or --verify_cert to change)", file=sys.stderr)
+ "(--no_verify_cert is set). Set --ca_cert or remove --no_verify_cert "
+ "to verify the server certificate.", file=sys.stderr)
else:
if options.verbose:
print("SSL is enabled", file=sys.stderr)
diff --git a/shell/impala_shell/impala_shell_config_defaults.py b/shell/impala_shell/impala_shell_config_defaults.py
index 6baf40c5b..4e56e12ca 100644
--- a/shell/impala_shell/impala_shell_config_defaults.py
+++ b/shell/impala_shell/impala_shell_config_defaults.py
@@ -50,7 +50,7 @@ impala_shell_defaults = {
'rpc_stdout': False,
'rpc_file': None,
'ssl': False,
- 'verify_cert': False,
+ 'verify_cert': True,
'use_kerberos': False,
'use_ldap': False,
'user': getpass.getuser(),
diff --git a/shell/impala_shell/option_parser.py b/shell/impala_shell/option_parser.py
index 65a21cdd1..a260e9912 100644
--- a/shell/impala_shell/option_parser.py
+++ b/shell/impala_shell/option_parser.py
@@ -247,22 +247,33 @@ def get_option_parser(defaults):
help="User to authenticate with.")
parser.add_option("--ssl", dest="ssl",
action="store_true",
- help="Connect to Impala via SSL-secured connection. Does not verify "
- "the server's TLS certificate by default (see --ca_cert and "
- "--verify_cert options).")
+ help="Connect to Impala via SSL-secured connection. By default the "
+ "server's TLS certificate is verified using the system's default CA "
+ "certificates (see the --ca_cert, --verify_cert and --no_verify_cert "
+ "options).")
parser.add_option("--ca_cert", dest="ca_cert",
help=("Full path to "
"certificate file used to authenticate Impala's SSL certificate."
" May either be a copy of Impala's certificate (for self-signed "
- "certs) or the certificate of a trusted third-party CA. If not set, "
- "but SSL is enabled, server verification is based on --verify_cert "
- "flag."))
+ "certs) or the certificate of a trusted third-party CA. If set, the "
+ "server certificate is always verified using this CA cert. If not "
+ "set, but SSL is enabled, the server certificate is verified using "
+ "the system's default CA certificates unless --no_verify_cert is "
+ "given."))
parser.add_option("--verify_cert", dest="verify_cert",
action="store_true",
help=("Verify the Impala server's TLS certificate using the "
"system's default CA certificates when SSL is enabled and --ca_cert "
- "is not set. If --ca_cert is set, the server certificate will always "
- "be verified using the provided CA cert regardless of this flag."))
+ "is not set. This is the default behavior; the flag is only useful "
+ "to override --no_verify_cert or verify_cert=false loaded from a "
+ "config file. If --ca_cert is set, the server certificate is always "
+ "verified using the provided CA cert regardless of this flag."))
+ parser.add_option("--no_verify_cert", dest="verify_cert",
+ action="store_false",
+ help=("Do not verify the Impala server's TLS certificate when SSL "
+ "is enabled and --ca_cert is not set. To disable verification from "
+ "a config file, set verify_cert=false instead. Ignored if --ca_cert "
+ "is set."))
parser.add_option("--config_file", dest="config_file",
help=("Specify the configuration file to load options. "
"The following sections are used: [impala], "
@@ -435,6 +446,10 @@ def get_option_parser(defaults):
# (print disable_live_progress is false since live_progress is true)
elif option == parser.get_option('--disable_live_progress'):
option.help += " [default: %s]" % (not defaults['live_progress'])
+ # --no_verify_cert is the negation of verify_cert, so print the opposite value of
+ # the verify_cert default (print no_verify_cert is false since verify_cert is true).
+ elif option == parser.get_option('--no_verify_cert'):
+ option.help += " [default: %s]" % (not defaults['verify_cert'])
elif option != parser.get_option('--help') and option.help is not SUPPRESS_HELP:
# don't want to print default value for help or options without help text
option.help += " [default: %default]"
@@ -453,6 +468,9 @@ def get_option_parser(defaults):
if '--verbose' in sys.argv and '--quiet' in sys.argv:
parser.error("options --verbose and --quiet are mutually exclusive")
+ if '--verify_cert' in sys.argv and '--no_verify_cert' in sys.argv:
+ parser.error("options --verify_cert and --no_verify_cert are mutually exclusive")
+
parser.set_defaults(**defaults)
return parser
diff --git a/tests/custom_cluster/test_client_keepalive.py b/tests/custom_cluster/test_client_keepalive.py
index a14b103cc..bd1c17ff1 100644
--- a/tests/custom_cluster/test_client_keepalive.py
+++ b/tests/custom_cluster/test_client_keepalive.py
@@ -74,6 +74,7 @@ class TestClientKeepalive(CustomClusterTestSuite):
shell_options = ["-q", "shell {0}".format(ss_command)]
if ssl:
shell_options.append("--ssl")
+ shell_options.append("--no_verify_cert")
result = run_impala_shell_cmd(vector, shell_options)
LOG.info("STDOUT: {0} STDERR: {1}".format(result.stdout, result.stderr))
diff --git a/tests/custom_cluster/test_client_ssl.py b/tests/custom_cluster/test_client_ssl.py
index 31b7f0702..7bb1d6ee0 100644
--- a/tests/custom_cluster/test_client_ssl.py
+++ b/tests/custom_cluster/test_client_ssl.py
@@ -92,7 +92,7 @@ class TestClientSsl(CustomClusterTestSuite):
impalad = ImpaladService(socket.getfqdn())
assert impalad.wait_for_num_in_flight_queries(0)
impalad.wait_for_metric_value('impala-server.backend-num-queries-executing', 0)
- p = ImpalaShell(vector, args=["--ssl"])
+ p = ImpalaShell(vector, args=["--ssl", "--no_verify_cert"])
p.send_cmd("SET DEBUG_ACTION=0:OPEN:WAIT")
p.send_cmd("select count(*) from functional.alltypes")
# Wait until the query has been planned and started executing, at which point it
@@ -254,12 +254,18 @@ class TestClientSsl(CustomClusterTestSuite):
# default CA certs from OS while test certs are self signed.
run_shell(["--ssl", "--verify_cert"])
+ # Certificate verification is on by default, so plain --ssl must also fail against
+ # the self-signed test certs (the OS default CA certs do not trust them).
+ run_shell(["--ssl"])
+
def _validate_positive_cases(self, vector, ca_cert="", host=None):
python3_10_version_re = re.compile(r"using Python 3\.1[0-9]")
shell_options = ["--ssl", "-q", "select 1 + 2"]
if host:
shell_options.extend(["-i", host])
- result = run_impala_shell_cmd(vector, shell_options)
+ # Without a --ca_cert the self-signed test certs cannot be verified against the OS
+ # default CA certs, so verification has to be explicitly disabled.
+ result = run_impala_shell_cmd(vector, shell_options + ["--no_verify_cert"])
for msg in [self.SSL_ENABLED, self.CONNECTED, self.FETCHED]:
assert msg in result.stderr
# Python >3.10 has deprecated ssl.PROTOCOL_TLS and impala-shell currently emits a
diff --git a/tests/custom_cluster/test_thrift_socket.py b/tests/custom_cluster/test_thrift_socket.py
index bb1aedf82..e9d671ab5 100644
--- a/tests/custom_cluster/test_thrift_socket.py
+++ b/tests/custom_cluster/test_thrift_socket.py
@@ -82,7 +82,7 @@ class TestThriftSocket(CustomClusterTestSuite):
# Iterate over test vector within test function to avoid restarting cluster.
for protocol_dim in create_client_protocol_dimension():
for vector in [ImpalaTestVector([protocol_dim])]:
- shell_args = ["-Q", "idle_session_timeout=1800", "--ssl"]
+ shell_args = ["-Q", "idle_session_timeout=1800", "--ssl", "--no_verify_cert"]
# This uses a longer idle time to verify IMPALA-12114, see comment above.
self._run_idle_shell(vector, shell_args, 12)
self.assert_impalad_log_contains('INFO',