You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Matthew Mulder (JIRA)" <ji...@apache.org> on 2017/09/21 00:03:00 UTC
[jira] [Created] (IMPALA-5968) load_nested.py fails on secure
cluster: no write access to /user on HDFS
Matthew Mulder created IMPALA-5968:
--------------------------------------
Summary: load_nested.py fails on secure cluster: no write access to /user on HDFS
Key: IMPALA-5968
URL: https://issues.apache.org/jira/browse/IMPALA-5968
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 2.10.0
Reporter: Matthew Mulder
Assignee: Matthew Mulder
Priority: Critical
Fix For: Impala 2.11.0
{code}testdata/bin/load_nested.py \
-s tpch_100_parquet \
-t tpch_100_nested_parquet \
--cm-host=vc0332.test \
-p 34 --use-kerberos --use-ssl
Output: 2017-09-20 14:18:06,174 INFO:db_connection[234]:Creating database tpch_100_nested_parquet
2017-09-20 14:18:06,737 INFO:load_nested[98]:Creating temp orders (chunk 1 of 34)
2017-09-20 14:18:31,420 INFO:load_nested[98]:Creating temp orders (chunk 2 of 34)
2017-09-20 14:18:52,146 INFO:load_nested[98]:Creating temp orders (chunk 3 of 34)
2017-09-20 14:19:12,398 INFO:load_nested[98]:Creating temp orders (chunk 4 of 34)
2017-09-20 14:19:32,109 INFO:load_nested[98]:Creating temp orders (chunk 5 of 34)
2017-09-20 14:19:51,823 INFO:load_nested[98]:Creating temp orders (chunk 6 of 34)
2017-09-20 14:20:12,521 INFO:load_nested[98]:Creating temp orders (chunk 7 of 34)
2017-09-20 14:20:33,220 INFO:load_nested[98]:Creating temp orders (chunk 8 of 34)
2017-09-20 14:20:53,417 INFO:load_nested[98]:Creating temp orders (chunk 9 of 34)
2017-09-20 14:21:13,143 INFO:load_nested[98]:Creating temp orders (chunk 10 of 34)
2017-09-20 14:21:34,361 INFO:load_nested[98]:Creating temp orders (chunk 11 of 34)
2017-09-20 14:21:53,048 INFO:load_nested[98]:Creating temp orders (chunk 12 of 34)
2017-09-20 14:22:13,743 INFO:load_nested[98]:Creating temp orders (chunk 13 of 34)
2017-09-20 14:22:34,439 INFO:load_nested[98]:Creating temp orders (chunk 14 of 34)
2017-09-20 14:22:55,160 INFO:load_nested[98]:Creating temp orders (chunk 15 of 34)
2017-09-20 14:23:15,868 INFO:load_nested[98]:Creating temp orders (chunk 16 of 34)
2017-09-20 14:23:37,075 INFO:load_nested[98]:Creating temp orders (chunk 17 of 34)
2017-09-20 14:23:57,260 INFO:load_nested[98]:Creating temp orders (chunk 18 of 34)
2017-09-20 14:24:18,967 INFO:load_nested[98]:Creating temp orders (chunk 19 of 34)
2017-09-20 14:24:39,680 INFO:load_nested[98]:Creating temp orders (chunk 20 of 34)
2017-09-20 14:24:59,364 INFO:load_nested[98]:Creating temp orders (chunk 21 of 34)
2017-09-20 14:25:20,556 INFO:load_nested[98]:Creating temp orders (chunk 22 of 34)
2017-09-20 14:25:40,240 INFO:load_nested[98]:Creating temp orders (chunk 23 of 34)
2017-09-20 14:26:00,949 INFO:load_nested[98]:Creating temp orders (chunk 24 of 34)
2017-09-20 14:26:21,146 INFO:load_nested[98]:Creating temp orders (chunk 25 of 34)
2017-09-20 14:26:41,841 INFO:load_nested[98]:Creating temp orders (chunk 26 of 34)
2017-09-20 14:27:01,530 INFO:load_nested[98]:Creating temp orders (chunk 27 of 34)
2017-09-20 14:27:21,219 INFO:load_nested[98]:Creating temp orders (chunk 28 of 34)
2017-09-20 14:27:42,921 INFO:load_nested[98]:Creating temp orders (chunk 29 of 34)
2017-09-20 14:28:04,617 INFO:load_nested[98]:Creating temp orders (chunk 30 of 34)
2017-09-20 14:28:25,808 INFO:load_nested[98]:Creating temp orders (chunk 31 of 34)
2017-09-20 14:28:48,031 INFO:load_nested[98]:Creating temp orders (chunk 32 of 34)
2017-09-20 14:29:08,213 INFO:load_nested[98]:Creating temp orders (chunk 33 of 34)
2017-09-20 14:29:29,918 INFO:load_nested[98]:Creating temp orders (chunk 34 of 34)
2017-09-20 14:29:50,610 INFO:load_nested[128]:Creating temp customers (chunk 1 of 34)
2017-09-20 14:30:18,747 INFO:load_nested[128]:Creating temp customers (chunk 2 of 34)
2017-09-20 14:30:44,058 INFO:load_nested[128]:Creating temp customers (chunk 3 of 34)
2017-09-20 14:31:07,818 INFO:load_nested[128]:Creating temp customers (chunk 4 of 34)
2017-09-20 14:31:31,565 INFO:load_nested[128]:Creating temp customers (chunk 5 of 34)
2017-09-20 14:31:54,799 INFO:load_nested[128]:Creating temp customers (chunk 6 of 34)
2017-09-20 14:32:18,529 INFO:load_nested[128]:Creating temp customers (chunk 7 of 34)
2017-09-20 14:32:41,762 INFO:load_nested[128]:Creating temp customers (chunk 8 of 34)
2017-09-20 14:33:04,974 INFO:load_nested[128]:Creating temp customers (chunk 9 of 34)
2017-09-20 14:33:28,713 INFO:load_nested[128]:Creating temp customers (chunk 10 of 34)
2017-09-20 14:33:51,923 INFO:load_nested[128]:Creating temp customers (chunk 11 of 34)
2017-09-20 14:34:15,656 INFO:load_nested[128]:Creating temp customers (chunk 12 of 34)
2017-09-20 14:34:38,900 INFO:load_nested[128]:Creating temp customers (chunk 13 of 34)
2017-09-20 14:35:02,628 INFO:load_nested[128]:Creating temp customers (chunk 14 of 34)
2017-09-20 14:35:26,347 INFO:load_nested[128]:Creating temp customers (chunk 15 of 34)
2017-09-20 14:35:51,079 INFO:load_nested[128]:Creating temp customers (chunk 16 of 34)
2017-09-20 14:36:14,804 INFO:load_nested[128]:Creating temp customers (chunk 17 of 34)
2017-09-20 14:36:38,535 INFO:load_nested[128]:Creating temp customers (chunk 18 of 34)
2017-09-20 14:37:01,765 INFO:load_nested[128]:Creating temp customers (chunk 19 of 34)
2017-09-20 14:37:25,512 INFO:load_nested[128]:Creating temp customers (chunk 20 of 34)
2017-09-20 14:37:49,745 INFO:load_nested[128]:Creating temp customers (chunk 21 of 34)
2017-09-20 14:38:13,980 INFO:load_nested[128]:Creating temp customers (chunk 22 of 34)
2017-09-20 14:38:37,706 INFO:load_nested[128]:Creating temp customers (chunk 23 of 34)
2017-09-20 14:39:01,428 INFO:load_nested[128]:Creating temp customers (chunk 24 of 34)
2017-09-20 14:39:25,652 INFO:load_nested[128]:Creating temp customers (chunk 25 of 34)
2017-09-20 14:39:49,368 INFO:load_nested[128]:Creating temp customers (chunk 26 of 34)
2017-09-20 14:40:13,600 INFO:load_nested[128]:Creating temp customers (chunk 27 of 34)
2017-09-20 14:40:38,902 INFO:load_nested[128]:Creating temp customers (chunk 28 of 34)
2017-09-20 14:41:03,117 INFO:load_nested[128]:Creating temp customers (chunk 29 of 34)
2017-09-20 14:41:26,847 INFO:load_nested[128]:Creating temp customers (chunk 30 of 34)
2017-09-20 14:41:51,068 INFO:load_nested[128]:Creating temp customers (chunk 31 of 34)
2017-09-20 14:42:15,300 INFO:load_nested[128]:Creating temp customers (chunk 32 of 34)
2017-09-20 14:42:39,527 INFO:load_nested[128]:Creating temp customers (chunk 33 of 34)
2017-09-20 14:43:03,753 INFO:load_nested[128]:Creating temp customers (chunk 34 of 34)
2017-09-20 14:43:28,187 INFO:load_nested[178]:Creating temp regions
2017-09-20 14:43:38,690 INFO:load_nested[208]:Creating temp suppliers
2017-09-20 14:44:51,084 INFO:load_nested[244]:Creating parts
2017-09-20 14:44:59,144 INFO:cluster[422]:kerberos module not found; attempting to install it...
2017-09-20 14:45:00,056 INFO:cluster[429]:kerberos installation complete.
/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
2017-09-20 14:45:21,562 ERROR:kerberos_[226]:handle_other(): Mutual authentication unavailable on 403 response
Traceback (most recent call last):
File "testdata/bin/load_nested.py", line 320, in <module>
load()
File "testdata/bin/load_nested.py", line 253, in load
cluster.hdfs.ensure_home_dir()
File "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/tests/comparison/cluster.py", line 402, in ensure_home_dir
client.makedirs(hdfs_dir)
File "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/client.py", line 866, in makedirs
self._mkdirs(hdfs_path, permission=permission)
File "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/client.py", line 90, in api_handler
**self.kwargs
File "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/ext/kerberos.py", line 98, in _request
return super(KerberosClient, self)._request(method, url, **kwargs)
File "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/client.py", line 182, in _request
return _on_error(response)
File "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/client.py", line 44, in _on_error
raise HdfsError(message)
hdfs.util.HdfsError: Permission denied: user=systest, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)