You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/09/19 22:25:00 UTC
[jira] [Commented] (IMPALA-7579)
TestObservability.test_query_profile_contains_all_events fails for S3 tests
[ https://issues.apache.org/jira/browse/IMPALA-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621272#comment-16621272 ]
ASF subversion and git services commented on IMPALA-7579:
---------------------------------------------------------
Commit e4b349e15a47fc113f20233fed19d2a494f5878c in impala's branch refs/heads/master from [~asherman]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=e4b349e ]
IMPALA-7579: fix test_query_profile_contains_all_events on S3
This bug was introduced by IMPALA-6568 which added the
test_query_profile_contains_all_events test. This test creates a file
in the filesystem so that it can be used by 'load data inpath'. The
test was using the hdfs_client object to do file system operations, but
this client only works with hdfs. Switch to using the filesystem_client
object which can work on other filesystems, including s3. Also create
the file that will be moved by 'load data inpath' under the unique
database directory; this means the file can be created without having to
check if it exists already and must be deleted.
Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95
Reviewed-on: http://gerrit.cloudera.org:8080/11461
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> TestObservability.test_query_profile_contains_all_events fails for S3 tests
> ---------------------------------------------------------------------------
>
> Key: IMPALA-7579
> URL: https://issues.apache.org/jira/browse/IMPALA-7579
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 3.1.0
> Reporter: Vuk Ercegovac
> Assignee: Andrew Sherman
> Priority: Blocker
>
> For S3 tests, the test introduced in [https://gerrit.cloudera.org/#/c/11387/] fails with:
> {noformat}
> query_test/test_observability.py:225: in test_query_profile_contains_all_events
> self.hdfs_client.delete_file_dir(path)
> util/hdfs_util.py:90: in delete_file_dir
> if not self.exists(path):
> util/hdfs_util.py:138: in exists
> self.get_file_dir_status(path)
> util/hdfs_util.py:102: in get_file_dir_status
> return super(PyWebHdfsClientWithChmod, self).get_file_dir_status(path)
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/infra/python/env/lib/python2.7/site-packages/pywebhdfs/webhdfs.py:335: in get_file_dir_status
> response = requests.get(uri, allow_redirects=True)
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/infra/python/env/lib/python2.7/site-packages/requests/api.py:69: in get
> return request('get', url, params=params, **kwargs)
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/infra/python/env/lib/python2.7/site-packages/requests/api.py:50: in request
> response = session.request(method=method, url=url, **kwargs)
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/infra/python/env/lib/python2.7/site-packages/requests/sessions.py:465: in request
> resp = self.send(prep, **send_kwargs)
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/infra/python/env/lib/python2.7/site-packages/requests/sessions.py:573: in send
> r = adapter.send(request, **kwargs)
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/infra/python/env/lib/python2.7/site-packages/requests/adapters.py:415: in send
> raise ConnectionError(err, request=request)
> E ConnectionError: ('Connection aborted.', error(111, 'Connection refused')){noformat}
> The dir delete might want to be guarded by an "if exists". The failure cases may differ between hdfs and s3, which is probably what this test ran into.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org