[jira] [Created] (IMPALA-5330) Impala tests never use or set a secondary FS, so TestMultipleFilesystems is always skipped

["David Knupp (JIRA)" <ji...@apache.org>]

David Knupp created IMPALA-5330:
-----------------------------------

             Summary: Impala tests never use or set a secondary FS, so TestMultipleFilesystems is always skipped
                 Key: IMPALA-5330
                 URL: https://issues.apache.org/jira/browse/IMPALA-5330
             Project: IMPALA
          Issue Type: Bug
          Components: Infrastructure
            Reporter: David Knupp
            Priority: Critical
             Fix For: Impala 2.9.0


This issue came up while investigating how can set up Impala + ADLS testing on remote clusters. Since testing Impala with ADLS is analogous to testing Impala with s3, I used that testing as a starting point.

When we change the TARGET_FILESYSTEM to 's3', the end result is to overwrite the defaultFS setting in core-site.xml to the s3 location. There's not a point (or doesn't appear to be a point) where we ever treat s3 as a true secondary filesystem.

Further more, if we look at our test suite that is supposed to check multiple filesystems simultaneously, we can see there's a pytest SkipIf decorator set to skip the test if a secondary FS is not set.

{noformat}
@SkipIf.no_secondary_fs
class TestMultipleFilesystems(ImpalaTestSuite)
{noformat}

From what I can see by going through old Jenkins console logs, this test never gets executed. It's skipped during exhaustive runs with HDFS as the default, and is also skipped during s3 and local FS test runs as well.

{noformat}
04:27:57 query_test/test_multiple_filesystems.py::TestMultipleFilesystems::test_multiple_filesystems[exec_option: {'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size': 0, 'num_nodes': 0} | table_format: text/none] SKIPPED
{noformat}




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)