[jira] [Commented] (BIGTOP-1750) Restructure TestCLI

["Martin Bukatovic (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BIGTOP-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356867#comment-14356867 ] 

Martin Bukatovic commented on BIGTOP-1750:
------------------------------------------

Ok, so it looks like it's time for me to explain hcfs convention I introduced
in BIGTOP-1342 bit better (you can check the original proposal with description
there).

I splitted TestCLI test cases into 2 groups:

 * *hcfs group* - those test cases are applicabe to any Hadoop filesystem
   (HDFS, GlusterFS, ...) - hence the name hcfs (which stands for Hadoop
   Compatible File System).
 * *hdfs group* - those tests are applicable to HDFS only (HDFS specific
   features are used there).

Why did I do that? I tried to make TestCLI test cases runnable on any
Hadoop filesystem while preserving the possibility to execute cases which are
HDFS specific.

So that I ended up with the following structure:

 * test {{org.apache.bigtop.itest.hadoop.hcfs.TestCLI}} contains all the logic
   and executes testcases from {{testHCFSConf.xml}} (this is the hcfs group)
 * test {{org.apache.bigtop.itest.hadoop.hdfs.TestHDFSCLI}} executes
   {{testHDFSConf.xml}} file which contains HDFS only cases (the hdfs group)

This means that for Hadoop cluster with HDFS, one would run both TestCLI and
TestHDFSCLI, while for cluster which uses alternative Hadoop filesystem (such
as GlusterFS), one would run TestCLI only.

Another consequence of this is that all the code is in hcfs module as
{{TestHDFSCLI}} just redefines one method to change the xml file with test
definitions. And that is where the test cases are actually stored, we have:

 * {{testHCFSConf.xml}} - tests for any hadoop filesystem
 * {{testHDFSConf.xml}} - tests for HDFS filesystem only
 * {{testConf.xml}} - this file is no longer used, I didn't delete it for
                      historical reasons

With this in mind, I believe that no further restructuration is needed.

{quote}
Currently, it includes tests that work with both HCFS and HDFS, the combination
of which occasionally causes problems.
{quote}

This is not correct, as the trash issue pointed out there is not actually
related to HCFS/HDFS split (since enabling trash would break tests on any
hadoop filesystem).

{quote}
I'm proposing we move all HDFS tests to TestHDFSCLI, which currently holds only
HDFS-specific tests.
{quote}

I don't think this is a good idea because this would mean enormous duplication
of the description of testcases - which not manageable in the long term.

{quote}
We can make configurational requirements specific to each FS's tests and
reorganize the xml files, since they're getting to be a bit too much (imho –
there's just too much of everything in TestHCFSConf.xml at this point).
{quote}

Configurational requirements are checked already in a correct way (imho).
And again, I don't think that reorganization of xml files is needed,
merging back everything into hdfs xml file would be much worse.

{quote}
And, unless I'm mistaken, we're not even using testConf.xml anymore?
{quote}

This is correct, we can delete this file to not confuse others.


> Restructure TestCLI
> -------------------
>
>                 Key: BIGTOP-1750
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-1750
>             Project: Bigtop
>          Issue Type: Bug
>          Components: tests
>    Affects Versions: 0.8.0
>            Reporter: Dasha Boudnik
>            Assignee: Dasha Boudnik
>             Fix For: 0.9.0
>
>
> As per the discussion in BIGTOP-1748, we want to restructure TestCLI. Currently, it includes tests that work with both HCFS and HDFS, the combination of which occasionally causes problems.
> I'm proposing we move all HDFS tests to TestHDFSCLI, which currently holds _only_ HDFS-specific tests.
> We can make configurational requirements specific to each FS's tests and reorganize the xml files, since they're getting to be a bit too much (imho -- there's just too much of everything in TestHCFSConf.xml at this point).
> And, unless I'm mistaken, we're not even using testConf.xml anymore?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)