Renaming prefix-named tests (IMPALA-4721)

[Lars Volker <lv...@cloudera.com>]

Hi all,

Some of our test names are also prefixes of other tests, e.g.
in tests/metadata/test_ddl.py we have *test_create_table* and
*test_create_table_as_select*. Selecting the former with "impala-py.test -k
test_create_table" will also select the latter.

In the past when I ran into these I renamed the prefix-named test, usually
by adding "_test" to make them unique. However this is somewhat unintuitive
and consequently needed explanation during reviews. To improve the
situation, I would like to propose changing all affected tests in a single
commit.

Before doing that, I'd like to ask for feedback. I created IMPALA-4721
<https://issues.cloudera.org/browse/IMPALA-4721> to track this.

Thanks, Lars

Re: Renaming prefix-named tests (IMPALA-4721)

[David Knupp <dk...@cloudera.com>]

There is a way to run a single test within a test module or class 
without -k  -- by instead using ::to resolve scope. As noted, using -k 
allows one to run tests according to keyword matching on test 
class/function names, but this can be inefficient if one test function 
name is a substring of another, or different test classes have similarly 
named functions. E.g., assume the following test module:


$ cat test_stuff/test_stuff.py
class TestMeOne(object):
     def test_me(self):
         assert True

     def test_me_one(self):
         assert True

     def test_me_two(self):
         assert True

     def test_me_three(self):
         assert True


class TestMeTwo(object):
     def test_me(self):
         assert True

     def test_me_one(self):
         assert True

     def test_me_two(self):
         assert True

     def test_me_three(self):
         assert True


If one simply points py.test at the test file itself, all eight tests 
will obviously run. But if I want to just run the test_me in the 
TestMeOne class, -k test_mewill obviously not help. All eight will still 
run.

$ impala-py.test test_stuff/test_stuff.py -k test_me
============================== test session starts 
==============================
platform linux2 -- Python 2.6.6 -- py-1.4.30 -- pytest-2.7.2
rootdir: /Users/dknupp/tmp/test_stuff, inifile:
collected 8 items

test_stuff/test_stuff.py ........

=========================== 8 passed in 0.01 seconds 
============================


However, py.test accepts the ::operator to resolve scope, and this can 
be used on the command line to isolate a single test function.

$ impala-py.testtest_stuff/test_stuff.py::TestMeOne::test_me
============================== test session starts 
==============================
platform linux2 -- Python 2.6.6 -- py-1.4.30 -- pytest-2.7.2
rootdir: /Users/dknupp/tmp/test_stuff, inifile:
collected 5 items

test_stuff/test_stuff.py .

=========================== 1 passed in 0.01 seconds 
============================


This is even easier to confirm if you use -v to run in verbose mode:

$ impala-py.test -v test_stuff/test_stuff.py::TestMeOne::test_me
============================== test session starts 
==============================
platform linux2 -- Python 2.6.6 -- py-1.4.30 -- pytest-2.7.2 -- 
/Users/dknupp/venv/dev/bin/python
cachedir: test_stuff/.cache
rootdir: /Users/dknupp/tmp/test_stuff, inifile:
collected 5 items

test_stuff/test_stuff.py::TestMeOne::test_me PASSED

=========================== 1 passed in 0.00 seconds 
============================

> Alex Behm <ma...@cloudera.com>
> January 4, 2017 at 11:54 AM
> Before we go down that route, isn't there a proper Pytest way to run 
> only a
> single test within a .py file?
> Our -k option is meant to filter based on a regex, not necessarily 
> select a
> single test.
>
>
> Lars Volker <ma...@cloudera.com>
> January 4, 2017 at 1:28 AM
> Hi all,
>
> Some of our test names are also prefixes of other tests, e.g.
> in tests/metadata/test_ddl.py we have *test_create_table* and
> *test_create_table_as_select*. Selecting the former with 
> "impala-py.test -k
> test_create_table" will also select the latter.
>
> In the past when I ran into these I renamed the prefix-named test, usually
> by adding "_test" to make them unique. However this is somewhat 
> unintuitive
> and consequently needed explanation during reviews. To improve the
> situation, I would like to propose changing all affected tests in a single
> commit.
>
> Before doing that, I'd like to ask for feedback. I created IMPALA-4721
> <https://issues.cloudera.org/browse/IMPALA-4721> to track this.
>
> Thanks, Lars
>

-- 
David Knupp
Software Engineer
Cloudera
415-312-1049
<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>

Re: Renaming prefix-named tests (IMPALA-4721)

[Alex Behm <al...@cloudera.com>]

Before we go down that route, isn't there a proper Pytest way to run only a
single test within a .py file?
Our -k option is meant to filter based on a regex, not necessarily select a
single test.

On Wed, Jan 4, 2017 at 1:28 AM, Lars Volker <lv...@cloudera.com> wrote:

> Hi all,
>
> Some of our test names are also prefixes of other tests, e.g.
> in tests/metadata/test_ddl.py we have *test_create_table* and
> *test_create_table_as_select*. Selecting the former with "impala-py.test -k
> test_create_table" will also select the latter.
>
> In the past when I ran into these I renamed the prefix-named test, usually
> by adding "_test" to make them unique. However this is somewhat unintuitive
> and consequently needed explanation during reviews. To improve the
> situation, I would like to propose changing all affected tests in a single
> commit.
>
> Before doing that, I'd like to ask for feedback. I created IMPALA-4721
> <https://issues.cloudera.org/browse/IMPALA-4721> to track this.
>
> Thanks, Lars
>