You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Alex Huang <Al...@citrix.com> on 2013/03/06 02:42:55 UTC
unit tests guidelines...
With the discussion in the BVT thread and other evidence from the check-ins, I think there are some confusion on when to and what to write in a unit test.
Unit tests are a philosophy thing and I usually stay away from things like that. If you are interested in writing the right type unit tests, here's my $.02.
Unit tests are for
- Guaranteeing the component is upholding its contract.
- Illustrating how the component is to be used.
- Mocking up failure scenarios
- Explaining something that people might not understand if they look inside your code.
When to write a unit test:
- Write it at the component level because that's where the value is. You can test everything under the sun but basically code changes all of the time. What you really want
- Write it for an interface. For example, if you have AgentManager and AgentManagerImpl, the methods you should test is in AgentManager. And a lot of this goes back to your design. I have seen quite a bit of code that just adds all the methods from the class to the interface. It's just something you do rather than something you think before doing. That's just wrong and it increases the number of unit tests you have to write.
These are usually what you need to test for when writing unit tests.
- Errors in the incoming parameters...<no duh!>
- Different positive paths for your code...<to state the obvious>.
- The one that people don't seem to do is to inject results into the components that the component being tested is dependent on. This forces component being tested to travel a different path. Most people recognize incoming parameters causing a different path but does not recognize that results from the components being used can cause a different path.
How to recognize a good unit test?
- The mock objects do not always return true or positive result.
- The unit test sets something for the mock object, test the method; sets something else for the mock objects and then test the method again. This means the tester is testing the component handling different returns from the components it is using.
- The unit test makes a call to the component, and checks the mock object to see if certain things are called. White box testing there.
- The unit test has asserts for catching exceptions or negative returns (negative paths are tested)
- The unit test has comments that tell you what and why you're testing
- The unit test asserts tells you why the assert should not fail
- You won the blame game by saying look I have a test case for exactly that.
Misconceptions about unit tests.
- You only write it when you write the component. Actually, a good unit test is one that's progressively built up. You found a bug, you write a test to make sure the bug is fixed. If you've gotten to that stage, then you've pretty much reach guru status.
--Alex
Re: unit tests guidelines...
Posted by Ahmad Emneina <ae...@gmail.com>.
+1 to fostering guruism.
On Tue, Mar 5, 2013 at 5:42 PM, Alex Huang <Al...@citrix.com> wrote:
> With the discussion in the BVT thread and other evidence from the
> check-ins, I think there are some confusion on when to and what to write in
> a unit test.
>
> Unit tests are a philosophy thing and I usually stay away from things like
> that. If you are interested in writing the right type unit tests, here's
> my $.02.
>
> Unit tests are for
>
> - Guaranteeing the component is upholding its contract.
>
> - Illustrating how the component is to be used.
>
> - Mocking up failure scenarios
>
> - Explaining something that people might not understand if they
> look inside your code.
>
> When to write a unit test:
>
> - Write it at the component level because that's where the value
> is. You can test everything under the sun but basically code changes all
> of the time. What you really want
>
> - Write it for an interface. For example, if you have
> AgentManager and AgentManagerImpl, the methods you should test is in
> AgentManager. And a lot of this goes back to your design. I have seen
> quite a bit of code that just adds all the methods from the class to the
> interface. It's just something you do rather than something you think
> before doing. That's just wrong and it increases the number of unit tests
> you have to write.
>
> These are usually what you need to test for when writing unit tests.
>
> - Errors in the incoming parameters...<no duh!>
>
> - Different positive paths for your code...<to state the obvious>.
>
> - The one that people don't seem to do is to inject results into
> the components that the component being tested is dependent on. This
> forces component being tested to travel a different path. Most people
> recognize incoming parameters causing a different path but does not
> recognize that results from the components being used can cause a different
> path.
>
> How to recognize a good unit test?
>
> - The mock objects do not always return true or positive result.
>
> - The unit test sets something for the mock object, test the
> method; sets something else for the mock objects and then test the method
> again. This means the tester is testing the component handling different
> returns from the components it is using.
>
> - The unit test makes a call to the component, and checks the
> mock object to see if certain things are called. White box testing there.
>
> - The unit test has asserts for catching exceptions or negative
> returns (negative paths are tested)
>
> - The unit test has comments that tell you what and why you're
> testing
>
> - The unit test asserts tells you why the assert should not fail
>
> - You won the blame game by saying look I have a test case for
> exactly that.
>
> Misconceptions about unit tests.
>
> - You only write it when you write the component. Actually, a
> good unit test is one that's progressively built up. You found a bug, you
> write a test to make sure the bug is fixed. If you've gotten to that
> stage, then you've pretty much reach guru status.
>
> --Alex
>
Re: unit tests guidelines...
Posted by Wido den Hollander <wi...@widodh.nl>.
On 03/06/2013 02:42 AM, Alex Huang wrote:
> With the discussion in the BVT thread and other evidence from the check-ins, I think there are some confusion on when to and what to write in a unit test.
>
> Unit tests are a philosophy thing and I usually stay away from things like that. If you are interested in writing the right type unit tests, here's my $.02.
>
Thanks for your input! Writing good tests is indeed a challenge, but
definitely worth it, although I'm not a guru-status yet.
Wido
> Unit tests are for
>
> - Guaranteeing the component is upholding its contract.
>
> - Illustrating how the component is to be used.
>
> - Mocking up failure scenarios
>
> - Explaining something that people might not understand if they look inside your code.
>
> When to write a unit test:
>
> - Write it at the component level because that's where the value is. You can test everything under the sun but basically code changes all of the time. What you really want
>
> - Write it for an interface. For example, if you have AgentManager and AgentManagerImpl, the methods you should test is in AgentManager. And a lot of this goes back to your design. I have seen quite a bit of code that just adds all the methods from the class to the interface. It's just something you do rather than something you think before doing. That's just wrong and it increases the number of unit tests you have to write.
>
> These are usually what you need to test for when writing unit tests.
>
> - Errors in the incoming parameters...<no duh!>
>
> - Different positive paths for your code...<to state the obvious>.
>
> - The one that people don't seem to do is to inject results into the components that the component being tested is dependent on. This forces component being tested to travel a different path. Most people recognize incoming parameters causing a different path but does not recognize that results from the components being used can cause a different path.
>
> How to recognize a good unit test?
>
> - The mock objects do not always return true or positive result.
>
> - The unit test sets something for the mock object, test the method; sets something else for the mock objects and then test the method again. This means the tester is testing the component handling different returns from the components it is using.
>
> - The unit test makes a call to the component, and checks the mock object to see if certain things are called. White box testing there.
>
> - The unit test has asserts for catching exceptions or negative returns (negative paths are tested)
>
> - The unit test has comments that tell you what and why you're testing
>
> - The unit test asserts tells you why the assert should not fail
>
> - You won the blame game by saying look I have a test case for exactly that.
>
> Misconceptions about unit tests.
>
> - You only write it when you write the component. Actually, a good unit test is one that's progressively built up. You found a bug, you write a test to make sure the bug is fixed. If you've gotten to that stage, then you've pretty much reach guru status.
>
> --Alex
>
Re: unit tests guidelines...
Posted by Chip Childers <ch...@sungard.com>.
On Tue, Mar 05, 2013 at 05:42:55PM -0800, Alex Huang wrote:
> With the discussion in the BVT thread and other evidence from the check-ins, I think there are some confusion on when to and what to write in a unit test.
>
> Unit tests are a philosophy thing and I usually stay away from things like that. If you are interested in writing the right type unit tests, here's my $.02.
>
> Unit tests are for
>
> - Guaranteeing the component is upholding its contract.
>
> - Illustrating how the component is to be used.
>
> - Mocking up failure scenarios
>
> - Explaining something that people might not understand if they look inside your code.
>
> When to write a unit test:
>
> - Write it at the component level because that's where the value is. You can test everything under the sun but basically code changes all of the time. What you really want
>
> - Write it for an interface. For example, if you have AgentManager and AgentManagerImpl, the methods you should test is in AgentManager. And a lot of this goes back to your design. I have seen quite a bit of code that just adds all the methods from the class to the interface. It's just something you do rather than something you think before doing. That's just wrong and it increases the number of unit tests you have to write.
>
> These are usually what you need to test for when writing unit tests.
>
> - Errors in the incoming parameters...<no duh!>
>
> - Different positive paths for your code...<to state the obvious>.
>
> - The one that people don't seem to do is to inject results into the components that the component being tested is dependent on. This forces component being tested to travel a different path. Most people recognize incoming parameters causing a different path but does not recognize that results from the components being used can cause a different path.
>
> How to recognize a good unit test?
>
> - The mock objects do not always return true or positive result.
>
> - The unit test sets something for the mock object, test the method; sets something else for the mock objects and then test the method again. This means the tester is testing the component handling different returns from the components it is using.
>
> - The unit test makes a call to the component, and checks the mock object to see if certain things are called. White box testing there.
>
> - The unit test has asserts for catching exceptions or negative returns (negative paths are tested)
>
> - The unit test has comments that tell you what and why you're testing
>
> - The unit test asserts tells you why the assert should not fail
>
> - You won the blame game by saying look I have a test case for exactly that.
>
> Misconceptions about unit tests.
>
> - You only write it when you write the component. Actually, a good unit test is one that's progressively built up. You found a bug, you write a test to make sure the bug is fixed. If you've gotten to that stage, then you've pretty much reach guru status.
>
> --Alex
+1 to this whole list. Want to add it to the wiki to institutionalize
it?
Re: unit tests guidelines...
Posted by Prasanna Santhanam <ts...@apache.org>.
On Wed, Mar 06, 2013 at 07:12:55AM +0530, Alex Huang wrote:
> With the discussion in the BVT thread and other evidence from the
> check-ins, I think there are some confusion on when to and what to
> write in a unit test.
>
Not just unit-tests alone, most testing in general. There was an
insightful thread about this on the testing-in-python lists that
discussed this in detail quite recently.
http://lists.idyll.org/pipermail/testing-in-python/2013-February/005399.html
Terms like bvt, smoke, uat, integration test, system test, functional
test are out in the wild!
Google apparently just uses small, medium, large for the complexity of
the test being written be it unit/functional etc.
--
Prasanna.,