You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tuscany.apache.org by James F Marino <ji...@gmail.com> on 2006/04/25 13:33:57 UTC
Java container JIRA issue reporting
I've noticed several issues have been logged against the Java
container where entire Maven projects or samples are used to
demonstrate a problem. Sometimes this may be unavoidable, but I would
like to propose that we get into the habit of submitting failing
testcases as a default instead. This is common practice in many
projects, and my reasons for suggesting this are:
- it helps diagnose the problem
- it provides verification when the issue is resolved
- it builds up our test suite which is woefully inadequate
- it is often less work than creating entire projects
Obviously, this can't be a literal policy but I believe it should be
"default" behavior for those directly involved in the project. For
non-commiters, we could encourage this behavior by noting that logged
issues with testcases generally get attention quicker.
Also, I would like to propose that while we may tolerate an issue
logged with a project which requires a particular IDE such as IntelliJ
or Eclipse, or requires "main" to run from those outside the project,
commiters should never do this.
Comments?
Re: Java container JIRA issue reporting
Posted by Mike Edwards <mi...@gmail.com>.
I think that the principle of building a test case that demonstrates the
problem is a good practice. If there is a problem, them it is almost
always the case that the current test suite does not find the problem.
To ensure a fix, then *someone* has to build the test, which needs to be
added to the test suite. While I dont want to stop people filing
problems, it is fair to share around the creation of testcases.
Whether a test is a unit test or an integration test depends on the
problem and how it was found. You can never have too many tests...
Yours, Mike.
Jim Marino wrote:
>
> On May 3, 2006, at 11:59 AM, Jean-Sebastien Delfino wrote:
>
>
>> Jim Marino wrote:
>>
>>
>>> I tried to distinguish between commiters and non-commiters. For
>>> *commiters* we should get into the habit of using unit tests to
>>> demonstrate problems since I believe we want to encourage this among
>>> non-comitters as well (note several Apache and many open source
>>> projects do this, including many of the Geronimo subprojects). I'm
>>> not saying showing how to reproduce the problem using a sample is
>>> somehow "inappropriate", I'm just asking that commiters adopt a
>>> default of using unit tests since it helps the project. Of course,
>>> there are times when this is not easy to do, but when it is (which
>>> was the case in the examples I referred to) it seems like a logical
>>> request to make. Also, please note I did not mean to imply that the
>>> bug fixer is free of the responsibility to write testcases
>>> associated with the fix (actually I would argue that if they must
>>> when checking in the fix).
>>>
>>> Do you have an issue with this approach? If so, could you please
>>> outline what specific concerns you have?
>>>
>>> Jim
>>>
>>>
>>> On May 3, 2006, at 8:43 AM, ant elder wrote:
>>>
>>>
>>>
>>>> I'm not sure I completely agree with this, if you make it hard to
>>>> report
>>>> problems then people don't report them. A bug reporter should have to
>>>> describe/provide just enough of a way to easily recreate a problem,
>>>> its down
>>>> to the bug fixer to write whatever regression and unit tests are
>>>> required by
>>>> the fix. If you can describe how to demonstrate a bug just by
>>>> running one of
>>>> our samples that's fine in my book.
>>>>
>>>> ...ant
>>>>
>>>> On 4/25/06, James F Marino <ji...@gmail.com> wrote:
>>>>
>>>>
>>>>
>>>>>
>>>>> I've noticed several issues have been logged against the Java
>>>>> container where entire Maven projects or samples are used to
>>>>> demonstrate a problem. Sometimes this may be unavoidable, but I
>>>>> would
>>>>> like to propose that we get into the habit of submitting failing
>>>>> testcases as a default instead. This is common practice in many
>>>>> projects, and my reasons for suggesting this are:
>>>>>
>>>>> - it helps diagnose the problem
>>>>> - it provides verification when the issue is resolved
>>>>> - it builds up our test suite which is woefully inadequate
>>>>> - it is often less work than creating entire projects
>>>>>
>>>>> Obviously, this can't be a literal policy but I believe it should be
>>>>> "default" behavior for those directly involved in the project. For
>>>>> non-commiters, we could encourage this behavior by noting that logged
>>>>> issues with testcases generally get attention quicker.
>>>>>
>>>>> Also, I would like to propose that while we may tolerate an issue
>>>>> logged with a project which requires a particular IDE such as
>>>>> IntelliJ
>>>>> or Eclipse, or requires "main" to run from those outside the project,
>>>>> commiters should never do this.
>>>>>
>>>>> Comments?
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>
>> Jim, could you please confirm what you mean by "test case" in this
>> context? Are you asking for a function test case? an integration test
>> case? or are you really talking about a unit test case?
>>
>>
> This may depend on the issue but it is likely to be a unit or
> integration test case. If we do not catch something there, it is
> generally a problem with test coverage, although there are exceptions
> to the rule (no pun intended). For example, if we can't use a unit or
> integration test to check something, it's often (but not always)
> reflects a design problem. An exception to this rule that comes to mind
> are platform-related issues.
>
>
>> Also, we've been looking at adding to our web site some guideline
>> documents for contributors (exception handling, logging, coding
>> guidelines, submitting patches etc), can you point us to any
>> guideline document from the mentioned Apache projects describing this
>> approach?
>>
>>
> Off the top of my head:
>
> ActiveMQ: http://www.activemq.org/Contributing
>
> Also, Spring (which obviously is not Apache).
>
> I'm sure I can dig up others if I looked but I think those should
> suffice. FYI on exception handling, Haleh was kind enough to revise
> the original project exception handling document - I have a todo to
> take a look at it.
>
> Jim
>
>
>> --
>> Jean-Sebastien
>>
>>
>>
>
>
>
Re: Java container JIRA issue reporting
Posted by Jim Marino <jm...@myromatours.com>.
On May 3, 2006, at 11:59 AM, Jean-Sebastien Delfino wrote:
> Jim Marino wrote:
>
>
>> I tried to distinguish between commiters and non-commiters. For
>> *commiters* we should get into the habit of using unit tests to
>> demonstrate problems since I believe we want to encourage this
>> among non-comitters as well (note several Apache and many open
>> source projects do this, including many of the Geronimo
>> subprojects). I'm not saying showing how to reproduce the problem
>> using a sample is somehow "inappropriate", I'm just asking that
>> commiters adopt a default of using unit tests since it helps the
>> project. Of course, there are times when this is not easy to do,
>> but when it is (which was the case in the examples I referred to)
>> it seems like a logical request to make. Also, please note I did
>> not mean to imply that the bug fixer is free of the responsibility
>> to write testcases associated with the fix (actually I would argue
>> that if they must when checking in the fix).
>>
>> Do you have an issue with this approach? If so, could you please
>> outline what specific concerns you have?
>>
>> Jim
>>
>>
>> On May 3, 2006, at 8:43 AM, ant elder wrote:
>>
>>
>>
>>> I'm not sure I completely agree with this, if you make it hard to
>>> report
>>> problems then people don't report them. A bug reporter should
>>> have to
>>> describe/provide just enough of a way to easily recreate a
>>> problem, its down
>>> to the bug fixer to write whatever regression and unit tests are
>>> required by
>>> the fix. If you can describe how to demonstrate a bug just by
>>> running one of
>>> our samples that's fine in my book.
>>>
>>> ...ant
>>>
>>> On 4/25/06, James F Marino <ji...@gmail.com> wrote:
>>>
>>>
>>>
>>>>
>>>> I've noticed several issues have been logged against the Java
>>>> container where entire Maven projects or samples are used to
>>>> demonstrate a problem. Sometimes this may be unavoidable, but I
>>>> would
>>>> like to propose that we get into the habit of submitting failing
>>>> testcases as a default instead. This is common practice in many
>>>> projects, and my reasons for suggesting this are:
>>>>
>>>> - it helps diagnose the problem
>>>> - it provides verification when the issue is resolved
>>>> - it builds up our test suite which is woefully inadequate
>>>> - it is often less work than creating entire projects
>>>>
>>>> Obviously, this can't be a literal policy but I believe it
>>>> should be
>>>> "default" behavior for those directly involved in the project. For
>>>> non-commiters, we could encourage this behavior by noting that
>>>> logged
>>>> issues with testcases generally get attention quicker.
>>>>
>>>> Also, I would like to propose that while we may tolerate an issue
>>>> logged with a project which requires a particular IDE such as
>>>> IntelliJ
>>>> or Eclipse, or requires "main" to run from those outside the
>>>> project,
>>>> commiters should never do this.
>>>>
>>>> Comments?
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>>
>
> Jim, could you please confirm what you mean by "test case" in this
> context? Are you asking for a function test case? an integration
> test case? or are you really talking about a unit test case?
>
>
This may depend on the issue but it is likely to be a unit or
integration test case. If we do not catch something there, it is
generally a problem with test coverage, although there are exceptions
to the rule (no pun intended). For example, if we can't use a unit or
integration test to check something, it's often (but not always)
reflects a design problem. An exception to this rule that comes to
mind are platform-related issues.
> Also, we've been looking at adding to our web site some guideline
> documents for contributors (exception handling, logging, coding
> guidelines, submitting patches etc), can you point us to any
> guideline document from the mentioned Apache projects describing
> this approach?
>
>
Off the top of my head:
ActiveMQ: http://www.activemq.org/Contributing
Also, Spring (which obviously is not Apache).
I'm sure I can dig up others if I looked but I think those should
suffice. FYI on exception handling, Haleh was kind enough to revise
the original project exception handling document - I have a todo to
take a look at it.
Jim
> --
> Jean-Sebastien
>
>
>
Re: Java container JIRA issue reporting
Posted by Jean-Sebastien Delfino <js...@apache.org>.
Jim Marino wrote:
> I tried to distinguish between commiters and non-commiters. For
> *commiters* we should get into the habit of using unit tests to
> demonstrate problems since I believe we want to encourage this among
> non-comitters as well (note several Apache and many open source
> projects do this, including many of the Geronimo subprojects). I'm not
> saying showing how to reproduce the problem using a sample is somehow
> "inappropriate", I'm just asking that commiters adopt a default of
> using unit tests since it helps the project. Of course, there are
> times when this is not easy to do, but when it is (which was the case
> in the examples I referred to) it seems like a logical request to
> make. Also, please note I did not mean to imply that the bug fixer is
> free of the responsibility to write testcases associated with the fix
> (actually I would argue that if they must when checking in the fix).
>
> Do you have an issue with this approach? If so, could you please
> outline what specific concerns you have?
>
> Jim
>
>
> On May 3, 2006, at 8:43 AM, ant elder wrote:
>
>> I'm not sure I completely agree with this, if you make it hard to report
>> problems then people don't report them. A bug reporter should have to
>> describe/provide just enough of a way to easily recreate a problem,
>> its down
>> to the bug fixer to write whatever regression and unit tests are
>> required by
>> the fix. If you can describe how to demonstrate a bug just by running
>> one of
>> our samples that's fine in my book.
>>
>> ...ant
>>
>> On 4/25/06, James F Marino <ji...@gmail.com> wrote:
>>
>>>
>>> I've noticed several issues have been logged against the Java
>>> container where entire Maven projects or samples are used to
>>> demonstrate a problem. Sometimes this may be unavoidable, but I would
>>> like to propose that we get into the habit of submitting failing
>>> testcases as a default instead. This is common practice in many
>>> projects, and my reasons for suggesting this are:
>>>
>>> - it helps diagnose the problem
>>> - it provides verification when the issue is resolved
>>> - it builds up our test suite which is woefully inadequate
>>> - it is often less work than creating entire projects
>>>
>>> Obviously, this can't be a literal policy but I believe it should be
>>> "default" behavior for those directly involved in the project. For
>>> non-commiters, we could encourage this behavior by noting that logged
>>> issues with testcases generally get attention quicker.
>>>
>>> Also, I would like to propose that while we may tolerate an issue
>>> logged with a project which requires a particular IDE such as IntelliJ
>>> or Eclipse, or requires "main" to run from those outside the project,
>>> commiters should never do this.
>>>
>>> Comments?
>>>
>>
>
>
Jim, could you please confirm what you mean by "test case" in this
context? Are you asking for a function test case? an integration test
case? or are you really talking about a unit test case?
Also, we've been looking at adding to our web site some guideline
documents for contributors (exception handling, logging, coding
guidelines, submitting patches etc), can you point us to any guideline
document from the mentioned Apache projects describing this approach?
--
Jean-Sebastien
Re: Java container JIRA issue reporting
Posted by Jim Marino <jm...@myromatours.com>.
I tried to distinguish between commiters and non-commiters. For
*commiters* we should get into the habit of using unit tests to
demonstrate problems since I believe we want to encourage this among
non-comitters as well (note several Apache and many open source
projects do this, including many of the Geronimo subprojects). I'm
not saying showing how to reproduce the problem using a sample is
somehow "inappropriate", I'm just asking that commiters adopt a
default of using unit tests since it helps the project. Of course,
there are times when this is not easy to do, but when it is (which
was the case in the examples I referred to) it seems like a logical
request to make. Also, please note I did not mean to imply that the
bug fixer is free of the responsibility to write testcases associated
with the fix (actually I would argue that if they must when checking
in the fix).
Do you have an issue with this approach? If so, could you please
outline what specific concerns you have?
Jim
On May 3, 2006, at 8:43 AM, ant elder wrote:
> I'm not sure I completely agree with this, if you make it hard to
> report
> problems then people don't report them. A bug reporter should have to
> describe/provide just enough of a way to easily recreate a problem,
> its down
> to the bug fixer to write whatever regression and unit tests are
> required by
> the fix. If you can describe how to demonstrate a bug just by
> running one of
> our samples that's fine in my book.
>
> ...ant
>
> On 4/25/06, James F Marino <ji...@gmail.com> wrote:
>
>>
>> I've noticed several issues have been logged against the Java
>> container where entire Maven projects or samples are used to
>> demonstrate a problem. Sometimes this may be unavoidable, but I
>> would
>> like to propose that we get into the habit of submitting failing
>> testcases as a default instead. This is common practice in many
>> projects, and my reasons for suggesting this are:
>>
>> - it helps diagnose the problem
>> - it provides verification when the issue is resolved
>> - it builds up our test suite which is woefully inadequate
>> - it is often less work than creating entire projects
>>
>> Obviously, this can't be a literal policy but I believe it should be
>> "default" behavior for those directly involved in the project. For
>> non-commiters, we could encourage this behavior by noting that logged
>> issues with testcases generally get attention quicker.
>>
>> Also, I would like to propose that while we may tolerate an issue
>> logged with a project which requires a particular IDE such as
>> IntelliJ
>> or Eclipse, or requires "main" to run from those outside the project,
>> commiters should never do this.
>>
>> Comments?
>>
>
Re: Java container JIRA issue reporting
Posted by ant elder <an...@gmail.com>.
I'm not sure I completely agree with this, if you make it hard to report
problems then people don't report them. A bug reporter should have to
describe/provide just enough of a way to easily recreate a problem, its down
to the bug fixer to write whatever regression and unit tests are required by
the fix. If you can describe how to demonstrate a bug just by running one of
our samples that's fine in my book.
...ant
On 4/25/06, James F Marino <ji...@gmail.com> wrote:
>
> I've noticed several issues have been logged against the Java
> container where entire Maven projects or samples are used to
> demonstrate a problem. Sometimes this may be unavoidable, but I would
> like to propose that we get into the habit of submitting failing
> testcases as a default instead. This is common practice in many
> projects, and my reasons for suggesting this are:
>
> - it helps diagnose the problem
> - it provides verification when the issue is resolved
> - it builds up our test suite which is woefully inadequate
> - it is often less work than creating entire projects
>
> Obviously, this can't be a literal policy but I believe it should be
> "default" behavior for those directly involved in the project. For
> non-commiters, we could encourage this behavior by noting that logged
> issues with testcases generally get attention quicker.
>
> Also, I would like to propose that while we may tolerate an issue
> logged with a project which requires a particular IDE such as IntelliJ
> or Eclipse, or requires "main" to run from those outside the project,
> commiters should never do this.
>
> Comments?
>
Re: Java container JIRA issue reporting
Posted by Mike Edwards <mi...@gmail.com>.
+1 from me, Jim
James F Marino wrote:
> I've noticed several issues have been logged against the Java
> container where entire Maven projects or samples are used to
> demonstrate a problem. Sometimes this may be unavoidable, but I would
> like to propose that we get into the habit of submitting failing
> testcases as a default instead. This is common practice in many
> projects, and my reasons for suggesting this are:
>
> - it helps diagnose the problem
> - it provides verification when the issue is resolved
> - it builds up our test suite which is woefully inadequate
> - it is often less work than creating entire projects
>
> Obviously, this can't be a literal policy but I believe it should be
> "default" behavior for those directly involved in the project. For
> non-commiters, we could encourage this behavior by noting that logged
> issues with testcases generally get attention quicker.
>
> Also, I would like to propose that while we may tolerate an issue
> logged with a project which requires a particular IDE such as IntelliJ
> or Eclipse, or requires "main" to run from those outside the project,
> commiters should never do this.
>
> Comments?
>