You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Brett Porter <br...@apache.org> on 2006/03/07 21:10:53 UTC
making docs and tests suck less
This is not too long, and important. Please read.
I've just spent some time discussing this on users@, and felt it was
time to bring it here to look at the practical steps going forward.
Basically, I think we've all known for a long time that both need work.
There was a big push around 2.0 but it fizzled out afterwards.
I don't want to focus on what we are missing now - we can address that
with what is already in place. I also want to put aside the
documentation and testing efforts that are already under way as they've
been taken into consideration. What I want to focus on is going forward
- new work. I think we need to change the culture. I realise this is
hard given the timing because there isn't a lot of new work going on -
because we're all doing bug fixes, docs and testing! - but hopefully
because of this it will be apparent why it is important to do them as we go.
I've been trying to push for this for a long time, but haven't lived up
to it myself which is one reason why it fails. I hate hypocrisy so can't
really call other people on it (though I probably do anyway), and it
requires everyone to buy in.
So, I've committed r383963 which emphasises these requirements on patch
contributions:
* Whether it works and does what is intended.
* Whether it fits the spirit of the project.
* Whether it contains tests. It is expected that any patches relating to
functionality will be accompanied by unit tests and/or integration
tests. It is strongly desired (and will be requested) for bug fixes too,
but will not be the basis for not applying it. At a bare minimum, the
change should not decrease the amount of automated test coverage.
* Whether it contains documentation. All new functionality needs to be
documented for users, even if it is very rough for someone to expand on
later. While rough is acceptable, incomplete is not.
In the following, I'll discuss a technical veto. That's a -1 that means
that the issue needs to be resolved before a release can occur. It
doesn't mean someone rolls it back immediately (though a release manager
may if it blocks a branch being released). It also doesn't mean anything
negative about the committer in question, and it's important we keep it
that way so that people aren't afraid to contribute or to call people on
theese things.
So my questions are, can we institute the following:
* new functionality without automated test coverage can and should be
technically vetoed. We can measure this with code coverage tools, and
the measure will be that the % does not decline. It will actually break
the build, being an automatic technical veto :)
* new functionality without documentation can and should be technically
vetoed. We can't really measure this with tools, so need to be vigilant
on it ourselves. We also need to be understanding that docs may follow
the code once it is finished, so we should use JIRA to track it (keep
the original ticket open until documented, or create a new, blocking issue).
* new code without javadoc/code documentation can be technically vetoed.
I'm less tied to this one, though for public facing APIs I think Javadoc
is imperative. It may be that we can use Checkstyle to enforce this to a
degree.
* code that doesn't meet current metrics can and should be technically
vetoed. Again, we should set these up to fail the build, using
Checkstyle, PMD and Clirr.
Of course, I'll incorporate this into the dev process after some discussion.
Thoughts?
- Brett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: making docs and tests suck less
Posted by Stephane Nicoll <st...@gmail.com>.
+1
On 3/7/06, Carlos Sanchez <ca...@apache.org> wrote:
> I agree. Just one comment about javadocs, they are not important just
> for public apis, they should be there for almost all methods so we
> won't have to go into the details of the code when bug fixing or
> writing new code.
>
> On 3/7/06, Brett Porter <br...@apache.org> wrote:
> > This is not too long, and important. Please read.
> >
> > I've just spent some time discussing this on users@, and felt it was
> > time to bring it here to look at the practical steps going forward.
> >
> > Basically, I think we've all known for a long time that both need work.
> > There was a big push around 2.0 but it fizzled out afterwards.
> >
> > I don't want to focus on what we are missing now - we can address that
> > with what is already in place. I also want to put aside the
> > documentation and testing efforts that are already under way as they've
> > been taken into consideration. What I want to focus on is going forward
> > - new work. I think we need to change the culture. I realise this is
> > hard given the timing because there isn't a lot of new work going on -
> > because we're all doing bug fixes, docs and testing! - but hopefully
> > because of this it will be apparent why it is important to do them as we go.
> >
> > I've been trying to push for this for a long time, but haven't lived up
> > to it myself which is one reason why it fails. I hate hypocrisy so can't
> > really call other people on it (though I probably do anyway), and it
> > requires everyone to buy in.
> >
> > So, I've committed r383963 which emphasises these requirements on patch
> > contributions:
> >
> > * Whether it works and does what is intended.
> > * Whether it fits the spirit of the project.
> > * Whether it contains tests. It is expected that any patches relating to
> > functionality will be accompanied by unit tests and/or integration
> > tests. It is strongly desired (and will be requested) for bug fixes too,
> > but will not be the basis for not applying it. At a bare minimum, the
> > change should not decrease the amount of automated test coverage.
> > * Whether it contains documentation. All new functionality needs to be
> > documented for users, even if it is very rough for someone to expand on
> > later. While rough is acceptable, incomplete is not.
> >
> > In the following, I'll discuss a technical veto. That's a -1 that means
> > that the issue needs to be resolved before a release can occur. It
> > doesn't mean someone rolls it back immediately (though a release manager
> > may if it blocks a branch being released). It also doesn't mean anything
> > negative about the committer in question, and it's important we keep it
> > that way so that people aren't afraid to contribute or to call people on
> > theese things.
> >
> > So my questions are, can we institute the following:
> >
> > * new functionality without automated test coverage can and should be
> > technically vetoed. We can measure this with code coverage tools, and
> > the measure will be that the % does not decline. It will actually break
> > the build, being an automatic technical veto :)
> >
> > * new functionality without documentation can and should be technically
> > vetoed. We can't really measure this with tools, so need to be vigilant
> > on it ourselves. We also need to be understanding that docs may follow
> > the code once it is finished, so we should use JIRA to track it (keep
> > the original ticket open until documented, or create a new, blocking issue).
> >
> > * new code without javadoc/code documentation can be technically vetoed.
> > I'm less tied to this one, though for public facing APIs I think Javadoc
> > is imperative. It may be that we can use Checkstyle to enforce this to a
> > degree.
> >
> > * code that doesn't meet current metrics can and should be technically
> > vetoed. Again, we should set these up to fail the build, using
> > Checkstyle, PMD and Clirr.
> >
> > Of course, I'll incorporate this into the dev process after some discussion.
> >
> > Thoughts?
> >
> > - Brett
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
>
> --
> I could give you my word as a Spaniard.
> No good. I've known too many Spaniards.
> -- The Princess Bride
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
.::You're welcome ::.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: making docs and tests suck less
Posted by Carlos Sanchez <ca...@apache.org>.
I agree. Just one comment about javadocs, they are not important just
for public apis, they should be there for almost all methods so we
won't have to go into the details of the code when bug fixing or
writing new code.
On 3/7/06, Brett Porter <br...@apache.org> wrote:
> This is not too long, and important. Please read.
>
> I've just spent some time discussing this on users@, and felt it was
> time to bring it here to look at the practical steps going forward.
>
> Basically, I think we've all known for a long time that both need work.
> There was a big push around 2.0 but it fizzled out afterwards.
>
> I don't want to focus on what we are missing now - we can address that
> with what is already in place. I also want to put aside the
> documentation and testing efforts that are already under way as they've
> been taken into consideration. What I want to focus on is going forward
> - new work. I think we need to change the culture. I realise this is
> hard given the timing because there isn't a lot of new work going on -
> because we're all doing bug fixes, docs and testing! - but hopefully
> because of this it will be apparent why it is important to do them as we go.
>
> I've been trying to push for this for a long time, but haven't lived up
> to it myself which is one reason why it fails. I hate hypocrisy so can't
> really call other people on it (though I probably do anyway), and it
> requires everyone to buy in.
>
> So, I've committed r383963 which emphasises these requirements on patch
> contributions:
>
> * Whether it works and does what is intended.
> * Whether it fits the spirit of the project.
> * Whether it contains tests. It is expected that any patches relating to
> functionality will be accompanied by unit tests and/or integration
> tests. It is strongly desired (and will be requested) for bug fixes too,
> but will not be the basis for not applying it. At a bare minimum, the
> change should not decrease the amount of automated test coverage.
> * Whether it contains documentation. All new functionality needs to be
> documented for users, even if it is very rough for someone to expand on
> later. While rough is acceptable, incomplete is not.
>
> In the following, I'll discuss a technical veto. That's a -1 that means
> that the issue needs to be resolved before a release can occur. It
> doesn't mean someone rolls it back immediately (though a release manager
> may if it blocks a branch being released). It also doesn't mean anything
> negative about the committer in question, and it's important we keep it
> that way so that people aren't afraid to contribute or to call people on
> theese things.
>
> So my questions are, can we institute the following:
>
> * new functionality without automated test coverage can and should be
> technically vetoed. We can measure this with code coverage tools, and
> the measure will be that the % does not decline. It will actually break
> the build, being an automatic technical veto :)
>
> * new functionality without documentation can and should be technically
> vetoed. We can't really measure this with tools, so need to be vigilant
> on it ourselves. We also need to be understanding that docs may follow
> the code once it is finished, so we should use JIRA to track it (keep
> the original ticket open until documented, or create a new, blocking issue).
>
> * new code without javadoc/code documentation can be technically vetoed.
> I'm less tied to this one, though for public facing APIs I think Javadoc
> is imperative. It may be that we can use Checkstyle to enforce this to a
> degree.
>
> * code that doesn't meet current metrics can and should be technically
> vetoed. Again, we should set these up to fail the build, using
> Checkstyle, PMD and Clirr.
>
> Of course, I'll incorporate this into the dev process after some discussion.
>
> Thoughts?
>
> - Brett
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
I could give you my word as a Spaniard.
No good. I've known too many Spaniards.
-- The Princess Bride
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: making docs and tests suck less
Posted by Raphaël Piéroni <ra...@gmail.com>.
Brett Porter a écrit :
> This is not too long, and important. Please read.
>
> I've just spent some time discussing this on users@, and felt it was
> time to bring it here to look at the practical steps going forward.
>
> Basically, I think we've all known for a long time that both need work.
> There was a big push around 2.0 but it fizzled out afterwards.
>
> I don't want to focus on what we are missing now - we can address that
> with what is already in place. I also want to put aside the
> documentation and testing efforts that are already under way as they've
> been taken into consideration. What I want to focus on is going forward
> - new work. I think we need to change the culture. I realise this is
> hard given the timing because there isn't a lot of new work going on -
> because we're all doing bug fixes, docs and testing! - but hopefully
> because of this it will be apparent why it is important to do them as we go.
>
> I've been trying to push for this for a long time, but haven't lived up
> to it myself which is one reason why it fails. I hate hypocrisy so can't
> really call other people on it (though I probably do anyway), and it
> requires everyone to buy in.
>
> So, I've committed r383963 which emphasises these requirements on patch
> contributions:
>
> * Whether it works and does what is intended.
> * Whether it fits the spirit of the project.
> * Whether it contains tests. It is expected that any patches relating to
> functionality will be accompanied by unit tests and/or integration
> tests. It is strongly desired (and will be requested) for bug fixes too,
> but will not be the basis for not applying it. At a bare minimum, the
> change should not decrease the amount of automated test coverage.
> * Whether it contains documentation. All new functionality needs to be
> documented for users, even if it is very rough for someone to expand on
> later. While rough is acceptable, incomplete is not.
>
> In the following, I'll discuss a technical veto. That's a -1 that means
> that the issue needs to be resolved before a release can occur. It
> doesn't mean someone rolls it back immediately (though a release manager
> may if it blocks a branch being released). It also doesn't mean anything
> negative about the committer in question, and it's important we keep it
> that way so that people aren't afraid to contribute or to call people on
> theese things.
>
> So my questions are, can we institute the following:
>
> * new functionality without automated test coverage can and should be
> technically vetoed. We can measure this with code coverage tools, and
> the measure will be that the % does not decline. It will actually break
> the build, being an automatic technical veto :)
>
> * new functionality without documentation can and should be technically
> vetoed. We can't really measure this with tools, so need to be vigilant
> on it ourselves. We also need to be understanding that docs may follow
> the code once it is finished, so we should use JIRA to track it (keep
> the original ticket open until documented, or create a new, blocking issue).
>
> * new code without javadoc/code documentation can be technically vetoed.
> I'm less tied to this one, though for public facing APIs I think Javadoc
> is imperative. It may be that we can use Checkstyle to enforce this to a
> degree.
>
> * code that doesn't meet current metrics can and should be technically
> vetoed. Again, we should set these up to fail the build, using
> Checkstyle, PMD and Clirr.
>
> Of course, I'll incorporate this into the dev process after some discussion.
>
> Thoughts?
>
what about a reminder mail (automatic) every week which
gives the url of the list of documentation and javadoc issues
for the existing code and show the number of still undocumented
issues. Sawing advancement is motivating.
Raphaël
> - Brett
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: making docs and tests suck less
Posted by Lukas Theussl <lt...@apache.org>.
+1 for everything and I hope this will be enforced (by all of us)
-Lukas
Brett Porter wrote:
> This is not too long, and important. Please read.
>
> I've just spent some time discussing this on users@, and felt it was
> time to bring it here to look at the practical steps going forward.
>
> Basically, I think we've all known for a long time that both need work.
> There was a big push around 2.0 but it fizzled out afterwards.
>
> I don't want to focus on what we are missing now - we can address that
> with what is already in place. I also want to put aside the
> documentation and testing efforts that are already under way as they've
> been taken into consideration. What I want to focus on is going forward
> - new work. I think we need to change the culture. I realise this is
> hard given the timing because there isn't a lot of new work going on -
> because we're all doing bug fixes, docs and testing! - but hopefully
> because of this it will be apparent why it is important to do them as we go.
>
> I've been trying to push for this for a long time, but haven't lived up
> to it myself which is one reason why it fails. I hate hypocrisy so can't
> really call other people on it (though I probably do anyway), and it
> requires everyone to buy in.
>
> So, I've committed r383963 which emphasises these requirements on patch
> contributions:
>
> * Whether it works and does what is intended.
> * Whether it fits the spirit of the project.
> * Whether it contains tests. It is expected that any patches relating to
> functionality will be accompanied by unit tests and/or integration
> tests. It is strongly desired (and will be requested) for bug fixes too,
> but will not be the basis for not applying it. At a bare minimum, the
> change should not decrease the amount of automated test coverage.
> * Whether it contains documentation. All new functionality needs to be
> documented for users, even if it is very rough for someone to expand on
> later. While rough is acceptable, incomplete is not.
>
> In the following, I'll discuss a technical veto. That's a -1 that means
> that the issue needs to be resolved before a release can occur. It
> doesn't mean someone rolls it back immediately (though a release manager
> may if it blocks a branch being released). It also doesn't mean anything
> negative about the committer in question, and it's important we keep it
> that way so that people aren't afraid to contribute or to call people on
> theese things.
>
> So my questions are, can we institute the following:
>
> * new functionality without automated test coverage can and should be
> technically vetoed. We can measure this with code coverage tools, and
> the measure will be that the % does not decline. It will actually break
> the build, being an automatic technical veto :)
>
> * new functionality without documentation can and should be technically
> vetoed. We can't really measure this with tools, so need to be vigilant
> on it ourselves. We also need to be understanding that docs may follow
> the code once it is finished, so we should use JIRA to track it (keep
> the original ticket open until documented, or create a new, blocking issue).
>
> * new code without javadoc/code documentation can be technically vetoed.
> I'm less tied to this one, though for public facing APIs I think Javadoc
> is imperative. It may be that we can use Checkstyle to enforce this to a
> degree.
>
> * code that doesn't meet current metrics can and should be technically
> vetoed. Again, we should set these up to fail the build, using
> Checkstyle, PMD and Clirr.
>
> Of course, I'll incorporate this into the dev process after some discussion.
>
> Thoughts?
>
> - Brett
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: making docs and tests suck less
Posted by Rinku <ra...@gmail.com>.
Hi,
My response inline...
Tim O'Brien wrote:
> Developers don't write great documentation. Don't raise the bar so high
> that people are discouraged from submitting doco-less patches. Just create
> a structure to address documentation deficiency.
>
Sorry, but IMHO that's too much of a generalization on developer's
'documentation' skills and I don't agree... and definately not for for a
project like Maven. Ideas jotted in Brett's email are worth implementing
and I commend him and the team on putting them forth for a discussion
here. I am sure Maven community will notice if the bar is set too high!
> 1. Create a documentation posse - identify people who will focus solely on
> doco, reduce the barriers for them to commit. Or, move documentation to an
> external project with lower barriers to entry.
>
-1 for external project, documentation is intrinsic to the project, not
sure what's the benefit of having another external project.
> 2. Consider a separate list - people removed from the development process
> produce better documentation.
>
-1, user and dev lists are suited enough for doc related
questions/discussions. Enough mails are floating on these lists to
gather ideas and content for documentation. I am not aware of an
instance where this generalization above about producing better
documentation by people not in development applies.
> 3. Add "Waiting for Documentation" as a stage in a customized JIRA
> workflow. Instead of creating separate issues for "Document what I just did
> in the Site Plugin", give developers the opportunity to just mark something
> as implemented but undocumented (otherwise know n as useless and invisible)
>
[snip]
Cheers,
Rahul
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: making docs and tests suck less
Posted by Brett Porter <br...@apache.org>.
Thanks for taking an interest. I've got lots of replies to catch up on.
Tim O'Brien wrote:
> Developers don't write great documentation. Don't raise the bar so high
> that people are discouraged from submitting doco-less patches. Just create
> a structure to address documentation deficiency.
I wasn't aiming to discourage submitting patches, that's why I said
"even if it is very rough for someone to expand on later. While rough is
acceptable, incomplete is not."
>
> 1. Create a documentation posse - identify people who will focus solely on
> doco, reduce the barriers for them to commit. Or, move documentation to an
> external project with lower barriers to entry.
You can't just do
new DocumentationPosse().startInBackground();
People need to be interested. In all honestly, I've seen maybe 2 or 3
specifically interested and it in all the time I've been here. I think
we all need to lift our own games, to make tying it together achievable.
We should put the call out, though. Are you volunteering to lead the
posse? :)
As for lowering barriers - what exact barriers do you think there are,
and how can we lower them. Creating a separate project is unnecessary
(the doc should go where it should go, access can be granted
accordingly). We can't just hand out commit access like candy - people
need to show they know what they are doing and are going to stick around
just like anyone else.
The only barrier I see is to apply patches immediately as they are at
greater risk of getting stale. I've been doing this recently.
>
> 2. Consider a separate list - people removed from the development process
> produce better documentation.
I absolutely disagree. The developers will stop participating in
documentation altogether. The documenters need to know how development
affects them. The two need to be closely related.
Do you have any factual basis or data for the statement to contradict this?
>
> 3. Add "Waiting for Documentation" as a stage in a customized JIRA
> workflow. Instead of creating separate issues for "Document what I just did
> in the Site Plugin", give developers the opportunity to just mark something
> as implemented but undocumented (otherwise know n as useless and invisible)
I don't think a workflow step is very helpful, myself. I'd say let
people go at it and decide on the best way to mark them through
experience. So far, I think the use of components suffice. I'd rather
add a custom field than a workflow state.
- Brett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: making docs and tests suck less
Posted by Stephen Duncan <st...@gmail.com>.
On 3/7/06, Tim O'Brien <to...@discursive.com> wrote:
> Developers don't write great documentation. Don't raise the bar so high
> that people are discouraged from submitting doco-less patches. Just create
> a structure to address documentation deficiency.
+1
Patches should be able to be SUBMITTED without documenation, but not
committed. Committers can decide if a patch with just a code fix is
valuable enough they'll spend the time to document or even add testing
for it. Anything that discourages non-committers from contributing is
a bad idea.
> 1. Create a documentation posse - identify people who will focus solely on
> doco, reduce the barriers for them to commit. Or, move documentation to an
> external project with lower barriers to entry.
>
> 2. Consider a separate list - people removed from the development process
> produce better documentation.
>
> 3. Add "Waiting for Documentation" as a stage in a customized JIRA
> workflow. Instead of creating separate issues for "Document what I just did
> in the Site Plugin", give developers the opportunity to just mark something
> as implemented but undocumented (otherwise know n as useless and invisible)
+1
Add something similar to indicate that it's waiting for test coverage also.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: making docs and tests suck less
Posted by Tim O'Brien <to...@discursive.com>.
Developers don't write great documentation. Don't raise the bar so high
that people are discouraged from submitting doco-less patches. Just create
a structure to address documentation deficiency.
1. Create a documentation posse - identify people who will focus solely on
doco, reduce the barriers for them to commit. Or, move documentation to an
external project with lower barriers to entry.
2. Consider a separate list - people removed from the development process
produce better documentation.
3. Add "Waiting for Documentation" as a stage in a customized JIRA
workflow. Instead of creating separate issues for "Document what I just did
in the Site Plugin", give developers the opportunity to just mark something
as implemented but undocumented (otherwise know n as useless and invisible)
On 3/7/06, Brett Porter <br...@apache.org> wrote:
>
> This is not too long, and important. Please read.
>
> I've just spent some time discussing this on users@, and felt it was
> time to bring it here to look at the practical steps going forward.
>
> Basically, I think we've all known for a long time that both need work.
> There was a big push around 2.0 but it fizzled out afterwards.
>
> I don't want to focus on what we are missing now - we can address that
> with what is already in place. I also want to put aside the
> documentation and testing efforts that are already under way as they've
> been taken into consideration. What I want to focus on is going forward
> - new work. I think we need to change the culture. I realise this is
> hard given the timing because there isn't a lot of new work going on -
> because we're all doing bug fixes, docs and testing! - but hopefully
> because of this it will be apparent why it is important to do them as we
> go.
>
> I've been trying to push for this for a long time, but haven't lived up
> to it myself which is one reason why it fails. I hate hypocrisy so can't
> really call other people on it (though I probably do anyway), and it
> requires everyone to buy in.
>
> So, I've committed r383963 which emphasises these requirements on patch
> contributions:
>
> * Whether it works and does what is intended.
> * Whether it fits the spirit of the project.
> * Whether it contains tests. It is expected that any patches relating to
> functionality will be accompanied by unit tests and/or integration
> tests. It is strongly desired (and will be requested) for bug fixes too,
> but will not be the basis for not applying it. At a bare minimum, the
> change should not decrease the amount of automated test coverage.
> * Whether it contains documentation. All new functionality needs to be
> documented for users, even if it is very rough for someone to expand on
> later. While rough is acceptable, incomplete is not.
>
> In the following, I'll discuss a technical veto. That's a -1 that means
> that the issue needs to be resolved before a release can occur. It
> doesn't mean someone rolls it back immediately (though a release manager
> may if it blocks a branch being released). It also doesn't mean anything
> negative about the committer in question, and it's important we keep it
> that way so that people aren't afraid to contribute or to call people on
> theese things.
>
> So my questions are, can we institute the following:
>
> * new functionality without automated test coverage can and should be
> technically vetoed. We can measure this with code coverage tools, and
> the measure will be that the % does not decline. It will actually break
> the build, being an automatic technical veto :)
>
> * new functionality without documentation can and should be technically
> vetoed. We can't really measure this with tools, so need to be vigilant
> on it ourselves. We also need to be understanding that docs may follow
> the code once it is finished, so we should use JIRA to track it (keep
> the original ticket open until documented, or create a new, blocking
> issue).
>
> * new code without javadoc/code documentation can be technically vetoed.
> I'm less tied to this one, though for public facing APIs I think Javadoc
> is imperative. It may be that we can use Checkstyle to enforce this to a
> degree.
>
> * code that doesn't meet current metrics can and should be technically
> vetoed. Again, we should set these up to fail the build, using
> Checkstyle, PMD and Clirr.
>
> Of course, I'll incorporate this into the dev process after some
> discussion.
>
> Thoughts?
>
> - Brett
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: making docs and tests suck less
Posted by Brett Porter <br...@apache.org>.
John Casey wrote:
>> * Whether it works and does what is intended.
>> * Whether it fits the spirit of the project.
>
> This seems like it would be hard for a user to assess. Not sure how a
> patch contributor is supposed to handle it.
The first one is obvious, so I assume just just mean #2. Not for them to
determine. It's reasons a patch might be rejected that they should be
prepared for and consider. If they aren't sure, they can ask before
doing the work.
> What I wind up doing in cases where I'm applying a patch is create tests
> to go along with them if there are none.
That's a good idea, but how often do you hesitate to apply a patch
because you don't want to do the tests? I don't think we are any better
off for it.
> HOWERVER, I'd like to suggest
> another guideline, for submitting a bug report:
>
> Steps to reproduce.
Absolutely.
> I agree that documentation is important, but making it a basis for
> rejecting a patch isn't necessarily the way to go IMO. In many cases,
> patches will be small enough that its documentation won't serve to
> improve the lives of users. It's still a good idea to have doco in code
> comments for developers to use, but if you're going to apply a patch,
> you should be able to write a one- or two-line code comment describing a
> particular change...otherwise, you probably shouldn't be applying the
> patch.
This was directed solely at user-facing docs, and only applies when that
is deemed necessary. It's not that extreme.
I agree, its rare for patch submitters to be affected by this, so its
not that onerous. IT's much more to show what applies to the *developers*.
>> * code that doesn't meet current metrics can and should be technically
>> vetoed. Again, we should set these up to fail the build, using
>> Checkstyle, PMD and Clirr.
>
> I think we need to define what those metrics are in some way that we all
> can reference, and *then* make sure we have the proper plugins in place
> to break the build when the metrics are violated.
Well, obviously ;)
Thanks,
Brett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: making docs and tests suck less
Posted by John Casey <jd...@yahoo.com>.
I think this is all pretty good in principle, but I have some concerns
about specific points, which I'll summarize below. I agree that we need
to raise our minimum quality at the release level, and even at the
individual commit level. However, I'm hesitant to add too much burden to
patch contributors, since I'd hate to discourage such involvement.
-john
Brett Porter wrote:
> So, I've committed r383963 which emphasises these requirements on patch
> contributions:
>
> * Whether it works and does what is intended.
> * Whether it fits the spirit of the project.
This seems like it would be hard for a user to assess. Not sure how a
patch contributor is supposed to handle it.
> * Whether it contains tests. It is expected that any patches relating to
> functionality will be accompanied by unit tests and/or integration
> tests. It is strongly desired (and will be requested) for bug fixes too,
> but will not be the basis for not applying it. At a bare minimum, the
> change should not decrease the amount of automated test coverage.
What I wind up doing in cases where I'm applying a patch is create tests
to go along with them if there are none. HOWERVER, I'd like to suggest
another guideline, for submitting a bug report:
Steps to reproduce.
This is fairly simple in most cases, and if you cannot reliably tell
someone else how you happened on the bug, I'd say you can't be certain
you have a bug at all. If you have a failing test case you can attach to
the issue, even better, since this will often speak much louder than the
issue description for describing the bug.
> * Whether it contains documentation. All new functionality needs to be
> documented for users, even if it is very rough for someone to expand on
> later. While rough is acceptable, incomplete is not.
I agree that documentation is important, but making it a basis for
rejecting a patch isn't necessarily the way to go IMO. In many cases,
patches will be small enough that its documentation won't serve to
improve the lives of users. It's still a good idea to have doco in code
comments for developers to use, but if you're going to apply a patch,
you should be able to write a one- or two-line code comment describing a
particular change...otherwise, you probably shouldn't be applying the patch.
>
> In the following, I'll discuss a technical veto. That's a -1 that means
> that the issue needs to be resolved before a release can occur. It
> doesn't mean someone rolls it back immediately (though a release manager
> may if it blocks a branch being released). It also doesn't mean anything
> negative about the committer in question, and it's important we keep it
> that way so that people aren't afraid to contribute or to call people on
> theese things.
>
> So my questions are, can we institute the following:
>
> * new functionality without automated test coverage can and should be
> technically vetoed. We can measure this with code coverage tools, and
> the measure will be that the % does not decline. It will actually break
> the build, being an automatic technical veto :)
+1000000
>
> * new functionality without documentation can and should be technically
> vetoed. We can't really measure this with tools, so need to be vigilant
> on it ourselves. We also need to be understanding that docs may follow
> the code once it is finished, so we should use JIRA to track it (keep
> the original ticket open until documented, or create a new, blocking issue).
+1. We just need to watch commit messages, and I *really* like the idea
of adding a workflow step to JIRA that says something like "Awaiting
Documentation" or something...Tim's idea, I believe.
>
> * new code without javadoc/code documentation can be technically vetoed.
> I'm less tied to this one, though for public facing APIs I think Javadoc
> is imperative. It may be that we can use Checkstyle to enforce this to a
> degree.
+1, though I'd say that Maven's componentized design means nearly any
method marked "public" or "protected" should be considered part of the
public API...either that, or we need to clearly delineate what
constitutes public vs. private APIs. For maven-artifact-* stuff, I'd
expect that none of it would be private API, since it can be reused in
other places, and since the container composes the component
relationships at runtime...
>
> * code that doesn't meet current metrics can and should be technically
> vetoed. Again, we should set these up to fail the build, using
> Checkstyle, PMD and Clirr.
I think we need to define what those metrics are in some way that we all
can reference, and *then* make sure we have the proper plugins in place
to break the build when the metrics are violated. We may want to run
these assessments in CI if they become too heavy, as the bootstrap takes
quite awhile already. However, it would be nice to make them required
before code can be committed, just like it would be nice to require that
all ITs pass before committing.
>
> Of course, I'll incorporate this into the dev process after some discussion.
>
> Thoughts?
>
> - Brett
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: making docs and tests suck less
Posted by David Sag <ds...@epo.org>.
see my comments inline below but in general +1
Kind regards,
Dave Sag
Brett Porter <br...@apache.org> wrote on 07-03-2006 21:10:53:
> This is not too long, and important. Please read.
>
> I've just spent some time discussing this on users@, and felt it was
> time to bring it here to look at the practical steps going forward.
>
> Basically, I think we've all known for a long time that both need work.
> There was a big push around 2.0 but it fizzled out afterwards.
>
> I don't want to focus on what we are missing now - we can address that
> with what is already in place. I also want to put aside the
> documentation and testing efforts that are already under way as they've
> been taken into consideration. What I want to focus on is going forward
> - new work. I think we need to change the culture. I realise this is
> hard given the timing because there isn't a lot of new work going on -
> because we're all doing bug fixes, docs and testing! - but hopefully
> because of this it will be apparent why it is important to do them as we
go.
>
> I've been trying to push for this for a long time, but haven't lived up
> to it myself which is one reason why it fails. I hate hypocrisy so can't
> really call other people on it (though I probably do anyway), and it
> requires everyone to buy in.
>
> So, I've committed r383963 which emphasises these requirements on patch
> contributions:
>
> * Whether it works and does what is intended.
> * Whether it fits the spirit of the project.
> * Whether it contains tests. It is expected that any patches relating to
> functionality will be accompanied by unit tests and/or integration
> tests. It is strongly desired (and will be requested) for bug fixes too,
> but will not be the basis for not applying it. At a bare minimum, the
> change should not decrease the amount of automated test coverage.
> * Whether it contains documentation. All new functionality needs to be
> documented for users, even if it is very rough for someone to expand on
> later. While rough is acceptable, incomplete is not.
>
> In the following, I'll discuss a technical veto. That's a -1 that means
> that the issue needs to be resolved before a release can occur. It
> doesn't mean someone rolls it back immediately (though a release manager
> may if it blocks a branch being released). It also doesn't mean anything
> negative about the committer in question, and it's important we keep it
> that way so that people aren't afraid to contribute or to call people on
> theese things.
>
> So my questions are, can we institute the following:
>
> * new functionality without automated test coverage can and should be
> technically vetoed. We can measure this with code coverage tools, and
> the measure will be that the % does not decline. It will actually break
> the build, being an automatic technical veto :)
>
Yes please. Which code coverage tool will you use? If cobertura then
will that mean that the outstanding cobertura plugin bugs will be
addressed soon :-)
eg http://jira.codehaus.org/browse/MCOBERTURA-2 and
http://jira.codehaus.org/browse/MCOBERTURA-5
> * new functionality without documentation can and should be technically
> vetoed. We can't really measure this with tools, so need to be vigilant
> on it ourselves. We also need to be understanding that docs may follow
> the code once it is finished, so we should use JIRA to track it (keep
> the original ticket open until documented, or create a new, blocking
issue).
absolutely YES!
> * new code without javadoc/code documentation can be technically vetoed.
> I'm less tied to this one, though for public facing APIs I think Javadoc
> is imperative. It may be that we can use Checkstyle to enforce this to a
> degree.
I am in favour of being quite hard-core on this. Writing javadocs isn't
hard.
> * code that doesn't meet current metrics can and should be technically
> vetoed. Again, we should set these up to fail the build, using
> Checkstyle, PMD and Clirr.
We have found it's also quite useful to pipe the output of checkstyle et
al thru QALab so you get some very nice visuals showing metrics vs time.
cheers
dave