[DISCUSS] Goals for 2013

[Noah Slater <ns...@apache.org>]

Hello community,

I'd like to start a thread and collect a list of goals for 2013.

We have a few exciting things in the pipeline. BigCouch is about to land.
Benoit's rcouch is also going to land. We're probably going to be moving to
V8. And if Jan has anything to do with it, we'll be getting a plugin system.

These are roadmap items. (Which are very important to have!) But I am
thinking about other things which we might want to make a commitment to.
Things to do with the community, and marketing, and docs, etc.

Some of the things I think we could aim for:

 * Increase month over month commits. Current activity is slowing down
month over month. Let's re-energize the project. How do we do this though?
I think by accepting more patches, closing more tickets, getting more
committers, and generating excitement.

I actually think the rest of the goals I want to propose all feed into
that...

 * Get a JIRA triage team and Github/patch team up and running. Stay on top
of this stuff. Bring the JIRA queue down. Anyone want to put forward a
number or a metric we can actually work towards here?

 * Increase the size of the committership. Perhaps we could aim to double
it? I think we should go into mass recruitment mode. To the point even
where we might want to suggest handing out a commit bit to anyone who asks
for it. I'm talking about lowering the barrier to entry as much as is
possible.

 * Release monthly. Activity already around getting this process
bootstrapped.

 * Increase marketing activity. Social media, blogs, etc. What can we do
here? Weekly news is one great idea that's already been mentioned this
week. What other metrics can we use?

Think of this like a set of personal development goals you might get at a
job, except we generate it for the community as a whole. And we can all
work towards them.

Things which are * measurable* are ideal. It's hard to measure "excitement"
but it's easy to measure social media mentions, and blog subscribers,
releases, committers, patches, tickets, etc.

We could break these down into quarters. If we manage to agree on some
metrics this month, we could set quarter by quarter goals for Q2, Q3, and
Q4.

Thoughts?

-- 
NS

Re: [DISCUSS] Goals for 2013

[Dave Cottlehuber <dc...@jsonified.com>]

Great thread. I've nothing specific to add except some comments from
my experience as a manager.

1. You can expect your contribution (in hours / head space) to be
roughly the same this year as last year unless you had Major Life
Change. Don't expect New Year's miracles.

2.  We need to focus on a very small number of things, as a team, and
get them done. Then pick the next bunch. Don't let ourselves get
distracted. I'm as guilty as the next person on this, but the point
still stands. e.g.:

- futon/fauxton made huge strides as it coalesced from 6+ individual
hacks on it into a group of people pursuing a common goal.
- docs streaked ahead when we had a few people focusing on it at the same time.
- cors rocketed ahead with several people working on it together from
different angles. It's also the first feature that had docs added when
it landed.

Above all, the first committed code was what drove the change & activity.

3. Nothing except security issues should prevent us from rolling
releases. We are not good on this at the moment, as Dirkjan rightly
pointed out.  A small number of active features is also easier to get
into the hands of our community, and less hassle than reviewing a mega
release.

So, we have a few things that I think we are collectively committed to already:

- the merges of bigcouchy refugey otp goodness
- fauxton super-powers
- CI integration
- the node/v8 exploration

That's already a big ask. I'll do what I can to help anybody
interested in tackling areas outside that of course, but I'd like to
see consensus around our "big 3" targets. And I'd like to ensure my
efforts are mostly aligned with those objectives.

A+
Dave

Re: [DISCUSS] Goals for 2013

[Benoit Chesneau <bc...@gmail.com>]

My personal technical and community goals for this year :

- give more visibility of what's going on. ANd for me it is just about
to let you know what do each others (and specifically the devs) on the
ml or on the media I am. This isn't marketing. I just want do do it
because I also miss that these days and I'm partially responsible.

- having couchdb full OTP: my rcouch branch should land later today or
tomorrow, i have been sidetracked by a thing yesterday.

- having a real couchapp engine around. But something new. that should
use the full power of OTP and can be embedded easily: message passing,
node distributions, websockets, ...

- improve the view protocol to make it more efficient. And if possible
fully embedded in couchdb with only some line of C if really needed.
Some code experimentations last week gave me the convinction  we could
use v8 without having to embed a bloated nodejs. Also we can propose
an alternative and innovate rather than embrace the world and do like
others. I also think we should distinct views from the couchapp engine
by itself. They require distinct features. Hopefully I will be able to
push some code at the end of the month.

- improve the HTTP layer which goes imo by revisiting our internal API
to only use HTTP as a transport. Today we are mixing some core db code
in the HTTP api

- revisit auth handling. remove all users references in the core  and
only handle auth at upper level (ie remove ddoc load, embedded authz,
security doc...) but propose some callbacks or events to allows
transport on top to manage authentication. Ie only having authz done
at HTTP level for the HTTP api. I've started to experiment that here:

https://github.com/benoitc/libcouch

- improve the replicator code: handle partial replications, speed
(maybe use websockets) and improve a lot the way you can track a
replication task activity.

- improve the user  & dev documentation. and collect all the
experiments around to give a full panorama of the couchdb usage.

I have other things in view, but these above are enough for me.

For the rest , the more we will make couchdb interresting for the
users at a technical level but also as a concept, the more we will
gain traction. But it doesn't go necessarily by doing like others (ex
take nodejs because most are using it, if we do that should be for
real technical reason) or having more "marketing". The best evangelism
is when you don't have to do it.  And we didn't have to do it until we
pushed new features and interesting concepts in Apache CouchDB. We
should however improve the user experiment and that could be done by
addressing each concerns: users of couchdb "just as a db" and couchapp
users at a technical level but also documentation.

Just some quick notes about these 2 concerns:

- couchapp. there is a trend among some users to say that couchapps
are useless or not enough by themselves and then either remove them or
use them just as a way to replicate your app and handle most of the
logic either on the browser or by using external code on top (proxy,
...) and behind (workers...) . Both are partials responses and mostly
hack that doesn't really address the concern. And i'm quite annoyed to
hear people talking about it like a thing written in stone that can't
be changed. I think we can do better today on the couchapp side and
use the power of replication, html5 to propose a new framework
designed for the coming web (webrtc, websockets, ...) with new routing
possibilities. All embedded. External scripts should be used when
erlang isn't enough by itself or you need to use specific library for
the work (like in machine learning or image processing).

- db level: for those who are using couchdb "just" as a database.
bigcouch, rcouch will give more feature on that part but there are a
lot of improvements around that can be made. also really wish that the
core of couchdb can be still used as a single db without cluster need.
I don't want to use couchdb only for big dataset.s I also want it to
use on my small device or my mobile. And sorry no I don't want to use
pouchdb, touchdb for that. I want to use the same core code in most
places.

Voilà. Hopefully, I will be able to help on that topics, if not in
apache couchdb at least in rcouch as new addons. And of course others
are welcome to help!


- benoît

On Wed, Mar 6, 2013 at 7:14 PM, Noah Slater <ns...@apache.org> wrote:
> Hello community,
>
> I'd like to start a thread and collect a list of goals for 2013.
>
> We have a few exciting things in the pipeline. BigCouch is about to land.
> Benoit's rcouch is also going to land. We're probably going to be moving to
> V8. And if Jan has anything to do with it, we'll be getting a plugin system.
>
> These are roadmap items. (Which are very important to have!) But I am
> thinking about other things which we might want to make a commitment to.
> Things to do with the community, and marketing, and docs, etc.
>
> Some of the things I think we could aim for:
>
>  * Increase month over month commits. Current activity is slowing down
> month over month. Let's re-energize the project. How do we do this though?
> I think by accepting more patches, closing more tickets, getting more
> committers, and generating excitement.
>
> I actually think the rest of the goals I want to propose all feed into
> that...
>
>  * Get a JIRA triage team and Github/patch team up and running. Stay on top
> of this stuff. Bring the JIRA queue down. Anyone want to put forward a
> number or a metric we can actually work towards here?
>
>  * Increase the size of the committership. Perhaps we could aim to double
> it? I think we should go into mass recruitment mode. To the point even
> where we might want to suggest handing out a commit bit to anyone who asks
> for it. I'm talking about lowering the barrier to entry as much as is
> possible.
>
>  * Release monthly. Activity already around getting this process
> bootstrapped.
>
>  * Increase marketing activity. Social media, blogs, etc. What can we do
> here? Weekly news is one great idea that's already been mentioned this
> week. What other metrics can we use?
>
> Think of this like a set of personal development goals you might get at a
> job, except we generate it for the community as a whole. And we can all
> work towards them.
>
> Things which are * measurable* are ideal. It's hard to measure "excitement"
> but it's easy to measure social media mentions, and blog subscribers,
> releases, committers, patches, tickets, etc.
>
> We could break these down into quarters. If we manage to agree on some
> metrics this month, we could set quarter by quarter goals for Q2, Q3, and
> Q4.
>
> Thoughts?
>
> --
> NS

Re: [DISCUSS] Goals for 2013

[Russell Branca <ch...@gmail.com>]

My goals right now for CouchDB are focused around how users interact with
the system.

Specifically, I want to see fauxton fully functional with a vibrant plugin
ecosystem.

The other big area is how users extend functionality of CouchDB with
javascript. I think the first big step is switching to a Node.js based
javascript engine, but continuing from there, I would love to see a well
supported reference implementation of a  worker system that does event
based and time based job execution, and makes it easy for new users to
build full applications with CouchDB. There are literally dozens of half
completed implementations of this idea floating around, and I think we
should make a solid, well documented, and easily extensible workers system
that new users and veteran users alike can put to use for common tasks.
Whether this is hoodie workers, or gardener, or whatever, we should choose
one and make it great.

Also, I want to put together a CouchDB Seattle conference this summer
(tentatively looking at sometime in July).


-Russell


On Thu, Mar 7, 2013 at 1:22 PM, Jan Lehnardt <ja...@apache.org> wrote:

> Thanks for calling this to action Noah!
>
> My personal list for this year:
>
> -1. Get people excited to contribute to CouchDB.
>
>  0. Help remove roadblocks for any sort of contribution, docs, code,
> tests, bugs, whatever it takes.
>
>  1. A bunch of features that I’d like to rally some activity around, as I
> sure can’t write them all on my own:
>
>   - A plugin system as roughly outlined in that November/December thread,
> but with better timing.
>
>   - Full continuous integration suite, run tests suits on various systems
> automatically on every commit in release and integration branches.
>
>   - Help provide great system and binary packages for CouchDB in the
> various distributions and standalone.
>     (In my dream future, the continuous integration system spits out
> binaries for all supported systems so people who report at bug can quickly
> test if it solves the issue without having to build from source etc.)
>
>   - Move the JS engine to Node.js. Allow for writing db-tools / plugins /
> utilities in Node.
>
>  2. Get a Website/PR/Marketing team together. Refine the focus of the
> CouchDB mission statement. I have some ideas that I share when the time is
> right.
>
>  3. Support a JIRA/PR triage team in every which way I can.
>
>  4. Get a developer’s handbook started to the CouchDB internals.
>
>  5. Help build local real-world CouchDB community meetups like the ones we
> do in Berlin (http://berlin.couchdb.org), supply folks with guides on how
> to run a meetup, resources to talk about, help out with organising shwag
> etc.
>
>  6. Get an evangelism team together: bundle and make available for anyone
> slides and teaching materials that can be used to give talks at
> conferences, meetups and the like.
>
>  5.+ 6. = 11. Connect this all online, so we can share experiences and
> improve each other’s materials.
>
>
> That’s just off the back of my head. I will personally focus a lot on -1.
> and 0. as well as getting the rest started, but I will need all your help
> to move things to completion. I’m not yet comfortable to commit to
> promising all of that done by the end of 2013, but if most of it is on the
> way and going well, I think we are in good shape.
>
>
> If you want to help out with any of these, please get in touch.
>
>
> <3
>
> Jan
> --
>
>
>
>
> On Mar 6, 2013, at 19:14 , Noah Slater <ns...@apache.org> wrote:
>
> > Hello community,
> >
> > I'd like to start a thread and collect a list of goals for 2013.
> >
> > We have a few exciting things in the pipeline. BigCouch is about to land.
> > Benoit's rcouch is also going to land. We're probably going to be moving
> to
> > V8. And if Jan has anything to do with it, we'll be getting a plugin
> system.
> >
> > These are roadmap items. (Which are very important to have!) But I am
> > thinking about other things which we might want to make a commitment to.
> > Things to do with the community, and marketing, and docs, etc.
> >
> > Some of the things I think we could aim for:
> >
> > * Increase month over month commits. Current activity is slowing down
> > month over month. Let's re-energize the project. How do we do this
> though?
> > I think by accepting more patches, closing more tickets, getting more
> > committers, and generating excitement.
> >
> > I actually think the rest of the goals I want to propose all feed into
> > that...
> >
> > * Get a JIRA triage team and Github/patch team up and running. Stay on
> top
> > of this stuff. Bring the JIRA queue down. Anyone want to put forward a
> > number or a metric we can actually work towards here?
> >
> > * Increase the size of the committership. Perhaps we could aim to double
> > it? I think we should go into mass recruitment mode. To the point even
> > where we might want to suggest handing out a commit bit to anyone who
> asks
> > for it. I'm talking about lowering the barrier to entry as much as is
> > possible.
> >
> > * Release monthly. Activity already around getting this process
> > bootstrapped.
> >
> > * Increase marketing activity. Social media, blogs, etc. What can we do
> > here? Weekly news is one great idea that's already been mentioned this
> > week. What other metrics can we use?
> >
> > Think of this like a set of personal development goals you might get at a
> > job, except we generate it for the community as a whole. And we can all
> > work towards them.
> >
> > Things which are * measurable* are ideal. It's hard to measure
> "excitement"
> > but it's easy to measure social media mentions, and blog subscribers,
> > releases, committers, patches, tickets, etc.
> >
> > We could break these down into quarters. If we manage to agree on some
> > metrics this month, we could set quarter by quarter goals for Q2, Q3, and
> > Q4.
> >
> > Thoughts?
> >
> > --
> > NS
>
>

Re: [DISCUSS] Goals for 2013

[Jan Lehnardt <ja...@apache.org>]

Thanks for calling this to action Noah!

My personal list for this year:

-1. Get people excited to contribute to CouchDB.

 0. Help remove roadblocks for any sort of contribution, docs, code, tests, bugs, whatever it takes.

 1. A bunch of features that I’d like to rally some activity around, as I sure can’t write them all on my own:

  - A plugin system as roughly outlined in that November/December thread, but with better timing.

  - Full continuous integration suite, run tests suits on various systems automatically on every commit in release and integration branches.

  - Help provide great system and binary packages for CouchDB in the various distributions and standalone.
    (In my dream future, the continuous integration system spits out binaries for all supported systems so people who report at bug can quickly test if it solves the issue without having to build from source etc.)

  - Move the JS engine to Node.js. Allow for writing db-tools / plugins / utilities in Node.

 2. Get a Website/PR/Marketing team together. Refine the focus of the CouchDB mission statement. I have some ideas that I share when the time is right.

 3. Support a JIRA/PR triage team in every which way I can.

 4. Get a developer’s handbook started to the CouchDB internals.

 5. Help build local real-world CouchDB community meetups like the ones we do in Berlin (http://berlin.couchdb.org), supply folks with guides on how to run a meetup, resources to talk about, help out with organising shwag etc.

 6. Get an evangelism team together: bundle and make available for anyone slides and teaching materials that can be used to give talks at conferences, meetups and the like.

 5.+ 6. = 11. Connect this all online, so we can share experiences and improve each other’s materials.


That’s just off the back of my head. I will personally focus a lot on -1. and 0. as well as getting the rest started, but I will need all your help to move things to completion. I’m not yet comfortable to commit to promising all of that done by the end of 2013, but if most of it is on the way and going well, I think we are in good shape.


If you want to help out with any of these, please get in touch.


<3

Jan
-- 




On Mar 6, 2013, at 19:14 , Noah Slater <ns...@apache.org> wrote:

> Hello community,
> 
> I'd like to start a thread and collect a list of goals for 2013.
> 
> We have a few exciting things in the pipeline. BigCouch is about to land.
> Benoit's rcouch is also going to land. We're probably going to be moving to
> V8. And if Jan has anything to do with it, we'll be getting a plugin system.
> 
> These are roadmap items. (Which are very important to have!) But I am
> thinking about other things which we might want to make a commitment to.
> Things to do with the community, and marketing, and docs, etc.
> 
> Some of the things I think we could aim for:
> 
> * Increase month over month commits. Current activity is slowing down
> month over month. Let's re-energize the project. How do we do this though?
> I think by accepting more patches, closing more tickets, getting more
> committers, and generating excitement.
> 
> I actually think the rest of the goals I want to propose all feed into
> that...
> 
> * Get a JIRA triage team and Github/patch team up and running. Stay on top
> of this stuff. Bring the JIRA queue down. Anyone want to put forward a
> number or a metric we can actually work towards here?
> 
> * Increase the size of the committership. Perhaps we could aim to double
> it? I think we should go into mass recruitment mode. To the point even
> where we might want to suggest handing out a commit bit to anyone who asks
> for it. I'm talking about lowering the barrier to entry as much as is
> possible.
> 
> * Release monthly. Activity already around getting this process
> bootstrapped.
> 
> * Increase marketing activity. Social media, blogs, etc. What can we do
> here? Weekly news is one great idea that's already been mentioned this
> week. What other metrics can we use?
> 
> Think of this like a set of personal development goals you might get at a
> job, except we generate it for the community as a whole. And we can all
> work towards them.
> 
> Things which are * measurable* are ideal. It's hard to measure "excitement"
> but it's easy to measure social media mentions, and blog subscribers,
> releases, committers, patches, tickets, etc.
> 
> We could break these down into quarters. If we manage to agree on some
> metrics this month, we could set quarter by quarter goals for Q2, Q3, and
> Q4.
> 
> Thoughts?
> 
> -- 
> NS

Re: [DISCUSS] Goals for 2013

[Paul Davis <pa...@gmail.com>]

There's a lot of work that could be done to a lot of the internals of
CouchDB to clean things up and put us back in a place where we can be
quicker to innovate.

1. Rewrite the HTTP layer to not be a gobbledygook function call
spiderweb for request dispatching. Whether we use something like
webmachine or switch to Cowboy or whatever it takes. The HTTP layer is
entirely too complicated for what it does.

2. Cleanup the core database implementation. There's a lot of cruft
that's grown up in some of the core code that makes things
unreasonably hard to follow. Once you untangle the spaghetti you
realize that CouchDB is exceedingly simple, but not many people make
it through that initial learning investment. This also ties into #3.

3. Analyze hot code paths for dumbness to improve database
performance. Part of the code being complicated is that its really
hard to tell whats going on at various times. I've traced through
different parts debugging various things and have noticed a number of
lackluster behaviors (things like re-reading the btree root multiple
times for a single document update). Someone with enough interest
could start tracing through here and counting the ways we screw the
pooch on this count. I'm also still convinced that something in the
HTTP layer is an artificial bottleneck but haven't had time to
investigate.

4. Rewrite couchjs and the view server protocol. We've talked about
this for ages. Moving from multiple OS processes to a single process
that handles all requests for a given language asynchronously and
handles caching could turn into a significant performance benefit for
lots of database operations.

5. Rewrite the entire test suite. From scratch. In a way that doesn't
break things. There are some interesting ways to approach this. I've
played before with Erlang/Python hybrids for doing internal/external
testing. Nothing quite congealed into a cohesive platform but I'm
convinced that our current approach is lackluster at best.

6. Other things we've previously discussed. Rewrite the replicator to
focus on robustness. Change conflict beahviour so they can't be
ignored easily. Split couchapp stuff into a different project to
reinforce the separation of concerns. Remove metadata from doc bodies.
All the 2.0-y things.

On Thu, Mar 7, 2013 at 10:23 AM, Robert Newson <rn...@apache.org> wrote:
> +1. We are conscious of our release process failings but it's great to
> see an uninhibited reminder like this.
>
> We're aiming for regular, frequent releases after this one, as Noah
> says. I want to see that process so regular that it becomes boring.
> You should come to expect that a fix is out within a month and new
> features are out within a quarter.
>
> We are also making some excellent progress at Cloudant preparing the
> BigCouch merger. We hope to have something for folks to play with by
> the end of next week. I'll also add that when I say "BigCouch", I'm
> referring to the well-tendered version at Cloudant and not the 0.4 we
> cut over a year ago. This is *not* stale code, but the very well
> cared-for code we run in production. We'll have updates as major
> pieces come but we (Paul Davis and myself) are largely heads-down
> dealing with all kinds of lovely merge conflicts and stuff. :D
>
> B
>
> On 7 March 2013 09:54, Noah Slater <ns...@apache.org> wrote:
>> Agreed. Thanks for the honest feedback. Massively appreciated.
>>
>> I have been very frustrated with the 1.3.0 release. It was, and still is,
>> supposed to be the first of our regular releases. Unfortunately, the
>> release was mired with a raft of CVEs that came first, and then some thorny
>> blockers. Not making excuses. I felt the pain of this as much as anybody
>> here who cares about making this work.
>>
>> I've spent the last few weeks putting work in, and actually coming up with
>> an entirely new release procedure (which I will be announcing in tandem
>> with the release, in a day or two) that should address these problems. Most
>> significantly, I am moving the test suite to the voting stage, which will
>> allow me to cut and call votes very quickly.
>>
>> So, from my perspective, we are just about to emerge from release hell, and
>> I have my eye firmly on the next three quarters. As I mention in my
>> original email, basically, most of this hinges on activity. And activity
>> hinges on releases, as you say. But I've already put a lot of thought into
>> that, and I wanted to invite the community to start thinking about the
>> other things we can start to address. I think these things are important
>> too.
>>
>> Here's an example, look at the burn-down chart for tickets:
>>
>> https://issues.apache.org/jira/browse/CouchDB
>>
>> Abysmal.
>>
>> How do we solve it? Sure, release will stimulate activity. But again, this
>> is a known quantity. And I hope. Oh God, I hope. That we're going to nail
>> that one. So I am left asking, from my POV, what else?
>>
>> Well, the stuff I put in my email, I guess. I have a big backlog of project
>> management stuff to get through. Things I want to bring to the list. Things
>> that should hopefully get easier as activity picks up.
>>
>> Perhaps some of these are things you might be interested in? One of the
>> things we really really desperately need is someone to organise and run a
>> JIRA triage team. The goal being, I think, to just keep an eye on JIRA,
>> field incoming tickets, close them out where appropriate, or categorise and
>> assign them otherwise.
>>
>> We also need a docs team. And a marketing team. And a PM team. These teams
>> might just be one person to begin with. But I think we need someone to step
>> up to the plate and say "I will look after this."
>>
>> There are lots of ways to contribute that don't involve Erlang. I don't
>> know Erlang. :)
>>
>> Again, I think some metrics would be a good idea. For example, if we say,
>> okay, well, we want to double our committership by Q4. That invites a
>> conversation. How do we do that?
>>
>> These are important questions to be asking!
>>
>>
>> On 7 March 2013 11:04, Dirkjan Ochtman <di...@ochtman.nl> wrote:
>>
>>> (warning, grumpy rant forthcoming...)
>>>
>>> On Wed, Mar 6, 2013 at 7:14 PM, Noah Slater <ns...@apache.org> wrote:
>>> > Thoughts?
>>>
>>> It seems to me that a lot of this hinges on releases. Releases
>>> generate publicity and therefore developer interest, thus developer
>>> engagement. Releases make sure that work developers do gets into
>>> user's hands soon.
>>>
>>> CouchDB, and I am sorry to have been banging this drum for a few years
>>> now, is the worst open source project at doing releases I know.
>>>
>>> Here's a quote from this list, from February 13:
>>>
>>> > I plan to cut a release candidate from the 1.3.x branch this weekend.
>>>
>>> And another, from Nov 13:
>>>
>>> > when we set out to ship 1.3.0, we thought the cors and docs
>>> > branches were just around the corner. That was a couple of
>>> > weeks ago. We are just starting with this, but the whole
>>> > idea of time-based releases is that we do not wait for
>>> > feature branches to be ready.
>>> >
>>> > I’d like to propose that we ignore everything we’ve said
>>> > before and do the following:
>>> >
>>> >  - ship 1.3.0 as reflected in the 1.3.x branch now.
>>>
>>> Here's another, from October 23:
>>>
>>> > It's been a while since we released and I want to change this now. I
>>> > propose making a 1.3.x branch from today's master
>>> > (66529378dd06342929e04772370f3509cbe786a5) and building 1.3.0 there.
>>>
>>> Another, from June 16:
>>>
>>> > 1. We'd like to proposed formal time-based releases
>>>
>>> Et cetera. As an example, the EventSource bits, which are actually
>>> something which would be very useful for us at work, were committed to
>>> the tree on May 16. They are yet to be released.
>>>
>>> I feel a bit hypocritical for saying all this, since I don't
>>> contribute much to core CouchDB (although I did do a bunch of the doc
>>> conversion, try to be helpful around the edges and maintain
>>> couchdb-python). The reason for this is mostly that I thoroughly
>>> dislike Erlang as a language, and have no intention of learning it
>>> just to help out with CouchDB. I did some Futon work back in the day,
>>> but had lots of trouble getting it committed since no one's really
>>> owned Futon for a while (and right now, it looks like other people
>>> have that bit covered, though they're using a modern but also quite
>>> complex development stack that will make it unlikely for a lowly
>>> back-end web developer dude like me to be able to contribute much).
>>>
>>> So, IMNSH and annoyingly harsh O, stop talking so much about how to
>>> energize the project, and Just. Ship. It. Every month, preferably. Fix
>>> the test suite and whatever else makes it so damn hard to release.
>>> Insert story here about that dude who realized people were trying to
>>> fix the wrong problem [1].
>>>
>>> End of rant, cheers,
>>>
>>> Dirkjan
>>>
>>> [1] http://www.azarask.in/blog/post/the-wrong-problem/
>>>
>>
>>
>>
>> --
>> NS

Re: [DISCUSS] Goals for 2013

[Robert Newson <rn...@apache.org>]

+1. We are conscious of our release process failings but it's great to
see an uninhibited reminder like this.

We're aiming for regular, frequent releases after this one, as Noah
says. I want to see that process so regular that it becomes boring.
You should come to expect that a fix is out within a month and new
features are out within a quarter.

We are also making some excellent progress at Cloudant preparing the
BigCouch merger. We hope to have something for folks to play with by
the end of next week. I'll also add that when I say "BigCouch", I'm
referring to the well-tendered version at Cloudant and not the 0.4 we
cut over a year ago. This is *not* stale code, but the very well
cared-for code we run in production. We'll have updates as major
pieces come but we (Paul Davis and myself) are largely heads-down
dealing with all kinds of lovely merge conflicts and stuff. :D

B

On 7 March 2013 09:54, Noah Slater <ns...@apache.org> wrote:
> Agreed. Thanks for the honest feedback. Massively appreciated.
>
> I have been very frustrated with the 1.3.0 release. It was, and still is,
> supposed to be the first of our regular releases. Unfortunately, the
> release was mired with a raft of CVEs that came first, and then some thorny
> blockers. Not making excuses. I felt the pain of this as much as anybody
> here who cares about making this work.
>
> I've spent the last few weeks putting work in, and actually coming up with
> an entirely new release procedure (which I will be announcing in tandem
> with the release, in a day or two) that should address these problems. Most
> significantly, I am moving the test suite to the voting stage, which will
> allow me to cut and call votes very quickly.
>
> So, from my perspective, we are just about to emerge from release hell, and
> I have my eye firmly on the next three quarters. As I mention in my
> original email, basically, most of this hinges on activity. And activity
> hinges on releases, as you say. But I've already put a lot of thought into
> that, and I wanted to invite the community to start thinking about the
> other things we can start to address. I think these things are important
> too.
>
> Here's an example, look at the burn-down chart for tickets:
>
> https://issues.apache.org/jira/browse/CouchDB
>
> Abysmal.
>
> How do we solve it? Sure, release will stimulate activity. But again, this
> is a known quantity. And I hope. Oh God, I hope. That we're going to nail
> that one. So I am left asking, from my POV, what else?
>
> Well, the stuff I put in my email, I guess. I have a big backlog of project
> management stuff to get through. Things I want to bring to the list. Things
> that should hopefully get easier as activity picks up.
>
> Perhaps some of these are things you might be interested in? One of the
> things we really really desperately need is someone to organise and run a
> JIRA triage team. The goal being, I think, to just keep an eye on JIRA,
> field incoming tickets, close them out where appropriate, or categorise and
> assign them otherwise.
>
> We also need a docs team. And a marketing team. And a PM team. These teams
> might just be one person to begin with. But I think we need someone to step
> up to the plate and say "I will look after this."
>
> There are lots of ways to contribute that don't involve Erlang. I don't
> know Erlang. :)
>
> Again, I think some metrics would be a good idea. For example, if we say,
> okay, well, we want to double our committership by Q4. That invites a
> conversation. How do we do that?
>
> These are important questions to be asking!
>
>
> On 7 March 2013 11:04, Dirkjan Ochtman <di...@ochtman.nl> wrote:
>
>> (warning, grumpy rant forthcoming...)
>>
>> On Wed, Mar 6, 2013 at 7:14 PM, Noah Slater <ns...@apache.org> wrote:
>> > Thoughts?
>>
>> It seems to me that a lot of this hinges on releases. Releases
>> generate publicity and therefore developer interest, thus developer
>> engagement. Releases make sure that work developers do gets into
>> user's hands soon.
>>
>> CouchDB, and I am sorry to have been banging this drum for a few years
>> now, is the worst open source project at doing releases I know.
>>
>> Here's a quote from this list, from February 13:
>>
>> > I plan to cut a release candidate from the 1.3.x branch this weekend.
>>
>> And another, from Nov 13:
>>
>> > when we set out to ship 1.3.0, we thought the cors and docs
>> > branches were just around the corner. That was a couple of
>> > weeks ago. We are just starting with this, but the whole
>> > idea of time-based releases is that we do not wait for
>> > feature branches to be ready.
>> >
>> > I’d like to propose that we ignore everything we’ve said
>> > before and do the following:
>> >
>> >  - ship 1.3.0 as reflected in the 1.3.x branch now.
>>
>> Here's another, from October 23:
>>
>> > It's been a while since we released and I want to change this now. I
>> > propose making a 1.3.x branch from today's master
>> > (66529378dd06342929e04772370f3509cbe786a5) and building 1.3.0 there.
>>
>> Another, from June 16:
>>
>> > 1. We'd like to proposed formal time-based releases
>>
>> Et cetera. As an example, the EventSource bits, which are actually
>> something which would be very useful for us at work, were committed to
>> the tree on May 16. They are yet to be released.
>>
>> I feel a bit hypocritical for saying all this, since I don't
>> contribute much to core CouchDB (although I did do a bunch of the doc
>> conversion, try to be helpful around the edges and maintain
>> couchdb-python). The reason for this is mostly that I thoroughly
>> dislike Erlang as a language, and have no intention of learning it
>> just to help out with CouchDB. I did some Futon work back in the day,
>> but had lots of trouble getting it committed since no one's really
>> owned Futon for a while (and right now, it looks like other people
>> have that bit covered, though they're using a modern but also quite
>> complex development stack that will make it unlikely for a lowly
>> back-end web developer dude like me to be able to contribute much).
>>
>> So, IMNSH and annoyingly harsh O, stop talking so much about how to
>> energize the project, and Just. Ship. It. Every month, preferably. Fix
>> the test suite and whatever else makes it so damn hard to release.
>> Insert story here about that dude who realized people were trying to
>> fix the wrong problem [1].
>>
>> End of rant, cheers,
>>
>> Dirkjan
>>
>> [1] http://www.azarask.in/blog/post/the-wrong-problem/
>>
>
>
>
> --
> NS

Re: [DISCUSS] Goals for 2013

[Noah Slater <ns...@apache.org>]

Agreed. Thanks for the honest feedback. Massively appreciated.

I have been very frustrated with the 1.3.0 release. It was, and still is,
supposed to be the first of our regular releases. Unfortunately, the
release was mired with a raft of CVEs that came first, and then some thorny
blockers. Not making excuses. I felt the pain of this as much as anybody
here who cares about making this work.

I've spent the last few weeks putting work in, and actually coming up with
an entirely new release procedure (which I will be announcing in tandem
with the release, in a day or two) that should address these problems. Most
significantly, I am moving the test suite to the voting stage, which will
allow me to cut and call votes very quickly.

So, from my perspective, we are just about to emerge from release hell, and
I have my eye firmly on the next three quarters. As I mention in my
original email, basically, most of this hinges on activity. And activity
hinges on releases, as you say. But I've already put a lot of thought into
that, and I wanted to invite the community to start thinking about the
other things we can start to address. I think these things are important
too.

Here's an example, look at the burn-down chart for tickets:

https://issues.apache.org/jira/browse/CouchDB

Abysmal.

How do we solve it? Sure, release will stimulate activity. But again, this
is a known quantity. And I hope. Oh God, I hope. That we're going to nail
that one. So I am left asking, from my POV, what else?

Well, the stuff I put in my email, I guess. I have a big backlog of project
management stuff to get through. Things I want to bring to the list. Things
that should hopefully get easier as activity picks up.

Perhaps some of these are things you might be interested in? One of the
things we really really desperately need is someone to organise and run a
JIRA triage team. The goal being, I think, to just keep an eye on JIRA,
field incoming tickets, close them out where appropriate, or categorise and
assign them otherwise.

We also need a docs team. And a marketing team. And a PM team. These teams
might just be one person to begin with. But I think we need someone to step
up to the plate and say "I will look after this."

There are lots of ways to contribute that don't involve Erlang. I don't
know Erlang. :)

Again, I think some metrics would be a good idea. For example, if we say,
okay, well, we want to double our committership by Q4. That invites a
conversation. How do we do that?

These are important questions to be asking!


On 7 March 2013 11:04, Dirkjan Ochtman <di...@ochtman.nl> wrote:

> (warning, grumpy rant forthcoming...)
>
> On Wed, Mar 6, 2013 at 7:14 PM, Noah Slater <ns...@apache.org> wrote:
> > Thoughts?
>
> It seems to me that a lot of this hinges on releases. Releases
> generate publicity and therefore developer interest, thus developer
> engagement. Releases make sure that work developers do gets into
> user's hands soon.
>
> CouchDB, and I am sorry to have been banging this drum for a few years
> now, is the worst open source project at doing releases I know.
>
> Here's a quote from this list, from February 13:
>
> > I plan to cut a release candidate from the 1.3.x branch this weekend.
>
> And another, from Nov 13:
>
> > when we set out to ship 1.3.0, we thought the cors and docs
> > branches were just around the corner. That was a couple of
> > weeks ago. We are just starting with this, but the whole
> > idea of time-based releases is that we do not wait for
> > feature branches to be ready.
> >
> > I’d like to propose that we ignore everything we’ve said
> > before and do the following:
> >
> >  - ship 1.3.0 as reflected in the 1.3.x branch now.
>
> Here's another, from October 23:
>
> > It's been a while since we released and I want to change this now. I
> > propose making a 1.3.x branch from today's master
> > (66529378dd06342929e04772370f3509cbe786a5) and building 1.3.0 there.
>
> Another, from June 16:
>
> > 1. We'd like to proposed formal time-based releases
>
> Et cetera. As an example, the EventSource bits, which are actually
> something which would be very useful for us at work, were committed to
> the tree on May 16. They are yet to be released.
>
> I feel a bit hypocritical for saying all this, since I don't
> contribute much to core CouchDB (although I did do a bunch of the doc
> conversion, try to be helpful around the edges and maintain
> couchdb-python). The reason for this is mostly that I thoroughly
> dislike Erlang as a language, and have no intention of learning it
> just to help out with CouchDB. I did some Futon work back in the day,
> but had lots of trouble getting it committed since no one's really
> owned Futon for a while (and right now, it looks like other people
> have that bit covered, though they're using a modern but also quite
> complex development stack that will make it unlikely for a lowly
> back-end web developer dude like me to be able to contribute much).
>
> So, IMNSH and annoyingly harsh O, stop talking so much about how to
> energize the project, and Just. Ship. It. Every month, preferably. Fix
> the test suite and whatever else makes it so damn hard to release.
> Insert story here about that dude who realized people were trying to
> fix the wrong problem [1].
>
> End of rant, cheers,
>
> Dirkjan
>
> [1] http://www.azarask.in/blog/post/the-wrong-problem/
>



-- 
NS

Re: [DISCUSS] Goals for 2013

[Jan Lehnardt <ja...@apache.org>]

On Mar 7, 2013, at 12:04 , Dirkjan Ochtman <di...@ochtman.nl> wrote:

> (warning, grumpy rant forthcoming...)

Oh, and thanks for speaking your mind, and being patient with us. Your
contributions are very much appreciated! :)

Jan
-- 

> 
> On Wed, Mar 6, 2013 at 7:14 PM, Noah Slater <ns...@apache.org> wrote:
>> Thoughts?
> 
> It seems to me that a lot of this hinges on releases. Releases
> generate publicity and therefore developer interest, thus developer
> engagement. Releases make sure that work developers do gets into
> user's hands soon.
> 
> CouchDB, and I am sorry to have been banging this drum for a few years
> now, is the worst open source project at doing releases I know.
> 
> Here's a quote from this list, from February 13:
> 
>> I plan to cut a release candidate from the 1.3.x branch this weekend.
> 
> And another, from Nov 13:
> 
>> when we set out to ship 1.3.0, we thought the cors and docs
>> branches were just around the corner. That was a couple of
>> weeks ago. We are just starting with this, but the whole
>> idea of time-based releases is that we do not wait for
>> feature branches to be ready.
>> 
>> I’d like to propose that we ignore everything we’ve said
>> before and do the following:
>> 
>> - ship 1.3.0 as reflected in the 1.3.x branch now.
> 
> Here's another, from October 23:
> 
>> It's been a while since we released and I want to change this now. I
>> propose making a 1.3.x branch from today's master
>> (66529378dd06342929e04772370f3509cbe786a5) and building 1.3.0 there.
> 
> Another, from June 16:
> 
>> 1. We'd like to proposed formal time-based releases
> 
> Et cetera. As an example, the EventSource bits, which are actually
> something which would be very useful for us at work, were committed to
> the tree on May 16. They are yet to be released.
> 
> I feel a bit hypocritical for saying all this, since I don't
> contribute much to core CouchDB (although I did do a bunch of the doc
> conversion, try to be helpful around the edges and maintain
> couchdb-python). The reason for this is mostly that I thoroughly
> dislike Erlang as a language, and have no intention of learning it
> just to help out with CouchDB. I did some Futon work back in the day,
> but had lots of trouble getting it committed since no one's really
> owned Futon for a while (and right now, it looks like other people
> have that bit covered, though they're using a modern but also quite
> complex development stack that will make it unlikely for a lowly
> back-end web developer dude like me to be able to contribute much).
> 
> So, IMNSH and annoyingly harsh O, stop talking so much about how to
> energize the project, and Just. Ship. It. Every month, preferably. Fix
> the test suite and whatever else makes it so damn hard to release.
> Insert story here about that dude who realized people were trying to
> fix the wrong problem [1].
> 
> End of rant, cheers,
> 
> Dirkjan
> 
> [1] http://www.azarask.in/blog/post/the-wrong-problem/

Re: [DISCUSS] Goals for 2013

[Jan Lehnardt <ja...@apache.org>]

On Mar 7, 2013, at 12:04 , Dirkjan Ochtman <di...@ochtman.nl> wrote:

> (warning, grumpy rant forthcoming...)

Dirkjan,

understood, loud and clear. And I think we all agree with you :)

In fact, we made a few changes to the way we manage the code and releases
to improve all of that, but I think we failed at communicating this clearly.

1.3.0 will be the last of the traditional “random” releases of CouchDB.

We are switching to time based releases and aim to release a new version
of CouchDB every three months for feature releases and every month, if
we can manage, for bugfix releases. This has a few consequences:

 - Moar releases.
 - Smaller releases, upgrades will be more straightforward as differences
   will be smaller.
 - New features, when merged into master will at most sit 90 days on the
   shelf before hitting a release.
 - Bugfixes get out even faster.


We’ve written this up with more details on the wiki:

 - http://wiki.apache.org/couchdb/Roadmap_Process
 - http://wiki.apache.org/couchdb/Merge_Procedure

To allow for this, we had to streamline the release process, unfortunately
that has taken longer than expected, but Noah made good progress.

The test suite is still a place of contention, and if you would like to
help resolving this, please speak up.

* * *

Just to be clear, I am not trying to defend the status quo, I agree it sucks.

I just wanted to highlight the things we started doing to resolve this and
I realise that it is a bit of a mistake to have communicated this earlier.

Best
Jan
-- 




> On Wed, Mar 6, 2013 at 7:14 PM, Noah Slater <ns...@apache.org> wrote:
>> Thoughts?
> 
> It seems to me that a lot of this hinges on releases. Releases
> generate publicity and therefore developer interest, thus developer
> engagement. Releases make sure that work developers do gets into
> user's hands soon.
> 
> CouchDB, and I am sorry to have been banging this drum for a few years
> now, is the worst open source project at doing releases I know.
> 
> Here's a quote from this list, from February 13:
> 
>> I plan to cut a release candidate from the 1.3.x branch this weekend.
> 
> And another, from Nov 13:
> 
>> when we set out to ship 1.3.0, we thought the cors and docs
>> branches were just around the corner. That was a couple of
>> weeks ago. We are just starting with this, but the whole
>> idea of time-based releases is that we do not wait for
>> feature branches to be ready.
>> 
>> I’d like to propose that we ignore everything we’ve said
>> before and do the following:
>> 
>> - ship 1.3.0 as reflected in the 1.3.x branch now.
> 
> Here's another, from October 23:
> 
>> It's been a while since we released and I want to change this now. I
>> propose making a 1.3.x branch from today's master
>> (66529378dd06342929e04772370f3509cbe786a5) and building 1.3.0 there.
> 
> Another, from June 16:
> 
>> 1. We'd like to proposed formal time-based releases
> 
> Et cetera. As an example, the EventSource bits, which are actually
> something which would be very useful for us at work, were committed to
> the tree on May 16. They are yet to be released.
> 
> I feel a bit hypocritical for saying all this, since I don't
> contribute much to core CouchDB (although I did do a bunch of the doc
> conversion, try to be helpful around the edges and maintain
> couchdb-python). The reason for this is mostly that I thoroughly
> dislike Erlang as a language, and have no intention of learning it
> just to help out with CouchDB. I did some Futon work back in the day,
> but had lots of trouble getting it committed since no one's really
> owned Futon for a while (and right now, it looks like other people
> have that bit covered, though they're using a modern but also quite
> complex development stack that will make it unlikely for a lowly
> back-end web developer dude like me to be able to contribute much).
> 
> So, IMNSH and annoyingly harsh O, stop talking so much about how to
> energize the project, and Just. Ship. It. Every month, preferably. Fix
> the test suite and whatever else makes it so damn hard to release.
> Insert story here about that dude who realized people were trying to
> fix the wrong problem [1].
> 
> End of rant, cheers,
> 
> Dirkjan
> 
> [1] http://www.azarask.in/blog/post/the-wrong-problem/

Re: [DISCUSS] Goals for 2013

[Dirkjan Ochtman <di...@ochtman.nl>]

(warning, grumpy rant forthcoming...)

On Wed, Mar 6, 2013 at 7:14 PM, Noah Slater <ns...@apache.org> wrote:
> Thoughts?

It seems to me that a lot of this hinges on releases. Releases
generate publicity and therefore developer interest, thus developer
engagement. Releases make sure that work developers do gets into
user's hands soon.

CouchDB, and I am sorry to have been banging this drum for a few years
now, is the worst open source project at doing releases I know.

Here's a quote from this list, from February 13:

> I plan to cut a release candidate from the 1.3.x branch this weekend.

And another, from Nov 13:

> when we set out to ship 1.3.0, we thought the cors and docs
> branches were just around the corner. That was a couple of
> weeks ago. We are just starting with this, but the whole
> idea of time-based releases is that we do not wait for
> feature branches to be ready.
>
> I’d like to propose that we ignore everything we’ve said
> before and do the following:
>
>  - ship 1.3.0 as reflected in the 1.3.x branch now.

Here's another, from October 23:

> It's been a while since we released and I want to change this now. I
> propose making a 1.3.x branch from today's master
> (66529378dd06342929e04772370f3509cbe786a5) and building 1.3.0 there.

Another, from June 16:

> 1. We'd like to proposed formal time-based releases

Et cetera. As an example, the EventSource bits, which are actually
something which would be very useful for us at work, were committed to
the tree on May 16. They are yet to be released.

I feel a bit hypocritical for saying all this, since I don't
contribute much to core CouchDB (although I did do a bunch of the doc
conversion, try to be helpful around the edges and maintain
couchdb-python). The reason for this is mostly that I thoroughly
dislike Erlang as a language, and have no intention of learning it
just to help out with CouchDB. I did some Futon work back in the day,
but had lots of trouble getting it committed since no one's really
owned Futon for a while (and right now, it looks like other people
have that bit covered, though they're using a modern but also quite
complex development stack that will make it unlikely for a lowly
back-end web developer dude like me to be able to contribute much).

So, IMNSH and annoyingly harsh O, stop talking so much about how to
energize the project, and Just. Ship. It. Every month, preferably. Fix
the test suite and whatever else makes it so damn hard to release.
Insert story here about that dude who realized people were trying to
fix the wrong problem [1].

End of rant, cheers,

Dirkjan

[1] http://www.azarask.in/blog/post/the-wrong-problem/