You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Chris Anderson <jc...@apache.org> on 2009/08/17 22:58:56 UTC
Ready for 0.10 Release?
Couchers,
We've been sprinting extra hard for the last week to get a lot of new
power into CouchDB. I think we're close on Windows support, we just
got native view servers, we've added a bunch of robustness around
external process handling. There's a bunch more that's happened since
0.9 but the point of this thread is not to list the changes, but to
see if there are patches that people feel should go in now, so they
make the 0.10 release. Hopefully this is a short list, as I think most
of the last-minute patching has been done, but if something's
important and easy, we shouldn't forget it.
Best,
Chris
--
Chris Anderson
http://jchrisa.net
http://couch.io
Re: Ready for 0.10 Release?
Posted by Paul Davis <pa...@gmail.com>.
On Mon, Aug 17, 2009 at 4:58 PM, Chris Anderson<jc...@apache.org> wrote:
> Couchers,
>
> We've been sprinting extra hard for the last week to get a lot of new
> power into CouchDB. I think we're close on Windows support, we just
> got native view servers, we've added a bunch of robustness around
> external process handling. There's a bunch more that's happened since
> 0.9 but the point of this thread is not to list the changes, but to
> see if there are patches that people feel should go in now, so they
> make the 0.10 release. Hopefully this is a short list, as I think most
> of the last-minute patching has been done, but if something's
> important and easy, we shouldn't forget it.
>
> Best,
> Chris
>
I'd like to see the stats patch go in before 0.10. Bob Dionne went
over it this morning and didn't report finding any errors. I'd like to
swing through and add a couple more tests to double check the stat
calculations tonight or tomorrow. Once that's done I'll be satisfied
that its ready.
Paul
Re: Ready for 0.10 Release?
Posted by Robert Newson <ro...@gmail.com>.
Do ensure you're using the latest patch (the 3rd one I've posted to
the jira ticket) as it uses short, random leaps between ids to prevent
monotonicity (the concern being that ids are too guessable). i noticed
no reduction in improvement with that over strictly monotonic
increments, but I'd appreciate a double check.
B.
On Mon, Aug 17, 2009 at 11:14 PM, Robert
Dionne<di...@dionne-associates.com> wrote:
> I've tested with this quite a bit now and it's much faster, in some cases 4
> times, but it is sensitive to both doc size and batch sizes so in some cases
> the speed may only be about 4-10% better. It's definitely much improved in
> the file footprint resulting in faster compaction also.
>
> It's interesting to note that a good amount of the speed improvement comes
> from a reduced number of calls to couch_util:to_hex because it's only used
> for the prefix and not for every id as it is in the new_uuid() case. In one
> of the hovercraft tests I ran this method was cut from 36% to 25% of the run
> time. You can see this using eprof. This method is also used in the rev
> stuff so it's a good candidate to look at for optimization.
>
> So I"m +1 on this, but I think the other devs ought to noodle it some to
> double check it conceptually, .ie. that we have global uniqueness and it
> won't present issues elsewhere. Definitely a nice hack.
>
> Cheers,
>
> Bob
>
>
>
> On Aug 17, 2009, at 5:18 PM, Robert Newson wrote:
>
>> I was hoping for more feedback on COUCHDB-465 as the alternate uuid
>> generation algorithm there seems to double insertion speed and reduces
>> the need to run compaction.
>>
>> B.
>>
>> https://issues.apache.org/jira/browse/COUCHDB-465
>>
>> On Mon, Aug 17, 2009 at 10:10 PM, Noah Slater<ns...@apache.org> wrote:
>>>
>>> On Mon, Aug 17, 2009 at 01:58:56PM -0700, Chris Anderson wrote:
>>>>
>>>> We've been sprinting extra hard for the last week to get a lot of new
>>>> power into CouchDB. I think we're close on Windows support, we just
>>>> got native view servers, we've added a bunch of robustness around
>>>> external process handling. There's a bunch more that's happened since
>>>> 0.9 but the point of this thread is not to list the changes, but to
>>>> see if there are patches that people feel should go in now, so they
>>>> make the 0.10 release. Hopefully this is a short list, as I think most
>>>> of the last-minute patching has been done, but if something's
>>>> important and easy, we shouldn't forget it.
>>>
>>> I don't want to be the damp squid[1] here, but for now, shipping with the
>>> recent
>>> changes for Windows, or at least claiming Windows support, will get my -0
>>> or -1
>>> vote depending on how convincing people's arguments get.
>>>
>>> I wasn't even aware that this had properly landed yet, and it would
>>> certainly
>>> need to be simmered in trunk for a few months, with a wide call for
>>> testing,
>>> before I would be happy shipping it.
>>>
>>> [1] http://eggcorns.lascribe.net/english/416/squid/
>>>
>>> Best,
>>>
>>> --
>>> Noah Slater, http://tumbolia.org/nslater
>>>
>
>
Re: Ready for 0.10 Release?
Posted by Robert Dionne <di...@dionne-associates.com>.
I've tested with this quite a bit now and it's much faster, in some
cases 4 times, but it is sensitive to both doc size and batch sizes so
in some cases the speed may only be about 4-10% better. It's
definitely much improved in the file footprint resulting in faster
compaction also.
It's interesting to note that a good amount of the speed improvement
comes from a reduced number of calls to couch_util:to_hex because it's
only used for the prefix and not for every id as it is in the
new_uuid() case. In one of the hovercraft tests I ran this method was
cut from 36% to 25% of the run time. You can see this using eprof.
This method is also used in the rev stuff so it's a good candidate to
look at for optimization.
So I"m +1 on this, but I think the other devs ought to noodle it some
to double check it conceptually, .ie. that we have global uniqueness
and it won't present issues elsewhere. Definitely a nice hack.
Cheers,
Bob
On Aug 17, 2009, at 5:18 PM, Robert Newson wrote:
> I was hoping for more feedback on COUCHDB-465 as the alternate uuid
> generation algorithm there seems to double insertion speed and reduces
> the need to run compaction.
>
> B.
>
> https://issues.apache.org/jira/browse/COUCHDB-465
>
> On Mon, Aug 17, 2009 at 10:10 PM, Noah Slater<ns...@apache.org>
> wrote:
>> On Mon, Aug 17, 2009 at 01:58:56PM -0700, Chris Anderson wrote:
>>> We've been sprinting extra hard for the last week to get a lot of
>>> new
>>> power into CouchDB. I think we're close on Windows support, we just
>>> got native view servers, we've added a bunch of robustness around
>>> external process handling. There's a bunch more that's happened
>>> since
>>> 0.9 but the point of this thread is not to list the changes, but to
>>> see if there are patches that people feel should go in now, so they
>>> make the 0.10 release. Hopefully this is a short list, as I think
>>> most
>>> of the last-minute patching has been done, but if something's
>>> important and easy, we shouldn't forget it.
>>
>> I don't want to be the damp squid[1] here, but for now, shipping
>> with the recent
>> changes for Windows, or at least claiming Windows support, will get
>> my -0 or -1
>> vote depending on how convincing people's arguments get.
>>
>> I wasn't even aware that this had properly landed yet, and it would
>> certainly
>> need to be simmered in trunk for a few months, with a wide call for
>> testing,
>> before I would be happy shipping it.
>>
>> [1] http://eggcorns.lascribe.net/english/416/squid/
>>
>> Best,
>>
>> --
>> Noah Slater, http://tumbolia.org/nslater
>>
Re: Ready for 0.10 Release?
Posted by Adam Kocoloski <ko...@apache.org>.
On Aug 17, 2009, at 5:23 PM, Paul Davis wrote:
> On Mon, Aug 17, 2009 at 5:18 PM, Robert
> Newson<ro...@gmail.com> wrote:
>> I was hoping for more feedback on COUCHDB-465 as the alternate uuid
>> generation algorithm there seems to double insertion speed and
>> reduces
>> the need to run compaction.
>>
>> B.
>>
>> https://issues.apache.org/jira/browse/COUCHDB-465
>
> Thanks for reminding me. I'll put this on the white board to review.
> Assuming its as simple as I'm guessing I'd be +1 for pushing to 0.10.
+1 from me too. We don't need all the crazy configurable options for
custom UUID generators or anything like that. This algorithm is just
flat-out better than what we're using right now.
Adam
Re: Ready for 0.10 Release?
Posted by Paul Davis <pa...@gmail.com>.
On Mon, Aug 17, 2009 at 5:18 PM, Robert Newson<ro...@gmail.com> wrote:
> I was hoping for more feedback on COUCHDB-465 as the alternate uuid
> generation algorithm there seems to double insertion speed and reduces
> the need to run compaction.
>
> B.
>
> https://issues.apache.org/jira/browse/COUCHDB-465
Thanks for reminding me. I'll put this on the white board to review.
Assuming its as simple as I'm guessing I'd be +1 for pushing to 0.10.
Paul Davis
Re: Ready for 0.10 Release?
Posted by Robert Newson <ro...@gmail.com>.
I was hoping for more feedback on COUCHDB-465 as the alternate uuid
generation algorithm there seems to double insertion speed and reduces
the need to run compaction.
B.
https://issues.apache.org/jira/browse/COUCHDB-465
On Mon, Aug 17, 2009 at 10:10 PM, Noah Slater<ns...@apache.org> wrote:
> On Mon, Aug 17, 2009 at 01:58:56PM -0700, Chris Anderson wrote:
>> We've been sprinting extra hard for the last week to get a lot of new
>> power into CouchDB. I think we're close on Windows support, we just
>> got native view servers, we've added a bunch of robustness around
>> external process handling. There's a bunch more that's happened since
>> 0.9 but the point of this thread is not to list the changes, but to
>> see if there are patches that people feel should go in now, so they
>> make the 0.10 release. Hopefully this is a short list, as I think most
>> of the last-minute patching has been done, but if something's
>> important and easy, we shouldn't forget it.
>
> I don't want to be the damp squid[1] here, but for now, shipping with the recent
> changes for Windows, or at least claiming Windows support, will get my -0 or -1
> vote depending on how convincing people's arguments get.
>
> I wasn't even aware that this had properly landed yet, and it would certainly
> need to be simmered in trunk for a few months, with a wide call for testing,
> before I would be happy shipping it.
>
> [1] http://eggcorns.lascribe.net/english/416/squid/
>
> Best,
>
> --
> Noah Slater, http://tumbolia.org/nslater
>
Re: Ready for 0.10 Release?
Posted by Noah Slater <ns...@apache.org>.
On Mon, Aug 17, 2009 at 05:21:30PM -0400, Paul Davis wrote:
> > I wasn't even aware that this had properly landed yet, and it would certainly
> > need to be simmered in trunk for a few months, with a wide call for testing,
> > before I would be happy shipping it.
>
> I'm pretty sure it hasn't landed quite yet, but Mark appears to be
> narrowing in on getting all of the build issues sorted. Is your
> concern mostly about testing the changes to the build system or
> testing the Windows build products? I could see putting in the
> necessary bits but not claiming full windows compatibility until its
> been in and tested over the duration of a release cycle or some such.
Changes like this should simmer in trunk for a few months.
I probably would ask the same for anything big.
I don't like the idea of rushing in loads of changes and then BOOM release.
That's a good way to ship lots of buggy code!
Thanks,
--
Noah Slater, http://tumbolia.org/nslater
Re: Ready for 0.10 Release?
Posted by Paul Davis <pa...@gmail.com>.
> I wasn't even aware that this had properly landed yet, and it would certainly
> need to be simmered in trunk for a few months, with a wide call for testing,
> before I would be happy shipping it.
I'm pretty sure it hasn't landed quite yet, but Mark appears to be
narrowing in on getting all of the build issues sorted. Is your
concern mostly about testing the changes to the build system or
testing the Windows build products? I could see putting in the
necessary bits but not claiming full windows compatibility until its
been in and tested over the duration of a release cycle or some such.
Paul Davis
Re: Ready for 0.10 Release?
Posted by Noah Slater <ns...@apache.org>.
On Mon, Aug 17, 2009 at 01:58:56PM -0700, Chris Anderson wrote:
> We've been sprinting extra hard for the last week to get a lot of new
> power into CouchDB. I think we're close on Windows support, we just
> got native view servers, we've added a bunch of robustness around
> external process handling. There's a bunch more that's happened since
> 0.9 but the point of this thread is not to list the changes, but to
> see if there are patches that people feel should go in now, so they
> make the 0.10 release. Hopefully this is a short list, as I think most
> of the last-minute patching has been done, but if something's
> important and easy, we shouldn't forget it.
I don't want to be the damp squid[1] here, but for now, shipping with the recent
changes for Windows, or at least claiming Windows support, will get my -0 or -1
vote depending on how convincing people's arguments get.
I wasn't even aware that this had properly landed yet, and it would certainly
need to be simmered in trunk for a few months, with a wide call for testing,
before I would be happy shipping it.
[1] http://eggcorns.lascribe.net/english/416/squid/
Best,
--
Noah Slater, http://tumbolia.org/nslater
Re: Ready for 0.10 Release?
Posted by Chris Widhelm <cw...@contego.net>.
+1
On Aug 17, 2009, at 4:39 PM, Noah Slater wrote:
> On Mon, Aug 17, 2009 at 02:30:26PM -0700, Chris Anderson wrote:
>> btw I'd be +1 on putting the windows stuff in 0.10.x, we won't know
>> until we try it how well it works. We can say Windows support is
>> alpha.
>
> I'm quite prepared for someone to make a really good argument that
> convinces me
> otherwise, but for now, I don't think this is how our releases
> should work. A
> release is not our way of saying to the wider community "here's all
> the new
> untested shit for you to break your system with" no matter how many
> alpha, beta,
> or similar Greek letters we shove in the NEWS file, which, and lets
> face it, no
> one bothers to read anyway. Subversion HEAD is the proper place to
> let things
> simmer and get fixed as bug reports come in. A release is our way of
> packaging
> up all the reasonably mature stuff and saying "here's some probably
> stable
> features for you to start using."
>
> Best,
>
> --
> Noah Slater, http://tumbolia.org/nslater
Re: Ready for 0.10 Release?
Posted by Noah Slater <ns...@apache.org>.
On Mon, Aug 17, 2009 at 04:26:13PM -0700, Damien Katz wrote:
> Then lets do the branch so we can start letting it simmer. Otherwise,
> the only way we can do a release is to have no new features checked into
> trunk for X weeks or months until we are sure everything else stable,
> which is unworkable.
Agreed.
--
Noah Slater, http://tumbolia.org/nslater
Re: Ready for 0.10 Release?
Posted by Damien Katz <da...@apache.org>.
On Aug 17, 2009, at 2:39 PM, Noah Slater wrote:
> On Mon, Aug 17, 2009 at 02:30:26PM -0700, Chris Anderson wrote:
>> btw I'd be +1 on putting the windows stuff in 0.10.x, we won't know
>> until we try it how well it works. We can say Windows support is
>> alpha.
>
> I'm quite prepared for someone to make a really good argument that
> convinces me
> otherwise, but for now, I don't think this is how our releases
> should work. A
> release is not our way of saying to the wider community "here's all
> the new
> untested shit for you to break your system with" no matter how many
> alpha, beta,
> or similar Greek letters we shove in the NEWS file, which, and lets
> face it, no
> one bothers to read anyway. Subversion HEAD is the proper place to
> let things
> simmer and get fixed as bug reports come in. A release is our way of
> packaging
> up all the reasonably mature stuff and saying "here's some probably
> stable
> features for you to start using."
Then lets do the branch so we can start letting it simmer. Otherwise,
the only way we can do a release is to have no new features checked
into trunk for X weeks or months until we are sure everything else
stable, which is unworkable.
-Damien
>
> Best,
>
> --
> Noah Slater, http://tumbolia.org/nslater
Re: Ready for 0.10 Release?
Posted by Paul Davis <pa...@gmail.com>.
> I'm quite prepared for someone to make a really good argument that convinces me
> otherwise, but for now, I don't think this is how our releases should work.
I definitely agree with your sentiment that we shouldn't just be
pushing untested features into releases and letting the user community
deal with any bugs that turn up, but I don't think that letting a
patch "simmer" is the right answer either. If you're want to only
include mature patches into a release, then we should settle on a way
to measure maturity.
The easiest method I can think of is making sure that features are
well tested before included into a release. I'd say that in general
we're not too bad on the testing front. There are a couple places that
we need to work on, and hopefully when the build bot instance comes up
I'll be able to start adding more tools to help us keep track of our
code base.
That said, the specific Windows patch mostly affects the build system
and introduces a bit of Windows specific code. Once that patch lands I
reckon its going to be pretty obvious whether something broke or not.
And if we introduce a non-obvious bug the fastest way I can think of
triggering it would be to make a release.
So basically, "Release Early, Release Often" [1].
HTH,
Paul Davis
[1] http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/ar01s04.html
Re: Ready for 0.10 Release?
Posted by Noah Slater <ns...@apache.org>.
On Mon, Aug 17, 2009 at 02:30:26PM -0700, Chris Anderson wrote:
> btw I'd be +1 on putting the windows stuff in 0.10.x, we won't know
> until we try it how well it works. We can say Windows support is
> alpha.
I'm quite prepared for someone to make a really good argument that convinces me
otherwise, but for now, I don't think this is how our releases should work. A
release is not our way of saying to the wider community "here's all the new
untested shit for you to break your system with" no matter how many alpha, beta,
or similar Greek letters we shove in the NEWS file, which, and lets face it, no
one bothers to read anyway. Subversion HEAD is the proper place to let things
simmer and get fixed as bug reports come in. A release is our way of packaging
up all the reasonably mature stuff and saying "here's some probably stable
features for you to start using."
Best,
--
Noah Slater, http://tumbolia.org/nslater
Re: Ready for 0.10 Release?
Posted by Chris Anderson <jc...@apache.org>.
On Mon, Aug 17, 2009 at 2:16 PM, Adam Kocoloski<ko...@apache.org> wrote:
> On Aug 17, 2009, at 4:58 PM, Chris Anderson wrote:
>
>> Couchers,
>>
>> We've been sprinting extra hard for the last week to get a lot of new
>> power into CouchDB. I think we're close on Windows support, we just
>> got native view servers, we've added a bunch of robustness around
>> external process handling. There's a bunch more that's happened since
>> 0.9 but the point of this thread is not to list the changes, but to
>> see if there are patches that people feel should go in now, so they
>> make the 0.10 release. Hopefully this is a short list, as I think most
>> of the last-minute patching has been done, but if something's
>> important and easy, we shouldn't forget it.
>
> Hi Chris, I thought we were going to branch 0.10.x but wait at least a few
> weeks for more testing and bugfixes before releasing 0.10.0. So much code
> has been flying around ...
>
> Adam
>
That's probably smart, just avoid the easy fixes that would otherwise
end up on 0.10.1
But good to see some feedback here about things that should /
shouldn't go into 0.10.x
btw I'd be +1 on putting the windows stuff in 0.10.x, we won't know
until we try it how well it works. We can say Windows support is
alpha.
Chris
--
Chris Anderson
http://jchrisa.net
http://couch.io
Re: Ready for 0.10 Release?
Posted by Adam Kocoloski <ko...@apache.org>.
On Aug 17, 2009, at 4:58 PM, Chris Anderson wrote:
> Couchers,
>
> We've been sprinting extra hard for the last week to get a lot of new
> power into CouchDB. I think we're close on Windows support, we just
> got native view servers, we've added a bunch of robustness around
> external process handling. There's a bunch more that's happened since
> 0.9 but the point of this thread is not to list the changes, but to
> see if there are patches that people feel should go in now, so they
> make the 0.10 release. Hopefully this is a short list, as I think most
> of the last-minute patching has been done, but if something's
> important and easy, we shouldn't forget it.
Hi Chris, I thought we were going to branch 0.10.x but wait at least a
few weeks for more testing and bugfixes before releasing 0.10.0. So
much code has been flying around ...
Adam
Re: Ready for 0.10 Release?
Posted by Dirkjan Ochtman <di...@ochtman.nl>.
On Tue, Aug 18, 2009 at 01:34, Jan Lehnardt<ja...@apache.org> wrote:
> I'd like the history patch to be simmering in the 0.10.x branch.
I'm working on another patch to provide view options in Futon. Would
it be possible to include that for 0.10 as well?
Also, I wonder if my HATEOAS patches ever went anywhere (I put them in
the tracker, don't remember where right now). Basically, it would be
nice if it was possible to find all the "special" URLs from the root
without having to know any beforehand.
Cheers,
Dirkjan
Re: Ready for 0.10 Release?
Posted by Jan Lehnardt <ja...@apache.org>.
Fair enough, thanks Damien, happy vacation! :)
Cheers
Jan
--
On 18 Aug 2009, at 01:42, Damien Katz wrote:
> I'd like to review the design and patch first, when I get back from
> vacation.
>
> -Damien
>
> On Aug 17, 2009, at 4:34 PM, Jan Lehnardt wrote:
>
>> I'd like the history patch to be simmering in the 0.10.x branch.
>>
>> Jason?
>>
>> Cheers
>> Jan
>> --
>>
>> On 17 Aug 2009, at 22:58, Chris Anderson wrote:
>>
>>> Couchers,
>>>
>>> We've been sprinting extra hard for the last week to get a lot of
>>> new
>>> power into CouchDB. I think we're close on Windows support, we just
>>> got native view servers, we've added a bunch of robustness around
>>> external process handling. There's a bunch more that's happened
>>> since
>>> 0.9 but the point of this thread is not to list the changes, but to
>>> see if there are patches that people feel should go in now, so they
>>> make the 0.10 release. Hopefully this is a short list, as I think
>>> most
>>> of the last-minute patching has been done, but if something's
>>> important and easy, we shouldn't forget it.
>>>
>>> Best,
>>> Chris
>>>
>>>
>>>
>>> --
>>> Chris Anderson
>>> http://jchrisa.net
>>> http://couch.io
>>>
>>
>
>
Re: Ready for 0.10 Release?
Posted by Damien Katz <da...@apache.org>.
I'd like to review the design and patch first, when I get back from
vacation.
-Damien
On Aug 17, 2009, at 4:34 PM, Jan Lehnardt wrote:
> I'd like the history patch to be simmering in the 0.10.x branch.
>
> Jason?
>
> Cheers
> Jan
> --
>
> On 17 Aug 2009, at 22:58, Chris Anderson wrote:
>
>> Couchers,
>>
>> We've been sprinting extra hard for the last week to get a lot of new
>> power into CouchDB. I think we're close on Windows support, we just
>> got native view servers, we've added a bunch of robustness around
>> external process handling. There's a bunch more that's happened since
>> 0.9 but the point of this thread is not to list the changes, but to
>> see if there are patches that people feel should go in now, so they
>> make the 0.10 release. Hopefully this is a short list, as I think
>> most
>> of the last-minute patching has been done, but if something's
>> important and easy, we shouldn't forget it.
>>
>> Best,
>> Chris
>>
>>
>>
>> --
>> Chris Anderson
>> http://jchrisa.net
>> http://couch.io
>>
>
Re: Ready for 0.10 Release?
Posted by Jan Lehnardt <ja...@apache.org>.
I'd like the history patch to be simmering in the 0.10.x branch.
Jason?
Cheers
Jan
--
On 17 Aug 2009, at 22:58, Chris Anderson wrote:
> Couchers,
>
> We've been sprinting extra hard for the last week to get a lot of new
> power into CouchDB. I think we're close on Windows support, we just
> got native view servers, we've added a bunch of robustness around
> external process handling. There's a bunch more that's happened since
> 0.9 but the point of this thread is not to list the changes, but to
> see if there are patches that people feel should go in now, so they
> make the 0.10 release. Hopefully this is a short list, as I think most
> of the last-minute patching has been done, but if something's
> important and easy, we shouldn't forget it.
>
> Best,
> Chris
>
>
>
> --
> Chris Anderson
> http://jchrisa.net
> http://couch.io
>