You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Joshua Bronson <ja...@gmail.com> on 2010/01/08 18:35:05 UTC

Re: 0.9 -> 0.10 view migration

On Tue, Dec 22, 2009 at 1:23 PM, Joshua Bronson <ja...@gmail.com> wrote:

> I just learned from Jan in #couchdb that 0.10 can read the 0.9 views also
> after running a simple migration to shift the header in the view file
>

It looks like this code is already written and committed! By Damien it looks
like. But apparently something is going wrong because when 0.10 reads a 0.9
view, a rebuild is caused rather than a simple header rewrite. Ticketed at
https://issues.apache.org/jira/browse/COUCHDB-616.

Re: 0.9 -> 0.10 view migration

Posted by Joshua Bronson <ja...@gmail.com>.
On Tue, Jan 19, 2010 at 6:39 PM, Joshua Bronson <ja...@gmail.com> wrote:

> On Tue, Jan 19, 2010 at 5:36 PM, Joshua Bronson <ja...@gmail.com>wrote:
>
>> we're going to give
>> http://github.com/kocolosk/couchdb/tree/0.9.2-and-view-compaction a shot
>>
>
> Uh oh, I just installed this branch, took down our system couch (0.9.0),
> got 0.9.2-and-view-compaction up and running (pointing at our live data),
> and browsed to a view in Futon, but no results. I checked the status page
> and the view group indexer is running. I expected to still be able to access
> the uncompacted views while couch was writing out the compacted ones. Did I
> misunderstand, Adam?
>

I looked in the logs to try to see what happened and put my findings up at
http://pylonshq.com/pasties/471b8769f43ceacd0999cfbc4c1355ef (annotations
underneath the excerpt).

I tried reproducing this behavior on a sandbox server (started a
0.9.2-and-v-c couch with a copy of 0.9.0 data and then browsed around in
Futon) but I could not get it to trigger the erroneous view reindex.
Triggering the view compaction via curl worked as expected.

I was, however, able to reproduce this when I tried reading the data back
from an 0.9.0 couch:

$ curl localhost:5984/melkjug  # (the live db)
> {"error":"error","reason":"{{badmatch,unknown_header_type},\n
> [{couch_db_updater,init,1},\n  {gen_server,init_it,6},\n
>  {proc_lib,init_p_do_apply,3}]}"}
>

See http://pylonshq.com/pasties/07ebe42c7a75af52e9da701a79779fbd

Hope this helps,
Josh

Re: 0.9 -> 0.10 view migration

Posted by Joshua Bronson <ja...@gmail.com>.
On Tue, Jan 19, 2010 at 6:39 PM, Joshua Bronson <ja...@gmail.com> wrote:

> On Tue, Jan 19, 2010 at 5:36 PM, Joshua Bronson <ja...@gmail.com>wrote:
>
>> we're going to give
>> http://github.com/kocolosk/couchdb/tree/0.9.2-and-view-compaction a shot
>>
>
> Uh oh, I just installed this branch, took down our system couch (0.9.0),
> got 0.9.2-and-view-compaction up and running (pointing at our live data),
> and browsed to a view in Futon, but no results. I checked the status page
> and the view group indexer is running. I expected to still be able to access
> the uncompacted views while couch was writing out the compacted ones. Did I
> misunderstand, Adam?
>

I looked in the logs to try to see what happened and put my findings up at
http://pylonshq.com/pasties/471b8769f43ceacd0999cfbc4c1355ef (annotations
underneath the excerpt).

I tried reproducing this behavior on a sandbox server (started a
0.9.2-and-v-c couch with a copy of 0.9.0 data and then browsed around in
Futon) but I could not get it to trigger the erroneous view reindex.
Triggering the view compaction via curl worked as expected.

I was, however, able to reproduce this when I tried reading the data back
from an 0.9.0 couch:

$ curl localhost:5984/melkjug  # (the live db)
> {"error":"error","reason":"{{badmatch,unknown_header_type},\n
> [{couch_db_updater,init,1},\n  {gen_server,init_it,6},\n
>  {proc_lib,init_p_do_apply,3}]}"}
>

See http://pylonshq.com/pasties/07ebe42c7a75af52e9da701a79779fbd

Hope this helps,
Josh

Re: 0.9 -> 0.10 view migration

Posted by Chris Anderson <jc...@apache.org>.
On Tue, Jan 19, 2010 at 3:39 PM, Joshua Bronson <ja...@gmail.com> wrote:
> On Tue, Jan 19, 2010 at 5:36 PM, Joshua Bronson <ja...@gmail.com> wrote:
>
>> we're going to give
>> http://github.com/kocolosk/couchdb/tree/0.9.2-and-view-compaction a shot
>>
>
> Uh oh, I just installed this branch, took down our system couch (0.9.0), got
> 0.9.2-and-view-compaction up and running (pointing at our live data), and
> browsed to a view in Futon, but no results. I checked the status page and
> the view group indexer is running. I expected to still be able to access the
> uncompacted views while couch was writing out the compacted ones. Did I
> misunderstand, Adam?

The view group indexer shouldn't be running during compaction. It
sounds like something triggered a full view update.

It's hard to know what would do that from here (and without digging
into the sources).

If you still have that 0.10.0 replica laying around you might do well
to pick it back up. Once those db headers are upgraded you *might* be
able to go back down but not without writing custom code. (Did it
myself between 0.10 and a dev branch the other day.)

I didn't realize you were working with such big data that you couldn't
take backups.

We should have some documentation about how to do these tricky upgrades safely.

Let us know if we can help any more.

>
> I tried killing 0.9.2, started 0.9.0 back up again, but then:
>
> $ curl localhost:5984/melkjug  # (the live db)
> {"error":"error","reason":"{{badmatch,unknown_header_type},\n
> [{couch_db_updater,init,1},\n  {gen_server,init_it,6},\n
>  {proc_lib,init_p_do_apply,3}]}"}
>
> I thought the 0.9.2-and-view-compaction backport was supposed to preserve
> the header so it could still be read by 0.9.0. Again, did I misunderstand?
>
> I brought 0.9.2 back up and let it continue doing the compaction, but
> currently our site is down in the meantime. If there's any way to get couch
> to serve results from the uncompacted views while compaction is running,
> please holler back soon.
>
> Thanks,
> Josh
>



-- 
Chris Anderson
http://jchrisa.net
http://couch.io

Re: 0.9 -> 0.10 view migration

Posted by Joshua Bronson <ja...@gmail.com>.
On Tue, Jan 19, 2010 at 5:36 PM, Joshua Bronson <ja...@gmail.com> wrote:

> we're going to give
> http://github.com/kocolosk/couchdb/tree/0.9.2-and-view-compaction a shot
>

Uh oh, I just installed this branch, took down our system couch (0.9.0), got
0.9.2-and-view-compaction up and running (pointing at our live data), and
browsed to a view in Futon, but no results. I checked the status page and
the view group indexer is running. I expected to still be able to access the
uncompacted views while couch was writing out the compacted ones. Did I
misunderstand, Adam?

I tried killing 0.9.2, started 0.9.0 back up again, but then:

$ curl localhost:5984/melkjug  # (the live db)
{"error":"error","reason":"{{badmatch,unknown_header_type},\n
[{couch_db_updater,init,1},\n  {gen_server,init_it,6},\n
 {proc_lib,init_p_do_apply,3}]}"}

I thought the 0.9.2-and-view-compaction backport was supposed to preserve
the header so it could still be read by 0.9.0. Again, did I misunderstand?

I brought 0.9.2 back up and let it continue doing the compaction, but
currently our site is down in the meantime. If there's any way to get couch
to serve results from the uncompacted views while compaction is running,
please holler back soon.

Thanks,
Josh

Re: 0.9 -> 0.10 view migration

Posted by Joshua Bronson <ja...@gmail.com>.
On Tue, Jan 19, 2010 at 5:36 PM, Joshua Bronson <ja...@gmail.com> wrote:

> we're going to give
> http://github.com/kocolosk/couchdb/tree/0.9.2-and-view-compaction a shot
>

Uh oh, I just installed this branch, took down our system couch (0.9.0), got
0.9.2-and-view-compaction up and running (pointing at our live data), and
browsed to a view in Futon, but no results. I checked the status page and
the view group indexer is running. I expected to still be able to access the
uncompacted views while couch was writing out the compacted ones. Did I
misunderstand, Adam?

I tried killing 0.9.2, started 0.9.0 back up again, but then:

$ curl localhost:5984/melkjug  # (the live db)
{"error":"error","reason":"{{badmatch,unknown_header_type},\n
[{couch_db_updater,init,1},\n  {gen_server,init_it,6},\n
 {proc_lib,init_p_do_apply,3}]}"}

I thought the 0.9.2-and-view-compaction backport was supposed to preserve
the header so it could still be read by 0.9.0. Again, did I misunderstand?

I brought 0.9.2 back up and let it continue doing the compaction, but
currently our site is down in the meantime. If there's any way to get couch
to serve results from the uncompacted views while compaction is running,
please holler back soon.

Thanks,
Josh

Re: 0.9 -> 0.10 view migration

Posted by Joshua Bronson <ja...@gmail.com>.
On Tue, Jan 19, 2010 at 6:38 PM, Chris Anderson <jc...@apache.org> wrote:

> Thanks for the update. Damien's been busy with a cross-country move so
> he hasn't been able to look at this.
>

I figured as much, no worries. Hope the move's going well, Damien.

To avoid downtime you can replicate to 0.10 and build views, then
> switch over your processes to use it.
>

We actually tried this, got all the way through replicating to 0.10 and
partway through the initial view creation before it filled up the 200G nfs
mount we were replicating to. We just simply don't have enough space to
support this. Being able to just migrate the header successfully would be a
big win.

Re: 0.9 -> 0.10 view migration

Posted by Joshua Bronson <ja...@gmail.com>.
On Tue, Jan 19, 2010 at 6:38 PM, Chris Anderson <jc...@apache.org> wrote:

> Thanks for the update. Damien's been busy with a cross-country move so
> he hasn't been able to look at this.
>

I figured as much, no worries. Hope the move's going well, Damien.

To avoid downtime you can replicate to 0.10 and build views, then
> switch over your processes to use it.
>

We actually tried this, got all the way through replicating to 0.10 and
partway through the initial view creation before it filled up the 200G nfs
mount we were replicating to. We just simply don't have enough space to
support this. Being able to just migrate the header successfully would be a
big win.

Re: 0.9 -> 0.10 view migration

Posted by Chris Anderson <jc...@apache.org>.
On Tue, Jan 19, 2010 at 2:36 PM, Joshua Bronson <ja...@gmail.com> wrote:
> On Fri, Jan 8, 2010 at 12:35 PM, Joshua Bronson <ja...@gmail.com> wrote:
>
>> On Tue, Dec 22, 2009 at 1:23 PM, Joshua Bronson <ja...@gmail.com>wrote:
>>
>>> I just learned from Jan in #couchdb that 0.10 can read the 0.9 views also
>>> after running a simple migration to shift the header in the view file
>>>
>>
>> It looks like this code is already written and committed! By Damien it
>> looks like. But apparently something is going wrong because when 0.10 reads
>> a 0.9 view, a rebuild is caused rather than a simple header rewrite.
>> Ticketed at https://issues.apache.org/jira/browse/COUCHDB-616.
>>
>
> Just wanted to send an update: Since our disk is steadily filling up and
> there's been no progress yet with #616, we're going to give
> http://github.com/kocolosk/couchdb/tree/0.9.2-and-view-compaction a shot.
> Hopefully we still have enough space to hold the compacted views alongside
> the uncompacted ones while compaction is in progress. If not, we'll just
> have to delete and rebuild them while our site is down. :(
>
> Looking forward to seeing some work on view space and time
> efficiency<http://issues.apache.org/jira/browse/COUCHDB-623>,
> and contributing as much as I can.
>

Thanks for the update. Damien's been busy with a cross-country move so
he hasn't been able to look at this.

To avoid downtime you can replicate to 0.10 and build views, then
switch over your processes to use it.

Chris


-- 
Chris Anderson
http://jchrisa.net
http://couch.io

Re: 0.9 -> 0.10 view migration

Posted by Chris Anderson <jc...@apache.org>.
On Tue, Jan 19, 2010 at 2:36 PM, Joshua Bronson <ja...@gmail.com> wrote:
> On Fri, Jan 8, 2010 at 12:35 PM, Joshua Bronson <ja...@gmail.com> wrote:
>
>> On Tue, Dec 22, 2009 at 1:23 PM, Joshua Bronson <ja...@gmail.com>wrote:
>>
>>> I just learned from Jan in #couchdb that 0.10 can read the 0.9 views also
>>> after running a simple migration to shift the header in the view file
>>>
>>
>> It looks like this code is already written and committed! By Damien it
>> looks like. But apparently something is going wrong because when 0.10 reads
>> a 0.9 view, a rebuild is caused rather than a simple header rewrite.
>> Ticketed at https://issues.apache.org/jira/browse/COUCHDB-616.
>>
>
> Just wanted to send an update: Since our disk is steadily filling up and
> there's been no progress yet with #616, we're going to give
> http://github.com/kocolosk/couchdb/tree/0.9.2-and-view-compaction a shot.
> Hopefully we still have enough space to hold the compacted views alongside
> the uncompacted ones while compaction is in progress. If not, we'll just
> have to delete and rebuild them while our site is down. :(
>
> Looking forward to seeing some work on view space and time
> efficiency<http://issues.apache.org/jira/browse/COUCHDB-623>,
> and contributing as much as I can.
>

Thanks for the update. Damien's been busy with a cross-country move so
he hasn't been able to look at this.

To avoid downtime you can replicate to 0.10 and build views, then
switch over your processes to use it.

Chris


-- 
Chris Anderson
http://jchrisa.net
http://couch.io

Re: 0.9 -> 0.10 view migration

Posted by Joshua Bronson <ja...@gmail.com>.
On Fri, Jan 8, 2010 at 12:35 PM, Joshua Bronson <ja...@gmail.com> wrote:

> On Tue, Dec 22, 2009 at 1:23 PM, Joshua Bronson <ja...@gmail.com>wrote:
>
>> I just learned from Jan in #couchdb that 0.10 can read the 0.9 views also
>> after running a simple migration to shift the header in the view file
>>
>
> It looks like this code is already written and committed! By Damien it
> looks like. But apparently something is going wrong because when 0.10 reads
> a 0.9 view, a rebuild is caused rather than a simple header rewrite.
> Ticketed at https://issues.apache.org/jira/browse/COUCHDB-616.
>

Just wanted to send an update: Since our disk is steadily filling up and
there's been no progress yet with #616, we're going to give
http://github.com/kocolosk/couchdb/tree/0.9.2-and-view-compaction a shot.
Hopefully we still have enough space to hold the compacted views alongside
the uncompacted ones while compaction is in progress. If not, we'll just
have to delete and rebuild them while our site is down. :(

Looking forward to seeing some work on view space and time
efficiency<http://issues.apache.org/jira/browse/COUCHDB-623>,
and contributing as much as I can.

Re: 0.9 -> 0.10 view migration

Posted by Joshua Bronson <ja...@gmail.com>.
On Fri, Jan 8, 2010 at 12:35 PM, Joshua Bronson <ja...@gmail.com> wrote:

> On Tue, Dec 22, 2009 at 1:23 PM, Joshua Bronson <ja...@gmail.com>wrote:
>
>> I just learned from Jan in #couchdb that 0.10 can read the 0.9 views also
>> after running a simple migration to shift the header in the view file
>>
>
> It looks like this code is already written and committed! By Damien it
> looks like. But apparently something is going wrong because when 0.10 reads
> a 0.9 view, a rebuild is caused rather than a simple header rewrite.
> Ticketed at https://issues.apache.org/jira/browse/COUCHDB-616.
>

Just wanted to send an update: Since our disk is steadily filling up and
there's been no progress yet with #616, we're going to give
http://github.com/kocolosk/couchdb/tree/0.9.2-and-view-compaction a shot.
Hopefully we still have enough space to hold the compacted views alongside
the uncompacted ones while compaction is in progress. If not, we'll just
have to delete and rebuild them while our site is down. :(

Looking forward to seeing some work on view space and time
efficiency<http://issues.apache.org/jira/browse/COUCHDB-623>,
and contributing as much as I can.