You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Tito Ciuro <tc...@mac.com> on 2014/02/14 03:09:50 UTC
Why is the view index file size so huge?
Hello,
I have imported a total of 7 million documents into my sample database. The size is now 4.46GB. For reference, I have created the view using the following function (formatted for clarity):
{
"_id": "_design/test",
"views": {
"cityPath-byRating": {
"map": "function(doc) {
if (doc.type == 'test' && doc.rating && doc.testPath) {
emit([doc.testPath + ' ' + doc.rating, doc._id], null);
}
}"
}
}
}
A sample key being shown in futon looks like this:
> ["Country.United States.San Francisco 100", "cf6ed60fb2e8b6d96122da7b0d1a26f2"]
> ID: cf6ed60fb2e8b6d96122da7b0d1a26f2
To my surprise, the final view file size (found in /Users/tito/Library/Application Support/CouchDB/var/lib/couchdb/.example_design/mrview) is now 15.12GB! Four times the database size!? For one view alone? Is this expected or am I doing something really silly? By the way, the UUID type configured is 'sequential'.
Any ideas?
Thanks,
-- Tito
Re: Why is the view index file size so huge?
Posted by Robert Samuel Newson <rn...@apache.org>.
Leaving a note here for future mailing list readers, view compaction is a separate operation from database compaction.
B.
On 14 Feb 2014, at 11:23, Andy Wenk <an...@nms.de> wrote:
> keep in mind, that all revisions of documents are being kept as long as you
> do not run compaction. So you need to run compaction to throw away the old
> revisions. So what you saw is the normal behaviour of CouchDB.
>
> http://docs.couchdb.org/en/latest/maintenance/compaction.html?highlight=compaction
>
> http://docs.couchdb.org/en/latest/config/compaction.html?highlight=compaction
>
> Cheers
>
> Andy
>
>
> On 14 February 2014 03:12, Tito Ciuro <tc...@mac.com> wrote:
>
>> More news!
>>
>> I have finalized compacting the view and the size sits now at 781.9MB!
>> Sheesh! Now we're taking! I hope it helps someone having the same issue!
>>
>> </relieved>
>>
>> -- Tito
>>
>> On Feb 13, 2014, at 6:09 PM, Tito Ciuro <tc...@mac.com> wrote:
>>
>>> Hello,
>>>
>>> I have imported a total of 7 million documents into my sample database.
>> The size is now 4.46GB. For reference, I have created the view using the
>> following function (formatted for clarity):
>>>
>>> {
>>> "_id": "_design/test",
>>> "views": {
>>> "cityPath-byRating": {
>>> "map": "function(doc) {
>>> if (doc.type == 'test' && doc.rating &&
>> doc.testPath) {
>>> emit([doc.testPath + ' ' +
>> doc.rating, doc._id], null);
>>> }
>>> }"
>>> }
>>> }
>>> }
>>>
>>> A sample key being shown in futon looks like this:
>>>
>>>> ["Country.United States.San Francisco 100",
>> "cf6ed60fb2e8b6d96122da7b0d1a26f2"]
>>>> ID: cf6ed60fb2e8b6d96122da7b0d1a26f2
>>>
>>>
>>> To my surprise, the final view file size (found in
>> /Users/tito/Library/Application
>> Support/CouchDB/var/lib/couchdb/.example_design/mrview) is now 15.12GB!
>> Four times the database size!? For one view alone? Is this expected or am I
>> doing something really silly? By the way, the UUID type configured is
>> 'sequential'.
>>>
>>> Any ideas?
>>>
>>> Thanks,
>>>
>>> -- Tito
>>
>>
>
>
> --
> Andy Wenk
> Hamburg - Germany
> RockIt!
>
> http://www.couchdb-buch.de
> http://www.pg-praxisbuch.de
>
> GPG fingerprint: C044 8322 9E12 1483 4FEC 9452 B65D 6BE3 9ED3 9588
>
> https://people.apache.org/keys/committer/andywenk.asc
Re: Why is the view index file size so huge?
Posted by Andy Wenk <an...@nms.de>.
keep in mind, that all revisions of documents are being kept as long as you
do not run compaction. So you need to run compaction to throw away the old
revisions. So what you saw is the normal behaviour of CouchDB.
http://docs.couchdb.org/en/latest/maintenance/compaction.html?highlight=compaction
http://docs.couchdb.org/en/latest/config/compaction.html?highlight=compaction
Cheers
Andy
On 14 February 2014 03:12, Tito Ciuro <tc...@mac.com> wrote:
> More news!
>
> I have finalized compacting the view and the size sits now at 781.9MB!
> Sheesh! Now we're taking! I hope it helps someone having the same issue!
>
> </relieved>
>
> -- Tito
>
> On Feb 13, 2014, at 6:09 PM, Tito Ciuro <tc...@mac.com> wrote:
>
> > Hello,
> >
> > I have imported a total of 7 million documents into my sample database.
> The size is now 4.46GB. For reference, I have created the view using the
> following function (formatted for clarity):
> >
> > {
> > "_id": "_design/test",
> > "views": {
> > "cityPath-byRating": {
> > "map": "function(doc) {
> > if (doc.type == 'test' && doc.rating &&
> doc.testPath) {
> > emit([doc.testPath + ' ' +
> doc.rating, doc._id], null);
> > }
> > }"
> > }
> > }
> > }
> >
> > A sample key being shown in futon looks like this:
> >
> >> ["Country.United States.San Francisco 100",
> "cf6ed60fb2e8b6d96122da7b0d1a26f2"]
> >> ID: cf6ed60fb2e8b6d96122da7b0d1a26f2
> >
> >
> > To my surprise, the final view file size (found in
> /Users/tito/Library/Application
> Support/CouchDB/var/lib/couchdb/.example_design/mrview) is now 15.12GB!
> Four times the database size!? For one view alone? Is this expected or am I
> doing something really silly? By the way, the UUID type configured is
> 'sequential'.
> >
> > Any ideas?
> >
> > Thanks,
> >
> > -- Tito
>
>
--
Andy Wenk
Hamburg - Germany
RockIt!
http://www.couchdb-buch.de
http://www.pg-praxisbuch.de
GPG fingerprint: C044 8322 9E12 1483 4FEC 9452 B65D 6BE3 9ED3 9588
https://people.apache.org/keys/committer/andywenk.asc
Re: Why is the view index file size so huge?
Posted by Tito Ciuro <tc...@mac.com>.
More news!
I have finalized compacting the view and the size sits now at 781.9MB! Sheesh! Now we're taking! I hope it helps someone having the same issue!
</relieved>
-- Tito
On Feb 13, 2014, at 6:09 PM, Tito Ciuro <tc...@mac.com> wrote:
> Hello,
>
> I have imported a total of 7 million documents into my sample database. The size is now 4.46GB. For reference, I have created the view using the following function (formatted for clarity):
>
> {
> "_id": "_design/test",
> "views": {
> "cityPath-byRating": {
> "map": "function(doc) {
> if (doc.type == 'test' && doc.rating && doc.testPath) {
> emit([doc.testPath + ' ' + doc.rating, doc._id], null);
> }
> }"
> }
> }
> }
>
> A sample key being shown in futon looks like this:
>
>> ["Country.United States.San Francisco 100", "cf6ed60fb2e8b6d96122da7b0d1a26f2"]
>> ID: cf6ed60fb2e8b6d96122da7b0d1a26f2
>
>
> To my surprise, the final view file size (found in /Users/tito/Library/Application Support/CouchDB/var/lib/couchdb/.example_design/mrview) is now 15.12GB! Four times the database size!? For one view alone? Is this expected or am I doing something really silly? By the way, the UUID type configured is 'sequential'.
>
> Any ideas?
>
> Thanks,
>
> -- Tito