You are viewing a plain text version of this content. The canonical link for it is here.
Posted to xindice-users@xml.apache.org by Daniel Kroeger <ze...@zesolo.net> on 2002/06/13 13:18:54 UTC
Collection size after deleting
Hello,
is it normal, that the size of a collection doesn't shrink if you delete
documents:
I have a collection which has 12.288 bytes after its creation.
Than I add 100.000 documents, make an XPath query and delete the documents.
But the collection stays at 437.067.776 bytes.............which is IMHO a
bit too much for an empty collection.
Daniel
Re: AW: Collection size after deleting
Posted by "Mark J. Stang" <ma...@earthlink.net>.
Not being a Xindice developer, I am a developer who uses Xindice,
my guess is, yes. The problem, as I see it, is that at some point during
the lifetime of the collection it makes sense to recover some of the
space. Assuming of course that you are not going to reuse it
right away. Hard for Xindice to know what your intentions are ;-).
So as I was saying, at some point, it is time to recover/garbage
collect the empty space. The first step is that it has to rebuild
the binary tree that is the collection, in a way to leave empty space
at the end of the file and then truncate the file. This has to be done
when you don't need the collection or it has to be done continually.
My advice would be to:
1) Eat the space (probably not acceptable)
2) Create a temporary collection, muck around with it and transfer the
documents to a permanent collection.
3) Every now and then, copy the contents to a new collection, delete the
old, and rename the new back. Hard to tell when this should happen...
HTH,
Mark
Daniel Kroeger wrote:
> OK, wonderfull........let's buy some new harddisks!
>
> If I empty the complete collection, I can delete it anyway and free the
> space.
> But what if just delete 250.000 documents out of 500.000.
> Do I have to live with 1 GB of unused but not useable space, or is there
> another possibilty?
>
> Thanks
> Daniel
>
> > -----Ursprungliche Nachricht-----
> > Von: Mark J. Stang [mailto:markstang@earthlink.net]
> > Gesendet: Donnerstag, 13. Juni 2002 16:29
> > An: xindice-users@xml.apache.org
> > Betreff: Re: Collection size after deleting
> >
> >
> > If I remember right, Xindice doesn't free space when you delete
> > documents.
> >
> > Mark
> >
> > Daniel Kroeger wrote:
> >
> > > Hello,
> > >
> > > is it normal, that the size of a collection doesn't shrink if you delete
> > > documents:
> > >
> > > I have a collection which has 12.288 bytes after its creation.
> > > Than I add 100.000 documents, make an XPath query and delete
> > the documents.
> > > But the collection stays at 437.067.776 bytes.............which
> > is IMHO a
> > > bit too much for an empty collection.
> > >
> > > Daniel
> >
> > --
> > Mark J Stang
> > Architect
> > Cybershop Systems
> >
> >
--
Mark J Stang
Architect
Cybershop Systems
AW: AW: Collection size after deleting
Posted by Daniel Kroeger <ze...@zesolo.net>.
Thanks..........I love those workarounds!
> -----Ursprungliche Nachricht-----
> Von: KOZLOV Roman [mailto:r-kozlov@opencascade.com]
> Gesendet: Donnerstag, 13. Juni 2002 17:13
> An: xindice-users@xml.apache.org
> Betreff: Re: AW: Collection size after deleting
>
>
> Hi Daniel,
>
> You can try export -> delete collection -> import.
>
> Roman
>
> Daniel Kroeger wrote:
>
> > OK, wonderfull........let's buy some new harddisks!
> >
> > If I empty the complete collection, I can delete it anyway and free the
> > space.
> > But what if just delete 250.000 documents out of 500.000.
> > Do I have to live with 1 GB of unused but not useable space, or is there
> > another possibilty?
> >
> > Thanks
> > Daniel
> >
> > > -----Ursprungliche Nachricht-----
> > > Von: Mark J. Stang [mailto:markstang@earthlink.net]
> > > Gesendet: Donnerstag, 13. Juni 2002 16:29
> > > An: xindice-users@xml.apache.org
> > > Betreff: Re: Collection size after deleting
> > >
> > >
> > > If I remember right, Xindice doesn't free space when you delete
> > > documents.
> > >
> > > Mark
> > >
> > > Daniel Kroeger wrote:
> > >
> > > > Hello,
> > > >
> > > > is it normal, that the size of a collection doesn't shrink
> if you delete
> > > > documents:
> > > >
> > > > I have a collection which has 12.288 bytes after its creation.
> > > > Than I add 100.000 documents, make an XPath query and delete
> > > the documents.
> > > > But the collection stays at 437.067.776 bytes.............which
> > > is IMHO a
> > > > bit too much for an empty collection.
> > > >
> > > > Daniel
> > >
> > > --
> > > Mark J Stang
> > > Architect
> > > Cybershop Systems
> > >
> > >
>
Re: AW: Collection size after deleting
Posted by KOZLOV Roman <r-...@opencascade.com>.
Hi Daniel,
You can try export -> delete collection -> import.
Roman
Daniel Kroeger wrote:
> OK, wonderfull........let's buy some new harddisks!
>
> If I empty the complete collection, I can delete it anyway and free the
> space.
> But what if just delete 250.000 documents out of 500.000.
> Do I have to live with 1 GB of unused but not useable space, or is there
> another possibilty?
>
> Thanks
> Daniel
>
> > -----Ursprungliche Nachricht-----
> > Von: Mark J. Stang [mailto:markstang@earthlink.net]
> > Gesendet: Donnerstag, 13. Juni 2002 16:29
> > An: xindice-users@xml.apache.org
> > Betreff: Re: Collection size after deleting
> >
> >
> > If I remember right, Xindice doesn't free space when you delete
> > documents.
> >
> > Mark
> >
> > Daniel Kroeger wrote:
> >
> > > Hello,
> > >
> > > is it normal, that the size of a collection doesn't shrink if you delete
> > > documents:
> > >
> > > I have a collection which has 12.288 bytes after its creation.
> > > Than I add 100.000 documents, make an XPath query and delete
> > the documents.
> > > But the collection stays at 437.067.776 bytes.............which
> > is IMHO a
> > > bit too much for an empty collection.
> > >
> > > Daniel
> >
> > --
> > Mark J Stang
> > Architect
> > Cybershop Systems
> >
> >
AW: Collection size after deleting
Posted by Daniel Kroeger <ze...@zesolo.net>.
OK, wonderfull........let's buy some new harddisks!
If I empty the complete collection, I can delete it anyway and free the
space.
But what if just delete 250.000 documents out of 500.000.
Do I have to live with 1 GB of unused but not useable space, or is there
another possibilty?
Thanks
Daniel
> -----Ursprungliche Nachricht-----
> Von: Mark J. Stang [mailto:markstang@earthlink.net]
> Gesendet: Donnerstag, 13. Juni 2002 16:29
> An: xindice-users@xml.apache.org
> Betreff: Re: Collection size after deleting
>
>
> If I remember right, Xindice doesn't free space when you delete
> documents.
>
> Mark
>
> Daniel Kroeger wrote:
>
> > Hello,
> >
> > is it normal, that the size of a collection doesn't shrink if you delete
> > documents:
> >
> > I have a collection which has 12.288 bytes after its creation.
> > Than I add 100.000 documents, make an XPath query and delete
> the documents.
> > But the collection stays at 437.067.776 bytes.............which
> is IMHO a
> > bit too much for an empty collection.
> >
> > Daniel
>
> --
> Mark J Stang
> Architect
> Cybershop Systems
>
>
Re: Collection size after deleting
Posted by "Mark J. Stang" <ma...@earthlink.net>.
If I remember right, Xindice doesn't free space when you delete
documents.
Mark
Daniel Kroeger wrote:
> Hello,
>
> is it normal, that the size of a collection doesn't shrink if you delete
> documents:
>
> I have a collection which has 12.288 bytes after its creation.
> Than I add 100.000 documents, make an XPath query and delete the documents.
> But the collection stays at 437.067.776 bytes.............which is IMHO a
> bit too much for an empty collection.
>
> Daniel
--
Mark J Stang
Architect
Cybershop Systems
AW: Collection size after deleting
Posted by Dirk Bromberg <br...@tzi.de>.
thx.
dirk
-----Ursprungliche Nachricht-----
Von: Daniel Kroger [mailto:ZeSolo@zesolo.net]
Gesendet: Freitag, 14. Juni 2002 16:07
An: xindice-users@xml.apache.org
Betreff: AW: Collection size after deleting
In my tests regarding this problem the space was recycled, which means, if
you add 1000 docs, delete 500, and add 500 again, the collection has the
same size than after adding the 1000 in the beginning.
Daniel
> -----Ursprungliche Nachricht-----
> Von: Dirk Bromberg [mailto:bromberg@tzi.de]
> Gesendet: Freitag, 14. Juni 2002 15:55
> An: xindice-users@xml.apache.org
> Betreff: AW: Collection size after deleting
>
>
> Hi,
>
> when I've a collection with 1000 resources in it and i'm deleting 500 of
> them -> the size will not reduced.
> when i'm adding 500 new resource to this collection will the
> space recycled
> ?? or will the collection getting bigger ?? and the space form the 500
> deleted resource is lost ??
>
> thanks
>
> Dirk Bromberg
> bromberg@tzi.de
>
>
>
>
>
> -----Ursprungliche Nachricht-----
> Von: Kimbro Staken [mailto:kstaken@xmldatabases.org]
> Gesendet: Donnerstag, 13. Juni 2002 21:55
> An: xindice-users@xml.apache.org
> Betreff: Re: Collection size after deleting
>
>
>
> On Thursday, June 13, 2002, at 04:18 AM, Daniel Kroeger wrote:
>
> > Hello,
> >
> > is it normal, that the size of a collection doesn't shrink if you delete
> > documents:
>
> Yes it's normal, maybe not good, but normal.
>
> >
> > I have a collection which has 12.288 bytes after its creation.
> > Than I add 100.000 documents, make an XPath query and delete the
> > documents.
> > But the collection stays at 437.067.776 bytes.............which
> is IMHO a
> > bit too much for an empty collection.
>
> And you would be correct. The file grows in chunks as needed, the size of
> the file is always going to be larger then the amount of data in the file.
> As the engine matures I'm sure it will make better use of the files.
>
> >
> > Daniel
> >
> >
> Kimbro Staken
> Java and XML Software, Consulting and Writing http://www.xmldatabases.org/
> Apache Xindice native XML database http://xml.apache.org/xindice
> XML:DB Initiative http://www.xmldb.org
>
>
AW: Collection size after deleting
Posted by Daniel Kroger <Ze...@zesolo.net>.
In my tests regarding this problem the space was recycled, which means, if
you add 1000 docs, delete 500, and add 500 again, the collection has the
same size than after adding the 1000 in the beginning.
Daniel
> -----Ursprungliche Nachricht-----
> Von: Dirk Bromberg [mailto:bromberg@tzi.de]
> Gesendet: Freitag, 14. Juni 2002 15:55
> An: xindice-users@xml.apache.org
> Betreff: AW: Collection size after deleting
>
>
> Hi,
>
> when I've a collection with 1000 resources in it and i'm deleting 500 of
> them -> the size will not reduced.
> when i'm adding 500 new resource to this collection will the
> space recycled
> ?? or will the collection getting bigger ?? and the space form the 500
> deleted resource is lost ??
>
> thanks
>
> Dirk Bromberg
> bromberg@tzi.de
>
>
>
>
>
> -----Ursprungliche Nachricht-----
> Von: Kimbro Staken [mailto:kstaken@xmldatabases.org]
> Gesendet: Donnerstag, 13. Juni 2002 21:55
> An: xindice-users@xml.apache.org
> Betreff: Re: Collection size after deleting
>
>
>
> On Thursday, June 13, 2002, at 04:18 AM, Daniel Kroeger wrote:
>
> > Hello,
> >
> > is it normal, that the size of a collection doesn't shrink if you delete
> > documents:
>
> Yes it's normal, maybe not good, but normal.
>
> >
> > I have a collection which has 12.288 bytes after its creation.
> > Than I add 100.000 documents, make an XPath query and delete the
> > documents.
> > But the collection stays at 437.067.776 bytes.............which
> is IMHO a
> > bit too much for an empty collection.
>
> And you would be correct. The file grows in chunks as needed, the size of
> the file is always going to be larger then the amount of data in the file.
> As the engine matures I'm sure it will make better use of the files.
>
> >
> > Daniel
> >
> >
> Kimbro Staken
> Java and XML Software, Consulting and Writing http://www.xmldatabases.org/
> Apache Xindice native XML database http://xml.apache.org/xindice
> XML:DB Initiative http://www.xmldb.org
>
>
AW: Collection size after deleting
Posted by Dirk Bromberg <br...@tzi.de>.
Hi,
when I've a collection with 1000 resources in it and i'm deleting 500 of
them -> the size will not reduced.
when i'm adding 500 new resource to this collection will the space recycled
?? or will the collection getting bigger ?? and the space form the 500
deleted resource is lost ??
thanks
Dirk Bromberg
bromberg@tzi.de
-----Ursprungliche Nachricht-----
Von: Kimbro Staken [mailto:kstaken@xmldatabases.org]
Gesendet: Donnerstag, 13. Juni 2002 21:55
An: xindice-users@xml.apache.org
Betreff: Re: Collection size after deleting
On Thursday, June 13, 2002, at 04:18 AM, Daniel Kroeger wrote:
> Hello,
>
> is it normal, that the size of a collection doesn't shrink if you delete
> documents:
Yes it's normal, maybe not good, but normal.
>
> I have a collection which has 12.288 bytes after its creation.
> Than I add 100.000 documents, make an XPath query and delete the
> documents.
> But the collection stays at 437.067.776 bytes.............which is IMHO a
> bit too much for an empty collection.
And you would be correct. The file grows in chunks as needed, the size of
the file is always going to be larger then the amount of data in the file.
As the engine matures I'm sure it will make better use of the files.
>
> Daniel
>
>
Kimbro Staken
Java and XML Software, Consulting and Writing http://www.xmldatabases.org/
Apache Xindice native XML database http://xml.apache.org/xindice
XML:DB Initiative http://www.xmldb.org
Re: Collection size after deleting
Posted by Kimbro Staken <ks...@xmldatabases.org>.
On Thursday, June 13, 2002, at 04:18 AM, Daniel Kroeger wrote:
> Hello,
>
> is it normal, that the size of a collection doesn't shrink if you delete
> documents:
Yes it's normal, maybe not good, but normal.
>
> I have a collection which has 12.288 bytes after its creation.
> Than I add 100.000 documents, make an XPath query and delete the
> documents.
> But the collection stays at 437.067.776 bytes.............which is IMHO a
> bit too much for an empty collection.
And you would be correct. The file grows in chunks as needed, the size of
the file is always going to be larger then the amount of data in the file.
As the engine matures I'm sure it will make better use of the files.
>
> Daniel
>
>
Kimbro Staken
Java and XML Software, Consulting and Writing http://www.xmldatabases.org/
Apache Xindice native XML database http://xml.apache.org/xindice
XML:DB Initiative http://www.xmldb.org