You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Branko Čibej <br...@xbc.nu> on 2002/07/25 17:49:24 UTC
custom sorts? [was: dumper bug? need help.]
Ben Collins-Sussman wrote:
>This user sent me a copy of his repository. But I can't read the DB
>files on my Intel machine, due to endianness problems. The user then
>sent me dump files of his six tables, produced by 'db_dump'. However,
>I can't 'db_load' those -- I run out of locks and memory when I try
>(even when I set the maximum locks in DB_CONFIG from 2000 to 20000!).
>Cmpilato points out that db_loading probably wouldn't work anyway,
>since we depend on custom sorting routines which db_load wouldn't be
>using.
>
>
What?! We needed custom sorting in the Bad Old Days, to sort the node
IDs so that relatedness checks were faster. In these Enlightened Times
of the New Schema, no such customization should be necessary. Please
tell me that we're really not using custom sort algorithms any more, so
that db_dump and db_load can work.
Otherwise we'll need another repository reload before 1.0.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: custom sorts? [was: dumper bug? need help.]
Posted by Branko Čibej <br...@xbc.nu>.
cmpilato@collab.net wrote:
>Actually, this isn't true. I know because I tested it.
>
>I did:
>
> db_dump -kp nodes | grep -e '^ [0-9a-z]\+' > out.txt
> db_dump -kp nodes | grep -e '^ [0-9a-z]\+' | sort > out-sorted.txt
> diff -u out.txt out-sorted.txt
>
>And I got differences. Consider:
>
> [out.txt]
> 0.0.0
> 0.0.1
> 0.0.2
> 0.0.3
> ...
>
> [out-sorted.txt]
> 0.0.0
> 0.0.1
> 0.0.10
> 0.0.100
> ...
>
>
Wah. Of course. What a pity.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: custom sorts? [was: dumper bug? need help.]
Posted by cm...@collab.net.
=?UTF-8?B?QnJhbmtvIMSMaWJlag==?= <br...@xbc.nu> writes:
> cmpilato@collab.net wrote:
>
> >The custom sorting is still present. We sort by NODE_ID, then
> >COPY_ID, then TXN_ID. But it occurs to me that all we really need
> >nowadays is the `next-id' for determining the NODE-ID (just like the
> >other tables). The only thing that expects the sorted table is the
> >guy who generates new NODE-IDs.
> >
> >
>
> If the id is "[0-9a-z]\.[0-9a-z]\.[0-9a-z]", with the proper (node_id,
> copy_id, txn_id) order, then the default lexical sort will work the same
> way. +1 for getting rid of the custom sort, and using next-id.
Actually, this isn't true. I know because I tested it.
I did:
db_dump -kp nodes | grep -e '^ [0-9a-z]\+' > out.txt
db_dump -kp nodes | grep -e '^ [0-9a-z]\+' | sort > out-sorted.txt
diff -u out.txt out-sorted.txt
And I got differences. Consider:
[out.txt]
0.0.0
0.0.1
0.0.2
0.0.3
...
[out-sorted.txt]
0.0.0
0.0.1
0.0.10
0.0.100
...
> With a bit of luck, the change may not even require a reload. But even
> if it does, IMNSHO we should try to support the stock db_dump/db_load.
No such luck.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: custom sorts? [was: dumper bug? need help.]
Posted by Branko Čibej <br...@xbc.nu>.
cmpilato@collab.net wrote:
>The custom sorting is still present. We sort by NODE_ID, then
>COPY_ID, then TXN_ID. But it occurs to me that all we really need
>nowadays is the `next-id' for determining the NODE-ID (just like the
>other tables). The only thing that expects the sorted table is the
>guy who generates new NODE-IDs.
>
>
If the id is "[0-9a-z]\.[0-9a-z]\.[0-9a-z]", with the proper (node_id,
copy_id, txn_id) order, then the default lexical sort will work the same
way. +1 for getting rid of the custom sort, and using next-id.
With a bit of luck, the change may not even require a reload. But even
if it does, IMNSHO we should try to support the stock db_dump/db_load.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: custom sorts? [was: dumper bug? need help.]
Posted by cm...@collab.net.
=?UTF-8?B?QnJhbmtvIMSMaWJlag==?= <br...@xbc.nu> writes:
> Ben Collins-Sussman wrote:
>
> >This user sent me a copy of his repository. But I can't read the DB
> >files on my Intel machine, due to endianness problems. The user then
> >sent me dump files of his six tables, produced by 'db_dump'. However,
> >I can't 'db_load' those -- I run out of locks and memory when I try
> >(even when I set the maximum locks in DB_CONFIG from 2000 to 20000!).
> >Cmpilato points out that db_loading probably wouldn't work anyway,
> >since we depend on custom sorting routines which db_load wouldn't be
> >using.
> >
> >
>
> What?! We needed custom sorting in the Bad Old Days, to sort the node
> IDs so that relatedness checks were faster. In these Enlightened Times
> of the New Schema, no such customization should be necessary. Please
> tell me that we're really not using custom sort algorithms any more, so
> that db_dump and db_load can work.
>
> Otherwise we'll need another repository reload before 1.0.
The custom sorting is still present. We sort by NODE_ID, then
COPY_ID, then TXN_ID. But it occurs to me that all we really need
nowadays is the `next-id' for determining the NODE-ID (just like the
other tables). The only thing that expects the sorted table is the
guy who generates new NODE-IDs.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org