You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Donovan Aaron <do...@bah.com> on 2005/12/29 19:17:13 UTC
JE Directory/XA Transactions
I recently ported Andi Vajda's DbDirectory to the Java Edition of
Berkeley DB. The main reason being the JCA connector and XA
transactions. Initial results are great and shown below. I'm new to
contributing. What is the procedure for making this code available?
JEDirectory
Writing files byte by byte
1453 total milliseconds to create, 5214 kb/s
722 total milliseconds to read, 10494 kb/s
80 total milliseconds to delete
2255 total milliseconds
Writing files as one byte array
1032 total milliseconds to create, 7918 kb/s
311 total milliseconds to read, 26274 kb/s
60 total milliseconds to delete
1403 total milliseconds
DbDirectory
Writing files byte by byte
9879 total milliseconds to create, 766 kb/s
601 total milliseconds to read, 12607 kb/s
9188 total milliseconds to delete
19668 total milliseconds
Writing files as one byte array
12304 total milliseconds to create, 664 kb/s
281 total milliseconds to read, 29079 kb/s
9689 total milliseconds to delete
22274 total milliseconds
Regards,
Aaron Donovan
Re: JE Directory/XA Transactions
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
Aaron, this is great news!
As for contributing, the best thing to do is follow along with the
conventions set up in the contrib/db area. Maybe your contribution
could co-exist with that area? If it does make sense for it to co-
exist with DbDirectory, then put your files in your local copy there
getting all the build and dependency stuff taken care of, make a .zip
of the new files and a patch (svn diff > patchfile) for any changed
files. If it makes more sense for your contribution to be in a
separate contrib area, create that locally (perhaps contrib/bdbje?)
and create a .zip of it. Ensure all your files are licensed with the
Apache Software License, then attach your files to a new JIRA issue.
Thanks,
Erik
On Dec 29, 2005, at 1:17 PM, Donovan Aaron wrote:
> I recently ported Andi Vajda's DbDirectory to the Java Edition of
> Berkeley DB. The main reason being the JCA connector and XA
> transactions. Initial results are great and shown below. I'm new to
> contributing. What is the procedure for making this code available?
>
> JEDirectory
>
> Writing files byte by byte
> 1453 total milliseconds to create, 5214 kb/s
> 722 total milliseconds to read, 10494 kb/s
> 80 total milliseconds to delete
> 2255 total milliseconds
> Writing files as one byte array
> 1032 total milliseconds to create, 7918 kb/s
> 311 total milliseconds to read, 26274 kb/s
> 60 total milliseconds to delete
> 1403 total milliseconds
>
>
> DbDirectory
>
> Writing files byte by byte
> 9879 total milliseconds to create, 766 kb/s
> 601 total milliseconds to read, 12607 kb/s
> 9188 total milliseconds to delete
> 19668 total milliseconds
> Writing files as one byte array
> 12304 total milliseconds to create, 664 kb/s
> 281 total milliseconds to read, 29079 kb/s
> 9689 total milliseconds to delete
> 22274 total milliseconds
>
> Regards,
> Aaron Donovan
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
RE: JE Directory/XA Transactions
Posted by Robert Engels <re...@ix.netcom.com>.
I think JE transactions are held completely in memory, so this may be an
issue - although I have not reviewed your implementation yet... :)
-----Original Message-----
From: Andi Vajda [mailto:vajda@osafoundation.org]
Sent: Thursday, December 29, 2005 5:14 PM
To: java-dev@lucene.apache.org
Subject: Re: JE Directory/XA Transactions
On Thu, 29 Dec 2005, Donovan Aaron wrote:
> I recently ported Andi Vajda's DbDirectory to the Java Edition of
> Berkeley DB. The main reason being the JCA connector and XA
> transactions. Initial results are great and shown below. I'm new to
> contributing. What is the procedure for making this code available?
If you attach the Apache 2.0 license, I'd be glad to add your files
alongside
the C DB based implementation, since I'm currently the maintainer of the
'db'
Java Lucene contrib area.
Your timings look great but did you run DbDirectory in DB_AUTO_COMMIT mode
or
did you bracket the operations with your own transaction ? (it should make a
big difference to use a transaction for the whole indexing operation,
including IndexWriter.optimize(), instead of running in DB_AUTO_COMMIT mode,
where each write call gets its own transaction).
Andi..
>
> JEDirectory
>
> Writing files byte by byte
> 1453 total milliseconds to create, 5214 kb/s
> 722 total milliseconds to read, 10494 kb/s
> 80 total milliseconds to delete
> 2255 total milliseconds
> Writing files as one byte array
> 1032 total milliseconds to create, 7918 kb/s
> 311 total milliseconds to read, 26274 kb/s
> 60 total milliseconds to delete
> 1403 total milliseconds
>
>
> DbDirectory
>
> Writing files byte by byte
> 9879 total milliseconds to create, 766 kb/s
> 601 total milliseconds to read, 12607 kb/s
> 9188 total milliseconds to delete
> 19668 total milliseconds
> Writing files as one byte array
> 12304 total milliseconds to create, 664 kb/s
> 281 total milliseconds to read, 29079 kb/s
> 9689 total milliseconds to delete
> 22274 total milliseconds
>
> Regards,
> Aaron Donovan
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: JE Directory/XA Transactions
Posted by Andi Vajda <va...@osafoundation.org>.
On Thu, 29 Dec 2005, Donovan Aaron wrote:
> I recently ported Andi Vajda's DbDirectory to the Java Edition of
> Berkeley DB. The main reason being the JCA connector and XA
> transactions. Initial results are great and shown below. I'm new to
> contributing. What is the procedure for making this code available?
If you attach the Apache 2.0 license, I'd be glad to add your files alongside
the C DB based implementation, since I'm currently the maintainer of the 'db'
Java Lucene contrib area.
Your timings look great but did you run DbDirectory in DB_AUTO_COMMIT mode or
did you bracket the operations with your own transaction ? (it should make a
big difference to use a transaction for the whole indexing operation,
including IndexWriter.optimize(), instead of running in DB_AUTO_COMMIT mode,
where each write call gets its own transaction).
Andi..
>
> JEDirectory
>
> Writing files byte by byte
> 1453 total milliseconds to create, 5214 kb/s
> 722 total milliseconds to read, 10494 kb/s
> 80 total milliseconds to delete
> 2255 total milliseconds
> Writing files as one byte array
> 1032 total milliseconds to create, 7918 kb/s
> 311 total milliseconds to read, 26274 kb/s
> 60 total milliseconds to delete
> 1403 total milliseconds
>
>
> DbDirectory
>
> Writing files byte by byte
> 9879 total milliseconds to create, 766 kb/s
> 601 total milliseconds to read, 12607 kb/s
> 9188 total milliseconds to delete
> 19668 total milliseconds
> Writing files as one byte array
> 12304 total milliseconds to create, 664 kb/s
> 281 total milliseconds to read, 29079 kb/s
> 9689 total milliseconds to delete
> 22274 total milliseconds
>
> Regards,
> Aaron Donovan
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org