You are viewing a plain text version of this content. The canonical link for it is here.
Posted to xindice-dev@xml.apache.org by Gary Hallmark <Ga...@oracle.com> on 2002/01/09 20:27:46 UTC
pluggable storage layer for Xindice?
Do you have any thoughts, requirements, (or code) for plugging in an
alternate storage layer, say an RDBMS like mySQL or Oracle, in place of or
alongside the filer/indexer layers in Xindice?
Some benefits might include
- transactions
- online backup
- pushing some xpath subset down to the storage layer for evaluation, if
supported (Oracle9i supports an xpath subset, e.g.)
Where would be good places to "plug in" to Xindice, and what are the
potential gotchas?
Thanks!
Re: pluggable storage layer for Xindice?
Posted by Tom Bradford <br...@dbxmlgroup.com>.
On Wednesday, January 9, 2002, at 02:24 PM, Gary Hallmark wrote:
> Inconsistency can happen if there should be a SW or HW crash or
> transaction rollback.
> Databases support many indexing schemes -- better to keep the index and
> data in the same resource manager.
True, but an index is a disposable data structure that can be recreated
from scratch so long as the original data is available.
--
Tom Bradford - http://www.tbradford.org
Developer - Apache Xindice (Native XML Database)
Creator - Project Labrador (XML Object Broker)
Re: pluggable storage layer for Xindice?
Posted by Gary Hallmark <Ga...@oracle.com>.
Inconsistency can happen if there should be a SW or HW crash or transaction
rollback.
Databases support many indexing schemes -- better to keep the index and
data in the same resource manager.
At 02:01 PM 1/9/02 -0700, you wrote:
>On Wednesday, January 9, 2002, at 01:46 PM, Bertrand Delacretaz wrote:
>>What about a mixed architecture: Filer in an SQL database, and btree indexes?
>>Seems to me like this (provided a index rebuild utility is available or
>>created) would take advantage of the database's robustness with minimal
>>effort. Any thoughts?
>
>This is certainly possible, and if Xindice were the only client updating
>the database, the indexes would remain up to date as well (without manual
>rebuilds). It'd be nice to have a way to schedule rebuilds of BTree
>indexes for situations where the underlying relational database may change
>outside of Xindice's control.
>
>--
>Tom Bradford - http://www.tbradford.org
>Developer - Apache Xindice (Native XML Database)
>Creator - Project Labrador (XML Object Broker)
>
Re: pluggable storage layer for Xindice?
Posted by Gianugo Rabellino <gi...@apache.org>.
Bertrand Delacretaz wrote:
> This mixed storage sounds interesting for our project that requires maximal
> reliabilty of the data storage layer - I'll try to explore this further, and
> possibly contribute a JDBCFiler if we're successful.
You might want to explore how eXist (http://exist.sourceforge.net) did
the trick of storing XML content in a JDBC storage. Too bad eXist is
LGPL... maybe Wolfgang might be interested in sharing code and maybe
change the licence, given also that he's using XIndice's core filer for
its native backend. :)
Ciao,
Re: pluggable storage layer for Xindice?
Posted by Bertrand Delacretaz <bd...@codeconsult.ch>.
On Wednesday 09 January 2002 22:01, Tom Bradford wrote:
> . . .It'd be nice to have a way to schedule rebuilds of
> BTree indexes for situations where the underlying relational database
> may change outside of Xindice's control.
I tought about rebuilding indexes more for cases where they would get
corrupted, assuming this could happen with the current BTree indexes (in a
system crash for example).
This mixed storage sounds interesting for our project that requires maximal
reliabilty of the data storage layer - I'll try to explore this further, and
possibly contribute a JDBCFiler if we're successful.
--
-- Bertrand Delacrétaz, www.codeconsult.ch
-- web technologies consultant - OO, Java, XML, C++
Re: pluggable storage layer for Xindice?
Posted by Tom Bradford <br...@dbxmlgroup.com>.
On Wednesday, January 9, 2002, at 01:46 PM, Bertrand Delacretaz wrote:
> What about a mixed architecture: Filer in an SQL database, and btree
> indexes?
> Seems to me like this (provided a index rebuild utility is available or
> created) would take advantage of the database's robustness with minimal
> effort. Any thoughts?
This is certainly possible, and if Xindice were the only client updating
the database, the indexes would remain up to date as well (without
manual rebuilds). It'd be nice to have a way to schedule rebuilds of
BTree indexes for situations where the underlying relational database
may change outside of Xindice's control.
--
Tom Bradford - http://www.tbradford.org
Developer - Apache Xindice (Native XML Database)
Creator - Project Labrador (XML Object Broker)
Re: pluggable storage layer for Xindice?
Posted by Bertrand Delacretaz <bd...@codeconsult.ch>.
On Wednesday 09 January 2002 20:52, Tom Bradford wrote:
(about an alternate storage layer)
> This is already a part of the architecture. Look at
> org.apache.xindice.core.filer.Filer for a start.
After a first look at Filer, it seems fairly easy to write a
JDBCFiler that would take advantage of an underlying database's robustness
and online backup features (using collection path + Key.toString() as the
key, and storing Value as a binary object?).
If I understand right, however, an Indexer would also have to be created to
store the indexes in the database - at first look this seems a bit more
complicated, if I got it right the Indexer needs to implement index queries
in addition to storage?
What about a mixed architecture: Filer in an SQL database, and btree indexes?
Seems to me like this (provided a index rebuild utility is available or
created) would take advantage of the database's robustness with minimal
effort. Any thoughts?
--
-- Bertrand Delacrétaz, www.codeconsult.ch
-- web technologies consultant - OO, Java, XML, C++
Re: pluggable storage layer for Xindice?
Posted by Tom Bradford <br...@dbxmlgroup.com>.
On Wednesday, January 9, 2002, at 12:27 PM, Gary Hallmark wrote:
> Do you have any thoughts, requirements, (or code) for plugging in an
> alternate storage layer, say an RDBMS like mySQL or Oracle, in place of
> or alongside the filer/indexer layers in Xindice?
This is already a part of the architecture. Look at
org.apache.xindice.core.filer.Filer for a start. At one point, I wrote
an RDBMS mapping Filer as well as a Filer that layered on top of
Borland's JDataStore. There are also three additional filers included
with Xindice, a FSFiler for File System layering, a MemFiler for
in-memory temporary collections, and an experimental HashFiler that does
hashed file access instead of BTree.
The only major gotcha will be whether or not you want to emit text or
the dbXML compressed format. If it's only going to be text, you
shouldn't run into any major problems. If it's compressed, you'll have
to deal with the DOM Compression classes.
--
Tom Bradford - http://www.tbradford.org
Developer - Apache Xindice (Native XML Database)
Creator - Project Labrador (XML Object Broker)