using XSLT to generate SQL

[ishmael <is...@zendostudios.com>]

Hello all,

I'm new to both this list, XSL, and Xalan so forgive any mistakes or
ignorance on my part.

The Problem:
Basically, I have a large amount of data that exists in an XML format (w/
DTD). I need to get the XML data into a relational database which has an
existing schema. From what I've read so far it seems like I can use XSLT to
generate SQL statements that then get sent to the database via JDBC or
whatever. The real problem is the amount of data I have to deal with. To
start off there will be ~ 10 million records followed by weekly updates of ~
10k records or so. Each XML file will have to generate many INSERT, UPDATE,
and DELETE SQL statements all wrapped in a transaction, so that the whole
file can be rejected if something goes wrong.

Question #1: Is Xalan or even XSLT right for this job? It seems like this is
a fairly common problem there must be existing tools.

Question #2: Does anyone have any comments or suggestions for dealing with
large data sets and databases? Optimizations, things to watch out for and
what not. Does Xalan have to load the whole XML file into memory in order to
work with it?

Question #3: Another potential problem is that while parsing the document we
won't know if an entity should generate an UPDATE or an INSERT without first
checking the database to see if that record already exists. How will Xalan
handle this?


Thanks in advance for the abundance of wisdom, charity, and encouragement
that I'm about to receive from the good people of Xalan land. =)


-ishmael