[Fwd: Re: [J2] Vote: Torque schema generation problems - how to proceed?]

[Ate Douma <at...@douma.nu>]

I'm forwarding the reply from Henning because his is probably stuck in the moderators queue.
So expect the original message to arrive sometime later.

Ate

-------- Original Message --------
Subject: Re: [J2] Vote: Torque schema generation problems - how to proceed?
Date: Thu, 9 Dec 2004 11:58:51 +0000 (UTC)
From: Henning P. Schmiedehausen <hp...@intermeta.de>

Ate Douma <at...@douma.nu> writes:

>Henning P. Schmiedehausen wrote:

>> Chris Custine <ch...@gmail.com> writes:
>> 
>> 
>>>I think you may want to consider these items in the context of fixing
>>>the total nightmare of using torque-gen with Postgres as well.  I have
>>>posted several times about the hoops I jump through to get JS2 and
>>>Postgres to work together, and your suggestions look like a good
>>>opportunity to fix those problems with torque-gen and Postgres.  To be
>>>clear, no single project is at fault (Postgres, Torque, or JS2), but
>>>some minor bad choices in each are making Postgres support really
>>>crappy right now in JS2.
>> 
>> 
>> The Torque people right over there at torque-dev@db.apache.org would
>> be really happy to know what you consider "total nightmare".
>> 
>> (Personally I'm running a PostgreSQL based Project using sequences as
>> ID generators with ~4000 tables in 52 schemas inside a database quite
>> successful. All tables are generated an maintained by Torque and
>> accessed through Torque and Hibernate).
>> 
>> (Folks, if you have trouble with Torque and/or improvements or wishes,
>> LET US KNOW!). Not many Torque people scan the lists of <arbitrary
>> project>. Just come over to torque-dev or torque-user and tell us. We
>> are not only willing to listen, we are even willing to act and improve
>> our code to cater the needs of our users. That's what a project is all
>> about.
>Henning, thanks for the response and for scanning our list. Seems we aren't just
>an arbitrary project to you at least ;-)

I scan jetspeed-* because it is built on Turbine (at least 1.5) and I
still would like to find time (or someone that pays for it. ;-) ) to
work more with Jetspeed.

>In my initial message, http://nagoya.apache.org/eyebrowse/ReadMsg?listId=22&msgNo=19387,
>I described a few problems we are encountering with Torque right now.

Just read it.

>As far as I can see, Torque really shines where its Pear support is needed
>(like in Jetspeed-1). But, for our Jetspeed-2 requirements which mainly concern
>mapping an abstract schema definition and inserting initial configuration data,
>the current torque generator features are not really sufficient, nor flexible enough.

Yes. The static peers and the fixed connection of "table -> object"
really hurts (an example: In one of my projects I have a lot of tables
that all have the same structure but describe different things. I'd
really love to define the mapping between object and table only once
and then just say "table a maps to class foo. table b maps to class
foo. table c maps to class foo". I ended up with having all classes
that descibe a, b and c implement the same interface and just work
through the interfaces.

>After reviewing the generator and its templates I found the following lacking or missing:
>- separate generation and/or suppression of drop table statements

Yep. the drops should be turn-off-able using a property. They are not.

>- order of drop, create table statements causing conflicting result for
>   different database:
>   - for hsql a detail table must be generated (thus dropped) before the
>     parent table otherwise a fk constraint violation is thrown
>   - for mysql (also with innoDB) the foreign keys are generated inline and thus
>     changing the order doesn't help: we need to drop the child table (or constraints)
>     by hand, something Torque doesn't help us with

This is difficult to achieve IMHO because the sequence of the tables
is controlled by the Control.vm template and does not vary for the
different databases. It _should_ be possible to suppress foreign key
checking temporarily thus allowing arbitrary creation sequences with
some databases but I am no real wizard here. This needs more input
from other developers / users.

>- no support (as far as I could tell) for generating schema update scripts

Yes. Torque is "all or nothing".

>- no direct execution of ddl/data sql from xml, only through first written out
>   scripts. For execution at runtime from a portal/servlet application this isn't blocking
>   but not exactly what I really would like.

Correct. The whole generator has been designed as an additional
wart^W^Waddition to ant. It definitely needs refactoring and
personally I want to look into dropping the whole Velocity
templating/Texen shebang. Texen has given us nothing but headaches
starting with logging and error reporting and ending up with gobbling
up memory like mad... Personally, I believe an XSLT solution would be
nicer but I sorely lack time to do any work on this ATM.

Hibernate e.g. has a better solution by generating the creation SQL
directly at run-time. We still shine at the source code generation for
the classes/peers though.

>- data xml to sql translation is very limited. We at least need proper
>   date and timestamp (in configurable format) value transformation support, as well as
>   support for proper BIT mapping (an issue with PostgreSQL which seems to have
>   been reported several times on the Torque list). Furthermore, I'd like to be able
>   to use variables for values like $now or $yesterday, especially for date/timestamp
>   fields.

Support for custom functions would be nice, too (talk encrypting passwords...).

>- overriding only a few generator templates (like the table.vm and tablefk.vm for
>   to fix the mysql issue) requires us to maintain *all* templates: torque isn't able
>   to resolve missing templates from the classpath if templates from the file system are to be
>   used.

Texen again. I hope to become a velocity committer at some point (it
was basically agreed upon at ApacheCon with Geir and Daniel but no CfV
yet. :-( ).

>I've started reviewing commons-sql this morning. Although it is still in the commons sandbox,
>I think it might be a (far) better match for our requirements.

I've read about commons-sql (it will probably move to the DB project
(maybe make a short stint through the incubator) soon) yesterday. I
haven't looked at it yet, but everyone that I hear talking about it,
really likes it. So yes, I will definitely look at it.

>Especially direct ddl/sql execution is provided as well as schema update at runtime (!).
>Its datatype mapping support also looks far more extensive (including number, date and
>timestamp format support) than what I've seen in Torque. I expect the PostgreSQL problems
>Chris Custine has to be far less then we have right now with Torque.

>Finally, it also supports (and uses) templating. Although I only see jelly scripts (eek) in the
>codebase, the documentation also speaks of velocity support.

Even JvZ has admitted that introducing Jelly was a mistake. :-)

>I haven't tested it out yet, so it is far too early to conclude it is what I'm looking for,
>but it sure does look promising.

>No disregard to Torque or its team, but considering our requirements and the current features
>supported by Torque, do you think we should stick with it, or maybe better try commons-sql
>(or even something of our own which I don't really want if not needed).

Sure. Torque _is_ a smallish project and lacks in some aspects (you
did manage to put your fingers on quite a number of sore spots in the
generator. :-) ). Bad metadata for the DBs is another thing that I
really hate.

>We hope to put out the final release Jetspeed-2 early next year, and our current problems with
>the M1 release as well during development need to be addressed soon.
>If you do think Torque can meet our needs, how much work and time would it cost?

To clean up everything that you mentioned above? A lot. I freely
admit, that for some things that you like to have (update scripts
e.g.) Torque simply has not support. It was not really intended for
this. For you immediate needs, I cannot give you an estimate or time
frame, sorry. I will open a ticket for your points in the TTS so they
won't be forgotten.

>Thanks again for your response and I hope we can help each other out with this.
>I just subscribed to the Torque user and dev lists and also sending this response to the latter.

Cool.

>If you respond, I'd appreciate it if you would send it to the jetspeed-dev list too.

Let's see if cross-posting with this newsreader works out... :-)

	Regards
		Henning

-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen          INTERMETA GmbH
hps@intermeta.de        +49 9131 50 654 0   http://www.intermeta.de/

RedHat Certified Engineer -- Jakarta Turbine Development  -- hero for hire
    Linux, Java, perl, Solaris -- Consulting, Training, Development

What is more important to you...
    [ ] Product Security
or [ ] Quality of Sales and Marketing Support
               -- actual question from a Microsoft customer survey

---------------------------------------------------------------------
To unsubscribe, e-mail: torque-dev-unsubscribe@db.apache.org
For additional commands, e-mail: torque-dev-help@db.apache.org








---------------------------------------------------------------------
To unsubscribe, e-mail: jetspeed-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jetspeed-dev-help@jakarta.apache.org

Re: [Fwd: Re: [J2] Vote: Torque schema generation problems - how

["Henning P. Schmiedehausen" <hp...@intermeta.de>]

=?ISO-8859-1?Q?Rapha=EBl_Luta?= <ra...@apache.org> writes:

>Ate Douma wrote:

>> I'm forwarding the reply from Henning because his is probably stuck in 
>> the moderators queue.
>> So expect the original message to arrive sometime later.

>There's nothing from Henning in the moderation queue for this list.

Hm. So crossposting does not work with my news reader. :-/ Thanks for
checking.

	Regards
		Henning


-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen          INTERMETA GmbH
hps@intermeta.de        +49 9131 50 654 0   http://www.intermeta.de/

RedHat Certified Engineer -- Jakarta Turbine Development  -- hero for hire
   Linux, Java, perl, Solaris -- Consulting, Training, Development

What is more important to you...
   [ ] Product Security
or [ ] Quality of Sales and Marketing Support
              -- actual question from a Microsoft customer survey

---------------------------------------------------------------------
To unsubscribe, e-mail: jetspeed-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jetspeed-dev-help@jakarta.apache.org

Re: [Fwd: Re: [J2] Vote: Torque schema generation problems - how to proceed?]

[Raphaël Luta <ra...@apache.org>]

Ate Douma wrote:

> I'm forwarding the reply from Henning because his is probably stuck in 
> the moderators queue.
> So expect the original message to arrive sometime later.

There's nothing from Henning in the moderation queue for this list.

-- 
Raphaël Luta - raphael@apache.org
Apache Portals - Enterprise Portal in Java
http://portals.apache.org/


---------------------------------------------------------------------
To unsubscribe, e-mail: jetspeed-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jetspeed-dev-help@jakarta.apache.org