Re: Torque performance problems

[Chris Felaco <cf...@netscape.net>]

Stephen Haberman <st...@beachead.com> wrote:

>On Tue, Jan 14, 2003 at 10:37:37AM -0500, Chris Felaco wrote:
>> >- Separate the template based engine which creates java classes from
>> >  templates from the actual persistence code.
>> 
>> I'm not entirely sure what you're proposing here, but I think
>> you're suggesting keeping the save() and related methods out of
>> the OM objects entirely.
>
>No, I think he was suggesting that as it is now, the templates look
>like:
>
>public class $blah {
>
>#if x
>  protected int foo;
>#if y
>  public int getFoo()
>#if z
>    return x
>#else
>    return w
>#endif
>#endif
>#endif
>
>So, this isn't a real code snippet. But having to do both
>persistence code generation and Java source generation at the same
>time sucks. My interpretation was that the Java source generation
>would be put in a seperate template based engine, then Torque would
>interact with a nice API and plug in the various bits of persistence
>code here and there (as templates).
>

It still does not compute with me, perhaps because I haven't been following the list much.  Are you suggesting that instead of 1 template per source file, you would have templates for the code snippets and Torque decides where to use them and what data to pass them?

>> I'm not in favor of simply replacing the existing persistence
>> layer with any other persistence layer out there.  There are so
>> many APIs out there that it's hard to make sense of them all and
>> compare and contrast them.
>
>Precisely why we don't need to make another one. As long as we pick
>a good one, with community support, that 1-2-3 years from now will
>still be around and active/semi-active, I think it is a great boon
>for us not having to re-invent and re-debug and re-maintain the
>wheel.

Who is going to decide which API will still be around and active/semi-active?  I don't think you will ever make universal consensus on that, and I can almost guarantee you'll regret the choice down the road.  There are so many APIs, and most of them are good, but differ only in the fine points of how they do things.  Some have abstractions for SQL queries, some don't.  Some attempt to be a generic persistence mechanism using XML or serialization on the backend, while others (like Torque) are pretty much tied to SQL.  I just think betting the future of Torque on some other persistence API (especially one not managed by the Jakarta project) is not a good idea.  What you end up with is Torque as just an adapter for this other API, and not a stand-alone product.

>If you want to spend the next few months/years designing,
>implementing, and maintaining your very own persistance mechancism
>embedded specifically into Torque, go ahead. But given that there
>are already viable, generic alternatives, I'd rather do that.

There really isn't all that much to design if we're just reimplementing the currently published API. What I want to do is just fix the egregious performance problems in Torque so that the basics of what is there now can be used.  If you for instance wanted to scrap the Criteria code and use some other better SQL query abstraction, then that's a different story.  But, if that's the case, then start something new and don't call it Torque.  I don't want to change the overall design philosophy behind Torque, I just want to make it work better.

>> Torque should be self-sufficient if it has any chance of surviving
>> on its own merits.  If Torque is at the mercy of bugs in another
>> API, it's less viable. 
>
>If you pick a good API that is stable and released, this won't
>happen anymore than we have bugs in our own solution.

Increasing the number of LOCs, and the number of interactions between APIs, will increase the overall complexity of the software and therefore make it buggier and more difficult to maintain.  One abstraction layer is better than two.  We all know JDBC, we all understand JDBC, why should it be so hard to build Torque just using JDBC?  Why do you want to force potential code contributors to Torque to have to learn yet-another 3rd party API?

Here's an example of what I'm talking about.  In Turbine 2.1, I ran into problems updating NUMBER columns in my Oracle DB simply because Village 1.3 was not using BigDecimal where it should have.  I had a hell of a time just finding the Village source and any info about it.  I ended up finding the problem by using jad to disassemble the classes.  I toyed with the idea of just fixing the code in the disassembled class and packaging my own jar, but I couldn't bring myself to do it.  Then somehow by some miracle I was able to find the 1.5.3 version which had fixed the problem.  The whole episode cost me at least a day of work to fix something that would have taken an hour to track down if the code had just been in Turbine.  And I still don't know how to find the source for village 2.0 which is included in Torque 3.0.

- Chris


__________________________________________________________________
The NEW Netscape 7.0 browser is now available. Upgrade now! http://channels.netscape.com/ns/browsers/download.jsp 

Get your own FREE, personal Netscape Mail account today at http://webmail.netscape.com/

Re: Torque performance problems

[Stephen Haberman <st...@beachead.com>]

On Tue, Jan 14, 2003 at 03:40:26PM -0500, Chris Felaco wrote:
> It still does not compute with me, perhaps because I haven't been
> following the list much.  Are you suggesting that instead of 1
> template per source file, you would have templates for the code
> snippets and Torque decides where to use them and what data to
> pass them?

Kinda. I toyed with the idea awhile ago and it did not completely
fall together, but my basic idea was to make it layers. E.g. have
one layer, like BasicObjectLayer, that goes through each table in
the schema and creates a Java object for it (e.g JavaClass).

Then you have another layer, like GetterSetterLayer, that goes
through the schema, and for each table, add the private data members
and public getters/setters to the previously-created JavaClass.

Then if you want complex object model, e.g. doing the joins, etc.,
have another layer like AddSimpleJoinsLayer, then you go on to
AddSaveLayer, etc.

Seperating each of the aspects into their own given layer against
Java objects instead of hacking out raw source code would make it a
lot more elegant.

The one tricky part that I don't have elegantly figured out is how
to define the contents of methods. To remain usable, the Java
objects you are creating from the schema would be like JavaClass,
JavaField, JavaMethod, JavaImport, and that's about all you want.
You don't want to start generating JavaInt, JavaAdd, etc.

So to stay with the strength of Velocity, I was thinking method
contents could be defined as Velocity snippets which then when you
pay all of the JavaClasses to the code generation library, it does
all of the raw source code generation and includes the Velocity
snippets from the methods as it goes along.

> Who is going to decide which API will still be around and
> active/semi-active? 
<snip/>
> I just think betting the future of Torque on some other
> persistence API (especially one not managed by the Jakarta
> project) is not a good idea.

Jarkarta does have a persistence layer, OJB, that I'm willing to bet
will be around for while. They have a great community, which is why
they were let into Jakarta.

> Increasing the number of LOCs, and the number of interactions
> between APIs, will increase the overall complexity of the software
> and therefore make it buggier and more difficult to maintain.  One
> abstraction layer is better than two.  We all know JDBC, we all
> understand JDBC, why should it be so hard to build Torque just
> using JDBC?

Why do you think persistence frameworks are so popular? People see
how many LOC using a persistence framework saves them and come to
hate hand-coding JDBC when you can save entire native Java objects
with a few LOC.

I can guarantee that hacking out all of the various persistence code
and dropping in a framework like OJB would drastically reduce the
LOC in Torque.

>  Why do you want to force potential code contributors to Torque to
>  have to learn yet-another 3rd party API?

So you're saying any given project should only use it's own code?
And cross-project reuse should never happen? A developer of Torque
should never use an outside library, such as anything from commons,
as it is a 3rd party API that can't be trusted?

I'm sorry for taking your position to the extreme, but this is
effectively what you are saying: instead of relying on
currently-stable API's and implementations from the likes of OJB, we
should roll our own and/or evolve our existing dodgy implementation
to duplicate _the same functionality_ as OJB?

> Here's an example of what I'm talking about.  In Turbine 2.1, I
> ran into problems updating NUMBER columns in my Oracle DB simply
> because Village 1.3 was not using BigDecimal where it should have.
> I had a hell of a time just finding the Village source and any
> info about it.  I ended up finding the problem by using jad to
> disassemble the classes.  I toyed with the idea of just fixing the
> code in the disassembled class and packaging my own jar, but I
> couldn't bring myself to do it.  Then somehow by some miracle I
> was able to find the 1.5.3 version which had fixed the problem.
> The whole episode cost me at least a day of work to fix something
> that would have taken an hour to track down if the code had just
> been in Turbine.  And I still don't know how to find the source
> for village 2.0 which is included in Torque 3.0.

Yes, using Village is an iffy thing that was decided long ago in
Torque's past. E.g. three/four years?

Some reuse works, some fails. Village failed. But the reuse of ant,
commons-beantutils, commons-collections, commons-configuration,
commons-dbcp, commons-lang, commons-logging, commons-pool, jcs,
log4j, velocity, xerces, and junit, have all went well.

Given the ratio, I'd say, providing smart decisions were made about
the entire matter, reusing a persistence layer has a good chance of
succeeding.

(Also note that all of my comments are not items that I see being
done to Torque in the near future; evolution of the likes you
suggest is certainly welcomed to improve what is currently there.
But there is an awful lot of cruft that ideally when/if our two
current heavyweights, Henning and Martin, choose to fix up Torque,
they can hack out and replace with a cleaner design. And of that
cleaner design, I'm making the argument that a persistence layer
should be apart of it).

So currently this entire discussion is not that important, I'm
mostly just making my point for the sake of making my point. If you
still maintain your position that Torque should be it's very own
persistence layer, that is fine, I can understand and accept your
position. I just don't agree with it. :-)

- Stephen

RE: Torque performance problems

[Gonzalo Diethelm <go...@aditiva.com>]

> >If you want to spend the next few months/years designing,
> >implementing, and maintaining your very own persistence mechanism
> >embedded specifically into Torque, go ahead. But given that there
> >are already viable, generic alternatives, I'd rather do that.
> 
> There really isn't all that much to design if we're just 
> reimplementing the currently published API. What I want to do is 
> just fix the egregious performance problems in Torque so that the 
> basics of what is there now can be used.  If you for instance 
> wanted to scrap the Criteria code and use some other better SQL 
> query abstraction, then that's a different story.  But, if that's 
> the case, then start something new and don't call it Torque.  I 
> don't want to change the overall design philosophy behind Torque, 
> I just want to make it work better.

If you could abstract a generic API for persistence, you could
make the persistence mechanism pluggable. Big "if" here...

Something else, related to Criteria: it is well known that the
design of this class is really lacking, and makes it very
difficult to compose complex criteria. Criterion was a good
step forward, but this is definitely an area of Torque that
could use some redesigning.


-- 
Gonzalo A. Diethelm
gonzalo.diethelm@aditiva.com

RE: Torque performance problems

[Gonzalo Diethelm <go...@aditiva.com>]

> >If you want to spend the next few months/years designing,
> >implementing, and maintaining your very own persistence mechanism
> >embedded specifically into Torque, go ahead. But given that there
> >are already viable, generic alternatives, I'd rather do that.
> 
> There really isn't all that much to design if we're just 
> reimplementing the currently published API. What I want to do is 
> just fix the egregious performance problems in Torque so that the 
> basics of what is there now can be used.  If you for instance 
> wanted to scrap the Criteria code and use some other better SQL 
> query abstraction, then that's a different story.  But, if that's 
> the case, then start something new and don't call it Torque.  I 
> don't want to change the overall design philosophy behind Torque, 
> I just want to make it work better.

If you could abstract a generic API for persistence, you could
make the persistence mechanism pluggable. Big "if" here...

Something else, related to Criteria: it is well known that the
design of this class is really lacking, and makes it very
difficult to compose complex criteria. Criterion was a good
step forward, but this is definitely an area of Torque that
could use some redesigning.


-- 
Gonzalo A. Diethelm
gonzalo.diethelm@aditiva.com


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>