You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Christopher Oliver <re...@verizon.net> on 2003/03/03 20:57:19 UTC
Flow Layer Database API
I've just committed [experimental] changes that provide a simple
database API for the Cocoon flow layer modeled after JSTL
(http://java.sun.com/products/jsp/jstl). This will allow you to perform
a database query in a flow script that produces a bean-like object for
use by your presentation layer (i.e. you can pass it to sendPage*() or
use it as part of your XMLForm model), without requiring a heavyweight
object-relational mapping.
In addition, hopefully this will remove the need for ESQL and satisfy
issues like the one raised here:
http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=104533819604446&w=2
The cocoon object now supports a new function
getConnection([String] dataSourceName);
that returns a Database connection object for any configured data-source
in cocoon.xconf. The returned object provides two methods, analagous to
JSTL's <sql:update> and <sql:query> tags:
update([String] sql)
query([String] sql, [Number] startRow, [Number] maxRows)
The update() function can be used to execute INSERT, UPDATE, and DELETE
statements, as well as statements that create or remove database
objects, such as CREATE TABLE and DROP TABLE. It returns the number of
rows affected by the statement.
The query() function executes a SQL SELECT statement. You can limit the
result with startRow and maxRows (which are optional). The object
returned by query() contains the same properties as in JSTL:
rows An array with a case-insensitive object per row
with properties matching column names and values
matching column values.
rowsByIndex An array with an array per row with column values.
columnNames An array with column names.
rowCount The number of rows in the result.
limitedByMaxRows True if not all matching rows are included due to
reaching a specified max rows limit.
Here is an example script:
function test() {
var conn = cocoon.getConnection("hsql");
conn.update("create table employee (first varchar(15), last
varchar(20), address varchar(30), city varchar(20), state varchar(20))");
conn.update("insert into employee values ('joe', 'blow', '1 Main
St.', 'Los Angeles', 'CA')");
conn.update("insert into employee values ('joe', 'smith', '10 North
St.', 'Sacramento', 'CA')");
var emps = conn.query("select * from employee");
sendPage("templates/employee.vm", {emps: emps})
}
where employee.vm looks like this:
?xml version="1.0"?>
<page>
<title>Employees</title>
<content>
#foreach ($emp in $emps.rows)
<para>$emp.first $emp.last from $emp.city, $emp.state</para>
#end
</content>
</page>
Let me know what you think.
Regards,
Chris
Re: Flow Layer Database API
Posted by Antonio Gallardo <ag...@agsoftware.dnsalias.com>.
> Antonio Gallardo wrote:
>>>On a personal note, it is unfortunate because I think your new
>>>techniques will be more attractive to my Client! Even though it may
>>> not be the best way to work, they will find it easier to understand,
>>> being closer to what they used to do. So even though I might prefer
>>> O-R Mapping, I don't think they will.
>>
>> I think O/R Mapping at the first sight looks a doom. But when I closer
>> see it. It is the same stuff we do when we write modular Database
>> Actions. The database.xml is a real O/R mapping without Beans.
>
> Ok, let me tell you something. I've done consulting for a company that
> relies rather havily on AS/400 machines. For those of you that don't
> know it, AS/400 is soooooo relational-oriented that even the filesystem
> is bunch of tables.
>
> They rely on SQL so much thay have SQL-wizards in house that program in
> RPG (think about a horrible programming language... mix it with cobol...
> stir... not even close!) as you and I speak. Ah, note that AS/400 has a
> 8.3 char limitation on variable names, so you have wonderful things
> like
>
> select ASNAMEW, AWCHORD from FOCLI2
>
> In short, their business logic is RPG+SQL, *that* kind of SQL. After
> seven years (and millions of euros) of producing their own patches on
> top of a commercial ERP, and several people hired to match that
> technological needs, they have no OO in-house programmer, nor they want
> it.
>
> The problem is they have to connect that data (and the business logic
> that is connected to it) on their intranet.
>
> As weird as it might seem, any O/R tool will drive them nuts for two
> reasons:
>
> 1) none of them think in terms of objects
How many of them think in terms of pipelines? This question was posted
just to show that we cannot meet every request from the users. If you are
really thinking to use Cocoon, there are some concepts you must learn.
This apply for many technologies.
Please note: When I come to Cocoon, I never before used XML, XSL. But I
must learn it in order to use Cocoon.
> 2) there is no E/R model of their database!
I think there must be a model. Always is a model. The other question is if
they already had a graphic model of the database. Maybe there are not
"clear" relations between tables, but if you think a little bit about it I
am sure you will find the relations.
> 3) only a few people know enough of that database to really extract
> something meaningful.
If you dont know the database, then how they build the needed queries?
Sorry Stefano but things are not to opaque as you try to point it out. ;-)
>
> Ok, first answer to their problem is: you are hopeless and walk away.
I dont think this is the solution.
>
> Second answer (that would make me look nicer and them call me again)
> would be something like Chris' solution.
Maybe you are right.... But, I started to believe in Beans, because
someone posted a mail with the convenience to have a clear data layer. The
beans do this for you. Beans are not to hard to write. There are classes
with just a data structure of database fields with a set of "get" and
"set" methods. That is all. This trivial classes let us to separate the
data from the presentation. This is the power I saw here. It does not have
info about how your data are structured into your database. To do this
work you need the O/R map. This map "glue" every data field of the Bean
with his equivalent into the Database. Please note: Maybe I am wrong, but
this is my point of view about the beans and O/R maps (called Hibernate,
Torque, OJB, etc.)
The Beans allow us to be scalable and to separate presentation from data
layer. the use if Beans help us to win independence from propetary SQL
constructions. Of course it does not comes for free, there is a
performance penality when we add Beans.
>
> This example is to shows you a few things:
>
> 1) there is no clear-cut solution to every problem. Elegance is never an
> absolute metric.
I partially agree. I can try to find a way to build database web based
applications. The old approach of Cocoon (FormValidation + Database
Actions) is not complete. You need to use Javascript on the client side to
handle some situations. This Javascript code is hard to mantain.....
>
> 2) on the other hand, SoC *IS* an absolute metric, only that concern
> analysis must be performed on an 'ad-hoc' basis: it can't be done once
> for every potential problem space.
>
> This said, O/R mapping is *ONE* of the ways to solve the problem of
> connecting flow logic with business logic and assumes several things:
>
> - objects are easier to deal with than SQL by the people involved in the
> development process. This is not always true.
>
> - it is possible (in terms of economical feasibility) to perform this
> mapping and to keep it updated. This is not always true.
I think this is true. I believe that if you change the database you ONLY
need to chanhe the O/R map. You dont need to change the XForm and the
Bean.
>
> - database abstraction and cross-database portability is seen as a
> value. In most cases, it's not.
Please explain your point a little more.
>
> - the people are willing to pay the development costs of an additional
> decoupling layer between their data abstraction and their database
> representation. In most cases, they aren't.
You will not expend many time to do this. This is why I thinked we can
build a tool that will create the Beans for us. Of course you always will
need to create and manage the O/R map.
>
> NOTE: SQL was invented *exactly* to provide that portable abstraction.
> Today, it's considered nothing different from proprietary language with
> lots of similarities between products in the same market niche.
>
> The result is that people will use direct SQL mapping much more than any
> other solution.
You are right if you are build a product for a specific Database Engine.
But I think people are trying to gain some independence from providers
(please read OS provider, Database provider and so on.). This is the main
reason to use Java instead of a tight plataform language or tool. (I
strongly believe in this). They are trying to gain independence from the
Database Engines too. This is why we need to use Beans too.
I faced a 2 years ago a similar problem. We had a tight integrated
plataform with M$. Then we will need to rewrite all the stuff again in
order to change to another OS. That means that the cheap at the beginning
is not alway cheap at the end. In the other way if will expend a little
more at the beginning creating O/R maps and Beans then we can easily
migrate to another Database Engine.
What this people will do when migrate to another plataform outside OS/400?
>
> They will do it in flow, or in SQLTransformer, or in ESQL, it doesn't
> matter where, but they will. And we can't do (nor should, if this is a
> value for them) anything about it.
>
> Now, does this mean that there are no better alternatives? no, I don't
> think so.
>
> Is O/R the solution? could be.
>
> How would O/R work? so that it is as transparent as possible. This
> probably rules out Torque-like pre-compilation-based approaches, leading
> to more dynamically-oriented proxy-based approaches. So OBJ seems more
> advanced in this context.
>
> Is O/R the solution to every datamapping problem? no way! but it might
> solve lots of issues with structured-data mapping. AS for
> unstructure-data mappings, a JSR-170-like repository will hopefully do
> the job.
>
> But I really don't know, I just want to see people experiment and try
> things out for a while and for sure there won't be *one*
> one-size-fits-all solution, but several that will hopefully reduce their
> overlap to the bare minimum.
To be honest I hope it will help us. Soonly I will to write a new
application. This time I will try to use some O/R mapping + XForm + Flow.
I hope that after creating the correct databased web-based framework. We
can archieve some database block or components. For example: customers,
vendors, invoices, etc.
After you create this components you can simply put this stuff into your
new application and it will work.
Best Regards,
Antonio Gallardo
>
> --
> Stefano Mazzocchi <st...@apache.org>
> Pluralitas non est ponenda sine necessitate [William of Ockham]
> --------------------------------------------------------------------
Re: Flow Layer Database API
Posted by Stefano Mazzocchi <st...@apache.org>.
Antonio Gallardo wrote:
>>On a personal note, it is unfortunate because I think your new
>>techniques will be more attractive to my Client! Even though it may not
>>be the best way to work, they will find it easier to understand, being
>>closer to what they used to do. So even though I might prefer O-R
>>Mapping, I don't think they will.
>
> I think O/R Mapping at the first sight looks a doom. But when I closer see
> it. It is the same stuff we do when we write modular Database Actions. The
> database.xml is a real O/R mapping without Beans.
Ok, let me tell you something. I've done consulting for a company that
relies rather havily on AS/400 machines. For those of you that don't
know it, AS/400 is soooooo relational-oriented that even the filesystem
is bunch of tables.
They rely on SQL so much thay have SQL-wizards in house that program in
RPG (think about a horrible programming language... mix it with cobol...
stir... not even close!) as you and I speak. Ah, note that AS/400 has a
8.3 char limitation on variable names, so you have wonderful things like
select ASNAMEW, AWCHORD from FOCLI2
In short, their business logic is RPG+SQL, *that* kind of SQL. After
seven years (and millions of euros) of producing their own patches on
top of a commercial ERP, and several people hired to match that
technological needs, they have no OO in-house programmer, nor they want it.
The problem is they have to connect that data (and the business logic
that is connected to it) on their intranet.
As weird as it might seem, any O/R tool will drive them nuts for two
reasons:
1) none of them think in terms of objects
2) there is no E/R model of their database!
3) only a few people know enough of that database to really extract
something meaningful.
Ok, first answer to their problem is: you are hopeless and walk away.
Second answer (that would make me look nicer and them call me again)
would be something like Chris' solution.
This example is to shows you a few things:
1) there is no clear-cut solution to every problem. Elegance is never an
absolute metric.
2) on the other hand, SoC *IS* an absolute metric, only that concern
analysis must be performed on an 'ad-hoc' basis: it can't be done once
for every potential problem space.
This said, O/R mapping is *ONE* of the ways to solve the problem of
connecting flow logic with business logic and assumes several things:
- objects are easier to deal with than SQL by the people involved in the
development process. This is not always true.
- it is possible (in terms of economical feasibility) to perform this
mapping and to keep it updated. This is not always true.
- database abstraction and cross-database portability is seen as a
value. In most cases, it's not.
- the people are willing to pay the development costs of an additional
decoupling layer between their data abstraction and their database
representation. In most cases, they aren't.
NOTE: SQL was invented *exactly* to provide that portable abstraction.
Today, it's considered nothing different from proprietary language with
lots of similarities between products in the same market niche.
The result is that people will use direct SQL mapping much more than any
other solution.
They will do it in flow, or in SQLTransformer, or in ESQL, it doesn't
matter where, but they will. And we can't do (nor should, if this is a
value for them) anything about it.
Now, does this mean that there are no better alternatives? no, I don't
think so.
Is O/R the solution? could be.
How would O/R work? so that it is as transparent as possible. This
probably rules out Torque-like pre-compilation-based approaches, leading
to more dynamically-oriented proxy-based approaches. So OBJ seems more
advanced in this context.
Is O/R the solution to every datamapping problem? no way! but it might
solve lots of issues with structured-data mapping. AS for
unstructure-data mappings, a JSR-170-like repository will hopefully do
the job.
But I really don't know, I just want to see people experiment and try
things out for a while and for sure there won't be *one*
one-size-fits-all solution, but several that will hopefully reduce their
overlap to the bare minimum.
--
Stefano Mazzocchi <st...@apache.org>
Pluralitas non est ponenda sine necessitate [William of Ockham]
--------------------------------------------------------------------
Re: Flow Layer Database API
Posted by Jeremy Quinn <je...@media.demon.co.uk>.
On Tuesday, March 4, 2003, at 11:10 AM, Antonio Gallardo wrote:
>> On a personal note, it is unfortunate because I think your new
>> techniques will be more attractive to my Client! Even though it may
>> not
>> be the best way to work, they will find it easier to understand, being
>> closer to what they used to do. So even though I might prefer O-R
>> Mapping, I don't think they will.
>
>
> I think O/R Mapping at the first sight looks a doom. But when I closer
> see
> it. It is the same stuff we do when we write modular Database Actions.
> The
> database.xml is a real O/R mapping without Beans.
I would like to understand this better ;)
But unfortunately, ModularDatabase stuff stubbornly remains completely
opaque to me! I cannot get my head around it (but then I am a complete
beginner to RDBMS programming).
I heartily encourage you to send in some examples :)
regards Jeremy
Re: Flow Layer Database API
Posted by Antonio Gallardo <ag...@agsoftware.dnsalias.com>.
> On a personal note, it is unfortunate because I think your new
> techniques will be more attractive to my Client! Even though it may not
> be the best way to work, they will find it easier to understand, being
> closer to what they used to do. So even though I might prefer O-R
> Mapping, I don't think they will.
I think O/R Mapping at the first sight looks a doom. But when I closer see
it. It is the same stuff we do when we write modular Database Actions. The
database.xml is a real O/R mapping without Beans.
Antonio Gallardo.
Re: Flow Layer Database API
Posted by Jeremy Quinn <je...@media.demon.co.uk>.
On Monday, March 3, 2003, at 11:52 PM, Christopher Oliver wrote:
> You know what? You're right. My description below was wrong. What I
> should have said is that I created a simple _JavaScript_ database API
> that can be used to implement business objects in JavaScript.
<snip>PetStore Example</snip>
> I believe if used properly this can still provide the correct SOC.
> What do you think?
That new example does look a lot better!
TBH, your first example set alarms off in my head ;)
I think this looks very useful for rapid prototyping, but it also
allows you to work in a bad way .... and unfortunately this way of
doing it is so much closer to the way you'd do it in 'other
environments' it will be attractive to newcomers, which is both good
and bad ;)
On a personal note, it is unfortunate because I think your new
techniques will be more attractive to my Client! Even though it may not
be the best way to work, they will find it easier to understand, being
closer to what they used to do. So even though I might prefer O-R
Mapping, I don't think they will.
Anyway, please do not take this as criticism!
regards Jeremy
Re: Flow Layer Database API
Posted by Christopher Oliver <re...@verizon.net>.
You know what? You're right. My description below was wrong. What I
should have said is that I created a simple _JavaScript_ database API
that can be used to implement business objects in JavaScript. This
really has nothing to do with the "flow" layer per se, which is about
controlling page flow. Instead, it aims to satisfy the niche that
Sylvain identified and allow non-Java, non-O/R experts, a rapid
development tool to also program business objects that can used by the
Cocoon flow and presentation layers.
For example, if I were writing a "Petstore" application I could program
both the business layer, and flow layer in JavaScript. For the business
layer I might define a business object like this:
function PetStore(poolId) {
this.poolId = poolId;
}
PetStore.prototype.getProductListByCategory = function(key,
skipResults, maxResults) {
var conn = cocoon.getConnection(this.poolId);
var result = conn.query("select * from PRODUCT where CATEGORY =
" + key,
skipResults, maxResults);
conn.close();
return result;
}
// etc...
And for the flow layer I might write code like this:
var petStore = new Petstore(...);
function viewCategory() {
var categoryId = cocoon.request.get("categoryId");
var skipResults = 0;
var maxResults = 10;
do {
var productList =
petStore.getProductListByCategory(categoryId,
skipResults,
maxResults);
skipResults += productList.rowCount;
sendPageAndWait("view/Category.xml",
{productList: productList});
} while (productList.limitedByMaxRows);
}
// etc ...
I believe if used properly this can still provide the correct SOC. What
do you think?
Regards,
Chris
Gianugo Rabellino wrote:
> Christopher Oliver wrote:
>
>> I've just committed [experimental] changes that provide a simple
>> database API for the Cocoon flow layer modeled after JSTL
>
>
> [...]
>
>>
>> Let me know what you think.
>>
>
> OK, first of all I must confess that I have little or no experience with
> the flow, so probably I should just shut up and let you guys proceed
> with your impressive work, eagerly waiting to have some time to catch up
> and join the party.
>
> This said, I just wanted to share a couple of "belly thoughts", since
> the recent additions to the flow are ringing a bell in my head, and I
> start getting a bit puzzled. One of the first concerns we had when we
> started talking about flow was about people abusing it and how to stop
> them. The answer has always been that the flow would have played the
> role of "pure glue", providing, well, flow control and little more: in
> particular it was take for granted that the business objects would have
> been outside of the flow scope.
>
Re: Flow Layer Database API
Posted by Gianugo Rabellino <gi...@apache.org>.
Christopher Oliver wrote:
> I've just committed [experimental] changes that provide a simple
> database API for the Cocoon flow layer modeled after JSTL
[...]
>
> Let me know what you think.
>
OK, first of all I must confess that I have little or no experience with
the flow, so probably I should just shut up and let you guys proceed
with your impressive work, eagerly waiting to have some time to catch up
and join the party.
This said, I just wanted to share a couple of "belly thoughts", since
the recent additions to the flow are ringing a bell in my head, and I
start getting a bit puzzled. One of the first concerns we had when we
started talking about flow was about people abusing it and how to stop
them. The answer has always been that the flow would have played the
role of "pure glue", providing, well, flow control and little more: in
particular it was take for granted that the business objects would have
been outside of the flow scope.
Now, this database addition makes me think that we are approaching the
"all purpose" scenario (who said that every application must grow until
it's able to connecto to a database? :-)), with a lot of possible (user)
abuses (I perfectly understand that users will abuse it nevertheless, I
just don't want to help them too much in that).
Please understand me: I might well be wrong with this, since I'm not
seeing the whole picture... I'm just feeling that there might be
something wrong here. Basically I don't see the reason for having
specialized flow components that are somehow duplicating Cocoon ones
(take Velocity as an example): if I'm not able to use Cocoon components
inside the flow (better: if using Cocoon components is hard enough to
make duplication a viable option) then I'm wondering if we aren't going
sooner or later to crash against a wall.
Anyway, take this as an old aunt rant, just 0.02c of random thought from
an almost complete ignorant. And, of course, thank you for the great job
you have been doing until now: I surely hope to join the next
discussions with a more thorough knowledge of what's going on under the
hood.
Ciao,
--
Gianugo Rabellino
Pro-netics s.r.l.
http://www.pro-netics.com
Re: Flow Layer Database API
Posted by Christopher Oliver <re...@verizon.net>.
Sylvain Wallez wrote:
> Christopher Oliver wrote:
>
>>
>> The cocoon object now supports a new function
>>
>> getConnection([String] dataSourceName);
>
>
>
> This looks cool... but, continuing my rant about oversimplification, why
> does the _cocoon_ object hold methods to access database connections ?
> AFAIK, this object is meant to give access to the enclosing context :
> request, response, manager (but *not* the environment, which IMO should
> be removed). So why JDBC stuff there ?
>
> Shouldn't we start to structure the JS stuff in libraries that people
> load whenever (and only when) they need them rather than mix everything
> in a single object ?
>
You are absolutely correct. However, for now I'd prefer not to invent
new objects, and this was the most convenient place to put it, so others
could try it.
Re: Flow Layer Database API
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
Christopher Oliver wrote:
> I've just committed [experimental] changes that provide a simple
> database API for the Cocoon flow layer modeled after JSTL
> (http://java.sun.com/products/jsp/jstl). This will allow you to
> perform a database query in a flow script that produces a bean-like
> object for use by your presentation layer (i.e. you can pass it to
> sendPage*() or use it as part of your XMLForm model), without
> requiring a heavyweight object-relational mapping.
>
> In addition, hopefully this will remove the need for ESQL and satisfy
> issues like the one raised here:
> http://marc.theaimsgroup.com/?l=xml-cocoon-dev&m=104533819604446&w=2
>
> The cocoon object now supports a new function
>
> getConnection([String] dataSourceName);
This looks cool... but, continuing my rant about oversimplification, why
does the _cocoon_ object hold methods to access database connections ?
AFAIK, this object is meant to give access to the enclosing context :
request, response, manager (but *not* the environment, which IMO should
be removed). So why JDBC stuff there ?
Shouldn't we start to structure the JS stuff in libraries that people
load whenever (and only when) they need them rather than mix everything
in a single object ?
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
Re: Flow Layer Database API
Posted by Tony Collen <tc...@neuagency.com>.
On Mon, 3 Mar 2003, Christopher Oliver wrote:
> I've just committed [experimental] changes that provide a simple
> database API for the Cocoon flow layer modeled after JSTL
> (http://java.sun.com/products/jsp/jstl). This will allow you to perform
> a database query in a flow script that produces a bean-like object for
> use by your presentation layer (i.e. you can pass it to sendPage*() or
> use it as part of your XMLForm model), without requiring a heavyweight
> object-relational mapping.
Sweet! I was starting to wonder when we'd get some DB conectivity in the
flow. I look forward to checking it out..
Regards,
Tony