You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Nick Kew <ni...@webthing.com> on 2004/12/03 13:33:59 UTC
Re: Towards a generic database connection API
I've updated the proposed DBD API, and written a better description
of it at <URL:http://www.apache.org/~niq/dbd.html>. This defines
only the API, decoupled from my software implementing it.
The only material change to the software is that I have added
random-access to SELECT result sets, as suggested by several people.
I've also retained my originally-proposed row-iterator access,
as it has a performance advantage for at least MySQL.
For the purposes of this exercise, I'm ruling out APR-ising it:
that's a separate project. Bearing this in mind, I'd like to
invite comments again.
Summary: database applications get the following for free:
* Multi-database support
* Optimised access to all MPMs
Should we seek to standardise this?
--
Nick Kew
Re: Towards a generic database connection API
Posted by Nick Kew <ni...@webthing.com>.
On Mon, 13 Dec 2004, Greg Stein wrote:
> On Fri, Dec 03, 2004 at 12:33:59PM +0000, Nick Kew wrote:
> >
> > I've updated the proposed DBD API, and written a better description
> > of it at <URL:http://www.apache.org/~niq/dbd.html>. This defines
> > only the API, decoupled from my software implementing it.
> >...
> > Bearing this in mind, I'd like to
> > invite comments again.
> >
> > Summary: database applications get the following for free:
> > * Multi-database support
> > * Optimised access to all MPMs
> >
> > Should we seek to standardise this?
>
> Feel free, but I'm -1 on it being part of httpd. The API doesn't
> support most of the things needed for high-performance database
> access. There is no statement preparation, no input/output parameter
> binding, no array access, etc. It also appears to be bound to
> httpd-isms (e.g. acquire takes a request_rec -- should be an arbitrary
> pool). Cursors need to be supported, and it would be great to have
> non-string values within the API.
Do you not follow dev@apr? I moved it there in the light of feedback
and it now supports statement preparation and parameter binding.
The httpd-specific part is now a much smaller module that just deals
with MPM and resource lifetime&management issues.
> These things don't have to make the API complicated -- it is possible
> to have the API scale up with complexity. But it should be possible to
> do these things. When Michael Lorton and I designed the Python DBAPI
> back in '96, we took a lot of time to ensure that the simple things
> were still simple, but it was possible to do serious full-on high-perf
> work. Our Python-based data loader was just as fast as Oracle's loader
> because we were able to use array-inserts via Python.
Sounds like you're someone this discussion could really use on dev@apr:-)
> I also believe the right level for an API like this would be in APR.
> Whether APR itself, or APRUTIL... not sure. I'd think the latter. The
http://marc.theaimsgroup.com/?l=apr-dev&m=110259565527659&w=2
http://marc.theaimsgroup.com/?l=apr-dev&m=110279584404570&w=2
(the latter is the most up-to-date)
You'll notice the code in apr_dbd.c is very minimal and comprises mostly,
handling of driver loading/init (APR_HAS_DSO and threads) while apr_dbd.h
and the drivers are bigger.
> helped a ton. We also said, "do the fancy stuff at a higher layer; we
> are simply trying to provide some basic interop."
Yep.
> Oh, and the notion of swap one provider for another? Pipe dream.
Well, I've done this to a limited extent with real apps. Site Valet
uses a C++ SQL base class + drivers that instantiate it. That includes
functions to generate statements where the SQL syntaxes differ - e.g.
MySQL's "INSERT IGNORE" vs INSERT triggering a stored func if the
operation would violate a uniqueness constraint. I'm not proposing
that for APR - just the basic API for apps to build on.
> Apps
> will always get coded for one back-end or another. If they are
> independent, then they're using the database in very light ways.
I'm aware of that: when I had to convert some stuff originally
implemented with PostgreSQL to run on Windows, the conversion to
MSSQL/ODBC was more work than the original, despite having used
Perl's DBI. But that's a learning curve: when you've been there
once, you have a better idea what to expect.
I feel sure we can do this with common Apache modules including
Authentication and Logging, and that a high proportion of LAMP
applications would fit at least the lightness criteria.
--
Nick Kew
Re: Towards a generic database connection API
Posted by Greg Stein <gs...@lyra.org>.
On Fri, Dec 03, 2004 at 12:33:59PM +0000, Nick Kew wrote:
>
> I've updated the proposed DBD API, and written a better description
> of it at <URL:http://www.apache.org/~niq/dbd.html>. This defines
> only the API, decoupled from my software implementing it.
>...
> Bearing this in mind, I'd like to
> invite comments again.
>
> Summary: database applications get the following for free:
> * Multi-database support
> * Optimised access to all MPMs
>
> Should we seek to standardise this?
Feel free, but I'm -1 on it being part of httpd. The API doesn't
support most of the things needed for high-performance database
access. There is no statement preparation, no input/output parameter
binding, no array access, etc. It also appears to be bound to
httpd-isms (e.g. acquire takes a request_rec -- should be an arbitrary
pool). Cursors need to be supported, and it would be great to have
non-string values within the API.
These things don't have to make the API complicated -- it is possible
to have the API scale up with complexity. But it should be possible to
do these things. When Michael Lorton and I designed the Python DBAPI
back in '96, we took a lot of time to ensure that the simple things
were still simple, but it was possible to do serious full-on high-perf
work. Our Python-based data loader was just as fast as Oracle's loader
because we were able to use array-inserts via Python.
I also believe the right level for an API like this would be in APR.
Whether APR itself, or APRUTIL... not sure. I'd think the latter. The
API itself wouldn't have a lot of code; it's going to all be in the
providers. I would also advise an API that is *very* easy for
providers to implement, or you won't get any of them built. That was a
design goal for the Python DBAPI -- make it easy for providers. It
helped a ton. We also said, "do the fancy stuff at a higher layer; we
are simply trying to provide some basic interop."
Oh, and the notion of swap one provider for another? Pipe dream. Apps
will always get coded for one back-end or another. If they are
independent, then they're using the database in very light ways.
That's alright, but it is also very rare. Most apps are very tied to a
database, even if they use a "common API".
Cheers,
-g
--
Greg Stein, http://www.lyra.org/