You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by "Mikhail T." <mi...@aldan.algebra.com> on 2012/08/21 21:27:15 UTC
Limitations of mod_dbd - single server per vhost
From reading the module's documentation
<http://httpd.apache.org/docs/2.4/mod/mod_dbd.html>, it seems, a single vhost
can only use one DB-server (and only with one set of credentials): only a single
DBDriver and DBDParams is possible, for example.
Also, the syntax of mod_rewrite's dbd/fastdbd maps
<http://httpd.apache.org/docs/2.4/rewrite/rewritemap.html#dbd> doesn't even
allow to specify the DB-server, implying, once again, there can only be one in a
given context (entire server or virtual host). The missing capability could be
useful to someone, who would want, for example, to check a local SQLite-database
for data and, failing to find anything, attempt a query against a remote MySQL
or Sybase server. Other enterprises may want to talk to multiple remote servers
(potentially using different drivers).
I would've filed an enhancement-request, but, perhaps, this was already
discussed here and there are some good reasons for the limitation? Thanks! Yours,
-mi
Re: Fixing apr_dbd_freetds
Posted by "Mikhail T." <mi...@aldan.algebra.com>.
On 22.08.2012 12:55, Jeff Trawick wrote:
> and/ordev@apr.apache.org
You are right. Unfortunately, that list (like this one), requires one to
subscribe before posting. I deeply resent such requirements and try to avoid
such lists, when at all possible :-(
Yours,
-mi
Re: Fixing apr_dbd_freetds (Re: Limitations of mod_dbd - single
server per vhost)
Posted by Jeff Trawick <tr...@gmail.com>.
On Wed, Aug 22, 2012 at 12:26 PM, Mikhail T. <mi...@aldan.algebra.com> wrote:
> Perhaps, this discussion should be happening on the ticket itself? Oh, well
> :) Comments inline:
and/or dev@apr.apache.org
>
> On 22.08.2012 10:47, Nick Kew wrote:
>
> I think I've pointed a few people at the ODBC driver as an alternative.
> Do you have a strong reason to use FreeTDS rather than ODBC?
>
> Yes, ODBC is yet another layer of abstraction. I'd understand, if apr_dbd.c
> used purely ODBC, leaving the ODBC implementation to deal with the backends'
> native drivers. But if apr_dbd offers "native" drivers of its own, better to
> use those, if possible. That's my "strong" reason :-)
>
> One question here may be, does anyone have the platforms to test-drive
> your patch?
>
> I would not know... But, in my opinion, even without getting tested by
> anyone else, my patch is still a vast improvement over the current
> situation...
>
> It seems to do rather more than just fix a simple bug:
>
> That's because the "bug" was not simple -- the driver remained broken ever
> since the apr_dbd.c was changed to do the parsing of query-templates
> centrally instead of delegating the job to each driver. Other drivers got
> updated back then, but not the FreeTDS one :-( It still tried to do its own
> parsing, and was failing...
>
> you've removed the untainting code which was part of protecting against
> injection attacks.
>
> I don't think, the untainting was ever properly implemented (you can't do it
> right without prepared statements) and I don't believe, it is the driver's
> job to do it anyway. None of the other drivers do it either -- though they
> rely on the respective client-libraries. Sybase/FreeTDS client does not
> offer such checks -- so be it... The untainting was costly (done for each
> call) and not guaranteed -- better to leave it to the caller, IMHO.
>
> Have you implemented prepared statements properly for all backends?
>
> To the best of my knowledge, neither the Sybase's db-client library nor the
> FreeTDS reimplementation of it offer prepared statements, unfortunately. The
> current implementation certainly does not use them. Sybase offers a newer
> client interface (ct-lib), that does have prepared statements (see
> ct_dynamic()), but FreeTDS does not provide it, so, for the driver to be
> compatible with both, it has to use the old db-interface... To me this seems
> like an acceptable caveat, as long as it is known and documented.
>
> My patch does not touch any of the other backends.
>
> Perhaps, some day I (or someone else) will implement apr_dbd_sybase.c --
> using the ct-interface, which would give prepared statements and other
> improvements. But that would have to be maintained outside of Apache,
> because, foolishly, Sybase would not open-source the client... It would also
> be Sybase-only (no MS SQL Server).
>
> For the time being, my patch offers a major improvement over the status quo
> -- the driver becomes usable. I am, in fact, preparing to use it with
> RewriteMaps to do SEO for a giant site, that still uses an old Sybase-backed
> CMS. The RewriteRules will ensure, no tainted keys are passed to the queries
> and the DB-user's credentials will be limited to only SELECTs and only for a
> particular table to mitigate the risk of any SQL-injection attack.
>
> Yours,
>
> -mi
--
Born in Roswell... married an alien...
http://emptyhammock.com/
Fixing apr_dbd_freetds (Re: Limitations of mod_dbd - single server
per vhost)
Posted by "Mikhail T." <mi...@aldan.algebra.com>.
Perhaps, this discussion should be happening on theticket <https://issues.apache.org/bugzilla/show_bug.cgi?id=53666> itself? Oh, well :) Comments inline:
On 22.08.2012 10:47, Nick Kew wrote:
> I think I've pointed a few people at the ODBC driver as an alternative.
> Do you have a strong reason to use FreeTDS rather than ODBC?
Yes, ODBC is yet another layer of abstraction. I'd understand, if apr_dbd.c used
purely ODBC, leaving the ODBC implementation to deal with the backends' native
drivers. But if apr_dbd offers "native" drivers of its own, better to use those,
if possible. That's my "strong" reason :-)
> One question here may be, does anyone have the platforms to test-drive
> your patch?
I would not know... But, in my opinion, even without getting tested by anyone
else, my patch is still a vast improvement over the current situation...
> It seems to do rather more than just fix a simple bug:
That's because the "bug" was not simple -- the driver remained broken ever since
the apr_dbd.c was changed to do the parsing of query-templates centrally instead
of delegating the job to each driver. Other drivers got updated back then, but
not the FreeTDS one :-( It still tried to do its own parsing, and was failing...
> you've removed the untainting code which was part of protecting against
> injection attacks.
I don't think, the untainting was ever properly implemented (you can't do it
right without prepared statements) and I don't believe, it is the driver's job
to do it anyway. None of the other drivers do it either -- though they rely on
the respective client-libraries. Sybase/FreeTDS client does not offer such
checks -- so be it... The untainting was costly (done for each call) and not
guaranteed -- better to leave it to the caller, IMHO.
> Have you implemented prepared statements properly for all backends?
To the best of my knowledge, neither the Sybase's db-client library nor the
FreeTDS reimplementation of it offer prepared statements
<http://lists.ibiblio.org/pipermail/freetds/2010q1/025493.html>, unfortunately.
The current implementation certainly does not use them. Sybase offers a newer
client interface (ct-lib), that does have prepared statements (see ct_dynamic()
<http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.help.sdk_12.5.1.ctref/html/ctref/CHDIGFHG.htm>),
but FreeTDS does not provide it, so, for the driver to be compatible with both,
it has to use the old db-interface... To me this seems like an acceptable
caveat, as long as it is known and documented.
My patch does not touch any of the other backends.
Perhaps, some day I (or someone else) will implement apr_dbd_sybase.c -- using
the ct-interface, which would give prepared statements and other improvements.
But that would have to be maintained outside of Apache, because, foolishly,
Sybase would not open-source the client... It would also be Sybase-only (no MS
SQL Server).
For the time being, my patch offers a major improvement over the status quo --
the driver becomes usable. I am, in fact, preparing to use it with RewriteMaps
to do SEO for a giant site, that still uses an old Sybase-backed CMS. The
RewriteRules will ensure, no tainted keys are passed to the queries and the
DB-user's credentials will be limited to only SELECTs and only for a particular
table to mitigate the risk of any SQL-injection attack.
Yours,
-mi
Re: Limitations of mod_dbd - single server per vhost
Posted by Nick Kew <ni...@webthing.com>.
On Wed, 22 Aug 2012 09:21:16 -0400
"Mikhail T." <mi...@aldan.algebra.com> wrote:
> On 22.08.2012 07:24, Nick Kew wrote:
> > The implementation needs object persistence, of the kind we have
> > in the per-server configuration but not per-directory. That was the
> > original design constraint.
> I'd say, my complaint was a little misunderstood. It is not, that I wish
> to see separate DBD-configurations per location/directory, though it may
> be a good thing too.
Yes, the discussion got sidetracked. Probably my fault!
> Then RewriteMaps can then changed to operate thus:
>
> RewriteMap first "dbd:first:SELECT ...."
> RewriteMap second "dbd:second:SELECT ...."
Indeed that would follow if we had multiple backends. Indeed,
it could go further and capture multiple return fields, either
by name or in the manner of regexp backreferences. This kind
of incremental advance happens when someone feels motivated to
hack it.
> I don't think, I'm the best person to implement this -- whoever
> "quick-hacked" the dbd-maps in recently is, probably, better positioned
> to do this ;-)
It wasn't actually very recent:
https://svn.eu.apache.org/viewvc?view=revision&revision=454683
> P.S. BTW, I fixed the FreeTDS/Sybase driver
> <https://issues.apache.org/bugzilla/show_bug.cgi?id=53666> for APR/APU
> somewhat -- it has been absolutely broken and unusable for years, no
> kidding -- but don't see any movement on the ticket... Does anyone care?
To be fair, I think that driver carried some health warnings!
I think I've pointed a few people at the ODBC driver as an alternative.
Do you have a strong reason to use FreeTDS rather than ODBC?
One question here may be, does anyone have the platforms to test-drive
your patch? It seems to do rather more than just fix a simple bug:
you've removed the untainting code which was part of protecting against
injection attacks. Have you implemented prepared statements properly
for all backends?
It worked for its original users at the time it was written, but I guess
it's withered on the vine since then. Maybe it's some stumbling on some
difference between the different FreeTDS backends: you might even be
the first person to report trying it with Sybase!
--
Nick Kew
Re: Limitations of mod_dbd - single server per vhost
Posted by "Mikhail T." <mi...@aldan.algebra.com>.
On 22.08.2012 07:24, Nick Kew wrote:
> The implementation needs object persistence, of the kind we have
> in the per-server configuration but not per-directory. That was the
> original design constraint.
I'd say, my complaint was a little misunderstood. It is not, that I wish
to see separate DBD-configurations per location/directory, though it may
be a good thing too.
It is that I want to be able to use more than one server per vhost. This
collection of servers can be maintained in the persistent object you had
in mind, where the single server is maintained now.
Perhaps, the DBD-statements should begin accepting one more (optional)
argument -- the server's label:
DBDriver first freetds
DBDriver second mysql
DBDParams first "...."
DBDParams second "..........."
Then RewriteMaps can then changed to operate thus:
RewriteMap first "dbd:first:SELECT ...."
RewriteMap second "dbd:second:SELECT ...."
I don't think, I'm the best person to implement this -- whoever
"quick-hacked" the dbd-maps in recently is, probably, better positioned
to do this ;-) My employer's needs here are satisfied for now -- we
decided to use text-maps for local cache and thus need only a single
dbd-map for talking to database (if the local cache fails). But I can
give it a try, if no one else does.
Yours,
-mi
P.S. BTW, I fixed the FreeTDS/Sybase driver
<https://issues.apache.org/bugzilla/show_bug.cgi?id=53666> for APR/APU
somewhat -- it has been absolutely broken and unusable for years, no
kidding -- but don't see any movement on the ticket... Does anyone care?
Re: Limitations of mod_dbd - single server per vhost
Posted by Igor Galić <i....@brainsware.org>.
> > My rusty memory tells me that someone submitted a patch to fix this
> > a while ago?
https://issues.apache.org/bugzilla/show_bug.cgi?id=45456
Has someone had a chance to review/test it so far?
> Now that you mention it, wasn't something committed? I think that
> whatever we have is either there in the code or excluded because
> there were more problems with it than anyone was prepared to fix.
> But ICBW.
i
--
Igor Galić
Tel: +43 (0) 664 886 22 883
Mail: i.galic@brainsware.org
URL: http://brainsware.org/
GPG: 6880 4155 74BD FD7C B515 2EA5 4B1D 9E08 A097 C9AE
Re: Limitations of mod_dbd - single server per vhost
Posted by Nick Kew <ni...@webthing.com>.
On 22 Aug 2012, at 11:43, Graham Leggett wrote:
> On 22 Aug 2012, at 10:25 AM, Nick Kew wrote:
>
>>> From reading the module's documentation, it seems, a single vhost can only use one DB-server (and only with one set of credentials): only a single DBDriver and DBDParams is possible, for example.
>>
>> That is by design: it is expected to meet most needs.
>> If you wants more generality, go ahead and hack it.
>
> This is one of the things I'd like see minimised in the server implementation, the inclusion of arbitrary restrictions that later have to be lifted.
My words you quoted are somewhat oversimplified. The design didn't say
"we'll only allow one backend ...", it was just a case of not introducing
extra complexity.
The implementation needs object persistence, of the kind we have
in the per-server configuration but not per-directory. That was the
original design constraint.
> I went through the proxy and cache implementations for httpd v2.4 and removed all the per-virtual-host restrictions from all directives that didn't require it, as too often I was approached saying "we can't use the cache because feature X can only be set server wide and that breaks site Y". Ideally directives should be per-directory or per-location, unless there is a concrete reason otherwise.
Indeed, I seem to recollect doing some of that for ProxyPass & family
right back in the days of 2.0, when it was indeed a bug[1]! But that's a
different kettle of fish.
> My rusty memory tells me that someone submitted a patch to fix this a while ago?
Now that you mention it, wasn't something committed? I think that
whatever we have is either there in the code or excluded because
there were more problems with it than anyone was prepared to fix.
But ICBW.
[1] starting with r231355 and
http://marc.info/?l=apache-httpd-dev&m=110726027118798&w=2
--
Nick Kew
Re: Limitations of mod_dbd - single server per vhost
Posted by Graham Leggett <mi...@sharp.fm>.
On 22 Aug 2012, at 10:25 AM, Nick Kew wrote:
>> From reading the module's documentation, it seems, a single vhost can only use one DB-server (and only with one set of credentials): only a single DBDriver and DBDParams is possible, for example.
>
> That is by design: it is expected to meet most needs.
> If you wants more generality, go ahead and hack it.
This is one of the things I'd like see minimised in the server implementation, the inclusion of arbitrary restrictions that later have to be lifted. I went through the proxy and cache implementations for httpd v2.4 and removed all the per-virtual-host restrictions from all directives that didn't require it, as too often I was approached saying "we can't use the cache because feature X can only be set server wide and that breaks site Y". Ideally directives should be per-directory or per-location, unless there is a concrete reason otherwise.
My rusty memory tells me that someone submitted a patch to fix this a while ago?
Regards,
Graham
--
Re: Limitations of mod_dbd - single server per vhost
Posted by Nick Kew <ni...@webthing.com>.
On 21 Aug 2012, at 20:27, Mikhail T. wrote:
> From reading the module's documentation, it seems, a single vhost can only use one DB-server (and only with one set of credentials): only a single DBDriver and DBDParams is possible, for example.
That is by design: it is expected to meet most needs.
If you wants more generality, go ahead and hack it.
> Also, the syntax of mod_rewrite's dbd/fastdbd maps doesn't even allow to specify the DB-server,
Indeed, dbd support in rewritemap was only ever a quick hack.
Some of us think mod_rewrite is long overdue retirement.
> implying, once again, there can only be one in a given context (entire server or virtual host). The missing capability could be useful to someone, who would want, for example, to check a local SQLite-database for data and, failing to find anything, attempt a query against a remote MySQL or Sybase server. Other enterprises may want to talk to multiple remote servers (potentially using different drivers).
Well, you could hack it into mod_rewrite, but isn't that traveling
further down the road of using mod_rewrite as a pseudo-
programming language? The provision of <If "expr"> syntax
and of mod_lua is supposed to give you better tools than
single-IF (RewriteCond) and GOTO (RewriteRule).
Also bear in mind that without a query cacheing regime comparable
to what mod_authn_socache provides for authentication, that kind
of complexity would most likely mean rather a steep overhead.
> I would've filed an enhancement-request,
Feel free to file an enhancement request. Feel free to submit a patch.
--
Nick Kew