You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Paul Querna <ch...@force-elite.com> on 2004/11/17 19:34:41 UTC
PCRE in 2.1/2.2
Is there any reason to keep libpcre in the httpd tree?
I believe we should use the PCRE installed on the system. If its not on
the system, the user can install it. There are several bugs from 3rd
party modules that also try to use pcre, and get conflicting symbols
because of the version embedded within apache 2.0.
No comments in a week, and Ill start taking it out.
-Paul Querna
Re: PCRE in 2.1/2.2
Posted by Joe Orton <jo...@redhat.com>.
On Wed, Nov 17, 2004 at 05:01:48PM -0800, Brian Pane wrote:
> About two years ago, I made some performance optimizations to the copy
> of pcre in the httpd-2.0 tree. I submitted the diffs to the pcre
> maintainer, but I don't know if they've made it into a subsequent pcre
> release.
Yep, these have been in upstream releasessince pcre 4.0, with
SMALL_NMATCH renamed to POSIX_MALLOC_THRESHOLD.
joe
Re: PCRE in 2.1/2.2
Posted by Brian Pane <br...@apache.org>.
Paul Querna wrote:
> Is there any reason to keep libpcre in the httpd tree?
>
> I believe we should use the PCRE installed on the system. If its not
> on the system, the user can install it. There are several bugs from
> 3rd party modules that also try to use pcre, and get conflicting
> symbols because of the version embedded within apache 2.0.
>
> No comments in a week, and Ill start taking it out.
>
> -Paul Querna
About two years ago, I made some performance optimizations to the copy
of pcre
in the httpd-2.0 tree. I submitted the diffs to the pcre maintainer,
but I don't know
if they've made it into a subsequent pcre release. The changes, as I
recall, were to
remove some mallocs/frees in the regexec function. They may or may not
still be
needed with newer versions of pcre, if the pcre code has changed
substantially
since then.
<http://marc.theaimsgroup.com/?t=101659005500001>
Brian
Re: PCRE in 2.1/2.2
Posted by Richard Jones <ri...@annexia.org>.
On Mon, Dec 13, 2004 at 12:57:36AM -0700, Paul Querna wrote:
> I agree with you however, but this is the current state of httpd.
> Patches to allow using of external libs for everything we currently
> embed are extremely welcome.
For external PCRE:
http://issues.apache.org/bugzilla/show_bug.cgi?id=27550
http://issues.apache.org/bugzilla/attachment.cgi?id=13377
Rich.
--
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
>>> http://www.team-notepad.com/ - collaboration tools for teams <<<
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
Write Apache modules in OCaml - http://www.merjis.com/developers/mod_caml
Re: PCRE in 2.1/2.2
Posted by Enrico Weigelt <we...@metux.de>.
* Paul Querna <ch...@force-elite.com> wrote:
> Enrico Weigelt wrote:
> <snip>
> >>Dependencies are to be tracked by a vendor, not by our users who
> >>download the source from us.
> >
> >Ah, great. And when will be the day when glibc is included in httpd ?
>
> Never. glibc is GPL/LGPLed, and the httpd server project cannot
> distribute it as part of our license and ASF policy. The BSD libc
> however...</tonge-in-cheek>
Aha, if glibc was released under asf license, it would already
be in ?
...
Tip: too much sun makes silly ...
cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service
phone: +49 36207 519931 www: http://www.metux.de/
fax: +49 36207 519932 email: contact@metux.de
cellphone: +49 174 7066481
---------------------------------------------------------------------
-- DSL ab 0 Euro. -- statische IP -- UUCP -- Hosting -- Webshops --
---------------------------------------------------------------------
Re: PCRE in 2.1/2.2
Posted by Paul Querna <ch...@force-elite.com>.
Enrico Weigelt wrote:
<snip>
>>Dependencies are to be tracked by a vendor, not by our users who
>>download the source from us.
>
> Ah, great. And when will be the day when glibc is included in httpd ?
Never. glibc is GPL/LGPLed, and the httpd server project cannot
distribute it as part of our license and ASF policy. The BSD libc
however...</tonge-in-cheek>
-Paul
Re: PCRE in 2.1/2.2
Posted by Enrico Weigelt <we...@metux.de>.
* Paul Querna <ch...@force-elite.com> wrote:
<snip>
> srclib/apr-util/configure --help | grep expat
> --with-expat=DIR specify Expat location
Ah, yeah. I've grepped for --with-pcre, which is missing.
I'm now trying it on my buildfarm ...
<snip>
> >btw: why is it a goal to put all dependencies into the package
> >just to be able to do "make && make install" w/o any preparations ?!
> >
> >dependencies are to be tracked by the computer, 100% automatically.
>
> Dependencies are to be tracked by a vendor, not by our users who
> download the source from us.
Ah, great. And when will be the day when glibc is included in httpd ?
Dependencies are to be tracked by the installation system.
> I agree with you however, but this is the current state of httpd.
> Patches to allow using of external libs for everything we currently
> embed are extremely welcome.
If the configure script was something carefully hand-written and not
autoshitt'ed, I would go ahead and do it. But I refuse to spend more
time on autoshit as I'm already urged to.
cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service
phone: +49 36207 519931 www: http://www.metux.de/
fax: +49 36207 519932 email: contact@metux.de
cellphone: +49 174 7066481
---------------------------------------------------------------------
-- DSL ab 0 Euro. -- statische IP -- UUCP -- Hosting -- Webshops --
---------------------------------------------------------------------
Re: PCRE in 2.1/2.2
Posted by Paul Querna <ch...@force-elite.com>.
Enrico Weigelt wrote:
> * Paul Querna <ch...@force-elite.com> wrote:
>
> <snip>
>
>>>shot yourself.
>>>apache2 requires expat, which is less common than pcre.
>>
>>does not. APR-Util embeds expat.
>
> eh? uglier than i thouht!
>
> why isnt there an --with-expat=... option for enforcing use of
> external expat ?
>
There is:
srclib/apr-util/configure --help | grep expat
--with-expat=DIR specify Expat location
> btw: why is it a goal to put all dependencies into the package
> just to be able to do "make && make install" w/o any preparations ?!
>
> dependencies are to be tracked by the computer, 100% automatically.
Dependencies are to be tracked by a vendor, not by our users who
download the source from us.
I agree with you however, but this is the current state of httpd.
Patches to allow using of external libs for everything we currently
embed are extremely welcome.
-Paul
Re: PCRE in 2.1/2.2
Posted by Enrico Weigelt <we...@metux.de>.
* Paul Querna <ch...@force-elite.com> wrote:
<snip>
> >shot yourself.
> >apache2 requires expat, which is less common than pcre.
>
> does not. APR-Util embeds expat.
eh? uglier than i thouht!
why isnt there an --with-expat=... option for enforcing use of
external expat ?
embedding standard libraries adds unnecessary complexity, wastes
resources and limits flexibility.
btw: why is it a goal to put all dependencies into the package
just to be able to do "make && make install" w/o any preparations ?!
dependencies are to be tracked by the computer, 100% automatically.
cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service
phone: +49 36207 519931 www: http://www.metux.de/
fax: +49 36207 519932 email: contact@metux.de
cellphone: +49 174 7066481
---------------------------------------------------------------------
-- DSL ab 0 Euro. -- statische IP -- UUCP -- Hosting -- Webshops --
---------------------------------------------------------------------
Re: PCRE in 2.1/2.2
Posted by Paul Querna <ch...@force-elite.com>.
Enrico Weigelt wrote:
> * Joe Orton <jo...@redhat.com> wrote:
>
> <snip>
>
>>>Is there any reason to keep libpcre in the httpd tree?
>>
>>untar && configure && make must "just work" on any random Unix system.
>
>
> shot yourself.
> apache2 requires expat, which is less common than pcre.
does not. APR-Util embeds expat.
-Paul
Re: PCRE in 2.1/2.2
Posted by Enrico Weigelt <we...@metux.de>.
* Joe Orton <jo...@redhat.com> wrote:
<snip>
> > Is there any reason to keep libpcre in the httpd tree?
>
> untar && configure && make must "just work" on any random Unix system.
shot yourself.
apache2 requires expat, which is less common than pcre.
<snip>
> Those are bugs which can be fixed by optionally using an external PCRE
> library (configure --with-pcre) and/or properly namespacing the symbols
Are you shure this option really exists ?
> defined by the bundled copy of PCRE. Removing the bundled PCRE is not
> necessary to that goal or otherwise desirable, though.
The original question was exactly the opposite: is it necessary always
to have a bundled pcre ?
cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service
phone: +49 36207 519931 www: http://www.metux.de/
fax: +49 36207 519932 email: contact@metux.de
cellphone: +49 174 7066481
---------------------------------------------------------------------
-- DSL ab 0 Euro. -- statische IP -- UUCP -- Hosting -- Webshops --
---------------------------------------------------------------------
Re: PCRE in 2.1/2.2
Posted by Brian Pane <br...@apache.org>.
Joe Orton wrote:
>On Fri, Nov 26, 2004 at 11:55:35PM -0800, Brian Pane wrote:
>
>
>>Yep. I just did that, and it worked as expected. I ended up with a
>>clean merge: the only files that needed merge conflict resolution were
>>those that had been modified in the srclib/pcre copy:
>>
>>
>
>Looks good, thanks Brian. Do you think it's worth keeping all ~1Mb of
>the doc/ directory in srclib/pcre? I think it might as well be removed,
>they won't get installed and developers can refer to the vendor/pcre
>copies if necessary. svn merge will happily ignore the removed files
>when merging to future versions so there's no issue there.
>
>
+1, I'll remove docs/* from srclib/pcre and add a README pointing to
vendor/pcre.
Brian
Re: PCRE in 2.1/2.2
Posted by Joe Orton <jo...@redhat.com>.
On Fri, Nov 26, 2004 at 11:55:35PM -0800, Brian Pane wrote:
> Yep. I just did that, and it worked as expected. I ended up with a
> clean merge: the only files that needed merge conflict resolution were
> those that had been modified in the srclib/pcre copy:
Looks good, thanks Brian. Do you think it's worth keeping all ~1Mb of
the doc/ directory in srclib/pcre? I think it might as well be removed,
they won't get installed and developers can refer to the vendor/pcre
copies if necessary. svn merge will happily ignore the removed files
when merging to future versions so there's no issue there.
joe
Re: PCRE in 2.1/2.2
Posted by Brian Pane <br...@apache.org>.
Joe Orton wrote:
>I also spent some time recently working out how to apply that process to
>srclib/pcre and it seemed fairly straight-forward. It seems like the
>process works OK even if the step to copy the original vendor
>pcre/current to srclib/pcre is skipped, so the current srclib/pcre can
>be used to merge into. The steps I came up with would be just:
>
> import pcre-3.9 to vendor/pcre/current
> cp vendor/pcre/current vendor/pcre/pcre-3.9
> unpack pcre-5.0 into vendor/pcre/current, svn add everything then commit
> verify that vendor/pcre/current exactly matches contents of pcre-5.0.tar.gz
> cp vendor/pcre/current vendor/pcre/pcre-5.0
>
>and then the merge between pcre-3.9 and current can happen. You concur?
>
>
Yep. I just did that, and it worked as expected. I ended up with a
clean merge: the only files that needed merge conflict resolution were
those that had been modified in the srclib/pcre copy:
pcreposix.c (conflicts due to my old SMALL_NMATCH change, as expected; I
replaced this whole file with the 5.0 version)
Makefile.in
configure.in
For the latter two files, I hand-merged just the essential changes from
5.0, leaving out the code that would require aclocal.
Brian
Re: PCRE in 2.1/2.2
Posted by Joe Orton <jo...@redhat.com>.
On Thu, Nov 25, 2004 at 04:00:46PM -0800, Brian Pane wrote:
> I volunteer to upgrade the copy of PCRE in httpd-2.1 to the latest
> version. I'll have a bit of time to work on this over the next few days.
>
> One question, though...
>
> I'd like to follow the vendor-branch approach outlined in the Subversion
> book, <http://svnbook.red-bean.com/en/1.0/ch07s04.html>. In this
> approach, a clean copy of the 3rd party code is maintained on a branch
> under some unique path, so that it's easy to generate a diff between
> versions M and N of the 3rd party code that can then be merged into our
> code base.
Sounds great!
I also spent some time recently working out how to apply that process to
srclib/pcre and it seemed fairly straight-forward. It seems like the
process works OK even if the step to copy the original vendor
pcre/current to srclib/pcre is skipped, so the current srclib/pcre can
be used to merge into. The steps I came up with would be just:
import pcre-3.9 to vendor/pcre/current
cp vendor/pcre/current vendor/pcre/pcre-3.9
unpack pcre-5.0 into vendor/pcre/current, svn add everything then commit
verify that vendor/pcre/current exactly matches contents of pcre-5.0.tar.gz
cp vendor/pcre/current vendor/pcre/pcre-5.0
and then the merge between pcre-3.9 and current can happen. You concur?
> Do we have a standard yet for where vendor branches should be located
> within the httpd Subversion repository? E.g., should the clean PCRE
> releases be stored under svn.apache.org/repos/asf/httpd/vendor/pcre? or
> svn.apache.org/repos/vendor/pcre? Or something else entirely?
I think /asf/httpd/httpd/vendor is the right place, putting it alongside
the trunk and branches, giving:
/repos/asf/httpd/httpd/vendor
/repos/asf/httpd/httpd/vendor/pcre
/repos/asf/httpd/httpd/vendor/pcre/current
/repos/asf/httpd/httpd/vendor/pcre/pcre-3.9
/repos/asf/httpd/httpd/vendor/pcre/pcre-5.0
Regards,
joe
Re: PCRE in 2.1/2.2
Posted by André Malo <nd...@perlig.de>.
* Brian Pane wrote:
> Okay, I checked out the patch attached to that bug. Supporting the use
> of an external copy of PCRE makes sense.
+1 on concept.
> The patch also changes various
> core PCRE function names to ap_*. That definitely helps prevent
> collisions with libc or libpthread versions of the regex functions on
> some platforms, but changing the PCRE function names may be
> controversial... would any other committers care to chime in with their
> thoughts on this?
I'm wondering if there isn't a better solution for this. But if we'd need to
change the names, I think we should do this more centralized, i.e. a header
file, which contains some defines and is included in proper files. This
would keep the patches against the upstream more clear. Shouldn't that
work?
> Meanwhile, now that we have the latest release of PCRE in httpd-2.1, one
> option would help somewhat would be to backport it into httpd-2.0.
I'm -1 on backporting PCRE 5.0 to 2.0.
nd
--
Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook! Ook? Ook. Ook? Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook. Ook.
Ook. Ook. Ook. Ook. Ook? Ook. Ook! Ook! Ook? Ook! Ook. Ook? Ook. Ook.
Ook. Ook. Ook. Ook. Ook! Ook. Ook! Ook! Ook! Ook! Ook! Ook.
Re: PCRE in 2.1/2.2
Posted by André Malo <nd...@perlig.de>.
* Richard Jones wrote:
> Is this (binary compatibility) really a policy?
Yes.
nd
--
"Umfassendes Werk (auch fuer Umsteiger vom Apache 1.3)"
-- aus einer Rezension
<http://pub.perlig.de/books.html#apache2>
Re: PCRE in 2.1/2.2
Posted by Brian Pane <br...@apache.org>.
Justin Erenkrantz wrote:
> --On Sunday, November 28, 2004 3:36 PM -0800 Brian Pane
> <br...@apache.org> wrote:
>
>> Meanwhile, now that we have the latest release of PCRE in httpd-2.1, one
>> option would help somewhat would be to backport it into httpd-2.0.
>
>
> I doubt we'd be able to maintain binary compatibility if we did that.
> So, I'd strongly urge that we don't touch PCRE in 2.0.x. -- justin
From a binary compatibility perspective, what's the specific PCRE API
change you're concerned about?
Thanks,
Brian
Re: PCRE in 2.1/2.2
Posted by Richard Jones <ri...@annexia.org>.
On Sun, Nov 28, 2004 at 03:39:33PM -0800, Justin Erenkrantz wrote:
> --On Sunday, November 28, 2004 3:36 PM -0800 Brian Pane <br...@apache.org>
> wrote:
>
> >Meanwhile, now that we have the latest release of PCRE in httpd-2.1, one
> >option would help somewhat would be to backport it into httpd-2.0.
>
> I doubt we'd be able to maintain binary compatibility if we did that. So,
> I'd strongly urge that we don't touch PCRE in 2.0.x. -- justin
Is this (binary compatibility) really a policy? The problem right now
is that people are asking to run my software on Apache 2, and I cannot
offer that because my software links with the system PCRE via an
external library dependency.
Rich.
--
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
>>> http://www.team-notepad.com/ - collaboration tools for teams <<<
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
http://winwinsales.co.uk/ - CRM consultancy
Re: PCRE in 2.1/2.2
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
--On Sunday, November 28, 2004 3:36 PM -0800 Brian Pane <br...@apache.org>
wrote:
> Meanwhile, now that we have the latest release of PCRE in httpd-2.1, one
> option would help somewhat would be to backport it into httpd-2.0.
I doubt we'd be able to maintain binary compatibility if we did that. So, I'd
strongly urge that we don't touch PCRE in 2.0.x. -- justin
Re: PCRE in 2.1/2.2
Posted by Brian Pane <br...@apache.org>.
Brian Pane wrote:
> Richard Jones wrote:
>
>> On Thu, Nov 25, 2004 at 04:00:46PM -0800, Brian Pane wrote:
>>
>>
>>> I volunteer to upgrade the copy of PCRE in httpd-2.1 to the latest
>>> version. I'll have a bit of time to work on this over the next few
>>> days.
>>>
>>
>>
>> Any chance of integrating the fix in bug 27550 at the same time?
>>
>> http://issues.apache.org/bugzilla/show_bug.cgi?id=27550
>>
>> Rich.
>>
>
> I'll take a look at the patch in 27550 some time this weekend if
> nobody else gets to it first.
Okay, I checked out the patch attached to that bug. Supporting the use
of an external copy of PCRE makes sense. The patch also changes various
core PCRE function names to ap_*. That definitely helps prevent
collisions with libc or libpthread versions of the regex functions on
some platforms, but changing the PCRE function names may be
controversial... would any other committers care to chime in with their
thoughts on this?
Meanwhile, now that we have the latest release of PCRE in httpd-2.1, one
option would help somewhat would be to backport it into httpd-2.0.
Brian
Re: PCRE in 2.1/2.2
Posted by Brian Pane <br...@apache.org>.
Richard Jones wrote:
>On Thu, Nov 25, 2004 at 04:00:46PM -0800, Brian Pane wrote:
>
>
>>I volunteer to upgrade the copy of PCRE in httpd-2.1 to the latest
>>version. I'll have a bit of time to work on this over the next few days.
>>
>>
>
>Any chance of integrating the fix in bug 27550 at the same time?
>
>http://issues.apache.org/bugzilla/show_bug.cgi?id=27550
>
>Rich.
>
I'll take a look at the patch in 27550 some time this weekend if nobody
else gets to it first.
Brian
Re: PCRE in 2.1/2.2
Posted by Richard Jones <ri...@annexia.org>.
On Thu, Nov 25, 2004 at 04:00:46PM -0800, Brian Pane wrote:
> I volunteer to upgrade the copy of PCRE in httpd-2.1 to the latest
> version. I'll have a bit of time to work on this over the next few days.
Any chance of integrating the fix in bug 27550 at the same time?
http://issues.apache.org/bugzilla/show_bug.cgi?id=27550
Rich.
--
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
>>> http://www.team-notepad.com/ - collaboration tools for teams <<<
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
http://winwinsales.co.uk/ - CRM consultancy
Re: PCRE in 2.1/2.2
Posted by Brian Pane <br...@apache.org>.
I volunteer to upgrade the copy of PCRE in httpd-2.1 to the latest
version. I'll have a bit of time to work on this over the next few days.
One question, though...
I'd like to follow the vendor-branch approach outlined in the Subversion
book, <http://svnbook.red-bean.com/en/1.0/ch07s04.html>. In this
approach, a clean copy of the 3rd party code is maintained on a branch
under some unique path, so that it's easy to generate a diff between
versions M and N of the 3rd party code that can then be merged into our
code base.
Do we have a standard yet for where vendor branches should be located
within the httpd Subversion repository? E.g., should the clean PCRE
releases be stored under svn.apache.org/repos/asf/httpd/vendor/pcre? or
svn.apache.org/repos/vendor/pcre? Or something else entirely?
Thanks,
Brian
Re: PCRE in 2.1/2.2
Posted by Joe Orton <jo...@redhat.com>.
On Wed, Nov 17, 2004 at 09:05:38PM +0100, André Malo wrote:
> But we should take the effort and try to keep up to date. Current PCRE version
> is 5.0 with many many bugfixes and optimizations since 3.9, which we're using.
>
> I've tried to update the stuff from time to time, but my autofoo knowledge
> seems to be too limited yet. I'd really appreciate if someone would work on
> such a task :)
Yeah I've also tried and ended up thinking "hmm this should really be
using a vendor branch" and then my head explodes... so maybe once the
CVS->SVN conversion it'll be easier and I'll have another go :). I did
notice the local SMALL_NMATCH changes have been merged upstream though
which is nice.
joe
Re: PCRE in 2.1/2.2
Posted by André Malo <nd...@perlig.de>.
* Joe Orton <jo...@redhat.com> wrote:
> On Wed, Nov 17, 2004 at 10:34:41AM -0800, Paul Querna wrote:
> > Is there any reason to keep libpcre in the httpd tree?
>
> untar && configure && make must "just work" on any random Unix system.
>
> > I believe we should use the PCRE installed on the system. If its not on
> > the system, the user can install it. There are several bugs from 3rd
> > party modules that also try to use pcre, and get conflicting symbols
> > because of the version embedded within apache 2.0.
>
> Those are bugs which can be fixed by optionally using an external PCRE
> library (configure --with-pcre) and/or properly namespacing the symbols
> defined by the bundled copy of PCRE. Removing the bundled PCRE is not
> necessary to that goal or otherwise desirable, though.
But we should take the effort and try to keep up to date. Current PCRE version
is 5.0 with many many bugfixes and optimizations since 3.9, which we're using.
I've tried to update the stuff from time to time, but my autofoo knowledge
seems to be too limited yet. I'd really appreciate if someone would work on
such a task :)
nd
--
"Die Untergeschosse der Sempergalerie bleiben währenddessen aus
statistischen Gründen geflutet." -- Spiegel Online
Re: PCRE in 2.1/2.2
Posted by Joe Orton <jo...@redhat.com>.
On Wed, Nov 17, 2004 at 10:34:41AM -0800, Paul Querna wrote:
> Is there any reason to keep libpcre in the httpd tree?
untar && configure && make must "just work" on any random Unix system.
> I believe we should use the PCRE installed on the system. If its not on
> the system, the user can install it. There are several bugs from 3rd
> party modules that also try to use pcre, and get conflicting symbols
> because of the version embedded within apache 2.0.
Those are bugs which can be fixed by optionally using an external PCRE
library (configure --with-pcre) and/or properly namespacing the symbols
defined by the bundled copy of PCRE. Removing the bundled PCRE is not
necessary to that goal or otherwise desirable, though.
joe
Re: PCRE in 2.1/2.2
Posted by Bill Stoddard <bi...@wstoddard.com>.
Paul Querna wrote:
> Is there any reason to keep libpcre in the httpd tree?
>
> I believe we should use the PCRE installed on the system. If its not on
> the system, the user can install it. There are several bugs from 3rd
> party modules that also try to use pcre, and get conflicting symbols
> because of the version embedded within apache 2.0.
And relying on the OS reg ex support would make this problem go away? I think not. I prefer to use PCRE
because at least it is consistent across operating systems (it is, right?), more so than OSes are consistent
with each other in handling regexs. My $.02. And what about Windows? to the best of my knowledge, it does not
support a native regex library.
Bill