You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Luke Kenneth Casson Leighton <lk...@samba-tng.org> on 2001/06/06 16:28:43 UTC
XMLvL, apache, apr, mod_virgule origins: advice needed [LONG]
hi,
i need some help, and so i thought it best to ask here.
i am not subscribed to new-httpd@apache.org but i do
have post access, so if you reply, please cc me, but
i will keep an eye on the archives, just in case.
background
----------
raph levien developed mod_virgule (which runs advogato.org)
as a community-run forum engine.
it is 12,000 lines of hard-coded c that uses a tiny amount
of glib, a significant amount of libxml 1.8.x, and a lot
of ap_xxx routines.
raph's original comments, back in November 1999, were that
he wanted to see mod_virgule do less hard-coded c and more
scripting.
so, i started doing exactly that. four months later,
xvl 1.0.2 was released: http://virgule.sourceforge.net.
it's still only 12,000 or so lines of code.
development stages
------------------
mod_virgule was originally an apache module. it's actually
a web-server-in-a-web-server. its design is therefore a
little unusual: it generates dynamic content from site xml files.
it doesn't use apache authentication, in fact there's a whole
boat-load of stuff it can't use. it provides its own
authentication, from cookies stored in xml-based
user-profile files.
things i added:
- cgi-bin capability to run bash scripts: i wanted to
run sendmail, for example. this required cutting
large chunks of code across from mod_cgi.
- xml-based database capabilities. use one XML file
to specify the format of another XML file, for storage
and retrieval. forms can be generated to do data input.
- certification-based access control.
then, i started to get it. mod_virgule really _was_
a web-server-inside-a-web-server. so, i added:
- socket communication (both TCP/IP and unix-domains)
and also, i started to realise, huh, this is a bit like
PHP.EXE, so why not actually split out to a separate
executable, too?
so, i:
- created a unix-domain-socket version of mod_proxy,
with an XML syntax for the communication
- used the socket communication capability to create
a socket for the ux-ds-proxy to listen to.
- split out ap_sprintf.c, alloc.c, signals.c and a
_whole_ boat-load of apache 1.3.17 source code,
making it stand-alone compilable, removing block_signals()
and all sorts of horrible stuff that i wasn't going
to need in a unix fork-only environment.
- removed the cgi-bin capability because there was
absolutely no way i was going to cut the ap_exec()
code over, as well.
at the end of this coding spree, i had mod_virgule
as a tiny module that acts as a communication
mechanism between apache and other stand-alone
programs, which is an extremely useful thing to
have in its own right, and i had a stand-alone
executable - xvl - which was capable of running
as a mini-web-content server and a whole lot more.
things i didn't get around to doing:
- dual-compilation, ala mod_php and PHP.EXE
- re-enabling the cgi-bin capability
- working out how the hell php can rely on
so much apache code, which basically means
linking with libmain.a et al, when it's not
a public library, and how i could do the same,
instead of cutting across great swathes of
the apache codebase
- using APR and apr_util, not apache 1.3.17 code.
- using apache's HTTP code - or _whether_ to use
apache's HTTP code - in order to make xvl truly
capable of being a stand-alone web server.
problem statement
-----------------
in other words, xvl is _the_ perfect candidate
for significant integration / use of the apache
codebase, but the boundaries for doing such
radically different code generation - ala
mod_php, mod_snake, mod_xvl etc. - are not
clearly delineated or documented.
in order to progress xvl, i need to be able
to leverage the HTTP decoding and ap_xxx
codebase - without all the trappings, bells
and whistles of a fully-blown httpd
installation, if users don't want one, and
_also_ seamless integration into an existing
httpd installation, if they do.
i really want my cgi-bin capabilities back!
i want to be able to run sendmail from xvl
scripts!
i want my cake _and_ a knife, plate and spoon
_and_ the time to eat it at leisure [the cake].
please.
:)
so. any suggestions? what components already
exist? i'm using a brain-dead installation
of mandrake 7.0 with glibc 2.2 (so i get
error, cannot find symbol dl_init_next@@GLIBC_2_0
whenever i compile a shared library with -ldl,
that includes libxml2, apr, pretty much damn
well everything: anyone any clues? i'm
installing a lot of RPMs recently to overcome
this, _when_ they're available...)
xvl uses the ap_pool code, the table code,
list code, ap_psprintf, i need to exec
/ spawn programs, ideally i also need a
portable version of unix-domain-sockets, which
i understand doesn't exist in APR, yet.
Re: XMLvL, apache, apr, mod_virgule origins: advice needed [LONG]
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Thu, Jun 07, 2001 at 11:19:18AM +0200, Daniel Veillard wrote:
> On Thu, Jun 07, 2001 at 02:14:58AM -0700, Greg Stein wrote:
> > APRUTIL also locates Expat on the system for you, or builds its bundled copy
> > of Expat. You could drop your libxml2 dependency. Even better, there has
> > been a call for a SAX-like API in APRUTIL, backed by Expat, libxml, or
> > Xerces. If you're motivated, you could implement that API for APRUTIL :-)
>
> Actually I doubt mod_virgule would stick at the SAX level. I didn't
> looked at the code but from the exchanges I had with Raph Levien when
> he developped it, I'm pretty sure it uses the DOM tree build of libxml2.
DOM tree, libxml 1.8.x.
the version on http://virgule.sourceforge.net - xvl [a stand-alone
executable] - makes heavy usage of the SAX interface in 2.x.
i even use one of the _internal_ SAX functions, and i'm praying you
don't delete it from the library!
> Now whether switching to Xerces makes sense or not is a question I'm
> probably too biased to answer. The irony is that I started libxml in
> 99 when trying to build a small WebDAV implementation for Apache.
funny, neh :)
Re: XMLvL, apache, apr, mod_virgule origins: advice needed [LONG]
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Thu, Jun 07, 2001 at 11:19:18AM +0200, Daniel Veillard wrote:
> On Thu, Jun 07, 2001 at 02:14:58AM -0700, Greg Stein wrote:
> > APRUTIL also locates Expat on the system for you, or builds its bundled copy
> > of Expat. You could drop your libxml2 dependency. Even better, there has
> > been a call for a SAX-like API in APRUTIL, backed by Expat, libxml, or
> > Xerces. If you're motivated, you could implement that API for APRUTIL :-)
>
> Actually I doubt mod_virgule would stick at the SAX level. I didn't
> looked at the code but from the exchanges I had with Raph Levien when
> he developped it, I'm pretty sure it uses the DOM tree build of libxml2.
DOM tree, libxml 1.8.x.
the version on http://virgule.sourceforge.net - xvl [a stand-alone
executable] - makes heavy usage of the SAX interface in 2.x.
i even use one of the _internal_ SAX functions, and i'm praying you
don't delete it from the library!
> Now whether switching to Xerces makes sense or not is a question I'm
> probably too biased to answer. The irony is that I started libxml in
> 99 when trying to build a small WebDAV implementation for Apache.
funny, neh :)
Re: XMLvL, apache, apr, mod_virgule origins: advice needed [LONG]
Posted by Daniel Veillard <Da...@imag.fr>.
On Thu, Jun 07, 2001 at 02:14:58AM -0700, Greg Stein wrote:
> APRUTIL also locates Expat on the system for you, or builds its bundled copy
> of Expat. You could drop your libxml2 dependency. Even better, there has
> been a call for a SAX-like API in APRUTIL, backed by Expat, libxml, or
> Xerces. If you're motivated, you could implement that API for APRUTIL :-)
Actually I doubt mod_virgule would stick at the SAX level. I didn't
looked at the code but from the exchanges I had with Raph Levien when
he developped it, I'm pretty sure it uses the DOM tree build of libxml2.
Now whether switching to Xerces makes sense or not is a question I'm
probably too biased to answer. The irony is that I started libxml in
99 when trying to build a small WebDAV implementation for Apache.
Daniel
--
Daniel Veillard | Red Hat Network http://redhat.com/products/network/
veillard@redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Re: XMLvL, apache, apr, mod_virgule origins: advice needed [LONG]
Posted by Daniel Veillard <Da...@imag.fr>.
On Thu, Jun 07, 2001 at 02:14:58AM -0700, Greg Stein wrote:
> APRUTIL also locates Expat on the system for you, or builds its bundled copy
> of Expat. You could drop your libxml2 dependency. Even better, there has
> been a call for a SAX-like API in APRUTIL, backed by Expat, libxml, or
> Xerces. If you're motivated, you could implement that API for APRUTIL :-)
Actually I doubt mod_virgule would stick at the SAX level. I didn't
looked at the code but from the exchanges I had with Raph Levien when
he developped it, I'm pretty sure it uses the DOM tree build of libxml2.
Now whether switching to Xerces makes sense or not is a question I'm
probably too biased to answer. The irony is that I started libxml in
99 when trying to build a small WebDAV implementation for Apache.
Daniel
--
Daniel Veillard | Red Hat Network http://redhat.com/products/network/
veillard@redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
> > If the server spawns the programs, then you can also use regular pipes.
>
> I have been thinking of creating apr/rpc/... to handle stuff like this.
> However, right now, we have named pipes. They need to be implemented on
> more platforms, and that may require changing the API a bit, but please
> let's stick with what we have already.
>
> The only thing we can't do with named pipes today is communicate with
> different machines. IMHO, calling any cross-machine communication medium
> a named pipe is just going to confuse any unix programmer. Give it a
> different name.
hiya ryan,
i promised i'd get back to you after having had to leave quickly.
RPC. okay. building RPC directly into apr libs is something i can
strongly recommend you don't do: it could easily get out of hand,
make the code bulky, and most people would wonder what drugs you
were taking today [i.e. it goes against what i am beginning to understand
to be the APR ethos].
however, what i _can_ recommend is a slim wrapper - a *local*
named-pipe (ux-dom-sock-based, tcp-loopback-based, NT-NamedPipe-based,
doesn't matter what the implementation is) that feeds to a
separate program that is responsible for locating and communicating
remote procedures.
a bit like the epmapper on dce/rpc but also a transport as well.
it will take a bit of work to design, however what you end up with
is a very small library that apps have to link with, they also
have to run this [entirely separate] program, and voila you have
RPC.
now, if you're concerned about this design, well... uh... how
do you think NT provides RPC? :) :) :)
worse than that, they put these mechanisms *in the kernel*!
[services running as System, and of course they _have_ to
be optimised and of _course_ that means running parts in the
kernel, agh!]
[i.e., if you're concerned, be concerned about using NT :) ]
all best,
lukes
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 08:04:53AM -0700, rbb@covalent.net wrote:
> > > > BTW, APR already has named pipes. Just what are we trying to solve here?
> >
> > We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> > simply doesn't have an implementation for it.
>
> So, the solution is to implement them on other platforms, not to re-invent
> them. For historical completeness however, allow me to explain how we got
> where we are with named pipes. Unix has them becaues they are easy to
> implement. Windows doesn't have them because only certain versions of
> Windows supports named pipes. Namely, NT and 2000 have named pipes, 9x
> doesn't.
grep NamedPipes */*.c shows some code that uses NPs.
there is a nice comment about 9x not having some
feature [blocking or something], and it says, well,
uhhh.... TOUGH! :)
so, there does already exist code in apr that uses
CreateNamedPipe() - for different purposes.
> Originally, APR supported named pipes on Windows,
great!!!
> they were
> removed in revision 1.11. From looking at the log, I don't understand
> why.
*curious*.
> > > i looked at the named pipe code: i believe that you are thinking
> > > of 'unix' named pipes, which are a totally diffrerent beast
> > > from nt named pipes.
> >
> > Right. I don't recall all the bits about creating named pipes in Windows,
> > but it may be feasible to use the "filename" in the
> > apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> > other words, it may be a simple matter of writing the function for Windows.
> > But then again, the function params may not be rich enough for what is
> > needed.
>
> See above, this has worked in the past, all we need to do is copy the code
> back into place, and possibly tweak it.
yesplease! less work!
> > >...
> > > - a means to set up a server and have other programs (not
> > > remote programs) connect to and communicate with that server.
> > >
> > > the semantics must be identical to those of unix-domain-sockets,
> > > namely a listen, bind, accept, read, write and close.
> >
> > I would think you could use named pipes, TCP sockets, or Unix sockets.
> >
> > If the server spawns the programs, then you can also use regular pipes.
>
> I have been thinking of creating apr/rpc/... to handle stuff like this.
> However, right now, we have named pipes. They need to be implemented on
> more platforms, and that may require changing the API a bit, but please
> let's stick with what we have already.
>
> The only thing we can't do with named pipes today is communicate with
> different machines. IMHO, calling any cross-machine communication medium
> a named pipe is just going to confuse any unix programmer. Give it a
> different name.
okay.
i am familiar with how remote IPC works, from a few perspectives,
having implemented a subset of DCE/RPC that is MS-compatible,
over SMB, for Samba (ISBN 1578701503).
[eek, have bus to catch soon, gotta run].
i can attest to the complexity of implementing such: it's
not something to add to a simple library like APR without
a *lot* of prior thought - an implementation may be about
7,000 to 10,000 LOC - not trivial.
gotta run, more later.
all best,
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
> > If the server spawns the programs, then you can also use regular pipes.
>
> I have been thinking of creating apr/rpc/... to handle stuff like this.
> However, right now, we have named pipes. They need to be implemented on
> more platforms, and that may require changing the API a bit, but please
> let's stick with what we have already.
>
> The only thing we can't do with named pipes today is communicate with
> different machines. IMHO, calling any cross-machine communication medium
> a named pipe is just going to confuse any unix programmer. Give it a
> different name.
hiya ryan,
i promised i'd get back to you after having had to leave quickly.
RPC. okay. building RPC directly into apr libs is something i can
strongly recommend you don't do: it could easily get out of hand,
make the code bulky, and most people would wonder what drugs you
were taking today [i.e. it goes against what i am beginning to understand
to be the APR ethos].
however, what i _can_ recommend is a slim wrapper - a *local*
named-pipe (ux-dom-sock-based, tcp-loopback-based, NT-NamedPipe-based,
doesn't matter what the implementation is) that feeds to a
separate program that is responsible for locating and communicating
remote procedures.
a bit like the epmapper on dce/rpc but also a transport as well.
it will take a bit of work to design, however what you end up with
is a very small library that apps have to link with, they also
have to run this [entirely separate] program, and voila you have
RPC.
now, if you're concerned about this design, well... uh... how
do you think NT provides RPC? :) :) :)
worse than that, they put these mechanisms *in the kernel*!
[services running as System, and of course they _have_ to
be optimised and of _course_ that means running parts in the
kernel, agh!]
[i.e., if you're concerned, be concerned about using NT :) ]
all best,
lukes
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 08:04:53AM -0700, rbb@covalent.net wrote:
> > > > BTW, APR already has named pipes. Just what are we trying to solve here?
> >
> > We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> > simply doesn't have an implementation for it.
>
> So, the solution is to implement them on other platforms, not to re-invent
> them. For historical completeness however, allow me to explain how we got
> where we are with named pipes. Unix has them becaues they are easy to
> implement. Windows doesn't have them because only certain versions of
> Windows supports named pipes. Namely, NT and 2000 have named pipes, 9x
> doesn't.
grep NamedPipes */*.c shows some code that uses NPs.
there is a nice comment about 9x not having some
feature [blocking or something], and it says, well,
uhhh.... TOUGH! :)
so, there does already exist code in apr that uses
CreateNamedPipe() - for different purposes.
> Originally, APR supported named pipes on Windows,
great!!!
> they were
> removed in revision 1.11. From looking at the log, I don't understand
> why.
*curious*.
> > > i looked at the named pipe code: i believe that you are thinking
> > > of 'unix' named pipes, which are a totally diffrerent beast
> > > from nt named pipes.
> >
> > Right. I don't recall all the bits about creating named pipes in Windows,
> > but it may be feasible to use the "filename" in the
> > apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> > other words, it may be a simple matter of writing the function for Windows.
> > But then again, the function params may not be rich enough for what is
> > needed.
>
> See above, this has worked in the past, all we need to do is copy the code
> back into place, and possibly tweak it.
yesplease! less work!
> > >...
> > > - a means to set up a server and have other programs (not
> > > remote programs) connect to and communicate with that server.
> > >
> > > the semantics must be identical to those of unix-domain-sockets,
> > > namely a listen, bind, accept, read, write and close.
> >
> > I would think you could use named pipes, TCP sockets, or Unix sockets.
> >
> > If the server spawns the programs, then you can also use regular pipes.
>
> I have been thinking of creating apr/rpc/... to handle stuff like this.
> However, right now, we have named pipes. They need to be implemented on
> more platforms, and that may require changing the API a bit, but please
> let's stick with what we have already.
>
> The only thing we can't do with named pipes today is communicate with
> different machines. IMHO, calling any cross-machine communication medium
> a named pipe is just going to confuse any unix programmer. Give it a
> different name.
okay.
i am familiar with how remote IPC works, from a few perspectives,
having implemented a subset of DCE/RPC that is MS-compatible,
over SMB, for Samba (ISBN 1578701503).
[eek, have bus to catch soon, gotta run].
i can attest to the complexity of implementing such: it's
not something to add to a simple library like APR without
a *lot* of prior thought - an implementation may be about
7,000 to 10,000 LOC - not trivial.
gotta run, more later.
all best,
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 03:04:10AM -0700, Greg Stein wrote:
> On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> > On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
> >...
> > > BTW, APR already has named pipes. Just what are we trying to solve here?
>
> We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> simply doesn't have an implementation for it.
>
> > i looked at the named pipe code: i believe that you are thinking
> > of 'unix' named pipes, which are a totally diffrerent beast
> > from nt named pipes.
>
> Right. I don't recall all the bits about creating named pipes in Windows,
> but it may be feasible to use the "filename" in the
> apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> other words, it may be a simple matter of writing the function for Windows.
> But then again, the function params may not be rich enough for what is
> needed.
>
> >...
> > - a means to set up a server and have other programs (not
> > remote programs) connect to and communicate with that server.
> >
> > the semantics must be identical to those of unix-domain-sockets,
> > namely a listen, bind, accept, read, write and close.
>
> I would think you could use named pipes,
don't know anything about them: are they portable?
> TCP sockets,
known to be slow, even on 127.0.0.1 loopback.
> or Unix sockets.
not portable. [would do the trick, imo, for a ux-impl. of
NT NamedPipes.]
> If the server spawns the programs, then you can also use regular pipes.
>
> > - a means to transfer security context between
> > the server and the 'other programs'. i.e. if the
> > client program is running as 'foo', then it must be
> > possible for the server to be 'told' this - _if_ it
> > needs to know.
>
> I do not believe there is a way in any OS to say what *program* you are
the program is irrelevant: it's the named pipe that's important.
exactly in the same way as in ux-dom-socks, the program
is irrelevant [i have to say, i don't quite follow
what you mean or think i meant that prompted you to
mention 'programs, here: perhaps you could elaborate
so we don't mis-communicate?]
e.g i will call pp = apr_create_np("\PIPE\xvl\xmlvl.org", pool)
and it will create, on ux, a domsock /tmp/.apr/PIPE\xvl\xmlvl.org.
i will then call apr_bind(pp), apr_listen(pp) etc blah blah.
i will then, in a client-program (mod_virgule), call apr_open_np(
"\PIPE\xvl\xmlvl.org") etc. which opens me a connection, causes
the server to call apr_connect() blah blah, in all the usual
ways.
now, on NT, of course, this will not work: NT doesn't _have_
ux-dom-sock. but it _does_ have NamedPipes.
and i expect that using them could provide exactly the same
functionality [as described in above example].
> running (in an authenticated fashion). Reading man pages, it appears Linux
> 2.2 added a way to send another process the PID, UID, and GID. But that
> would obviously be non-portable.
the technique used in samba TNG's IPC mechanism
[which is ux-dom-sock-based], is to transfer the uid, gid and
ux-groups [plus the NT auth equivalents as well] over the
socket as ... 'out-of-band' data at setup time.
this info is recorded in a state-structure that can be
obtained by a get_sec_ctxt() -get security context - function.
so, the impl. would be, if you detect that lx 2.2 supports
transfer like this, use it. if not, send the uid, gid, ux-groups
at setup time.
simple :)
[btw, it so happens that the pid is also transferred, but that's
for other reasons that have nothing to do with emulating
NamedPipes, it's to do with SMB and the implementation
of Samba TNG.]
all best,
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 03:04:10AM -0700, Greg Stein wrote:
> On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> > On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
> >...
> > > BTW, APR already has named pipes. Just what are we trying to solve here?
>
> We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> simply doesn't have an implementation for it.
>
> > i looked at the named pipe code: i believe that you are thinking
> > of 'unix' named pipes, which are a totally diffrerent beast
> > from nt named pipes.
>
> Right. I don't recall all the bits about creating named pipes in Windows,
> but it may be feasible to use the "filename" in the
> apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> other words, it may be a simple matter of writing the function for Windows.
> But then again, the function params may not be rich enough for what is
> needed.
>
> >...
> > - a means to set up a server and have other programs (not
> > remote programs) connect to and communicate with that server.
> >
> > the semantics must be identical to those of unix-domain-sockets,
> > namely a listen, bind, accept, read, write and close.
>
> I would think you could use named pipes,
don't know anything about them: are they portable?
> TCP sockets,
known to be slow, even on 127.0.0.1 loopback.
> or Unix sockets.
not portable. [would do the trick, imo, for a ux-impl. of
NT NamedPipes.]
> If the server spawns the programs, then you can also use regular pipes.
>
> > - a means to transfer security context between
> > the server and the 'other programs'. i.e. if the
> > client program is running as 'foo', then it must be
> > possible for the server to be 'told' this - _if_ it
> > needs to know.
>
> I do not believe there is a way in any OS to say what *program* you are
the program is irrelevant: it's the named pipe that's important.
exactly in the same way as in ux-dom-socks, the program
is irrelevant [i have to say, i don't quite follow
what you mean or think i meant that prompted you to
mention 'programs, here: perhaps you could elaborate
so we don't mis-communicate?]
e.g i will call pp = apr_create_np("\PIPE\xvl\xmlvl.org", pool)
and it will create, on ux, a domsock /tmp/.apr/PIPE\xvl\xmlvl.org.
i will then call apr_bind(pp), apr_listen(pp) etc blah blah.
i will then, in a client-program (mod_virgule), call apr_open_np(
"\PIPE\xvl\xmlvl.org") etc. which opens me a connection, causes
the server to call apr_connect() blah blah, in all the usual
ways.
now, on NT, of course, this will not work: NT doesn't _have_
ux-dom-sock. but it _does_ have NamedPipes.
and i expect that using them could provide exactly the same
functionality [as described in above example].
> running (in an authenticated fashion). Reading man pages, it appears Linux
> 2.2 added a way to send another process the PID, UID, and GID. But that
> would obviously be non-portable.
the technique used in samba TNG's IPC mechanism
[which is ux-dom-sock-based], is to transfer the uid, gid and
ux-groups [plus the NT auth equivalents as well] over the
socket as ... 'out-of-band' data at setup time.
this info is recorded in a state-structure that can be
obtained by a get_sec_ctxt() -get security context - function.
so, the impl. would be, if you detect that lx 2.2 supports
transfer like this, use it. if not, send the uid, gid, ux-groups
at setup time.
simple :)
[btw, it so happens that the pid is also transferred, but that's
for other reasons that have nothing to do with emulating
NamedPipes, it's to do with SMB and the implementation
of Samba TNG.]
all best,
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 10:33:17AM -0500, William A. Rowe, Jr. wrote:
> Ok. Here are the not-so-subtle differences.
>
> First, I understand 9x actually supports reading and writing to a named pipe
> created on an NT/2000 machine, but they don't support them internally. And they
> are always blocking.
>
> Second, the naming conventions are entirely different. On win32, the name is
> \\.\pipe\somepipename while on unix (if I understand it) any directory can 'host'
> a named pipe. On unix, my pipe can be apache2.0/connectors/foo, while on win32
> it is all within that 'pipe' pseudofolder. On Win32, we can actually open the
> \\somemachine\pipe\somepipename from either 9x or WinNT, but the blocking features
> change.
>
> All this implies a common support for 'flat' entries (no directory tree, or the
> caviat that the 'path' will be ignored on win32.) It implies one of; an alternate
> file open call, a 'open pipe' flag bit, or returning the handles on open.
>
> Which is also a requirement. When we call namedpipe_create, we have to RETURN
> SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> But once that pipe handle is closed, the pipe evaporates, they are not persistant.
>
> So some semantics change. We don't 'rm' a pipe when we are finished with it, we just
> close the last handle. And pipes on win32 carry a 'suggested size', but the default
> is a paultry 8kb.
the reasons for this are historical, and to do with the fact that
the implementation uses SMB [even on loopback, i believe]
on top of this meagre data size, you can add your own protocol
that provides you with more data, and that has the added benefit
of guaranteeing your communication and latency timing without clogging
up the network.
> If this is what we want to use, let's start talking about what api we want. I'm happy
> to propose the changes to the _namedpipe_ functions that will get this working in the
> next day or two on Win32.
the basic functionality that i really need is to be able to do 'messages'.
definition of messages:
- send some data on a pipe, it is guaranteed to all be transferred.
followed by:
- receive some data from the pipe, even if the data received
is zero bytes [yes, this _is_ important], and it is guaranteed
to be all transferred, and the amount i am told that is
being received _is_ received.
i.e. on NT, TransactNamedPipe.
on ux-dom-sock, this can be achieved by sending / receiving a 4-byte
header indicating the size of the data to be transferred.
even if that size (send or receive) is zero bytes, the transaction
*has* occurred, which is communication in itself [hi, please
put this in a local file, and acknowledge it by sending me back
zero bytes to indicate that the transaction is completed].
i'm labouring the point, here, i'll shut up now.
> > The only thing we can't do with named pipes today is communicate with
> > different machines. IMHO, calling any cross-machine communication medium
> > a named pipe is just going to confuse any unix programmer. Give it a
> > different name.
>
> No, it effectively is a named pipe. They *really* don't differ all that much.
> The differences are in the naming rules, and Win9x compatibility.
>
> Implementors are just going to have to accept that 9x aren't Operating Systems
> in today's sense, but consumer appliances/gaming consoles,
seconded! 9x is a 'program-running-program' where the people developing
it burn out in 6 months, leave after 2 years MAX. NT is a mt-os where
the people developing it stay with MS for an average of 8 years, most
of them are now millionaires and are there out of a sense of responsibility
and duty to develop the best OS they can.
pick your poison...
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 10:33:17AM -0500, William A. Rowe, Jr. wrote:
> Ok. Here are the not-so-subtle differences.
>
> First, I understand 9x actually supports reading and writing to a named pipe
> created on an NT/2000 machine, but they don't support them internally. And they
> are always blocking.
>
> Second, the naming conventions are entirely different. On win32, the name is
> \\.\pipe\somepipename while on unix (if I understand it) any directory can 'host'
> a named pipe. On unix, my pipe can be apache2.0/connectors/foo, while on win32
> it is all within that 'pipe' pseudofolder. On Win32, we can actually open the
> \\somemachine\pipe\somepipename from either 9x or WinNT, but the blocking features
> change.
>
> All this implies a common support for 'flat' entries (no directory tree, or the
> caviat that the 'path' will be ignored on win32.) It implies one of; an alternate
> file open call, a 'open pipe' flag bit, or returning the handles on open.
>
> Which is also a requirement. When we call namedpipe_create, we have to RETURN
> SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> But once that pipe handle is closed, the pipe evaporates, they are not persistant.
>
> So some semantics change. We don't 'rm' a pipe when we are finished with it, we just
> close the last handle. And pipes on win32 carry a 'suggested size', but the default
> is a paultry 8kb.
the reasons for this are historical, and to do with the fact that
the implementation uses SMB [even on loopback, i believe]
on top of this meagre data size, you can add your own protocol
that provides you with more data, and that has the added benefit
of guaranteeing your communication and latency timing without clogging
up the network.
> If this is what we want to use, let's start talking about what api we want. I'm happy
> to propose the changes to the _namedpipe_ functions that will get this working in the
> next day or two on Win32.
the basic functionality that i really need is to be able to do 'messages'.
definition of messages:
- send some data on a pipe, it is guaranteed to all be transferred.
followed by:
- receive some data from the pipe, even if the data received
is zero bytes [yes, this _is_ important], and it is guaranteed
to be all transferred, and the amount i am told that is
being received _is_ received.
i.e. on NT, TransactNamedPipe.
on ux-dom-sock, this can be achieved by sending / receiving a 4-byte
header indicating the size of the data to be transferred.
even if that size (send or receive) is zero bytes, the transaction
*has* occurred, which is communication in itself [hi, please
put this in a local file, and acknowledge it by sending me back
zero bytes to indicate that the transaction is completed].
i'm labouring the point, here, i'll shut up now.
> > The only thing we can't do with named pipes today is communicate with
> > different machines. IMHO, calling any cross-machine communication medium
> > a named pipe is just going to confuse any unix programmer. Give it a
> > different name.
>
> No, it effectively is a named pipe. They *really* don't differ all that much.
> The differences are in the naming rules, and Win9x compatibility.
>
> Implementors are just going to have to accept that 9x aren't Operating Systems
> in today's sense, but consumer appliances/gaming consoles,
seconded! 9x is a 'program-running-program' where the people developing
it burn out in 6 months, leave after 2 years MAX. NT is a mt-os where
the people developing it stay with MS for an average of 8 years, most
of them are now millionaires and are there out of a sense of responsibility
and duty to develop the best OS they can.
pick your poison...
luke
Re: More migration of code from httpd to apr-util
Posted by Bill Stoddard <bi...@wstoddard.com>.
> > > > Which is also a requirement. When we call namedpipe_create, we have to RETURN
> > > > SOMETHING!
>
> Maybe not... What if we open a namedpipe in Windows in bidirectional mode. Then we call
> CreateFile(pipename,...) for mode read and CreateFile(pipename, ...) for write mode. Not
> tested but it might work.
Nope, doesn't work. The second CreateFile(pipename...) fails with message "All pipe
instances are busy". CreateNamedPipe on Win32 expects to return the server side of the
pipe and there is no way around that fact that I am aware of. This means that the
existing apr_file_namedpipe_create() function needs to change. Perhaps like this:
When server calls apr_file_namedpipe_create, return the server side pipe:
apr_file_t *server_pipe;
apr_file_namedpipe_create(&server_pipe, filename, perms, server_side_pipe_mode, pool)
Client the calls apr_file_open(filename) to get the client side of the pipe.
When client and server are done with the pipe, call apr_file_close(server_pipe |
client_pipe);
Bill
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Thu, Jun 21, 2001 at 07:58:07AM -0700, Justin Erenkrantz wrote:
> On Thu, Jun 21, 2001 at 02:54:43PM +0200, Luke Kenneth Casson Leighton wrote:
> > what i have been hinting at, and planning, is that i will actually
> > provide, with the TNG architecture, is the FULL \\servername\\PIPE\pipename
> > functionality. yes, that's right: i will write code that redirects
> > to TNG, which will farm out the data over SMB over to a remote
> > system FOR you.
> >
> > that's right: a unix apr application will be able to connect to
> > a *remote* NT apr server. that's *remote* platform independence,
> > not just local platform independence.
> >
> > and if you don't expose the full pipe-name in the APR api, i can't
> > do that.
>
> Heh, you attempted to answer my previous question before I asked it.
whoopsie :)
> Teaches me to read all of your posts before replying to any of them.
:) me too, i think i have enough experience at the consequences of
not reading ahead, now :)
> But, aren't named pipes typically local-only?
server-side, they can be nothing _but_ local-only.
client-side, no. it's possible to connect to
\\servername\PIPE\samr and make direct dce/rpc requests
to it, if you feel so inclined.
or to \\servername\PIPE\LANMAN, or whatever is running or
whatever someone chooses to make run, listening on a pipe.
> Maybe what you need is
> something outside of traditional named pipes (i.e. "remote" named
> pipes). Add the named pipe code and then add a "special" function in
> apr that allows you to get at the full namespace, if so desired.
.... which is why i'm advocating the NAL because then 'hooking' in
'special' functions becomes a trivial matter.
*thinks.
*thinks some more*.
AH! bill, i think you're right, regarding server-side, about
specifying _just_ the pipe name.
client-side, i'm not so sure.
does anyone have any working NT client / server pipe apps?
i'd be a lot happier / lot more confident if i could
see coding examples, native NT code.
and bill, do you envisage calling apr_namedpipe_create() on
_both_ the client-side _and_ the server-side, a bit like
when you create a socket?
because if so, i think the full syntax \\server\pipe\pipename
will be needed, and if you go 'server-side', then server
has to be '.' otherwise it's an error.
> But,
> it seems that this "remote" named pipe would only be available with an
> SMB library - which is TNG's domain, not APR's.
i envision that anyone could write a 'redirector' daemon,
and without such a daemon running, you simply don't _get_
the ability of client-side programs to connect to remote
pipes, you can only connect to \\.\PIPE\pipename.
> I guess I'm unsure whether named pipes will work from other machines on
> other platforms. If this works only with Win32 (or having an SMB
> protocol to handle this on the Unix end),
or your own 'redirector' daemon that runs on the client and on the
server, but to be honest, doing it via SMB is the simplest option:
take a look at the odbc-odbc-bridges that are out there, the principle
is sound - for only _one_ server - but as soon as you get to multiple
servers it becomes completely unworkable.
> this seems something that TNG
> could add outside of the apr space. -- justin
OS/2 also supports TransactNamedPipe and friends, i asked :)
all best,
luke
p.s you know, i really should just go and code this up :)
Re: More migration of code from httpd to apr-util
Posted by Justin Erenkrantz <je...@ebuilt.com>.
On Thu, Jun 21, 2001 at 02:54:43PM +0200, Luke Kenneth Casson Leighton wrote:
> what i have been hinting at, and planning, is that i will actually
> provide, with the TNG architecture, is the FULL \\servername\\PIPE\pipename
> functionality. yes, that's right: i will write code that redirects
> to TNG, which will farm out the data over SMB over to a remote
> system FOR you.
>
> that's right: a unix apr application will be able to connect to
> a *remote* NT apr server. that's *remote* platform independence,
> not just local platform independence.
>
> and if you don't expose the full pipe-name in the APR api, i can't
> do that.
Heh, you attempted to answer my previous question before I asked it.
Teaches me to read all of your posts before replying to any of them.
But, aren't named pipes typically local-only? Maybe what you need is
something outside of traditional named pipes (i.e. "remote" named
pipes). Add the named pipe code and then add a "special" function in
apr that allows you to get at the full namespace, if so desired. But,
it seems that this "remote" named pipe would only be available with an
SMB library - which is TNG's domain, not APR's.
I guess I'm unsure whether named pipes will work from other machines on
other platforms. If this works only with Win32 (or having an SMB
protocol to handle this on the Unix end), this seems something that TNG
could add outside of the apr space. -- justin
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Wed, Jun 20, 2001 at 10:50:03AM -0500, William A. Rowe, Jr. wrote:
> From: "Luke Kenneth Casson Leighton" <lk...@samba-tng.org>
> Sent: Wednesday, June 20, 2001 8:55 AM
>
>
> > > You miss my point. We at _least_ need to return a "Windows Acceptable Pipe Name", instead
> > > of some /PIPE/pluming style name.
> >
> > it is *important* that NT-style conventions are followed.
>
> ... internally
uhh... okay.
okay, it's important to me, given what i would like to achieve.
i explain below. sorry for not having mentioned it earlier.
> > the 'guiding light' *must* be the NT functions.
>
> No, the 'internal light' must be the platform-appropriate function.
>
> > i know that most of you may find this difficult to accept that
> > microsoft may be able to develop an API that's really useful,
> > and may feel 'tempted' to 'do better' by imposing 'unixy
> > style' conventions.
> >
> > please don't.
>
> We aren't. And you sir, are presuming that NT is the only 'obscure' platform
> out there. It isn't.
i wasn't, but i didn't say that, so you weren't to know, so i stand
corrected for not communicating enough.
> I'm suggesting that we pass a pipe name to create/locate a pipe. No hokey pathing,
> simply a pipe name that will be normalized \\.\PIPE\name or /dev/pipe/apr/name (or
> however it ought to be named on win32.)
what i have been hinting at, and planning, is that i will actually
provide, with the TNG architecture, is the FULL \\servername\\PIPE\pipename
functionality. yes, that's right: i will write code that redirects
to TNG, which will farm out the data over SMB over to a remote
system FOR you.
that's right: a unix apr application will be able to connect to
a *remote* NT apr server. that's *remote* platform independence,
not just local platform independence.
and if you don't expose the full pipe-name in the APR api, i can't
do that.
> And return an opaque structure that the
> platform may us to open handles on that pipe using an apr_filepipe_open() call.
ack to that.
> > you may be used to doing this the other way round: making NT do
> > what unix does.
>
> We aren't, we all agree that lcd coding is required on apr, and we offer extensions
> where they are needed.
>
> Heck, why could even accept unix "/dev/pipe/name" as a pipe, but it would be morphed
> into "\\.\PIPE\dev_pipe_name" in order to serve the data.
>
> > and if that means creating files like /tmp/.apr-socket/\PIPE\samr with
> > backslashes in them and i _know_ this _is_ possible to do on unix,
> > and i _know_ that there are no problems with doing so because winbindd
> > does it [i have a directory /home/DOMAIN\administrator on my linux
> > hard disk and even an entry in /etc/passwd to match it] then so be it.
>
> Why not simply a pipe name, and leave pathing to apr? Justification?
see above.
does that help?
luke
Re: More migration of code from httpd to apr-util
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
From: "Luke Kenneth Casson Leighton" <lk...@samba-tng.org>
Sent: Wednesday, June 20, 2001 8:55 AM
> > You miss my point. We at _least_ need to return a "Windows Acceptable Pipe Name", instead
> > of some /PIPE/pluming style name.
>
> it is *important* that NT-style conventions are followed.
... internally
> the 'guiding light' *must* be the NT functions.
No, the 'internal light' must be the platform-appropriate function.
> i know that most of you may find this difficult to accept that
> microsoft may be able to develop an API that's really useful,
> and may feel 'tempted' to 'do better' by imposing 'unixy
> style' conventions.
>
> please don't.
We aren't. And you sir, are presuming that NT is the only 'obscure' platform
out there. It isn't.
I'm suggesting that we pass a pipe name to create/locate a pipe. No hokey pathing,
simply a pipe name that will be normalized \\.\PIPE\name or /dev/pipe/apr/name (or
however it ought to be named on win32.) And return an opaque structure that the
platform may us to open handles on that pipe using an apr_filepipe_open() call.
> you may be used to doing this the other way round: making NT do
> what unix does.
We aren't, we all agree that lcd coding is required on apr, and we offer extensions
where they are needed.
Heck, why could even accept unix "/dev/pipe/name" as a pipe, but it would be morphed
into "\\.\PIPE\dev_pipe_name" in order to serve the data.
> and if that means creating files like /tmp/.apr-socket/\PIPE\samr with
> backslashes in them and i _know_ this _is_ possible to do on unix,
> and i _know_ that there are no problems with doing so because winbindd
> does it [i have a directory /home/DOMAIN\administrator on my linux
> hard disk and even an entry in /etc/passwd to match it] then so be it.
Why not simply a pipe name, and leave pathing to apr? Justification?
Bill
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
> You miss my point. We at _least_ need to return a "Windows Acceptable Pipe Name", instead
> of some /PIPE/pluming style name.
it is *important* that NT-style conventions are followed.
the 'guiding light' *must* be the NT functions.
i know that most of you may find this difficult to accept that
microsoft may be able to develop an API that's really useful,
and may feel 'tempted' to 'do better' by imposing 'unixy
style' conventions.
please don't.
pick the NT functions that are needed, then *make* the apr/unix/xxx.[ch]
functions do *exactly* the same job [and i will quite happily
write that code, btw: i already have most of the components needed
to do it].
you may be used to doing this the other way round: making NT do
what unix does.
please don't.
and if that means creating files like /tmp/.apr-socket/\PIPE\samr with
backslashes in them and i _know_ this _is_ possible to do on unix,
and i _know_ that there are no problems with doing so because winbindd
does it [i have a directory /home/DOMAIN\administrator on my linux
hard disk and even an entry in /etc/passwd to match it] then so be it.
luke
Re: More migration of code from httpd to apr-util
Posted by "William A. Rowe, Jr." <ad...@rowe-clan.net>.
From: "Bill Stoddard" <bi...@wstoddard.com>
Sent: Friday, June 15, 2001 2:15 PM
> > > > Which is also a requirement. When we call namedpipe_create, we have to RETURN
> > > > SOMETHING!
>
> Maybe not... What if we open a namedpipe in Windows in bidirectional mode. Then we call
> CreateFile(pipename,...) for mode read and CreateFile(pipename, ...) for write mode. Not
> tested but it might work.
You miss my point. We at _least_ need to return a "Windows Acceptable Pipe Name", instead
of some /PIPE/pluming style name.
Naming pipes is the biggest hassle, namespaces won't match. The subsequent apr_file_open()
calls need to have a legal name.
And we need to have a lifetime of that pipe. While it will default to the lifetime of the
pool that was passed, we should be able to close that pipe handle as soon as it's unneeded.
> >>>> Win32 will create a pipe handle (not the same as the read/write file
> > > > handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> > > > But once that pipe handle is closed, the pipe evaporates, they are not persistant.
> > >
> > > Not to be contrary, but can't we handle this by registering a cleanup with
> > > the handle? That way, the handle survives until we specifically clean it
> > > up, and we shouldn't have to change the API much. Not saying this is the
> > > right way to go, I am just offering suggestions.
> >
> > No, I don't think it's the right way. I'd expect that we add a placeholder of
> > apr_namedpipe_t that contains the name of the pipe on Unix (and nothing more),
> > or the pipe name and handle on Win32. Opaque, of course.
> >
> If the above works, then passing in an "open named pipe" flag on the apr_file_open() would
> enable APR to prepend the file://pipe/ stuff
Above, which?
Why are we fighting to make this platform specific within APR? If we simply add the
apr_filepipe_open() to compliment the create call, this becomes far more platform
independent.
Otherwise just use the Posix calls and be done with it.
Bill
Re: More migration of code from httpd to apr-util
Posted by Bill Stoddard <bi...@wstoddard.com>.
> > > Which is also a requirement. When we call namedpipe_create, we have to RETURN
> > > SOMETHING!
Maybe not... What if we open a namedpipe in Windows in bidirectional mode. Then we call
CreateFile(pipename,...) for mode read and CreateFile(pipename, ...) for write mode. Not
tested but it might work.
>>>> Win32 will create a pipe handle (not the same as the read/write file
> > > handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> > > But once that pipe handle is closed, the pipe evaporates, they are not persistant.
> >
> > Not to be contrary, but can't we handle this by registering a cleanup with
> > the handle? That way, the handle survives until we specifically clean it
> > up, and we shouldn't have to change the API much. Not saying this is the
> > right way to go, I am just offering suggestions.
>
> No, I don't think it's the right way. I'd expect that we add a placeholder of
> apr_namedpipe_t that contains the name of the pipe on Unix (and nothing more),
> or the pipe name and handle on Win32. Opaque, of course.
>
If the above works, then passing in an "open named pipe" flag on the apr_file_open() would
enable APR to prepend the file://pipe/ stuff
Bill
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 12:30:59PM -0500, William A. Rowe, Jr. wrote:
> From: <rb...@covalent.net>
> Sent: Monday, June 11, 2001 10:58 AM
>
>
> > > Which is also a requirement. When we call namedpipe_create, we have to RETURN
> > > SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> > > handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> > > But once that pipe handle is closed, the pipe evaporates, they are not persistant.
is there anybody out there who really understands, or has used,
pipes on NT?
CreateIoCompletionPort! that's the one!!! dammit, not
WaitNamedPipeHandleState, what am i talking about.
okay, just looking at the online MSDN on ms's site,
the function i am referring to that is somewhat equivalent
to listen/accept, esp. for use in multi-threaded code,
is CreateIoComplationPort.
i could be wrong: it may be possible to just use the
message-based functions CallNamedPipe, TransactNamedPipe
etc.
help, help, who really 'gets' this stuff?
[i understand what it looks like over-the-wire when
this stuff happens over SMB, but have no connection
back to the Win32 APIs that *generate* that network
traffic!]
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 12:30:59PM -0500, William A. Rowe, Jr. wrote:
> From: <rb...@covalent.net>
> Sent: Monday, June 11, 2001 10:58 AM
>
>
> > > Which is also a requirement. When we call namedpipe_create, we have to RETURN
> > > SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> > > handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> > > But once that pipe handle is closed, the pipe evaporates, they are not persistant.
is there anybody out there who really understands, or has used,
pipes on NT?
CreateIoCompletionPort! that's the one!!! dammit, not
WaitNamedPipeHandleState, what am i talking about.
okay, just looking at the online MSDN on ms's site,
the function i am referring to that is somewhat equivalent
to listen/accept, esp. for use in multi-threaded code,
is CreateIoComplationPort.
i could be wrong: it may be possible to just use the
message-based functions CallNamedPipe, TransactNamedPipe
etc.
help, help, who really 'gets' this stuff?
[i understand what it looks like over-the-wire when
this stuff happens over SMB, but have no connection
back to the Win32 APIs that *generate* that network
traffic!]
luke
Re: More migration of code from httpd to apr-util
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
From: <rb...@covalent.net>
Sent: Monday, June 11, 2001 10:58 AM
> > Which is also a requirement. When we call namedpipe_create, we have to RETURN
> > SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> > handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> > But once that pipe handle is closed, the pipe evaporates, they are not persistant.
>
> Not to be contrary, but can't we handle this by registering a cleanup with
> the handle? That way, the handle survives until we specifically clean it
> up, and we shouldn't have to change the API much. Not saying this is the
> right way to go, I am just offering suggestions.
No, I don't think it's the right way. I'd expect that we add a placeholder of
apr_namedpipe_t that contains the name of the pipe on Unix (and nothing more),
or the pipe name and handle on Win32. Opaque, of course.
> > > I have been thinking of creating apr/rpc/... to handle stuff like this.
> > > However, right now, we have named pipes. They need to be implemented on
> > > more platforms, and that may require changing the API a bit, but please
> > > let's stick with what we have already.
> > >
> > > The only thing we can't do with named pipes today is communicate with
> > > different machines. IMHO, calling any cross-machine communication medium
> > > a named pipe is just going to confuse any unix programmer. Give it a
> > > different name.
> >
> > No, it effectively is a named pipe. They *really* don't differ all that much.
> > The differences are in the naming rules, and Win9x compatibility.
> >
> > Implementors are just going to have to accept that 9x aren't Operating Systems
> > in today's sense, but consumer appliances/gaming consoles, so they have
> > significant drawbacks. This does ***NOT*** imply I'm against getting httpd up
> > and running on Win9x! It just means that the sort of advanced features that
> > cross-machine dce would implement can't be effectively targetted to Win9x, or
> > they must be coded around.
>
> My problem with calling this a named pipe, is that Unix named pipes can't
> do cross-machine communication. I think that if we call these things
> named pipes, then in people's (Unix programmer's) minds, there weill be
> limits that don't actually exist. Yes, they can learn what we mean by
> named pipes, but why make it harder than it needs to be?
Pointing out that they 'can' communicate between machines does _NOT_ mean we would
actually implement that feature :-) I'd suggest we plunge forward, with a return
type that lets us kill the pipe later (on Unix, that's nothing more than an
rm namedpipe->name, no? on Win32, closing the handle) and register a cleanup on
it, of course!
Bill
Re: More migration of code from httpd to apr-util
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
From: <rb...@covalent.net>
Sent: Monday, June 11, 2001 10:58 AM
> > Which is also a requirement. When we call namedpipe_create, we have to RETURN
> > SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> > handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> > But once that pipe handle is closed, the pipe evaporates, they are not persistant.
>
> Not to be contrary, but can't we handle this by registering a cleanup with
> the handle? That way, the handle survives until we specifically clean it
> up, and we shouldn't have to change the API much. Not saying this is the
> right way to go, I am just offering suggestions.
No, I don't think it's the right way. I'd expect that we add a placeholder of
apr_namedpipe_t that contains the name of the pipe on Unix (and nothing more),
or the pipe name and handle on Win32. Opaque, of course.
> > > I have been thinking of creating apr/rpc/... to handle stuff like this.
> > > However, right now, we have named pipes. They need to be implemented on
> > > more platforms, and that may require changing the API a bit, but please
> > > let's stick with what we have already.
> > >
> > > The only thing we can't do with named pipes today is communicate with
> > > different machines. IMHO, calling any cross-machine communication medium
> > > a named pipe is just going to confuse any unix programmer. Give it a
> > > different name.
> >
> > No, it effectively is a named pipe. They *really* don't differ all that much.
> > The differences are in the naming rules, and Win9x compatibility.
> >
> > Implementors are just going to have to accept that 9x aren't Operating Systems
> > in today's sense, but consumer appliances/gaming consoles, so they have
> > significant drawbacks. This does ***NOT*** imply I'm against getting httpd up
> > and running on Win9x! It just means that the sort of advanced features that
> > cross-machine dce would implement can't be effectively targetted to Win9x, or
> > they must be coded around.
>
> My problem with calling this a named pipe, is that Unix named pipes can't
> do cross-machine communication. I think that if we call these things
> named pipes, then in people's (Unix programmer's) minds, there weill be
> limits that don't actually exist. Yes, they can learn what we mean by
> named pipes, but why make it harder than it needs to be?
Pointing out that they 'can' communicate between machines does _NOT_ mean we would
actually implement that feature :-) I'd suggest we plunge forward, with a return
type that lets us kill the pipe later (on Unix, that's nothing more than an
rm namedpipe->name, no? on Win32, closing the handle) and register a cleanup on
it, of course!
Bill
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
> Which is also a requirement. When we call namedpipe_create, we have to RETURN
> SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> But once that pipe handle is closed, the pipe evaporates, they are not persistant.
Not to be contrary, but can't we handle this by registering a cleanup with
the handle? That way, the handle survives until we specifically clean it
up, and we shouldn't have to change the API much. Not saying this is the
right way to go, I am just offering suggestions.
> > I have been thinking of creating apr/rpc/... to handle stuff like this.
> > However, right now, we have named pipes. They need to be implemented on
> > more platforms, and that may require changing the API a bit, but please
> > let's stick with what we have already.
> >
> > The only thing we can't do with named pipes today is communicate with
> > different machines. IMHO, calling any cross-machine communication medium
> > a named pipe is just going to confuse any unix programmer. Give it a
> > different name.
>
> No, it effectively is a named pipe. They *really* don't differ all that much.
> The differences are in the naming rules, and Win9x compatibility.
>
> Implementors are just going to have to accept that 9x aren't Operating Systems
> in today's sense, but consumer appliances/gaming consoles, so they have
> significant drawbacks. This does ***NOT*** imply I'm against getting httpd up
> and running on Win9x! It just means that the sort of advanced features that
> cross-machine dce would implement can't be effectively targetted to Win9x, or
> they must be coded around.
My problem with calling this a named pipe, is that Unix named pipes can't
do cross-machine communication. I think that if we call these things
named pipes, then in people's (Unix programmer's) minds, there weill be
limits that don't actually exist. Yes, they can learn what we mean by
named pipes, but why make it harder than it needs to be?
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
> Which is also a requirement. When we call namedpipe_create, we have to RETURN
> SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> But once that pipe handle is closed, the pipe evaporates, they are not persistant.
Not to be contrary, but can't we handle this by registering a cleanup with
the handle? That way, the handle survives until we specifically clean it
up, and we shouldn't have to change the API much. Not saying this is the
right way to go, I am just offering suggestions.
> > I have been thinking of creating apr/rpc/... to handle stuff like this.
> > However, right now, we have named pipes. They need to be implemented on
> > more platforms, and that may require changing the API a bit, but please
> > let's stick with what we have already.
> >
> > The only thing we can't do with named pipes today is communicate with
> > different machines. IMHO, calling any cross-machine communication medium
> > a named pipe is just going to confuse any unix programmer. Give it a
> > different name.
>
> No, it effectively is a named pipe. They *really* don't differ all that much.
> The differences are in the naming rules, and Win9x compatibility.
>
> Implementors are just going to have to accept that 9x aren't Operating Systems
> in today's sense, but consumer appliances/gaming consoles, so they have
> significant drawbacks. This does ***NOT*** imply I'm against getting httpd up
> and running on Win9x! It just means that the sort of advanced features that
> cross-machine dce would implement can't be effectively targetted to Win9x, or
> they must be coded around.
My problem with calling this a named pipe, is that Unix named pipes can't
do cross-machine communication. I think that if we call these things
named pipes, then in people's (Unix programmer's) minds, there weill be
limits that don't actually exist. Yes, they can learn what we mean by
named pipes, but why make it harder than it needs to be?
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by "William A. Rowe, Jr." <ad...@rowe-clan.net>.
From: <rb...@covalent.net>
Sent: Monday, June 11, 2001 10:04 AM
> On Mon, 11 Jun 2001, Greg Stein wrote:
>
> > On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> > > On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
> > >...
> > > > BTW, APR already has named pipes. Just what are we trying to solve here?
> >
> > We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> > simply doesn't have an implementation for it.
>
> So, the solution is to implement them on other platforms, not to re-invent
> them. For historical completeness however, allow me to explain how we got
> where we are with named pipes. Unix has them becaues they are easy to
> implement. Windows doesn't have them because only certain versions of
> Windows supports named pipes. Namely, NT and 2000 have named pipes, 9x
> doesn't. Originally, APR supported named pipes on Windows, they were
> removed in revision 1.11. From looking at the log, I don't understand
> why.
Ok. Here are the not-so-subtle differences.
First, I understand 9x actually supports reading and writing to a named pipe
created on an NT/2000 machine, but they don't support them internally. And they
are always blocking.
Second, the naming conventions are entirely different. On win32, the name is
\\.\pipe\somepipename while on unix (if I understand it) any directory can 'host'
a named pipe. On unix, my pipe can be apache2.0/connectors/foo, while on win32
it is all within that 'pipe' pseudofolder. On Win32, we can actually open the
\\somemachine\pipe\somepipename from either 9x or WinNT, but the blocking features
change.
All this implies a common support for 'flat' entries (no directory tree, or the
caviat that the 'path' will be ignored on win32.) It implies one of; an alternate
file open call, a 'open pipe' flag bit, or returning the handles on open.
Which is also a requirement. When we call namedpipe_create, we have to RETURN
SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
But once that pipe handle is closed, the pipe evaporates, they are not persistant.
So some semantics change. We don't 'rm' a pipe when we are finished with it, we just
close the last handle. And pipes on win32 carry a 'suggested size', but the default
is a paultry 8kb.
If this is what we want to use, let's start talking about what api we want. I'm happy
to propose the changes to the _namedpipe_ functions that will get this working in the
next day or two on Win32.
> > > i looked at the named pipe code: i believe that you are thinking
> > > of 'unix' named pipes, which are a totally diffrerent beast
> > > from nt named pipes.
> >
> > Right. I don't recall all the bits about creating named pipes in Windows,
> > but it may be feasible to use the "filename" in the
> > apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> > other words, it may be a simple matter of writing the function for Windows.
> > But then again, the function params may not be rich enough for what is
> > needed.
>
> See above, this has worked in the past, all we need to do is copy the code
> back into place, and possibly tweak it.
I really doubt it. Time to start from scratch, for the reasons I cited above.
> > > - a means to set up a server and have other programs (not
> > > remote programs) connect to and communicate with that server.
> > >
> > > the semantics must be identical to those of unix-domain-sockets,
> > > namely a listen, bind, accept, read, write and close.
> >
> > I would think you could use named pipes, TCP sockets, or Unix sockets.
> >
> > If the server spawns the programs, then you can also use regular pipes.
>
> I have been thinking of creating apr/rpc/... to handle stuff like this.
> However, right now, we have named pipes. They need to be implemented on
> more platforms, and that may require changing the API a bit, but please
> let's stick with what we have already.
>
> The only thing we can't do with named pipes today is communicate with
> different machines. IMHO, calling any cross-machine communication medium
> a named pipe is just going to confuse any unix programmer. Give it a
> different name.
No, it effectively is a named pipe. They *really* don't differ all that much.
The differences are in the naming rules, and Win9x compatibility.
Implementors are just going to have to accept that 9x aren't Operating Systems
in today's sense, but consumer appliances/gaming consoles, so they have
significant drawbacks. This does ***NOT*** imply I'm against getting httpd up
and running on Win9x! It just means that the sort of advanced features that
cross-machine dce would implement can't be effectively targetted to Win9x, or
they must be coded around.
Bill
Re: More migration of code from httpd to apr-util
Posted by "William A. Rowe, Jr." <ad...@rowe-clan.net>.
From: <rb...@covalent.net>
Sent: Monday, June 11, 2001 10:04 AM
> On Mon, 11 Jun 2001, Greg Stein wrote:
>
> > On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> > > On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
> > >...
> > > > BTW, APR already has named pipes. Just what are we trying to solve here?
> >
> > We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> > simply doesn't have an implementation for it.
>
> So, the solution is to implement them on other platforms, not to re-invent
> them. For historical completeness however, allow me to explain how we got
> where we are with named pipes. Unix has them becaues they are easy to
> implement. Windows doesn't have them because only certain versions of
> Windows supports named pipes. Namely, NT and 2000 have named pipes, 9x
> doesn't. Originally, APR supported named pipes on Windows, they were
> removed in revision 1.11. From looking at the log, I don't understand
> why.
Ok. Here are the not-so-subtle differences.
First, I understand 9x actually supports reading and writing to a named pipe
created on an NT/2000 machine, but they don't support them internally. And they
are always blocking.
Second, the naming conventions are entirely different. On win32, the name is
\\.\pipe\somepipename while on unix (if I understand it) any directory can 'host'
a named pipe. On unix, my pipe can be apache2.0/connectors/foo, while on win32
it is all within that 'pipe' pseudofolder. On Win32, we can actually open the
\\somemachine\pipe\somepipename from either 9x or WinNT, but the blocking features
change.
All this implies a common support for 'flat' entries (no directory tree, or the
caviat that the 'path' will be ignored on win32.) It implies one of; an alternate
file open call, a 'open pipe' flag bit, or returning the handles on open.
Which is also a requirement. When we call namedpipe_create, we have to RETURN
SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
But once that pipe handle is closed, the pipe evaporates, they are not persistant.
So some semantics change. We don't 'rm' a pipe when we are finished with it, we just
close the last handle. And pipes on win32 carry a 'suggested size', but the default
is a paultry 8kb.
If this is what we want to use, let's start talking about what api we want. I'm happy
to propose the changes to the _namedpipe_ functions that will get this working in the
next day or two on Win32.
> > > i looked at the named pipe code: i believe that you are thinking
> > > of 'unix' named pipes, which are a totally diffrerent beast
> > > from nt named pipes.
> >
> > Right. I don't recall all the bits about creating named pipes in Windows,
> > but it may be feasible to use the "filename" in the
> > apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> > other words, it may be a simple matter of writing the function for Windows.
> > But then again, the function params may not be rich enough for what is
> > needed.
>
> See above, this has worked in the past, all we need to do is copy the code
> back into place, and possibly tweak it.
I really doubt it. Time to start from scratch, for the reasons I cited above.
> > > - a means to set up a server and have other programs (not
> > > remote programs) connect to and communicate with that server.
> > >
> > > the semantics must be identical to those of unix-domain-sockets,
> > > namely a listen, bind, accept, read, write and close.
> >
> > I would think you could use named pipes, TCP sockets, or Unix sockets.
> >
> > If the server spawns the programs, then you can also use regular pipes.
>
> I have been thinking of creating apr/rpc/... to handle stuff like this.
> However, right now, we have named pipes. They need to be implemented on
> more platforms, and that may require changing the API a bit, but please
> let's stick with what we have already.
>
> The only thing we can't do with named pipes today is communicate with
> different machines. IMHO, calling any cross-machine communication medium
> a named pipe is just going to confuse any unix programmer. Give it a
> different name.
No, it effectively is a named pipe. They *really* don't differ all that much.
The differences are in the naming rules, and Win9x compatibility.
Implementors are just going to have to accept that 9x aren't Operating Systems
in today's sense, but consumer appliances/gaming consoles, so they have
significant drawbacks. This does ***NOT*** imply I'm against getting httpd up
and running on Win9x! It just means that the sort of advanced features that
cross-machine dce would implement can't be effectively targetted to Win9x, or
they must be coded around.
Bill
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
On Mon, 11 Jun 2001, Greg Stein wrote:
> On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> > On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
> >...
> > > BTW, APR already has named pipes. Just what are we trying to solve here?
>
> We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> simply doesn't have an implementation for it.
So, the solution is to implement them on other platforms, not to re-invent
them. For historical completeness however, allow me to explain how we got
where we are with named pipes. Unix has them becaues they are easy to
implement. Windows doesn't have them because only certain versions of
Windows supports named pipes. Namely, NT and 2000 have named pipes, 9x
doesn't. Originally, APR supported named pipes on Windows, they were
removed in revision 1.11. From looking at the log, I don't understand
why.
> > i looked at the named pipe code: i believe that you are thinking
> > of 'unix' named pipes, which are a totally diffrerent beast
> > from nt named pipes.
>
> Right. I don't recall all the bits about creating named pipes in Windows,
> but it may be feasible to use the "filename" in the
> apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> other words, it may be a simple matter of writing the function for Windows.
> But then again, the function params may not be rich enough for what is
> needed.
See above, this has worked in the past, all we need to do is copy the code
back into place, and possibly tweak it.
> >...
> > - a means to set up a server and have other programs (not
> > remote programs) connect to and communicate with that server.
> >
> > the semantics must be identical to those of unix-domain-sockets,
> > namely a listen, bind, accept, read, write and close.
>
> I would think you could use named pipes, TCP sockets, or Unix sockets.
>
> If the server spawns the programs, then you can also use regular pipes.
I have been thinking of creating apr/rpc/... to handle stuff like this.
However, right now, we have named pipes. They need to be implemented on
more platforms, and that may require changing the API a bit, but please
let's stick with what we have already.
The only thing we can't do with named pipes today is communicate with
different machines. IMHO, calling any cross-machine communication medium
a named pipe is just going to confuse any unix programmer. Give it a
different name.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
On Mon, 11 Jun 2001, Greg Stein wrote:
> On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> > On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
> >...
> > > BTW, APR already has named pipes. Just what are we trying to solve here?
>
> We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> simply doesn't have an implementation for it.
So, the solution is to implement them on other platforms, not to re-invent
them. For historical completeness however, allow me to explain how we got
where we are with named pipes. Unix has them becaues they are easy to
implement. Windows doesn't have them because only certain versions of
Windows supports named pipes. Namely, NT and 2000 have named pipes, 9x
doesn't. Originally, APR supported named pipes on Windows, they were
removed in revision 1.11. From looking at the log, I don't understand
why.
> > i looked at the named pipe code: i believe that you are thinking
> > of 'unix' named pipes, which are a totally diffrerent beast
> > from nt named pipes.
>
> Right. I don't recall all the bits about creating named pipes in Windows,
> but it may be feasible to use the "filename" in the
> apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> other words, it may be a simple matter of writing the function for Windows.
> But then again, the function params may not be rich enough for what is
> needed.
See above, this has worked in the past, all we need to do is copy the code
back into place, and possibly tweak it.
> >...
> > - a means to set up a server and have other programs (not
> > remote programs) connect to and communicate with that server.
> >
> > the semantics must be identical to those of unix-domain-sockets,
> > namely a listen, bind, accept, read, write and close.
>
> I would think you could use named pipes, TCP sockets, or Unix sockets.
>
> If the server spawns the programs, then you can also use regular pipes.
I have been thinking of creating apr/rpc/... to handle stuff like this.
However, right now, we have named pipes. They need to be implemented on
more platforms, and that may require changing the API a bit, but please
let's stick with what we have already.
The only thing we can't do with named pipes today is communicate with
different machines. IMHO, calling any cross-machine communication medium
a named pipe is just going to confuse any unix programmer. Give it a
different name.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by Greg Stein <gs...@lyra.org>.
On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
>...
> > BTW, APR already has named pipes. Just what are we trying to solve here?
We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
simply doesn't have an implementation for it.
> i looked at the named pipe code: i believe that you are thinking
> of 'unix' named pipes, which are a totally diffrerent beast
> from nt named pipes.
Right. I don't recall all the bits about creating named pipes in Windows,
but it may be feasible to use the "filename" in the
apr_file_namedpipe_create function for the host/name of the pipe in NT. In
other words, it may be a simple matter of writing the function for Windows.
But then again, the function params may not be rich enough for what is
needed.
>...
> - a means to set up a server and have other programs (not
> remote programs) connect to and communicate with that server.
>
> the semantics must be identical to those of unix-domain-sockets,
> namely a listen, bind, accept, read, write and close.
I would think you could use named pipes, TCP sockets, or Unix sockets.
If the server spawns the programs, then you can also use regular pipes.
> - a means to transfer security context between
> the server and the 'other programs'. i.e. if the
> client program is running as 'foo', then it must be
> possible for the server to be 'told' this - _if_ it
> needs to know.
I do not believe there is a way in any OS to say what *program* you are
running (in an authenticated fashion). Reading man pages, it appears Linux
2.2 added a way to send another process the PID, UID, and GID. But that
would obviously be non-portable.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: More migration of code from httpd to apr-util
Posted by Greg Stein <gs...@lyra.org>.
On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
>...
> > BTW, APR already has named pipes. Just what are we trying to solve here?
We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
simply doesn't have an implementation for it.
> i looked at the named pipe code: i believe that you are thinking
> of 'unix' named pipes, which are a totally diffrerent beast
> from nt named pipes.
Right. I don't recall all the bits about creating named pipes in Windows,
but it may be feasible to use the "filename" in the
apr_file_namedpipe_create function for the host/name of the pipe in NT. In
other words, it may be a simple matter of writing the function for Windows.
But then again, the function params may not be rich enough for what is
needed.
>...
> - a means to set up a server and have other programs (not
> remote programs) connect to and communicate with that server.
>
> the semantics must be identical to those of unix-domain-sockets,
> namely a listen, bind, accept, read, write and close.
I would think you could use named pipes, TCP sockets, or Unix sockets.
If the server spawns the programs, then you can also use regular pipes.
> - a means to transfer security context between
> the server and the 'other programs'. i.e. if the
> client program is running as 'foo', then it must be
> possible for the server to be 'told' this - _if_ it
> needs to know.
I do not believe there is a way in any OS to say what *program* you are
running (in an authenticated fashion). Reading man pages, it appears Linux
2.2 added a way to send another process the PID, UID, and GID. But that
would obviously be non-portable.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
>
> mod_cgid should NOT use a named pipe. We started with a named pipe, and
> we moved away from that model for some very good reasons. Secondly,
> mod_cgid does NOT work on windows, and it shouldn't. The reason for
> mod_cgid, is that Unix's tend to not know how to fork threaded programs.
> Windows doesn't know how to fork any program, so using mod_cgid on Windows
> makes zero sense.
ack.
> BTW, APR already has named pipes. Just what are we trying to solve here?
i looked at the named pipe code: i believe that you are thinking
of 'unix' named pipes, which are a totally diffrerent beast
from nt named pipes.
... but i could - and am likely to - be wrong.
time to write requirements:
i am looking for, in apr:
- a means to set up a server and have other programs (not
remote programs) connect to and communicate with that server.
the semantics must be identical to those of unix-domain-sockets,
namely a listen, bind, accept, read, write and close.
- a means to transfer security context between
the server and the 'other programs'. i.e. if the
client program is running as 'foo', then it must be
possible for the server to be 'told' this - _if_ it
needs to know.
so, the implementation of this _tends_ to suggest
unix-domain-sockets on unix, and NamedPipes (_not_
unix named pipes) on NT.
however, if unix named pipes do exactly the same
job as is listed in re3quirements above, please
let me know and i'll shut up and code :)
btw, i notice that there is no NT version of
apr_named_pipe_create().
... yet?
luke
> Ryan
>
> On Sat, 9 Jun 2001, Luke Kenneth Casson Leighton wrote:
>
> > btw, this message goes primarily to the developers of
> > mod_cgid, but is related to the apr_create_named_pipe
> > proposal.
> >
> > my question is, does mod_cgid compile under win32, and
> > would you like it to?
> >
> > so far i count 2 immediate projects that need
> > an apr_create_named_pipe (xvl and mod_cgid) and
> > two long-term ones (tng and dce/rpc).
> >
> > an apr_wait_named_pipe_handle_state() which
> > emulates .h.. WaitNamedPipeHandle will also
> > be needed.
> >
> > i know how to do this in unix-domain-socket land,
> > i'm eager to hear if anyone wants to do the win32 version.[btw plare
> > please remember i'm not subscribed to new-httpd, thx!]
> >
> > luke
> >
> >
>
>
> _______________________________________________________________________________
> Ryan Bloom rbb@apache.org
> 406 29th St.
> San Francisco, CA 94131
> -------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
>
> mod_cgid should NOT use a named pipe. We started with a named pipe, and
> we moved away from that model for some very good reasons. Secondly,
> mod_cgid does NOT work on windows, and it shouldn't. The reason for
> mod_cgid, is that Unix's tend to not know how to fork threaded programs.
> Windows doesn't know how to fork any program, so using mod_cgid on Windows
> makes zero sense.
ack.
> BTW, APR already has named pipes. Just what are we trying to solve here?
i looked at the named pipe code: i believe that you are thinking
of 'unix' named pipes, which are a totally diffrerent beast
from nt named pipes.
... but i could - and am likely to - be wrong.
time to write requirements:
i am looking for, in apr:
- a means to set up a server and have other programs (not
remote programs) connect to and communicate with that server.
the semantics must be identical to those of unix-domain-sockets,
namely a listen, bind, accept, read, write and close.
- a means to transfer security context between
the server and the 'other programs'. i.e. if the
client program is running as 'foo', then it must be
possible for the server to be 'told' this - _if_ it
needs to know.
so, the implementation of this _tends_ to suggest
unix-domain-sockets on unix, and NamedPipes (_not_
unix named pipes) on NT.
however, if unix named pipes do exactly the same
job as is listed in re3quirements above, please
let me know and i'll shut up and code :)
btw, i notice that there is no NT version of
apr_named_pipe_create().
... yet?
luke
> Ryan
>
> On Sat, 9 Jun 2001, Luke Kenneth Casson Leighton wrote:
>
> > btw, this message goes primarily to the developers of
> > mod_cgid, but is related to the apr_create_named_pipe
> > proposal.
> >
> > my question is, does mod_cgid compile under win32, and
> > would you like it to?
> >
> > so far i count 2 immediate projects that need
> > an apr_create_named_pipe (xvl and mod_cgid) and
> > two long-term ones (tng and dce/rpc).
> >
> > an apr_wait_named_pipe_handle_state() which
> > emulates .h.. WaitNamedPipeHandle will also
> > be needed.
> >
> > i know how to do this in unix-domain-socket land,
> > i'm eager to hear if anyone wants to do the win32 version.[btw plare
> > please remember i'm not subscribed to new-httpd, thx!]
> >
> > luke
> >
> >
>
>
> _______________________________________________________________________________
> Ryan Bloom rbb@apache.org
> 406 29th St.
> San Francisco, CA 94131
> -------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
mod_cgid should NOT use a named pipe. We started with a named pipe, and
we moved away from that model for some very good reasons. Secondly,
mod_cgid does NOT work on windows, and it shouldn't. The reason for
mod_cgid, is that Unix's tend to not know how to fork threaded programs.
Windows doesn't know how to fork any program, so using mod_cgid on Windows
makes zero sense.
BTW, APR already has named pipes. Just what are we trying to solve here?
Ryan
On Sat, 9 Jun 2001, Luke Kenneth Casson Leighton wrote:
> btw, this message goes primarily to the developers of
> mod_cgid, but is related to the apr_create_named_pipe
> proposal.
>
> my question is, does mod_cgid compile under win32, and
> would you like it to?
>
> so far i count 2 immediate projects that need
> an apr_create_named_pipe (xvl and mod_cgid) and
> two long-term ones (tng and dce/rpc).
>
> an apr_wait_named_pipe_handle_state() which
> emulates .h.. WaitNamedPipeHandle will also
> be needed.
>
> i know how to do this in unix-domain-socket land,
> i'm eager to hear if anyone wants to do the win32 version.[btw plare
> please remember i'm not subscribed to new-httpd, thx!]
>
> luke
>
>
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
mod_cgid should NOT use a named pipe. We started with a named pipe, and
we moved away from that model for some very good reasons. Secondly,
mod_cgid does NOT work on windows, and it shouldn't. The reason for
mod_cgid, is that Unix's tend to not know how to fork threaded programs.
Windows doesn't know how to fork any program, so using mod_cgid on Windows
makes zero sense.
BTW, APR already has named pipes. Just what are we trying to solve here?
Ryan
On Sat, 9 Jun 2001, Luke Kenneth Casson Leighton wrote:
> btw, this message goes primarily to the developers of
> mod_cgid, but is related to the apr_create_named_pipe
> proposal.
>
> my question is, does mod_cgid compile under win32, and
> would you like it to?
>
> so far i count 2 immediate projects that need
> an apr_create_named_pipe (xvl and mod_cgid) and
> two long-term ones (tng and dce/rpc).
>
> an apr_wait_named_pipe_handle_state() which
> emulates .h.. WaitNamedPipeHandle will also
> be needed.
>
> i know how to do this in unix-domain-socket land,
> i'm eager to hear if anyone wants to do the win32 version.[btw plare
> please remember i'm not subscribed to new-httpd, thx!]
>
> luke
>
>
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
btw, this message goes primarily to the developers of
mod_cgid, but is related to the apr_create_named_pipe
proposal.
my question is, does mod_cgid compile under win32, and
would you like it to?
so far i count 2 immediate projects that need
an apr_create_named_pipe (xvl and mod_cgid) and
two long-term ones (tng and dce/rpc).
an apr_wait_named_pipe_handle_state() which
emulates .h.. WaitNamedPipeHandle will also
be needed.
i know how to do this in unix-domain-socket land,
i'm eager to hear if anyone wants to do the win32 version.[btw plare
please remember i'm not subscribed to new-httpd, thx!]
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Fri, Jun 08, 2001 at 11:37:46AM -0700, Greg Stein wrote:
> On Fri, Jun 08, 2001 at 10:52:34AM -0700, Justin Erenkrantz wrote:
> > On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
> > > On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> > > > what i perhaps should recommend is that a series of independent
> > > > libraries be created.
> > > >
> > > > e.g. libaprhttputil (i know it's a bit long).
> > >
> > > why?
because i forgot about te extract only the needed bits thing :)
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Fri, Jun 08, 2001 at 11:37:46AM -0700, Greg Stein wrote:
> On Fri, Jun 08, 2001 at 10:52:34AM -0700, Justin Erenkrantz wrote:
> > On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
> > > On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> > > > what i perhaps should recommend is that a series of independent
> > > > libraries be created.
> > > >
> > > > e.g. libaprhttputil (i know it's a bit long).
> > >
> > > why?
because i forgot about te extract only the needed bits thing :)
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
btw, this message goes primarily to the developers of
mod_cgid, but is related to the apr_create_named_pipe
proposal.
my question is, does mod_cgid compile under win32, and
would you like it to?
so far i count 2 immediate projects that need
an apr_create_named_pipe (xvl and mod_cgid) and
two long-term ones (tng and dce/rpc).
an apr_wait_named_pipe_handle_state() which
emulates .h.. WaitNamedPipeHandle will also
be needed.
i know how to do this in unix-domain-socket land,
i'm eager to hear if anyone wants to do the win32 version.[btw plare
please remember i'm not subscribed to new-httpd, thx!]
luke
Re: More migration of code from httpd to apr-util
Posted by Greg Stein <gs...@lyra.org>.
On Fri, Jun 08, 2001 at 10:52:34AM -0700, Justin Erenkrantz wrote:
> On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
> > On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> > > what i perhaps should recommend is that a series of independent
> > > libraries be created.
> > >
> > > e.g. libaprhttputil (i know it's a bit long).
> >
> > why?
> >
> > we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
> > ...
> >
> > the point of a library is that when you link against it, the linker will
> > extract only the portions which are used.
>
> So, then we should go ahead and add stuff like HTTP client support to
> apr-util? Where's the boundary line? Is there a limit to what can go
> in apr or apr-util? -- justin
IMO, the boundary line is where a specific chunk of functionality is *huge*,
it should go in its own library. HTTP client code is a very large task (to
do it right).
Otherwise, I'm fine with widely useful functionality going into APRUTIL.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: More migration of code from httpd to apr-util
Posted by Greg Stein <gs...@lyra.org>.
On Fri, Jun 08, 2001 at 10:52:34AM -0700, Justin Erenkrantz wrote:
> On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
> > On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> > > what i perhaps should recommend is that a series of independent
> > > libraries be created.
> > >
> > > e.g. libaprhttputil (i know it's a bit long).
> >
> > why?
> >
> > we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
> > ...
> >
> > the point of a library is that when you link against it, the linker will
> > extract only the portions which are used.
>
> So, then we should go ahead and add stuff like HTTP client support to
> apr-util? Where's the boundary line? Is there a limit to what can go
> in apr or apr-util? -- justin
IMO, the boundary line is where a specific chunk of functionality is *huge*,
it should go in its own library. HTTP client code is a very large task (to
do it right).
Otherwise, I'm fine with widely useful functionality going into APRUTIL.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: More migration of code from httpd to apr-util
Posted by dean gaudet <dg...@arctic.org>.
On Fri, 8 Jun 2001, Justin Erenkrantz wrote:
> On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
> >
> >
> > On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> >
> > > what i perhaps should recommend is that a series of independent
> > > libraries be created.
> > >
> > > e.g. libaprhttputil (i know it's a bit long).
> >
> > why?
> >
> > we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
> > ...
> >
> > the point of a library is that when you link against it, the linker will
> > extract only the portions which are used.
>
> So, then we should go ahead and add stuff like HTTP client support to
> apr-util? Where's the boundary line? Is there a limit to what can go
> in apr or apr-util? -- justin
i dunno, i just didn't want to see another dozen projects spring up with
one or two functions each :)
-dean
Re: More migration of code from httpd to apr-util
Posted by dean gaudet <dg...@arctic.org>.
On Fri, 8 Jun 2001, Justin Erenkrantz wrote:
> On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
> >
> >
> > On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> >
> > > what i perhaps should recommend is that a series of independent
> > > libraries be created.
> > >
> > > e.g. libaprhttputil (i know it's a bit long).
> >
> > why?
> >
> > we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
> > ...
> >
> > the point of a library is that when you link against it, the linker will
> > extract only the portions which are used.
>
> So, then we should go ahead and add stuff like HTTP client support to
> apr-util? Where's the boundary line? Is there a limit to what can go
> in apr or apr-util? -- justin
i dunno, i just didn't want to see another dozen projects spring up with
one or two functions each :)
-dean
Re: More migration of code from httpd to apr-util
Posted by Justin Erenkrantz <je...@ebuilt.com>.
On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
>
>
> On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
>
> > what i perhaps should recommend is that a series of independent
> > libraries be created.
> >
> > e.g. libaprhttputil (i know it's a bit long).
>
> why?
>
> we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
> ...
>
> the point of a library is that when you link against it, the linker will
> extract only the portions which are used.
So, then we should go ahead and add stuff like HTTP client support to
apr-util? Where's the boundary line? Is there a limit to what can go
in apr or apr-util? -- justin
Re: More migration of code from httpd to apr-util
Posted by Justin Erenkrantz <je...@ebuilt.com>.
On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
>
>
> On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
>
> > what i perhaps should recommend is that a series of independent
> > libraries be created.
> >
> > e.g. libaprhttputil (i know it's a bit long).
>
> why?
>
> we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
> ...
>
> the point of a library is that when you link against it, the linker will
> extract only the portions which are used.
So, then we should go ahead and add stuff like HTTP client support to
apr-util? Where's the boundary line? Is there a limit to what can go
in apr or apr-util? -- justin
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Fri, Jun 08, 2001 at 09:48:08AM -0700, rbb@covalent.net wrote:
>
> > > - date/time manipulation.
> > >
> > > at present, all i need to be able to do is get a
> > > time_t and display it in a nice localtime format.
> > > ripping code from httpd to do this is... uh.. silly?
> >
> > Submit a patch and I'll commit it into apr. I think date and time stuff
> > should be yanked out into apr-util. I've already yanked out date.
> > Submit something for time and I'll commit. We can probably find some
> > helpful dude with httpd commit access to remove those files from httpd
> > entirely.
>
> Ummmmm..... apr_ctime takes an apr_time_t, and returns a formatted
> string. If you need more, apr_strftime allows you to define the format
> for the string.
>
ack. thx ryan. that's another function down. only 12 or so
more to go :)
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Fri, Jun 08, 2001 at 09:48:08AM -0700, rbb@covalent.net wrote:
>
> > > - date/time manipulation.
> > >
> > > at present, all i need to be able to do is get a
> > > time_t and display it in a nice localtime format.
> > > ripping code from httpd to do this is... uh.. silly?
> >
> > Submit a patch and I'll commit it into apr. I think date and time stuff
> > should be yanked out into apr-util. I've already yanked out date.
> > Submit something for time and I'll commit. We can probably find some
> > helpful dude with httpd commit access to remove those files from httpd
> > entirely.
>
> Ummmmm..... apr_ctime takes an apr_time_t, and returns a formatted
> string. If you need more, apr_strftime allows you to define the format
> for the string.
>
ack. thx ryan. that's another function down. only 12 or so
more to go :)
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
> > my preference is to do the unix-implementation of the
> > apr_named_pipe API, if there's anyone else willing to
> > help parallel-develop the unix implementation, using
> > xvl as a test to prove the case.
>
> I'd certainly look over the code, but I'm not terribly certain
> if I'd use it. I'd just have to see and evaluate if named pipes
> make sense in my cases. If it does in yours, great. -- justin
ack. se my other post/question: do you want mod_cgid to
work on win32? :) :)
all best,
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
> > my preference is to do the unix-implementation of the
> > apr_named_pipe API, if there's anyone else willing to
> > help parallel-develop the unix implementation, using
> > xvl as a test to prove the case.
>
> I'd certainly look over the code, but I'm not terribly certain
> if I'd use it. I'd just have to see and evaluate if named pipes
> make sense in my cases. If it does in yours, great. -- justin
ack. se my other post/question: do you want mod_cgid to
work on win32? :) :)
all best,
luke
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
> > - date/time manipulation.
> >
> > at present, all i need to be able to do is get a
> > time_t and display it in a nice localtime format.
> > ripping code from httpd to do this is... uh.. silly?
>
> Submit a patch and I'll commit it into apr. I think date and time stuff
> should be yanked out into apr-util. I've already yanked out date.
> Submit something for time and I'll commit. We can probably find some
> helpful dude with httpd commit access to remove those files from httpd
> entirely.
Ummmmm..... apr_ctime takes an apr_time_t, and returns a formatted
string. If you need more, apr_strftime allows you to define the format
for the string.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
> > - date/time manipulation.
> >
> > at present, all i need to be able to do is get a
> > time_t and display it in a nice localtime format.
> > ripping code from httpd to do this is... uh.. silly?
>
> Submit a patch and I'll commit it into apr. I think date and time stuff
> should be yanked out into apr-util. I've already yanked out date.
> Submit something for time and I'll commit. We can probably find some
> helpful dude with httpd commit access to remove those files from httpd
> entirely.
Ummmmm..... apr_ctime takes an apr_time_t, and returns a formatted
string. If you need more, apr_strftime allows you to define the format
for the string.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by Justin Erenkrantz <je...@ebuilt.com>.
On Fri, Jun 08, 2001 at 02:45:09PM +0200, Luke Kenneth Casson Leighton wrote:
> what i perhaps should recommend is that a series of independent
> libraries be created.
>
> e.g. libaprhttputil (i know it's a bit long).
Aaron Bannert and I were discussing that as well yesterday. Yeah, we
kind of get that feeling too over here. But, again, I'm not sure what
the process would be for doing something like this.
> - string/path/cgi manipulation.
>
> this i need to be able to parse HTTP requests, to parse
> directory components. i know httpd does, too.
Depending upon how general it is (i.e. non-HTTP specific), this might
make sense to place in apr-util.
> - date/time manipulation.
>
> at present, all i need to be able to do is get a
> time_t and display it in a nice localtime format.
> ripping code from httpd to do this is... uh.. silly?
Submit a patch and I'll commit it into apr. I think date and time stuff
should be yanked out into apr-util. I've already yanked out date.
Submit something for time and I'll commit. We can probably find some
helpful dude with httpd commit access to remove those files from httpd
entirely.
> - an HTTP client.
>
> i need to be able to make http GETs / POSTs, and so
> originally i ripped entire bits of mod_proxy out into
> a separate file.
>
> i can believe that these two programs, xvl and mod_proxy
> are not the _only_ places where HTTP client GETs/POSTS
> are needed. for example, what about martin pool's
> work on mod_rproxy? [that's a bit like mod_proxy except
> it uses the rsync algorithm to sync up the file being
> requested with the last version obtained, etc., very cool
> work :)]
>
> EITHER :
>
> - an HTTP server-parser
Yeah, well, again, we're going to need it here at eBuilt as well (we're
*supposed* to be writing a load tester). So, I think it might make
sense to pool our resources and come up with an HTTP client API that
works on top of APR. I'm going to be writing this functionality
anyway. We could start with what is in mod_proxy (although I haven't
looked at this code yet). Again, this doesn't belong in APR or
APR-util, but I do believe it belongs somewhere. We keep writing this
over and over again.
And, I'll probably be writing a SAX API for apr-util as well. Hmm, that
may depend on how nice Xerces is.
> > I think the question is whether or not there is a predefined limit to
> > what goes in apr-util. We've hit this recently with the crypto/ stuff.
>
> so, create some new libraries. maybe apr-util isn't the
> best place for some of this stuff, but an apr-http-util is.
> or whatever. i don't mind, or really care [well, i do,
> but not enough to lose sleep over it if it doesn't happen :)]
>
> [and then think carefully about where and how to use them:
> you can't distribute anything that uses openssl / any crypto
> library, everywhere]
Yup. See above.
> i am trying to keep xvl's codebase down to an absolute, absolute
> minimum. the more LOCs i can take _out_, the happier i will be.
> duplicating or copying effort is a no-no in my book.
> :)
Those LOCs got to end up somewhere else though. They just don't
disappear.
> btw, who was it who recommended mod_cgid? [was it you,
> justin?]
I haven't been around *that* long. =)
> my preference is to do the unix-implementation of the
> apr_named_pipe API, if there's anyone else willing to
> help parallel-develop the unix implementation, using
> xvl as a test to prove the case.
I'd certainly look over the code, but I'm not terribly certain
if I'd use it. I'd just have to see and evaluate if named pipes
make sense in my cases. If it does in yours, great. -- justin
Re: More migration of code from httpd to apr-util
Posted by Justin Erenkrantz <je...@ebuilt.com>.
On Fri, Jun 08, 2001 at 02:45:09PM +0200, Luke Kenneth Casson Leighton wrote:
> what i perhaps should recommend is that a series of independent
> libraries be created.
>
> e.g. libaprhttputil (i know it's a bit long).
Aaron Bannert and I were discussing that as well yesterday. Yeah, we
kind of get that feeling too over here. But, again, I'm not sure what
the process would be for doing something like this.
> - string/path/cgi manipulation.
>
> this i need to be able to parse HTTP requests, to parse
> directory components. i know httpd does, too.
Depending upon how general it is (i.e. non-HTTP specific), this might
make sense to place in apr-util.
> - date/time manipulation.
>
> at present, all i need to be able to do is get a
> time_t and display it in a nice localtime format.
> ripping code from httpd to do this is... uh.. silly?
Submit a patch and I'll commit it into apr. I think date and time stuff
should be yanked out into apr-util. I've already yanked out date.
Submit something for time and I'll commit. We can probably find some
helpful dude with httpd commit access to remove those files from httpd
entirely.
> - an HTTP client.
>
> i need to be able to make http GETs / POSTs, and so
> originally i ripped entire bits of mod_proxy out into
> a separate file.
>
> i can believe that these two programs, xvl and mod_proxy
> are not the _only_ places where HTTP client GETs/POSTS
> are needed. for example, what about martin pool's
> work on mod_rproxy? [that's a bit like mod_proxy except
> it uses the rsync algorithm to sync up the file being
> requested with the last version obtained, etc., very cool
> work :)]
>
> EITHER :
>
> - an HTTP server-parser
Yeah, well, again, we're going to need it here at eBuilt as well (we're
*supposed* to be writing a load tester). So, I think it might make
sense to pool our resources and come up with an HTTP client API that
works on top of APR. I'm going to be writing this functionality
anyway. We could start with what is in mod_proxy (although I haven't
looked at this code yet). Again, this doesn't belong in APR or
APR-util, but I do believe it belongs somewhere. We keep writing this
over and over again.
And, I'll probably be writing a SAX API for apr-util as well. Hmm, that
may depend on how nice Xerces is.
> > I think the question is whether or not there is a predefined limit to
> > what goes in apr-util. We've hit this recently with the crypto/ stuff.
>
> so, create some new libraries. maybe apr-util isn't the
> best place for some of this stuff, but an apr-http-util is.
> or whatever. i don't mind, or really care [well, i do,
> but not enough to lose sleep over it if it doesn't happen :)]
>
> [and then think carefully about where and how to use them:
> you can't distribute anything that uses openssl / any crypto
> library, everywhere]
Yup. See above.
> i am trying to keep xvl's codebase down to an absolute, absolute
> minimum. the more LOCs i can take _out_, the happier i will be.
> duplicating or copying effort is a no-no in my book.
> :)
Those LOCs got to end up somewhere else though. They just don't
disappear.
> btw, who was it who recommended mod_cgid? [was it you,
> justin?]
I haven't been around *that* long. =)
> my preference is to do the unix-implementation of the
> apr_named_pipe API, if there's anyone else willing to
> help parallel-develop the unix implementation, using
> xvl as a test to prove the case.
I'd certainly look over the code, but I'm not terribly certain
if I'd use it. I'd just have to see and evaluate if named pipes
make sense in my cases. If it does in yours, great. -- justin
Re: More migration of code from httpd to apr-util
Posted by dean gaudet <dg...@arctic.org>.
On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> what i perhaps should recommend is that a series of independent
> libraries be created.
>
> e.g. libaprhttputil (i know it's a bit long).
why?
we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
...
the point of a library is that when you link against it, the linker will
extract only the portions which are used.
-dean
Re: More migration of code from httpd to apr-util
Posted by dean gaudet <dg...@arctic.org>.
On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> what i perhaps should recommend is that a series of independent
> libraries be created.
>
> e.g. libaprhttputil (i know it's a bit long).
why?
we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
...
the point of a library is that when you link against it, the linker will
extract only the portions which are used.
-dean
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Thu, Jun 07, 2001 at 12:10:15PM -0700, Justin Erenkrantz wrote:
> On Thu, Jun 07, 2001 at 01:33:15PM +0200, Luke Kenneth Casson Leighton wrote:
> > ack. if i make a [rather short] list of functions i am literally
> > duplicating line-for-line, would that suffice as motivation to,
> > say, add them to aprutil?
>
> Your list of potential functions looks mostly good to me (haven't sat
> down and thought too hard about it though).
:) btw, i deleted apr_to_strlower() because there is an apr_tolower()
so that's off the list.
what i perhaps should recommend is that a series of independent
libraries be created.
e.g. libaprhttputil (i know it's a bit long).
just for xvl, i need the following bits of code / functionality,
and i _know_ they're already partially in httpd because i ripped
entire sections out for use in xvl once before, already:
- string/path/cgi manipulation.
this i need to be able to parse HTTP requests, to parse
directory components. i know httpd does, too.
- date/time manipulation.
at present, all i need to be able to do is get a
time_t and display it in a nice localtime format.
ripping code from httpd to do this is... uh.. silly?
- an HTTP client.
i need to be able to make http GETs / POSTs, and so
originally i ripped entire bits of mod_proxy out into
a separate file.
i can believe that these two programs, xvl and mod_proxy
are not the _only_ places where HTTP client GETs/POSTS
are needed. for example, what about martin pool's
work on mod_rproxy? [that's a bit like mod_proxy except
it uses the rsync algorithm to sync up the file being
requested with the last version obtained, etc., very cool
work :)]
EITHER :
- an HTTP server-parser
yes, that's right: i need to be able to parse HTML requests
and the prospect of ripping and having to maintain entire
sections of HTTP parsing code is not one i will take on
lightly.
OR:
- a programmatic means to plug-in PROGRAMS - not cgi-bin
executables - DIRECTLY into httpd.
now, on unix, the obvious way to do this is unix domain
sockets, ala mysql, nssswitch, winbindd, tng domain architecture
X-windows and probably a whole boat-load of others.
on NT, it spells 'Named Pipes'.
dunno about Beos and OS/2, OS/2 probly supports both,
i suspect (?)
basically, the mod_xxx is all very well and good, until
you start wanting to plug in thing like... oh, i dunno,
8 MILLION lines of existing code [yes, there is such
a project].
... and you want to turn all these services, some of
which are #ifdef KERNEL based, into mod_xxxes? ha!
don't make me laugh :)
so, having an inter-process communication mechanism,
there, becomes essential, allowing for separate
programs to plug their data directly in as HTTP
requests.
i.e. httpd can become, effectively, a 'transport' layer
component.
[for those of you familiar with dce/rpc, look up
ncacn_http :) :) ]
> I think the question is whether or not there is a predefined limit to
> what goes in apr-util. We've hit this recently with the crypto/ stuff.
so, create some new libraries. maybe apr-util isn't the
best place for some of this stuff, but an apr-http-util is.
or whatever. i don't mind, or really care [well, i do,
but not enough to lose sleep over it if it doesn't happen :)]
[and then think carefully about where and how to use them:
you can't distribute anything that uses openssl / any crypto
library, everywhere]
> I think it probably means that some of the committers need to work on
> things outside of httpd itself. When I wrote the mod_mbox stuff and
> had to write a standalone indexer, I needed the date and uri functions
> from httpd. I initially copied them over, but then, since I now have
> commit access and no one screamed too loudly, I moved them into
> apr-util (with some help on the httpd side) and deleted my private
> copy. But, I think this will have to be done on a case-by-case basis
> with lots of thought for each one.
ack.
> There also needs to be a guiding vision for apr-util as well. I'm not
> sure what that is exactly. =) I think we might be able to define that
> though...
>
> I'd be curious to look at subversion to see if it is duplicating any
> code, but the domains are so different, I doubt there is a lot of
> overlap.
well, surely they have string / file / directory manipulation.
surely they do timestamps!
> The more users of apr-util, the more we can identify things
> that might be worth migrating from httpd into apr-util.
ack.
> But, you
> are always free to copy the code if we don't move it into apr-util.
i am trying to keep xvl's codebase down to an absolute, absolute
minimum. the more LOCs i can take _out_, the happier i will be.
duplicating or copying effort is a no-no in my book.
:)
btw, who was it who recommended mod_cgid? [was it you,
justin?]
i took a look at it, and... well, okay, what i _meant_ was,
as described above, i need a platform-independent inter-process-
communication mechanism, where process includes separate
programs
and i think that doing Named Pipes, ala NT, is the best way
to go about this, emulating the same functionality under
Unix, not the other way round. and i'm saying that from
the viewpoint of having developed Named Pipes in Samba TNG to
emulate the NP functionality, for unix, already.
sander and i discussed this another time [3 months ago]
related to transport APIs, on APR. the topic was inter-related
with buckets and filters etc, and was... well, a bit too
abstract, ambitious and not well understood or explained.
[best, therefore, i think, to go in small stages, using
xvl to work to that end.]
iow, i need to make a decision to remain with the hook into
httpd via a ux-dom-sock-XML-wrapped implementation of mod_proxy,
help you guys write an apr_named_pipe API, or to cut out
the middle man and shove in entire sections of httpd so
xvl can be a self-serving http server.
my preference is to do the unix-implementation of the
apr_named_pipe API, if there's anyone else willing to
help parallel-develop the unix implementation, using
xvl as a test to prove the case.
takers, anyone?
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Thu, Jun 07, 2001 at 12:10:15PM -0700, Justin Erenkrantz wrote:
> On Thu, Jun 07, 2001 at 01:33:15PM +0200, Luke Kenneth Casson Leighton wrote:
> > ack. if i make a [rather short] list of functions i am literally
> > duplicating line-for-line, would that suffice as motivation to,
> > say, add them to aprutil?
>
> Your list of potential functions looks mostly good to me (haven't sat
> down and thought too hard about it though).
:) btw, i deleted apr_to_strlower() because there is an apr_tolower()
so that's off the list.
what i perhaps should recommend is that a series of independent
libraries be created.
e.g. libaprhttputil (i know it's a bit long).
just for xvl, i need the following bits of code / functionality,
and i _know_ they're already partially in httpd because i ripped
entire sections out for use in xvl once before, already:
- string/path/cgi manipulation.
this i need to be able to parse HTTP requests, to parse
directory components. i know httpd does, too.
- date/time manipulation.
at present, all i need to be able to do is get a
time_t and display it in a nice localtime format.
ripping code from httpd to do this is... uh.. silly?
- an HTTP client.
i need to be able to make http GETs / POSTs, and so
originally i ripped entire bits of mod_proxy out into
a separate file.
i can believe that these two programs, xvl and mod_proxy
are not the _only_ places where HTTP client GETs/POSTS
are needed. for example, what about martin pool's
work on mod_rproxy? [that's a bit like mod_proxy except
it uses the rsync algorithm to sync up the file being
requested with the last version obtained, etc., very cool
work :)]
EITHER :
- an HTTP server-parser
yes, that's right: i need to be able to parse HTML requests
and the prospect of ripping and having to maintain entire
sections of HTTP parsing code is not one i will take on
lightly.
OR:
- a programmatic means to plug-in PROGRAMS - not cgi-bin
executables - DIRECTLY into httpd.
now, on unix, the obvious way to do this is unix domain
sockets, ala mysql, nssswitch, winbindd, tng domain architecture
X-windows and probably a whole boat-load of others.
on NT, it spells 'Named Pipes'.
dunno about Beos and OS/2, OS/2 probly supports both,
i suspect (?)
basically, the mod_xxx is all very well and good, until
you start wanting to plug in thing like... oh, i dunno,
8 MILLION lines of existing code [yes, there is such
a project].
... and you want to turn all these services, some of
which are #ifdef KERNEL based, into mod_xxxes? ha!
don't make me laugh :)
so, having an inter-process communication mechanism,
there, becomes essential, allowing for separate
programs to plug their data directly in as HTTP
requests.
i.e. httpd can become, effectively, a 'transport' layer
component.
[for those of you familiar with dce/rpc, look up
ncacn_http :) :) ]
> I think the question is whether or not there is a predefined limit to
> what goes in apr-util. We've hit this recently with the crypto/ stuff.
so, create some new libraries. maybe apr-util isn't the
best place for some of this stuff, but an apr-http-util is.
or whatever. i don't mind, or really care [well, i do,
but not enough to lose sleep over it if it doesn't happen :)]
[and then think carefully about where and how to use them:
you can't distribute anything that uses openssl / any crypto
library, everywhere]
> I think it probably means that some of the committers need to work on
> things outside of httpd itself. When I wrote the mod_mbox stuff and
> had to write a standalone indexer, I needed the date and uri functions
> from httpd. I initially copied them over, but then, since I now have
> commit access and no one screamed too loudly, I moved them into
> apr-util (with some help on the httpd side) and deleted my private
> copy. But, I think this will have to be done on a case-by-case basis
> with lots of thought for each one.
ack.
> There also needs to be a guiding vision for apr-util as well. I'm not
> sure what that is exactly. =) I think we might be able to define that
> though...
>
> I'd be curious to look at subversion to see if it is duplicating any
> code, but the domains are so different, I doubt there is a lot of
> overlap.
well, surely they have string / file / directory manipulation.
surely they do timestamps!
> The more users of apr-util, the more we can identify things
> that might be worth migrating from httpd into apr-util.
ack.
> But, you
> are always free to copy the code if we don't move it into apr-util.
i am trying to keep xvl's codebase down to an absolute, absolute
minimum. the more LOCs i can take _out_, the happier i will be.
duplicating or copying effort is a no-no in my book.
:)
btw, who was it who recommended mod_cgid? [was it you,
justin?]
i took a look at it, and... well, okay, what i _meant_ was,
as described above, i need a platform-independent inter-process-
communication mechanism, where process includes separate
programs
and i think that doing Named Pipes, ala NT, is the best way
to go about this, emulating the same functionality under
Unix, not the other way round. and i'm saying that from
the viewpoint of having developed Named Pipes in Samba TNG to
emulate the NP functionality, for unix, already.
sander and i discussed this another time [3 months ago]
related to transport APIs, on APR. the topic was inter-related
with buckets and filters etc, and was... well, a bit too
abstract, ambitious and not well understood or explained.
[best, therefore, i think, to go in small stages, using
xvl to work to that end.]
iow, i need to make a decision to remain with the hook into
httpd via a ux-dom-sock-XML-wrapped implementation of mod_proxy,
help you guys write an apr_named_pipe API, or to cut out
the middle man and shove in entire sections of httpd so
xvl can be a self-serving http server.
my preference is to do the unix-implementation of the
apr_named_pipe API, if there's anyone else willing to
help parallel-develop the unix implementation, using
xvl as a test to prove the case.
takers, anyone?
luke
More migration of code from httpd to apr-util
Posted by Justin Erenkrantz <je...@ebuilt.com>.
On Thu, Jun 07, 2001 at 01:33:15PM +0200, Luke Kenneth Casson Leighton wrote:
> ack. if i make a [rather short] list of functions i am literally
> duplicating line-for-line, would that suffice as motivation to,
> say, add them to aprutil?
Your list of potential functions looks mostly good to me (haven't sat
down and thought too hard about it though).
I think the question is whether or not there is a predefined limit to
what goes in apr-util. We've hit this recently with the crypto/ stuff.
I think it probably means that some of the committers need to work on
things outside of httpd itself. When I wrote the mod_mbox stuff and
had to write a standalone indexer, I needed the date and uri functions
from httpd. I initially copied them over, but then, since I now have
commit access and no one screamed too loudly, I moved them into
apr-util (with some help on the httpd side) and deleted my private
copy. But, I think this will have to be done on a case-by-case basis
with lots of thought for each one.
There also needs to be a guiding vision for apr-util as well. I'm not
sure what that is exactly. =) I think we might be able to define that
though...
I'd be curious to look at subversion to see if it is duplicating any
code, but the domains are so different, I doubt there is a lot of
overlap. The more users of apr-util, the more we can identify things
that might be worth migrating from httpd into apr-util. But, you
are always free to copy the code if we don't move it into apr-util.
-- justin
Re: XMLvL, apache, apr, mod_virgule origins: advice needed [LONG]
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Thu, Jun 07, 2001 at 02:44:52PM +0200, Daniel Veillard wrote:
> On Thu, Jun 07, 2001 at 01:33:15PM +0200, Luke Kenneth Casson Leighton wrote:
> > > Even better, there has
> > > been a call for a SAX-like API in APRUTIL, backed by Expat, libxml, or
> > > Xerces. If you're motivated, you could implement that API for APRUTIL :-)
> >
> > :)
> >
> > interesting concept.
> >
> > would keep _me_ happier too because i'm not entirely
> > ok with the clash between libxml and libxml2 caused
> > by the gnome people bundling libxml 1.8.x with their
> > codebase. rrrr :)
>
> At the risk of being slightly off-topic for this list, what clash ?
> You can install both libxml1 and libxml2 simultaneously, and use
> either of them when doing development. You cannot link with both
> but I doubt it's the issue here,
the header-files are identical?
i'm sorry: i assumed that there would be differences, what with
the upgrade-path that must be followed when doing libxml1 to
libxml2 development upgrading.
if that's not the case, sorry for my erroneous assumptions.
luke
Re: XMLvL, apache, apr, mod_virgule origins: advice needed [LONG]
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Thu, Jun 07, 2001 at 02:44:52PM +0200, Daniel Veillard wrote:
> On Thu, Jun 07, 2001 at 01:33:15PM +0200, Luke Kenneth Casson Leighton wrote:
> > > Even better, there has
> > > been a call for a SAX-like API in APRUTIL, backed by Expat, libxml, or
> > > Xerces. If you're motivated, you could implement that API for APRUTIL :-)
> >
> > :)
> >
> > interesting concept.
> >
> > would keep _me_ happier too because i'm not entirely
> > ok with the clash between libxml and libxml2 caused
> > by the gnome people bundling libxml 1.8.x with their
> > codebase. rrrr :)
>
> At the risk of being slightly off-topic for this list, what clash ?
> You can install both libxml1 and libxml2 simultaneously, and use
> either of them when doing development. You cannot link with both
> but I doubt it's the issue here,
the header-files are identical?
i'm sorry: i assumed that there would be differences, what with
the upgrade-path that must be followed when doing libxml1 to
libxml2 development upgrading.
if that's not the case, sorry for my erroneous assumptions.
luke
Re: XMLvL, apache, apr, mod_virgule origins: advice needed [LONG]
Posted by Daniel Veillard <Da...@imag.fr>.
On Thu, Jun 07, 2001 at 01:33:15PM +0200, Luke Kenneth Casson Leighton wrote:
> > Even better, there has
> > been a call for a SAX-like API in APRUTIL, backed by Expat, libxml, or
> > Xerces. If you're motivated, you could implement that API for APRUTIL :-)
>
> :)
>
> interesting concept.
>
> would keep _me_ happier too because i'm not entirely
> ok with the clash between libxml and libxml2 caused
> by the gnome people bundling libxml 1.8.x with their
> codebase. rrrr :)
At the risk of being slightly off-topic for this list, what clash ?
You can install both libxml1 and libxml2 simultaneously, and use
either of them when doing development. You cannot link with both
but I doubt it's the issue here,
Daniel
--
Daniel Veillard | Red Hat Network http://redhat.com/products/network/
veillard@redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
More migration of code from httpd to apr-util
Posted by Justin Erenkrantz <je...@ebuilt.com>.
On Thu, Jun 07, 2001 at 01:33:15PM +0200, Luke Kenneth Casson Leighton wrote:
> ack. if i make a [rather short] list of functions i am literally
> duplicating line-for-line, would that suffice as motivation to,
> say, add them to aprutil?
Your list of potential functions looks mostly good to me (haven't sat
down and thought too hard about it though).
I think the question is whether or not there is a predefined limit to
what goes in apr-util. We've hit this recently with the crypto/ stuff.
I think it probably means that some of the committers need to work on
things outside of httpd itself. When I wrote the mod_mbox stuff and
had to write a standalone indexer, I needed the date and uri functions
from httpd. I initially copied them over, but then, since I now have
commit access and no one screamed too loudly, I moved them into
apr-util (with some help on the httpd side) and deleted my private
copy. But, I think this will have to be done on a case-by-case basis
with lots of thought for each one.
There also needs to be a guiding vision for apr-util as well. I'm not
sure what that is exactly. =) I think we might be able to define that
though...
I'd be curious to look at subversion to see if it is duplicating any
code, but the domains are so different, I doubt there is a lot of
overlap. The more users of apr-util, the more we can identify things
that might be worth migrating from httpd into apr-util. But, you
are always free to copy the code if we don't move it into apr-util.
-- justin
Re: XMLvL, apache, apr, mod_virgule origins: advice needed [LONG]
Posted by Daniel Veillard <Da...@imag.fr>.
On Thu, Jun 07, 2001 at 01:33:15PM +0200, Luke Kenneth Casson Leighton wrote:
> > Even better, there has
> > been a call for a SAX-like API in APRUTIL, backed by Expat, libxml, or
> > Xerces. If you're motivated, you could implement that API for APRUTIL :-)
>
> :)
>
> interesting concept.
>
> would keep _me_ happier too because i'm not entirely
> ok with the clash between libxml and libxml2 caused
> by the gnome people bundling libxml 1.8.x with their
> codebase. rrrr :)
At the risk of being slightly off-topic for this list, what clash ?
You can install both libxml1 and libxml2 simultaneously, and use
either of them when doing development. You cannot link with both
but I doubt it's the issue here,
Daniel
--
Daniel Veillard | Red Hat Network http://redhat.com/products/network/
veillard@redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Re: XMLvL, apache, apr, mod_virgule origins: advice needed [LONG]
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Thu, Jun 07, 2001 at 02:14:58AM -0700, Greg Stein wrote:
> On Wed, Jun 06, 2001 at 04:28:43PM +0200, Luke Kenneth Casson Leighton wrote:
> >...
> > it doesn't use apache authentication, in fact there's a whole
> > boat-load of stuff it can't use. it provides its own
> > authentication, from cookies stored in xml-based
> > user-profile files.
>
> Side question: why don't you write mod_auth_FOO to authenticate against
> those files? Then you wouldn't need to bother with authentication in xvl.
> Users could also swap in/out their own authentication as they desired. Or,
> of course, use the default mod_auth_FOO mechanism.
... the xvl code (and the original mod_virgule) have
zero hooks back to using other mod_xxxes.
yes, i would dearly love to be able to hook into other
authentication mechanisms, and to that end i have already
split authentication from authorisation (acct.c and profile.c)
and user credentials and profile information.
once i have a clear idea of how to proceed to hook
into mod_auth_xxxes, i'll probably do it real quick :)
> > so. any suggestions? what components already
> > exist?
>
> Well, you obviously have an entire httpd server to grab code from.
yep!!!
> But in
> terms of isolated components? Nothing beyond APR and APRUTIL.
ack. if i make a [rather short] list of functions i am literally
duplicating line-for-line, would that suffice as motivation to,
say, add them to aprutil?
ahh, heck, why not just cp the file over now :)
these are the functions that i am duplicating - verbatim -
from httpd. needless to say, duplicating code makes
me very unhappy, and if anything can be done about that,
i'd be willing to help make it happen.
API_EXPORT(char *) ap_escape_html(apr_pool_t *p, const char *s);
API_EXPORT(char *) ap_os_escape_path(apr_pool_t *p, const char *path, int partial);
API_EXPORT(char *) ap_make_full_path(apr_pool_t *a, const char *src1,
const char *src2);
API_EXPORT(char *) ap_ht_time(apr_pool_t *p, time_t t, const char *fmt, int gmt);
API_EXPORT(char *) ap_gm_timestr_822(apr_pool_t *p, time_t sec);
API_EXPORT(char *) ap_getword(apr_pool_t *atrans, const char **line, char stop);
API_EXPORT(int) ap_count_dirs(const char *path);
API_EXPORT(char *) ap_make_dirstr_prefix(char *d, const char *s, int n);
API_EXPORT(char *) ap_make_dirstr_parent(apr_pool_t *p, const char *s);
API_EXPORT(int) ap_regexec(const regex_t *preg, const char *string,
size_t nmatch, regmatch_t pmatch[], int eflags);
API_EXPORT(size_t) ap_regerror(int errcode, const regex_t *preg, char *errbuf, size_t errbuf_size);
API_EXPORT(char *) ap_pregsub(apr_pool_t *p, const char *input, const char *source,
size_t nmatch, regmatch_t pmatch[]);
API_EXPORT(void) ap_str_tolower(char *str);
> > i'm using a brain-dead installation
> > of mandrake 7.0 with glibc 2.2 (so i get
> > error, cannot find symbol dl_init_next@@GLIBC_2_0
> > whenever i compile a shared library with -ldl,
> > that includes libxml2, apr, pretty much damn
> > well everything: anyone any clues? i'm
> > installing a lot of RPMs recently to overcome
> > this, _when_ they're available...)
>
> Hmm. No clue on that link error. Never seen it. Of course, I have glibc 2.1
> on my box. Have you tried dropping the -ldl? Maybe 2.2 includes that
> already.
yes i tried dropping -ldl: it says error, cannot find dlopen [and friends]
at link-time instead.
i have found a temporary solution: compile everything with
./configure --enable-shared=no
which doesn't exactly make me very happy, but there y'go.
i shouldn't be so damn stupid as to stuff up my development
machine, _should_ i? :)
> > xvl uses the ap_pool code, the table code,
> > list code, ap_psprintf, i need to exec
> > / spawn programs, ideally i also need a
> > portable version of unix-domain-sockets, which
> > i understand doesn't exist in APR, yet.
>
> APR does Unix domain sockets. We use them in mod_cgid.c.
hooray! i tried glib for this job, a while back, but its
'object-orientated' nature just... got in the way :)
> APRUTIL also locates Expat on the system for you, or builds its bundled copy
> of Expat. You could drop your libxml2 dependency.
ah, uh... libxml2 is a thorougly harsh dependency in xvl.
when i say harsh, i mean, in a 12,000-line codebase there
are 1,000 occurrences of lines with xml in it [grep xml *.c */*.c | wc]
one of the modules - the socket module - uses the SAX2
interface in such a way so as to provide an easily intuitive
[and not original] way to make data transfer look like XML that
took *four weeks* to implement.
i really don't have the kind of time available on my hands
that took up so much concentration to get that stupid code
right.
> Even better, there has
> been a call for a SAX-like API in APRUTIL, backed by Expat, libxml, or
> Xerces. If you're motivated, you could implement that API for APRUTIL :-)
:)
interesting concept.
would keep _me_ happier too because i'm not entirely
ok with the clash between libxml and libxml2 caused
by the gnome people bundling libxml 1.8.x with their
codebase. rrrr :)
> For exec/spawn of programs, take a look at the mod_cgi(d) code. There is
> also the "other child" features of APR.
uh... i am slightly embarrased. having solved the
issue of the dl_init_next@GLIBC2_0 thing, i got the
latest cvs apr and aprutil compiled and tested, and...
well... converted xvl to use it in about 2 hours flat.
including using testprog.c as the basis for getting the
cgi code up and running again.
so, _anyway_, apr is now the proud owner of yet another
project for its existence :)
[anyone interested: it's available at http://virgule.sourceforge.net.
we'll have a development site, run by xvl, up and going soon:
more later]
anyway: if it's ok with all y'all, i'll come back with
some more questions about portability, later? now that
i have the apr bit between my teeth, i'd like to carry
this through the whole hog and actually prove that
xvl can be compiled and run on different platforms.
that means using your ux-dom-socket code etc.
...does that work on win32, out of interest?
many thanks,
luke
Re: XMLvL, apache, apr, mod_virgule origins: advice needed [LONG]
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Thu, Jun 07, 2001 at 02:14:58AM -0700, Greg Stein wrote:
> On Wed, Jun 06, 2001 at 04:28:43PM +0200, Luke Kenneth Casson Leighton wrote:
> >...
> > it doesn't use apache authentication, in fact there's a whole
> > boat-load of stuff it can't use. it provides its own
> > authentication, from cookies stored in xml-based
> > user-profile files.
>
> Side question: why don't you write mod_auth_FOO to authenticate against
> those files? Then you wouldn't need to bother with authentication in xvl.
> Users could also swap in/out their own authentication as they desired. Or,
> of course, use the default mod_auth_FOO mechanism.
... the xvl code (and the original mod_virgule) have
zero hooks back to using other mod_xxxes.
yes, i would dearly love to be able to hook into other
authentication mechanisms, and to that end i have already
split authentication from authorisation (acct.c and profile.c)
and user credentials and profile information.
once i have a clear idea of how to proceed to hook
into mod_auth_xxxes, i'll probably do it real quick :)
> > so. any suggestions? what components already
> > exist?
>
> Well, you obviously have an entire httpd server to grab code from.
yep!!!
> But in
> terms of isolated components? Nothing beyond APR and APRUTIL.
ack. if i make a [rather short] list of functions i am literally
duplicating line-for-line, would that suffice as motivation to,
say, add them to aprutil?
ahh, heck, why not just cp the file over now :)
these are the functions that i am duplicating - verbatim -
from httpd. needless to say, duplicating code makes
me very unhappy, and if anything can be done about that,
i'd be willing to help make it happen.
API_EXPORT(char *) ap_escape_html(apr_pool_t *p, const char *s);
API_EXPORT(char *) ap_os_escape_path(apr_pool_t *p, const char *path, int partial);
API_EXPORT(char *) ap_make_full_path(apr_pool_t *a, const char *src1,
const char *src2);
API_EXPORT(char *) ap_ht_time(apr_pool_t *p, time_t t, const char *fmt, int gmt);
API_EXPORT(char *) ap_gm_timestr_822(apr_pool_t *p, time_t sec);
API_EXPORT(char *) ap_getword(apr_pool_t *atrans, const char **line, char stop);
API_EXPORT(int) ap_count_dirs(const char *path);
API_EXPORT(char *) ap_make_dirstr_prefix(char *d, const char *s, int n);
API_EXPORT(char *) ap_make_dirstr_parent(apr_pool_t *p, const char *s);
API_EXPORT(int) ap_regexec(const regex_t *preg, const char *string,
size_t nmatch, regmatch_t pmatch[], int eflags);
API_EXPORT(size_t) ap_regerror(int errcode, const regex_t *preg, char *errbuf, size_t errbuf_size);
API_EXPORT(char *) ap_pregsub(apr_pool_t *p, const char *input, const char *source,
size_t nmatch, regmatch_t pmatch[]);
API_EXPORT(void) ap_str_tolower(char *str);
> > i'm using a brain-dead installation
> > of mandrake 7.0 with glibc 2.2 (so i get
> > error, cannot find symbol dl_init_next@@GLIBC_2_0
> > whenever i compile a shared library with -ldl,
> > that includes libxml2, apr, pretty much damn
> > well everything: anyone any clues? i'm
> > installing a lot of RPMs recently to overcome
> > this, _when_ they're available...)
>
> Hmm. No clue on that link error. Never seen it. Of course, I have glibc 2.1
> on my box. Have you tried dropping the -ldl? Maybe 2.2 includes that
> already.
yes i tried dropping -ldl: it says error, cannot find dlopen [and friends]
at link-time instead.
i have found a temporary solution: compile everything with
./configure --enable-shared=no
which doesn't exactly make me very happy, but there y'go.
i shouldn't be so damn stupid as to stuff up my development
machine, _should_ i? :)
> > xvl uses the ap_pool code, the table code,
> > list code, ap_psprintf, i need to exec
> > / spawn programs, ideally i also need a
> > portable version of unix-domain-sockets, which
> > i understand doesn't exist in APR, yet.
>
> APR does Unix domain sockets. We use them in mod_cgid.c.
hooray! i tried glib for this job, a while back, but its
'object-orientated' nature just... got in the way :)
> APRUTIL also locates Expat on the system for you, or builds its bundled copy
> of Expat. You could drop your libxml2 dependency.
ah, uh... libxml2 is a thorougly harsh dependency in xvl.
when i say harsh, i mean, in a 12,000-line codebase there
are 1,000 occurrences of lines with xml in it [grep xml *.c */*.c | wc]
one of the modules - the socket module - uses the SAX2
interface in such a way so as to provide an easily intuitive
[and not original] way to make data transfer look like XML that
took *four weeks* to implement.
i really don't have the kind of time available on my hands
that took up so much concentration to get that stupid code
right.
> Even better, there has
> been a call for a SAX-like API in APRUTIL, backed by Expat, libxml, or
> Xerces. If you're motivated, you could implement that API for APRUTIL :-)
:)
interesting concept.
would keep _me_ happier too because i'm not entirely
ok with the clash between libxml and libxml2 caused
by the gnome people bundling libxml 1.8.x with their
codebase. rrrr :)
> For exec/spawn of programs, take a look at the mod_cgi(d) code. There is
> also the "other child" features of APR.
uh... i am slightly embarrased. having solved the
issue of the dl_init_next@GLIBC2_0 thing, i got the
latest cvs apr and aprutil compiled and tested, and...
well... converted xvl to use it in about 2 hours flat.
including using testprog.c as the basis for getting the
cgi code up and running again.
so, _anyway_, apr is now the proud owner of yet another
project for its existence :)
[anyone interested: it's available at http://virgule.sourceforge.net.
we'll have a development site, run by xvl, up and going soon:
more later]
anyway: if it's ok with all y'all, i'll come back with
some more questions about portability, later? now that
i have the apr bit between my teeth, i'd like to carry
this through the whole hog and actually prove that
xvl can be compiled and run on different platforms.
that means using your ux-dom-socket code etc.
...does that work on win32, out of interest?
many thanks,
luke
Re: XMLvL, apache, apr, mod_virgule origins: advice needed [LONG]
Posted by Greg Stein <gs...@lyra.org>.
On Wed, Jun 06, 2001 at 04:28:43PM +0200, Luke Kenneth Casson Leighton wrote:
>...
> it doesn't use apache authentication, in fact there's a whole
> boat-load of stuff it can't use. it provides its own
> authentication, from cookies stored in xml-based
> user-profile files.
Side question: why don't you write mod_auth_FOO to authenticate against
those files? Then you wouldn't need to bother with authentication in xvl.
Users could also swap in/out their own authentication as they desired. Or,
of course, use the default mod_auth_FOO mechanism.
>...
> problem statement
> -----------------
>
> in other words, xvl is _the_ perfect candidate
> for significant integration / use of the apache
> codebase, but the boundaries for doing such
> radically different code generation - ala
> mod_php, mod_snake, mod_xvl etc. - are not
> clearly delineated or documented.
>
> in order to progress xvl, i need to be able
> to leverage the HTTP decoding and ap_xxx
> codebase - without all the trappings, bells
> and whistles of a fully-blown httpd
> installation, if users don't want one, and
> _also_ seamless integration into an existing
> httpd installation, if they do.
>
> i really want my cgi-bin capabilities back!
> i want to be able to run sendmail from xvl
> scripts!
>
> i want my cake _and_ a knife, plate and spoon
> _and_ the time to eat it at leisure [the cake].
>
> please.
>
> :)
>
>
> so. any suggestions? what components already
> exist?
Well, you obviously have an entire httpd server to grab code from. But in
terms of isolated components? Nothing beyond APR and APRUTIL.
> i'm using a brain-dead installation
> of mandrake 7.0 with glibc 2.2 (so i get
> error, cannot find symbol dl_init_next@@GLIBC_2_0
> whenever i compile a shared library with -ldl,
> that includes libxml2, apr, pretty much damn
> well everything: anyone any clues? i'm
> installing a lot of RPMs recently to overcome
> this, _when_ they're available...)
Hmm. No clue on that link error. Never seen it. Of course, I have glibc 2.1
on my box. Have you tried dropping the -ldl? Maybe 2.2 includes that
already.
> xvl uses the ap_pool code, the table code,
> list code, ap_psprintf, i need to exec
> / spawn programs, ideally i also need a
> portable version of unix-domain-sockets, which
> i understand doesn't exist in APR, yet.
APR does Unix domain sockets. We use them in mod_cgid.c.
APRUTIL also locates Expat on the system for you, or builds its bundled copy
of Expat. You could drop your libxml2 dependency. Even better, there has
been a call for a SAX-like API in APRUTIL, backed by Expat, libxml, or
Xerces. If you're motivated, you could implement that API for APRUTIL :-)
For exec/spawn of programs, take a look at the mod_cgi(d) code. There is
also the "other child" features of APR.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: XMLvL, apache, apr, mod_virgule origins: advice needed [LONG]
Posted by Greg Stein <gs...@lyra.org>.
On Wed, Jun 06, 2001 at 04:28:43PM +0200, Luke Kenneth Casson Leighton wrote:
>...
> it doesn't use apache authentication, in fact there's a whole
> boat-load of stuff it can't use. it provides its own
> authentication, from cookies stored in xml-based
> user-profile files.
Side question: why don't you write mod_auth_FOO to authenticate against
those files? Then you wouldn't need to bother with authentication in xvl.
Users could also swap in/out their own authentication as they desired. Or,
of course, use the default mod_auth_FOO mechanism.
>...
> problem statement
> -----------------
>
> in other words, xvl is _the_ perfect candidate
> for significant integration / use of the apache
> codebase, but the boundaries for doing such
> radically different code generation - ala
> mod_php, mod_snake, mod_xvl etc. - are not
> clearly delineated or documented.
>
> in order to progress xvl, i need to be able
> to leverage the HTTP decoding and ap_xxx
> codebase - without all the trappings, bells
> and whistles of a fully-blown httpd
> installation, if users don't want one, and
> _also_ seamless integration into an existing
> httpd installation, if they do.
>
> i really want my cgi-bin capabilities back!
> i want to be able to run sendmail from xvl
> scripts!
>
> i want my cake _and_ a knife, plate and spoon
> _and_ the time to eat it at leisure [the cake].
>
> please.
>
> :)
>
>
> so. any suggestions? what components already
> exist?
Well, you obviously have an entire httpd server to grab code from. But in
terms of isolated components? Nothing beyond APR and APRUTIL.
> i'm using a brain-dead installation
> of mandrake 7.0 with glibc 2.2 (so i get
> error, cannot find symbol dl_init_next@@GLIBC_2_0
> whenever i compile a shared library with -ldl,
> that includes libxml2, apr, pretty much damn
> well everything: anyone any clues? i'm
> installing a lot of RPMs recently to overcome
> this, _when_ they're available...)
Hmm. No clue on that link error. Never seen it. Of course, I have glibc 2.1
on my box. Have you tried dropping the -ldl? Maybe 2.2 includes that
already.
> xvl uses the ap_pool code, the table code,
> list code, ap_psprintf, i need to exec
> / spawn programs, ideally i also need a
> portable version of unix-domain-sockets, which
> i understand doesn't exist in APR, yet.
APR does Unix domain sockets. We use them in mod_cgid.c.
APRUTIL also locates Expat on the system for you, or builds its bundled copy
of Expat. You could drop your libxml2 dependency. Even better, there has
been a call for a SAX-like API in APRUTIL, backed by Expat, libxml, or
Xerces. If you're motivated, you could implement that API for APRUTIL :-)
For exec/spawn of programs, take a look at the mod_cgi(d) code. There is
also the "other child" features of APR.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/