You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Justin Erenkrantz <je...@ebuilt.com> on 2001/06/07 21:10:15 UTC
More migration of code from httpd to apr-util
On Thu, Jun 07, 2001 at 01:33:15PM +0200, Luke Kenneth Casson Leighton wrote:
> ack. if i make a [rather short] list of functions i am literally
> duplicating line-for-line, would that suffice as motivation to,
> say, add them to aprutil?
Your list of potential functions looks mostly good to me (haven't sat
down and thought too hard about it though).
I think the question is whether or not there is a predefined limit to
what goes in apr-util. We've hit this recently with the crypto/ stuff.
I think it probably means that some of the committers need to work on
things outside of httpd itself. When I wrote the mod_mbox stuff and
had to write a standalone indexer, I needed the date and uri functions
from httpd. I initially copied them over, but then, since I now have
commit access and no one screamed too loudly, I moved them into
apr-util (with some help on the httpd side) and deleted my private
copy. But, I think this will have to be done on a case-by-case basis
with lots of thought for each one.
There also needs to be a guiding vision for apr-util as well. I'm not
sure what that is exactly. =) I think we might be able to define that
though...
I'd be curious to look at subversion to see if it is duplicating any
code, but the domains are so different, I doubt there is a lot of
overlap. The more users of apr-util, the more we can identify things
that might be worth migrating from httpd into apr-util. But, you
are always free to copy the code if we don't move it into apr-util.
-- justin
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
> > If the server spawns the programs, then you can also use regular pipes.
>
> I have been thinking of creating apr/rpc/... to handle stuff like this.
> However, right now, we have named pipes. They need to be implemented on
> more platforms, and that may require changing the API a bit, but please
> let's stick with what we have already.
>
> The only thing we can't do with named pipes today is communicate with
> different machines. IMHO, calling any cross-machine communication medium
> a named pipe is just going to confuse any unix programmer. Give it a
> different name.
hiya ryan,
i promised i'd get back to you after having had to leave quickly.
RPC. okay. building RPC directly into apr libs is something i can
strongly recommend you don't do: it could easily get out of hand,
make the code bulky, and most people would wonder what drugs you
were taking today [i.e. it goes against what i am beginning to understand
to be the APR ethos].
however, what i _can_ recommend is a slim wrapper - a *local*
named-pipe (ux-dom-sock-based, tcp-loopback-based, NT-NamedPipe-based,
doesn't matter what the implementation is) that feeds to a
separate program that is responsible for locating and communicating
remote procedures.
a bit like the epmapper on dce/rpc but also a transport as well.
it will take a bit of work to design, however what you end up with
is a very small library that apps have to link with, they also
have to run this [entirely separate] program, and voila you have
RPC.
now, if you're concerned about this design, well... uh... how
do you think NT provides RPC? :) :) :)
worse than that, they put these mechanisms *in the kernel*!
[services running as System, and of course they _have_ to
be optimised and of _course_ that means running parts in the
kernel, agh!]
[i.e., if you're concerned, be concerned about using NT :) ]
all best,
lukes
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 08:04:53AM -0700, rbb@covalent.net wrote:
> > > > BTW, APR already has named pipes. Just what are we trying to solve here?
> >
> > We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> > simply doesn't have an implementation for it.
>
> So, the solution is to implement them on other platforms, not to re-invent
> them. For historical completeness however, allow me to explain how we got
> where we are with named pipes. Unix has them becaues they are easy to
> implement. Windows doesn't have them because only certain versions of
> Windows supports named pipes. Namely, NT and 2000 have named pipes, 9x
> doesn't.
grep NamedPipes */*.c shows some code that uses NPs.
there is a nice comment about 9x not having some
feature [blocking or something], and it says, well,
uhhh.... TOUGH! :)
so, there does already exist code in apr that uses
CreateNamedPipe() - for different purposes.
> Originally, APR supported named pipes on Windows,
great!!!
> they were
> removed in revision 1.11. From looking at the log, I don't understand
> why.
*curious*.
> > > i looked at the named pipe code: i believe that you are thinking
> > > of 'unix' named pipes, which are a totally diffrerent beast
> > > from nt named pipes.
> >
> > Right. I don't recall all the bits about creating named pipes in Windows,
> > but it may be feasible to use the "filename" in the
> > apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> > other words, it may be a simple matter of writing the function for Windows.
> > But then again, the function params may not be rich enough for what is
> > needed.
>
> See above, this has worked in the past, all we need to do is copy the code
> back into place, and possibly tweak it.
yesplease! less work!
> > >...
> > > - a means to set up a server and have other programs (not
> > > remote programs) connect to and communicate with that server.
> > >
> > > the semantics must be identical to those of unix-domain-sockets,
> > > namely a listen, bind, accept, read, write and close.
> >
> > I would think you could use named pipes, TCP sockets, or Unix sockets.
> >
> > If the server spawns the programs, then you can also use regular pipes.
>
> I have been thinking of creating apr/rpc/... to handle stuff like this.
> However, right now, we have named pipes. They need to be implemented on
> more platforms, and that may require changing the API a bit, but please
> let's stick with what we have already.
>
> The only thing we can't do with named pipes today is communicate with
> different machines. IMHO, calling any cross-machine communication medium
> a named pipe is just going to confuse any unix programmer. Give it a
> different name.
okay.
i am familiar with how remote IPC works, from a few perspectives,
having implemented a subset of DCE/RPC that is MS-compatible,
over SMB, for Samba (ISBN 1578701503).
[eek, have bus to catch soon, gotta run].
i can attest to the complexity of implementing such: it's
not something to add to a simple library like APR without
a *lot* of prior thought - an implementation may be about
7,000 to 10,000 LOC - not trivial.
gotta run, more later.
all best,
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
> > If the server spawns the programs, then you can also use regular pipes.
>
> I have been thinking of creating apr/rpc/... to handle stuff like this.
> However, right now, we have named pipes. They need to be implemented on
> more platforms, and that may require changing the API a bit, but please
> let's stick with what we have already.
>
> The only thing we can't do with named pipes today is communicate with
> different machines. IMHO, calling any cross-machine communication medium
> a named pipe is just going to confuse any unix programmer. Give it a
> different name.
hiya ryan,
i promised i'd get back to you after having had to leave quickly.
RPC. okay. building RPC directly into apr libs is something i can
strongly recommend you don't do: it could easily get out of hand,
make the code bulky, and most people would wonder what drugs you
were taking today [i.e. it goes against what i am beginning to understand
to be the APR ethos].
however, what i _can_ recommend is a slim wrapper - a *local*
named-pipe (ux-dom-sock-based, tcp-loopback-based, NT-NamedPipe-based,
doesn't matter what the implementation is) that feeds to a
separate program that is responsible for locating and communicating
remote procedures.
a bit like the epmapper on dce/rpc but also a transport as well.
it will take a bit of work to design, however what you end up with
is a very small library that apps have to link with, they also
have to run this [entirely separate] program, and voila you have
RPC.
now, if you're concerned about this design, well... uh... how
do you think NT provides RPC? :) :) :)
worse than that, they put these mechanisms *in the kernel*!
[services running as System, and of course they _have_ to
be optimised and of _course_ that means running parts in the
kernel, agh!]
[i.e., if you're concerned, be concerned about using NT :) ]
all best,
lukes
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 08:04:53AM -0700, rbb@covalent.net wrote:
> > > > BTW, APR already has named pipes. Just what are we trying to solve here?
> >
> > We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> > simply doesn't have an implementation for it.
>
> So, the solution is to implement them on other platforms, not to re-invent
> them. For historical completeness however, allow me to explain how we got
> where we are with named pipes. Unix has them becaues they are easy to
> implement. Windows doesn't have them because only certain versions of
> Windows supports named pipes. Namely, NT and 2000 have named pipes, 9x
> doesn't.
grep NamedPipes */*.c shows some code that uses NPs.
there is a nice comment about 9x not having some
feature [blocking or something], and it says, well,
uhhh.... TOUGH! :)
so, there does already exist code in apr that uses
CreateNamedPipe() - for different purposes.
> Originally, APR supported named pipes on Windows,
great!!!
> they were
> removed in revision 1.11. From looking at the log, I don't understand
> why.
*curious*.
> > > i looked at the named pipe code: i believe that you are thinking
> > > of 'unix' named pipes, which are a totally diffrerent beast
> > > from nt named pipes.
> >
> > Right. I don't recall all the bits about creating named pipes in Windows,
> > but it may be feasible to use the "filename" in the
> > apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> > other words, it may be a simple matter of writing the function for Windows.
> > But then again, the function params may not be rich enough for what is
> > needed.
>
> See above, this has worked in the past, all we need to do is copy the code
> back into place, and possibly tweak it.
yesplease! less work!
> > >...
> > > - a means to set up a server and have other programs (not
> > > remote programs) connect to and communicate with that server.
> > >
> > > the semantics must be identical to those of unix-domain-sockets,
> > > namely a listen, bind, accept, read, write and close.
> >
> > I would think you could use named pipes, TCP sockets, or Unix sockets.
> >
> > If the server spawns the programs, then you can also use regular pipes.
>
> I have been thinking of creating apr/rpc/... to handle stuff like this.
> However, right now, we have named pipes. They need to be implemented on
> more platforms, and that may require changing the API a bit, but please
> let's stick with what we have already.
>
> The only thing we can't do with named pipes today is communicate with
> different machines. IMHO, calling any cross-machine communication medium
> a named pipe is just going to confuse any unix programmer. Give it a
> different name.
okay.
i am familiar with how remote IPC works, from a few perspectives,
having implemented a subset of DCE/RPC that is MS-compatible,
over SMB, for Samba (ISBN 1578701503).
[eek, have bus to catch soon, gotta run].
i can attest to the complexity of implementing such: it's
not something to add to a simple library like APR without
a *lot* of prior thought - an implementation may be about
7,000 to 10,000 LOC - not trivial.
gotta run, more later.
all best,
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 03:04:10AM -0700, Greg Stein wrote:
> On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> > On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
> >...
> > > BTW, APR already has named pipes. Just what are we trying to solve here?
>
> We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> simply doesn't have an implementation for it.
>
> > i looked at the named pipe code: i believe that you are thinking
> > of 'unix' named pipes, which are a totally diffrerent beast
> > from nt named pipes.
>
> Right. I don't recall all the bits about creating named pipes in Windows,
> but it may be feasible to use the "filename" in the
> apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> other words, it may be a simple matter of writing the function for Windows.
> But then again, the function params may not be rich enough for what is
> needed.
>
> >...
> > - a means to set up a server and have other programs (not
> > remote programs) connect to and communicate with that server.
> >
> > the semantics must be identical to those of unix-domain-sockets,
> > namely a listen, bind, accept, read, write and close.
>
> I would think you could use named pipes,
don't know anything about them: are they portable?
> TCP sockets,
known to be slow, even on 127.0.0.1 loopback.
> or Unix sockets.
not portable. [would do the trick, imo, for a ux-impl. of
NT NamedPipes.]
> If the server spawns the programs, then you can also use regular pipes.
>
> > - a means to transfer security context between
> > the server and the 'other programs'. i.e. if the
> > client program is running as 'foo', then it must be
> > possible for the server to be 'told' this - _if_ it
> > needs to know.
>
> I do not believe there is a way in any OS to say what *program* you are
the program is irrelevant: it's the named pipe that's important.
exactly in the same way as in ux-dom-socks, the program
is irrelevant [i have to say, i don't quite follow
what you mean or think i meant that prompted you to
mention 'programs, here: perhaps you could elaborate
so we don't mis-communicate?]
e.g i will call pp = apr_create_np("\PIPE\xvl\xmlvl.org", pool)
and it will create, on ux, a domsock /tmp/.apr/PIPE\xvl\xmlvl.org.
i will then call apr_bind(pp), apr_listen(pp) etc blah blah.
i will then, in a client-program (mod_virgule), call apr_open_np(
"\PIPE\xvl\xmlvl.org") etc. which opens me a connection, causes
the server to call apr_connect() blah blah, in all the usual
ways.
now, on NT, of course, this will not work: NT doesn't _have_
ux-dom-sock. but it _does_ have NamedPipes.
and i expect that using them could provide exactly the same
functionality [as described in above example].
> running (in an authenticated fashion). Reading man pages, it appears Linux
> 2.2 added a way to send another process the PID, UID, and GID. But that
> would obviously be non-portable.
the technique used in samba TNG's IPC mechanism
[which is ux-dom-sock-based], is to transfer the uid, gid and
ux-groups [plus the NT auth equivalents as well] over the
socket as ... 'out-of-band' data at setup time.
this info is recorded in a state-structure that can be
obtained by a get_sec_ctxt() -get security context - function.
so, the impl. would be, if you detect that lx 2.2 supports
transfer like this, use it. if not, send the uid, gid, ux-groups
at setup time.
simple :)
[btw, it so happens that the pid is also transferred, but that's
for other reasons that have nothing to do with emulating
NamedPipes, it's to do with SMB and the implementation
of Samba TNG.]
all best,
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 03:04:10AM -0700, Greg Stein wrote:
> On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> > On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
> >...
> > > BTW, APR already has named pipes. Just what are we trying to solve here?
>
> We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> simply doesn't have an implementation for it.
>
> > i looked at the named pipe code: i believe that you are thinking
> > of 'unix' named pipes, which are a totally diffrerent beast
> > from nt named pipes.
>
> Right. I don't recall all the bits about creating named pipes in Windows,
> but it may be feasible to use the "filename" in the
> apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> other words, it may be a simple matter of writing the function for Windows.
> But then again, the function params may not be rich enough for what is
> needed.
>
> >...
> > - a means to set up a server and have other programs (not
> > remote programs) connect to and communicate with that server.
> >
> > the semantics must be identical to those of unix-domain-sockets,
> > namely a listen, bind, accept, read, write and close.
>
> I would think you could use named pipes,
don't know anything about them: are they portable?
> TCP sockets,
known to be slow, even on 127.0.0.1 loopback.
> or Unix sockets.
not portable. [would do the trick, imo, for a ux-impl. of
NT NamedPipes.]
> If the server spawns the programs, then you can also use regular pipes.
>
> > - a means to transfer security context between
> > the server and the 'other programs'. i.e. if the
> > client program is running as 'foo', then it must be
> > possible for the server to be 'told' this - _if_ it
> > needs to know.
>
> I do not believe there is a way in any OS to say what *program* you are
the program is irrelevant: it's the named pipe that's important.
exactly in the same way as in ux-dom-socks, the program
is irrelevant [i have to say, i don't quite follow
what you mean or think i meant that prompted you to
mention 'programs, here: perhaps you could elaborate
so we don't mis-communicate?]
e.g i will call pp = apr_create_np("\PIPE\xvl\xmlvl.org", pool)
and it will create, on ux, a domsock /tmp/.apr/PIPE\xvl\xmlvl.org.
i will then call apr_bind(pp), apr_listen(pp) etc blah blah.
i will then, in a client-program (mod_virgule), call apr_open_np(
"\PIPE\xvl\xmlvl.org") etc. which opens me a connection, causes
the server to call apr_connect() blah blah, in all the usual
ways.
now, on NT, of course, this will not work: NT doesn't _have_
ux-dom-sock. but it _does_ have NamedPipes.
and i expect that using them could provide exactly the same
functionality [as described in above example].
> running (in an authenticated fashion). Reading man pages, it appears Linux
> 2.2 added a way to send another process the PID, UID, and GID. But that
> would obviously be non-portable.
the technique used in samba TNG's IPC mechanism
[which is ux-dom-sock-based], is to transfer the uid, gid and
ux-groups [plus the NT auth equivalents as well] over the
socket as ... 'out-of-band' data at setup time.
this info is recorded in a state-structure that can be
obtained by a get_sec_ctxt() -get security context - function.
so, the impl. would be, if you detect that lx 2.2 supports
transfer like this, use it. if not, send the uid, gid, ux-groups
at setup time.
simple :)
[btw, it so happens that the pid is also transferred, but that's
for other reasons that have nothing to do with emulating
NamedPipes, it's to do with SMB and the implementation
of Samba TNG.]
all best,
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 10:33:17AM -0500, William A. Rowe, Jr. wrote:
> Ok. Here are the not-so-subtle differences.
>
> First, I understand 9x actually supports reading and writing to a named pipe
> created on an NT/2000 machine, but they don't support them internally. And they
> are always blocking.
>
> Second, the naming conventions are entirely different. On win32, the name is
> \\.\pipe\somepipename while on unix (if I understand it) any directory can 'host'
> a named pipe. On unix, my pipe can be apache2.0/connectors/foo, while on win32
> it is all within that 'pipe' pseudofolder. On Win32, we can actually open the
> \\somemachine\pipe\somepipename from either 9x or WinNT, but the blocking features
> change.
>
> All this implies a common support for 'flat' entries (no directory tree, or the
> caviat that the 'path' will be ignored on win32.) It implies one of; an alternate
> file open call, a 'open pipe' flag bit, or returning the handles on open.
>
> Which is also a requirement. When we call namedpipe_create, we have to RETURN
> SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> But once that pipe handle is closed, the pipe evaporates, they are not persistant.
>
> So some semantics change. We don't 'rm' a pipe when we are finished with it, we just
> close the last handle. And pipes on win32 carry a 'suggested size', but the default
> is a paultry 8kb.
the reasons for this are historical, and to do with the fact that
the implementation uses SMB [even on loopback, i believe]
on top of this meagre data size, you can add your own protocol
that provides you with more data, and that has the added benefit
of guaranteeing your communication and latency timing without clogging
up the network.
> If this is what we want to use, let's start talking about what api we want. I'm happy
> to propose the changes to the _namedpipe_ functions that will get this working in the
> next day or two on Win32.
the basic functionality that i really need is to be able to do 'messages'.
definition of messages:
- send some data on a pipe, it is guaranteed to all be transferred.
followed by:
- receive some data from the pipe, even if the data received
is zero bytes [yes, this _is_ important], and it is guaranteed
to be all transferred, and the amount i am told that is
being received _is_ received.
i.e. on NT, TransactNamedPipe.
on ux-dom-sock, this can be achieved by sending / receiving a 4-byte
header indicating the size of the data to be transferred.
even if that size (send or receive) is zero bytes, the transaction
*has* occurred, which is communication in itself [hi, please
put this in a local file, and acknowledge it by sending me back
zero bytes to indicate that the transaction is completed].
i'm labouring the point, here, i'll shut up now.
> > The only thing we can't do with named pipes today is communicate with
> > different machines. IMHO, calling any cross-machine communication medium
> > a named pipe is just going to confuse any unix programmer. Give it a
> > different name.
>
> No, it effectively is a named pipe. They *really* don't differ all that much.
> The differences are in the naming rules, and Win9x compatibility.
>
> Implementors are just going to have to accept that 9x aren't Operating Systems
> in today's sense, but consumer appliances/gaming consoles,
seconded! 9x is a 'program-running-program' where the people developing
it burn out in 6 months, leave after 2 years MAX. NT is a mt-os where
the people developing it stay with MS for an average of 8 years, most
of them are now millionaires and are there out of a sense of responsibility
and duty to develop the best OS they can.
pick your poison...
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 10:33:17AM -0500, William A. Rowe, Jr. wrote:
> Ok. Here are the not-so-subtle differences.
>
> First, I understand 9x actually supports reading and writing to a named pipe
> created on an NT/2000 machine, but they don't support them internally. And they
> are always blocking.
>
> Second, the naming conventions are entirely different. On win32, the name is
> \\.\pipe\somepipename while on unix (if I understand it) any directory can 'host'
> a named pipe. On unix, my pipe can be apache2.0/connectors/foo, while on win32
> it is all within that 'pipe' pseudofolder. On Win32, we can actually open the
> \\somemachine\pipe\somepipename from either 9x or WinNT, but the blocking features
> change.
>
> All this implies a common support for 'flat' entries (no directory tree, or the
> caviat that the 'path' will be ignored on win32.) It implies one of; an alternate
> file open call, a 'open pipe' flag bit, or returning the handles on open.
>
> Which is also a requirement. When we call namedpipe_create, we have to RETURN
> SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> But once that pipe handle is closed, the pipe evaporates, they are not persistant.
>
> So some semantics change. We don't 'rm' a pipe when we are finished with it, we just
> close the last handle. And pipes on win32 carry a 'suggested size', but the default
> is a paultry 8kb.
the reasons for this are historical, and to do with the fact that
the implementation uses SMB [even on loopback, i believe]
on top of this meagre data size, you can add your own protocol
that provides you with more data, and that has the added benefit
of guaranteeing your communication and latency timing without clogging
up the network.
> If this is what we want to use, let's start talking about what api we want. I'm happy
> to propose the changes to the _namedpipe_ functions that will get this working in the
> next day or two on Win32.
the basic functionality that i really need is to be able to do 'messages'.
definition of messages:
- send some data on a pipe, it is guaranteed to all be transferred.
followed by:
- receive some data from the pipe, even if the data received
is zero bytes [yes, this _is_ important], and it is guaranteed
to be all transferred, and the amount i am told that is
being received _is_ received.
i.e. on NT, TransactNamedPipe.
on ux-dom-sock, this can be achieved by sending / receiving a 4-byte
header indicating the size of the data to be transferred.
even if that size (send or receive) is zero bytes, the transaction
*has* occurred, which is communication in itself [hi, please
put this in a local file, and acknowledge it by sending me back
zero bytes to indicate that the transaction is completed].
i'm labouring the point, here, i'll shut up now.
> > The only thing we can't do with named pipes today is communicate with
> > different machines. IMHO, calling any cross-machine communication medium
> > a named pipe is just going to confuse any unix programmer. Give it a
> > different name.
>
> No, it effectively is a named pipe. They *really* don't differ all that much.
> The differences are in the naming rules, and Win9x compatibility.
>
> Implementors are just going to have to accept that 9x aren't Operating Systems
> in today's sense, but consumer appliances/gaming consoles,
seconded! 9x is a 'program-running-program' where the people developing
it burn out in 6 months, leave after 2 years MAX. NT is a mt-os where
the people developing it stay with MS for an average of 8 years, most
of them are now millionaires and are there out of a sense of responsibility
and duty to develop the best OS they can.
pick your poison...
luke
Re: More migration of code from httpd to apr-util
Posted by Bill Stoddard <bi...@wstoddard.com>.
> > > > Which is also a requirement. When we call namedpipe_create, we have to RETURN
> > > > SOMETHING!
>
> Maybe not... What if we open a namedpipe in Windows in bidirectional mode. Then we call
> CreateFile(pipename,...) for mode read and CreateFile(pipename, ...) for write mode. Not
> tested but it might work.
Nope, doesn't work. The second CreateFile(pipename...) fails with message "All pipe
instances are busy". CreateNamedPipe on Win32 expects to return the server side of the
pipe and there is no way around that fact that I am aware of. This means that the
existing apr_file_namedpipe_create() function needs to change. Perhaps like this:
When server calls apr_file_namedpipe_create, return the server side pipe:
apr_file_t *server_pipe;
apr_file_namedpipe_create(&server_pipe, filename, perms, server_side_pipe_mode, pool)
Client the calls apr_file_open(filename) to get the client side of the pipe.
When client and server are done with the pipe, call apr_file_close(server_pipe |
client_pipe);
Bill
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Thu, Jun 21, 2001 at 07:58:07AM -0700, Justin Erenkrantz wrote:
> On Thu, Jun 21, 2001 at 02:54:43PM +0200, Luke Kenneth Casson Leighton wrote:
> > what i have been hinting at, and planning, is that i will actually
> > provide, with the TNG architecture, is the FULL \\servername\\PIPE\pipename
> > functionality. yes, that's right: i will write code that redirects
> > to TNG, which will farm out the data over SMB over to a remote
> > system FOR you.
> >
> > that's right: a unix apr application will be able to connect to
> > a *remote* NT apr server. that's *remote* platform independence,
> > not just local platform independence.
> >
> > and if you don't expose the full pipe-name in the APR api, i can't
> > do that.
>
> Heh, you attempted to answer my previous question before I asked it.
whoopsie :)
> Teaches me to read all of your posts before replying to any of them.
:) me too, i think i have enough experience at the consequences of
not reading ahead, now :)
> But, aren't named pipes typically local-only?
server-side, they can be nothing _but_ local-only.
client-side, no. it's possible to connect to
\\servername\PIPE\samr and make direct dce/rpc requests
to it, if you feel so inclined.
or to \\servername\PIPE\LANMAN, or whatever is running or
whatever someone chooses to make run, listening on a pipe.
> Maybe what you need is
> something outside of traditional named pipes (i.e. "remote" named
> pipes). Add the named pipe code and then add a "special" function in
> apr that allows you to get at the full namespace, if so desired.
.... which is why i'm advocating the NAL because then 'hooking' in
'special' functions becomes a trivial matter.
*thinks.
*thinks some more*.
AH! bill, i think you're right, regarding server-side, about
specifying _just_ the pipe name.
client-side, i'm not so sure.
does anyone have any working NT client / server pipe apps?
i'd be a lot happier / lot more confident if i could
see coding examples, native NT code.
and bill, do you envisage calling apr_namedpipe_create() on
_both_ the client-side _and_ the server-side, a bit like
when you create a socket?
because if so, i think the full syntax \\server\pipe\pipename
will be needed, and if you go 'server-side', then server
has to be '.' otherwise it's an error.
> But,
> it seems that this "remote" named pipe would only be available with an
> SMB library - which is TNG's domain, not APR's.
i envision that anyone could write a 'redirector' daemon,
and without such a daemon running, you simply don't _get_
the ability of client-side programs to connect to remote
pipes, you can only connect to \\.\PIPE\pipename.
> I guess I'm unsure whether named pipes will work from other machines on
> other platforms. If this works only with Win32 (or having an SMB
> protocol to handle this on the Unix end),
or your own 'redirector' daemon that runs on the client and on the
server, but to be honest, doing it via SMB is the simplest option:
take a look at the odbc-odbc-bridges that are out there, the principle
is sound - for only _one_ server - but as soon as you get to multiple
servers it becomes completely unworkable.
> this seems something that TNG
> could add outside of the apr space. -- justin
OS/2 also supports TransactNamedPipe and friends, i asked :)
all best,
luke
p.s you know, i really should just go and code this up :)
Re: More migration of code from httpd to apr-util
Posted by Justin Erenkrantz <je...@ebuilt.com>.
On Thu, Jun 21, 2001 at 02:54:43PM +0200, Luke Kenneth Casson Leighton wrote:
> what i have been hinting at, and planning, is that i will actually
> provide, with the TNG architecture, is the FULL \\servername\\PIPE\pipename
> functionality. yes, that's right: i will write code that redirects
> to TNG, which will farm out the data over SMB over to a remote
> system FOR you.
>
> that's right: a unix apr application will be able to connect to
> a *remote* NT apr server. that's *remote* platform independence,
> not just local platform independence.
>
> and if you don't expose the full pipe-name in the APR api, i can't
> do that.
Heh, you attempted to answer my previous question before I asked it.
Teaches me to read all of your posts before replying to any of them.
But, aren't named pipes typically local-only? Maybe what you need is
something outside of traditional named pipes (i.e. "remote" named
pipes). Add the named pipe code and then add a "special" function in
apr that allows you to get at the full namespace, if so desired. But,
it seems that this "remote" named pipe would only be available with an
SMB library - which is TNG's domain, not APR's.
I guess I'm unsure whether named pipes will work from other machines on
other platforms. If this works only with Win32 (or having an SMB
protocol to handle this on the Unix end), this seems something that TNG
could add outside of the apr space. -- justin
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Wed, Jun 20, 2001 at 10:50:03AM -0500, William A. Rowe, Jr. wrote:
> From: "Luke Kenneth Casson Leighton" <lk...@samba-tng.org>
> Sent: Wednesday, June 20, 2001 8:55 AM
>
>
> > > You miss my point. We at _least_ need to return a "Windows Acceptable Pipe Name", instead
> > > of some /PIPE/pluming style name.
> >
> > it is *important* that NT-style conventions are followed.
>
> ... internally
uhh... okay.
okay, it's important to me, given what i would like to achieve.
i explain below. sorry for not having mentioned it earlier.
> > the 'guiding light' *must* be the NT functions.
>
> No, the 'internal light' must be the platform-appropriate function.
>
> > i know that most of you may find this difficult to accept that
> > microsoft may be able to develop an API that's really useful,
> > and may feel 'tempted' to 'do better' by imposing 'unixy
> > style' conventions.
> >
> > please don't.
>
> We aren't. And you sir, are presuming that NT is the only 'obscure' platform
> out there. It isn't.
i wasn't, but i didn't say that, so you weren't to know, so i stand
corrected for not communicating enough.
> I'm suggesting that we pass a pipe name to create/locate a pipe. No hokey pathing,
> simply a pipe name that will be normalized \\.\PIPE\name or /dev/pipe/apr/name (or
> however it ought to be named on win32.)
what i have been hinting at, and planning, is that i will actually
provide, with the TNG architecture, is the FULL \\servername\\PIPE\pipename
functionality. yes, that's right: i will write code that redirects
to TNG, which will farm out the data over SMB over to a remote
system FOR you.
that's right: a unix apr application will be able to connect to
a *remote* NT apr server. that's *remote* platform independence,
not just local platform independence.
and if you don't expose the full pipe-name in the APR api, i can't
do that.
> And return an opaque structure that the
> platform may us to open handles on that pipe using an apr_filepipe_open() call.
ack to that.
> > you may be used to doing this the other way round: making NT do
> > what unix does.
>
> We aren't, we all agree that lcd coding is required on apr, and we offer extensions
> where they are needed.
>
> Heck, why could even accept unix "/dev/pipe/name" as a pipe, but it would be morphed
> into "\\.\PIPE\dev_pipe_name" in order to serve the data.
>
> > and if that means creating files like /tmp/.apr-socket/\PIPE\samr with
> > backslashes in them and i _know_ this _is_ possible to do on unix,
> > and i _know_ that there are no problems with doing so because winbindd
> > does it [i have a directory /home/DOMAIN\administrator on my linux
> > hard disk and even an entry in /etc/passwd to match it] then so be it.
>
> Why not simply a pipe name, and leave pathing to apr? Justification?
see above.
does that help?
luke
Re: More migration of code from httpd to apr-util
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
From: "Luke Kenneth Casson Leighton" <lk...@samba-tng.org>
Sent: Wednesday, June 20, 2001 8:55 AM
> > You miss my point. We at _least_ need to return a "Windows Acceptable Pipe Name", instead
> > of some /PIPE/pluming style name.
>
> it is *important* that NT-style conventions are followed.
... internally
> the 'guiding light' *must* be the NT functions.
No, the 'internal light' must be the platform-appropriate function.
> i know that most of you may find this difficult to accept that
> microsoft may be able to develop an API that's really useful,
> and may feel 'tempted' to 'do better' by imposing 'unixy
> style' conventions.
>
> please don't.
We aren't. And you sir, are presuming that NT is the only 'obscure' platform
out there. It isn't.
I'm suggesting that we pass a pipe name to create/locate a pipe. No hokey pathing,
simply a pipe name that will be normalized \\.\PIPE\name or /dev/pipe/apr/name (or
however it ought to be named on win32.) And return an opaque structure that the
platform may us to open handles on that pipe using an apr_filepipe_open() call.
> you may be used to doing this the other way round: making NT do
> what unix does.
We aren't, we all agree that lcd coding is required on apr, and we offer extensions
where they are needed.
Heck, why could even accept unix "/dev/pipe/name" as a pipe, but it would be morphed
into "\\.\PIPE\dev_pipe_name" in order to serve the data.
> and if that means creating files like /tmp/.apr-socket/\PIPE\samr with
> backslashes in them and i _know_ this _is_ possible to do on unix,
> and i _know_ that there are no problems with doing so because winbindd
> does it [i have a directory /home/DOMAIN\administrator on my linux
> hard disk and even an entry in /etc/passwd to match it] then so be it.
Why not simply a pipe name, and leave pathing to apr? Justification?
Bill
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
> You miss my point. We at _least_ need to return a "Windows Acceptable Pipe Name", instead
> of some /PIPE/pluming style name.
it is *important* that NT-style conventions are followed.
the 'guiding light' *must* be the NT functions.
i know that most of you may find this difficult to accept that
microsoft may be able to develop an API that's really useful,
and may feel 'tempted' to 'do better' by imposing 'unixy
style' conventions.
please don't.
pick the NT functions that are needed, then *make* the apr/unix/xxx.[ch]
functions do *exactly* the same job [and i will quite happily
write that code, btw: i already have most of the components needed
to do it].
you may be used to doing this the other way round: making NT do
what unix does.
please don't.
and if that means creating files like /tmp/.apr-socket/\PIPE\samr with
backslashes in them and i _know_ this _is_ possible to do on unix,
and i _know_ that there are no problems with doing so because winbindd
does it [i have a directory /home/DOMAIN\administrator on my linux
hard disk and even an entry in /etc/passwd to match it] then so be it.
luke
Re: More migration of code from httpd to apr-util
Posted by "William A. Rowe, Jr." <ad...@rowe-clan.net>.
From: "Bill Stoddard" <bi...@wstoddard.com>
Sent: Friday, June 15, 2001 2:15 PM
> > > > Which is also a requirement. When we call namedpipe_create, we have to RETURN
> > > > SOMETHING!
>
> Maybe not... What if we open a namedpipe in Windows in bidirectional mode. Then we call
> CreateFile(pipename,...) for mode read and CreateFile(pipename, ...) for write mode. Not
> tested but it might work.
You miss my point. We at _least_ need to return a "Windows Acceptable Pipe Name", instead
of some /PIPE/pluming style name.
Naming pipes is the biggest hassle, namespaces won't match. The subsequent apr_file_open()
calls need to have a legal name.
And we need to have a lifetime of that pipe. While it will default to the lifetime of the
pool that was passed, we should be able to close that pipe handle as soon as it's unneeded.
> >>>> Win32 will create a pipe handle (not the same as the read/write file
> > > > handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> > > > But once that pipe handle is closed, the pipe evaporates, they are not persistant.
> > >
> > > Not to be contrary, but can't we handle this by registering a cleanup with
> > > the handle? That way, the handle survives until we specifically clean it
> > > up, and we shouldn't have to change the API much. Not saying this is the
> > > right way to go, I am just offering suggestions.
> >
> > No, I don't think it's the right way. I'd expect that we add a placeholder of
> > apr_namedpipe_t that contains the name of the pipe on Unix (and nothing more),
> > or the pipe name and handle on Win32. Opaque, of course.
> >
> If the above works, then passing in an "open named pipe" flag on the apr_file_open() would
> enable APR to prepend the file://pipe/ stuff
Above, which?
Why are we fighting to make this platform specific within APR? If we simply add the
apr_filepipe_open() to compliment the create call, this becomes far more platform
independent.
Otherwise just use the Posix calls and be done with it.
Bill
Re: More migration of code from httpd to apr-util
Posted by Bill Stoddard <bi...@wstoddard.com>.
> > > Which is also a requirement. When we call namedpipe_create, we have to RETURN
> > > SOMETHING!
Maybe not... What if we open a namedpipe in Windows in bidirectional mode. Then we call
CreateFile(pipename,...) for mode read and CreateFile(pipename, ...) for write mode. Not
tested but it might work.
>>>> Win32 will create a pipe handle (not the same as the read/write file
> > > handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> > > But once that pipe handle is closed, the pipe evaporates, they are not persistant.
> >
> > Not to be contrary, but can't we handle this by registering a cleanup with
> > the handle? That way, the handle survives until we specifically clean it
> > up, and we shouldn't have to change the API much. Not saying this is the
> > right way to go, I am just offering suggestions.
>
> No, I don't think it's the right way. I'd expect that we add a placeholder of
> apr_namedpipe_t that contains the name of the pipe on Unix (and nothing more),
> or the pipe name and handle on Win32. Opaque, of course.
>
If the above works, then passing in an "open named pipe" flag on the apr_file_open() would
enable APR to prepend the file://pipe/ stuff
Bill
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 12:30:59PM -0500, William A. Rowe, Jr. wrote:
> From: <rb...@covalent.net>
> Sent: Monday, June 11, 2001 10:58 AM
>
>
> > > Which is also a requirement. When we call namedpipe_create, we have to RETURN
> > > SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> > > handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> > > But once that pipe handle is closed, the pipe evaporates, they are not persistant.
is there anybody out there who really understands, or has used,
pipes on NT?
CreateIoCompletionPort! that's the one!!! dammit, not
WaitNamedPipeHandleState, what am i talking about.
okay, just looking at the online MSDN on ms's site,
the function i am referring to that is somewhat equivalent
to listen/accept, esp. for use in multi-threaded code,
is CreateIoComplationPort.
i could be wrong: it may be possible to just use the
message-based functions CallNamedPipe, TransactNamedPipe
etc.
help, help, who really 'gets' this stuff?
[i understand what it looks like over-the-wire when
this stuff happens over SMB, but have no connection
back to the Win32 APIs that *generate* that network
traffic!]
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Mon, Jun 11, 2001 at 12:30:59PM -0500, William A. Rowe, Jr. wrote:
> From: <rb...@covalent.net>
> Sent: Monday, June 11, 2001 10:58 AM
>
>
> > > Which is also a requirement. When we call namedpipe_create, we have to RETURN
> > > SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> > > handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> > > But once that pipe handle is closed, the pipe evaporates, they are not persistant.
is there anybody out there who really understands, or has used,
pipes on NT?
CreateIoCompletionPort! that's the one!!! dammit, not
WaitNamedPipeHandleState, what am i talking about.
okay, just looking at the online MSDN on ms's site,
the function i am referring to that is somewhat equivalent
to listen/accept, esp. for use in multi-threaded code,
is CreateIoComplationPort.
i could be wrong: it may be possible to just use the
message-based functions CallNamedPipe, TransactNamedPipe
etc.
help, help, who really 'gets' this stuff?
[i understand what it looks like over-the-wire when
this stuff happens over SMB, but have no connection
back to the Win32 APIs that *generate* that network
traffic!]
luke
Re: More migration of code from httpd to apr-util
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
From: <rb...@covalent.net>
Sent: Monday, June 11, 2001 10:58 AM
> > Which is also a requirement. When we call namedpipe_create, we have to RETURN
> > SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> > handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> > But once that pipe handle is closed, the pipe evaporates, they are not persistant.
>
> Not to be contrary, but can't we handle this by registering a cleanup with
> the handle? That way, the handle survives until we specifically clean it
> up, and we shouldn't have to change the API much. Not saying this is the
> right way to go, I am just offering suggestions.
No, I don't think it's the right way. I'd expect that we add a placeholder of
apr_namedpipe_t that contains the name of the pipe on Unix (and nothing more),
or the pipe name and handle on Win32. Opaque, of course.
> > > I have been thinking of creating apr/rpc/... to handle stuff like this.
> > > However, right now, we have named pipes. They need to be implemented on
> > > more platforms, and that may require changing the API a bit, but please
> > > let's stick with what we have already.
> > >
> > > The only thing we can't do with named pipes today is communicate with
> > > different machines. IMHO, calling any cross-machine communication medium
> > > a named pipe is just going to confuse any unix programmer. Give it a
> > > different name.
> >
> > No, it effectively is a named pipe. They *really* don't differ all that much.
> > The differences are in the naming rules, and Win9x compatibility.
> >
> > Implementors are just going to have to accept that 9x aren't Operating Systems
> > in today's sense, but consumer appliances/gaming consoles, so they have
> > significant drawbacks. This does ***NOT*** imply I'm against getting httpd up
> > and running on Win9x! It just means that the sort of advanced features that
> > cross-machine dce would implement can't be effectively targetted to Win9x, or
> > they must be coded around.
>
> My problem with calling this a named pipe, is that Unix named pipes can't
> do cross-machine communication. I think that if we call these things
> named pipes, then in people's (Unix programmer's) minds, there weill be
> limits that don't actually exist. Yes, they can learn what we mean by
> named pipes, but why make it harder than it needs to be?
Pointing out that they 'can' communicate between machines does _NOT_ mean we would
actually implement that feature :-) I'd suggest we plunge forward, with a return
type that lets us kill the pipe later (on Unix, that's nothing more than an
rm namedpipe->name, no? on Win32, closing the handle) and register a cleanup on
it, of course!
Bill
Re: More migration of code from httpd to apr-util
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
From: <rb...@covalent.net>
Sent: Monday, June 11, 2001 10:58 AM
> > Which is also a requirement. When we call namedpipe_create, we have to RETURN
> > SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> > handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> > But once that pipe handle is closed, the pipe evaporates, they are not persistant.
>
> Not to be contrary, but can't we handle this by registering a cleanup with
> the handle? That way, the handle survives until we specifically clean it
> up, and we shouldn't have to change the API much. Not saying this is the
> right way to go, I am just offering suggestions.
No, I don't think it's the right way. I'd expect that we add a placeholder of
apr_namedpipe_t that contains the name of the pipe on Unix (and nothing more),
or the pipe name and handle on Win32. Opaque, of course.
> > > I have been thinking of creating apr/rpc/... to handle stuff like this.
> > > However, right now, we have named pipes. They need to be implemented on
> > > more platforms, and that may require changing the API a bit, but please
> > > let's stick with what we have already.
> > >
> > > The only thing we can't do with named pipes today is communicate with
> > > different machines. IMHO, calling any cross-machine communication medium
> > > a named pipe is just going to confuse any unix programmer. Give it a
> > > different name.
> >
> > No, it effectively is a named pipe. They *really* don't differ all that much.
> > The differences are in the naming rules, and Win9x compatibility.
> >
> > Implementors are just going to have to accept that 9x aren't Operating Systems
> > in today's sense, but consumer appliances/gaming consoles, so they have
> > significant drawbacks. This does ***NOT*** imply I'm against getting httpd up
> > and running on Win9x! It just means that the sort of advanced features that
> > cross-machine dce would implement can't be effectively targetted to Win9x, or
> > they must be coded around.
>
> My problem with calling this a named pipe, is that Unix named pipes can't
> do cross-machine communication. I think that if we call these things
> named pipes, then in people's (Unix programmer's) minds, there weill be
> limits that don't actually exist. Yes, they can learn what we mean by
> named pipes, but why make it harder than it needs to be?
Pointing out that they 'can' communicate between machines does _NOT_ mean we would
actually implement that feature :-) I'd suggest we plunge forward, with a return
type that lets us kill the pipe later (on Unix, that's nothing more than an
rm namedpipe->name, no? on Win32, closing the handle) and register a cleanup on
it, of course!
Bill
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
> Which is also a requirement. When we call namedpipe_create, we have to RETURN
> SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> But once that pipe handle is closed, the pipe evaporates, they are not persistant.
Not to be contrary, but can't we handle this by registering a cleanup with
the handle? That way, the handle survives until we specifically clean it
up, and we shouldn't have to change the API much. Not saying this is the
right way to go, I am just offering suggestions.
> > I have been thinking of creating apr/rpc/... to handle stuff like this.
> > However, right now, we have named pipes. They need to be implemented on
> > more platforms, and that may require changing the API a bit, but please
> > let's stick with what we have already.
> >
> > The only thing we can't do with named pipes today is communicate with
> > different machines. IMHO, calling any cross-machine communication medium
> > a named pipe is just going to confuse any unix programmer. Give it a
> > different name.
>
> No, it effectively is a named pipe. They *really* don't differ all that much.
> The differences are in the naming rules, and Win9x compatibility.
>
> Implementors are just going to have to accept that 9x aren't Operating Systems
> in today's sense, but consumer appliances/gaming consoles, so they have
> significant drawbacks. This does ***NOT*** imply I'm against getting httpd up
> and running on Win9x! It just means that the sort of advanced features that
> cross-machine dce would implement can't be effectively targetted to Win9x, or
> they must be coded around.
My problem with calling this a named pipe, is that Unix named pipes can't
do cross-machine communication. I think that if we call these things
named pipes, then in people's (Unix programmer's) minds, there weill be
limits that don't actually exist. Yes, they can learn what we mean by
named pipes, but why make it harder than it needs to be?
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
> Which is also a requirement. When we call namedpipe_create, we have to RETURN
> SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
> handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
> But once that pipe handle is closed, the pipe evaporates, they are not persistant.
Not to be contrary, but can't we handle this by registering a cleanup with
the handle? That way, the handle survives until we specifically clean it
up, and we shouldn't have to change the API much. Not saying this is the
right way to go, I am just offering suggestions.
> > I have been thinking of creating apr/rpc/... to handle stuff like this.
> > However, right now, we have named pipes. They need to be implemented on
> > more platforms, and that may require changing the API a bit, but please
> > let's stick with what we have already.
> >
> > The only thing we can't do with named pipes today is communicate with
> > different machines. IMHO, calling any cross-machine communication medium
> > a named pipe is just going to confuse any unix programmer. Give it a
> > different name.
>
> No, it effectively is a named pipe. They *really* don't differ all that much.
> The differences are in the naming rules, and Win9x compatibility.
>
> Implementors are just going to have to accept that 9x aren't Operating Systems
> in today's sense, but consumer appliances/gaming consoles, so they have
> significant drawbacks. This does ***NOT*** imply I'm against getting httpd up
> and running on Win9x! It just means that the sort of advanced features that
> cross-machine dce would implement can't be effectively targetted to Win9x, or
> they must be coded around.
My problem with calling this a named pipe, is that Unix named pipes can't
do cross-machine communication. I think that if we call these things
named pipes, then in people's (Unix programmer's) minds, there weill be
limits that don't actually exist. Yes, they can learn what we mean by
named pipes, but why make it harder than it needs to be?
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by "William A. Rowe, Jr." <ad...@rowe-clan.net>.
From: <rb...@covalent.net>
Sent: Monday, June 11, 2001 10:04 AM
> On Mon, 11 Jun 2001, Greg Stein wrote:
>
> > On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> > > On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
> > >...
> > > > BTW, APR already has named pipes. Just what are we trying to solve here?
> >
> > We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> > simply doesn't have an implementation for it.
>
> So, the solution is to implement them on other platforms, not to re-invent
> them. For historical completeness however, allow me to explain how we got
> where we are with named pipes. Unix has them becaues they are easy to
> implement. Windows doesn't have them because only certain versions of
> Windows supports named pipes. Namely, NT and 2000 have named pipes, 9x
> doesn't. Originally, APR supported named pipes on Windows, they were
> removed in revision 1.11. From looking at the log, I don't understand
> why.
Ok. Here are the not-so-subtle differences.
First, I understand 9x actually supports reading and writing to a named pipe
created on an NT/2000 machine, but they don't support them internally. And they
are always blocking.
Second, the naming conventions are entirely different. On win32, the name is
\\.\pipe\somepipename while on unix (if I understand it) any directory can 'host'
a named pipe. On unix, my pipe can be apache2.0/connectors/foo, while on win32
it is all within that 'pipe' pseudofolder. On Win32, we can actually open the
\\somemachine\pipe\somepipename from either 9x or WinNT, but the blocking features
change.
All this implies a common support for 'flat' entries (no directory tree, or the
caviat that the 'path' will be ignored on win32.) It implies one of; an alternate
file open call, a 'open pipe' flag bit, or returning the handles on open.
Which is also a requirement. When we call namedpipe_create, we have to RETURN
SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
But once that pipe handle is closed, the pipe evaporates, they are not persistant.
So some semantics change. We don't 'rm' a pipe when we are finished with it, we just
close the last handle. And pipes on win32 carry a 'suggested size', but the default
is a paultry 8kb.
If this is what we want to use, let's start talking about what api we want. I'm happy
to propose the changes to the _namedpipe_ functions that will get this working in the
next day or two on Win32.
> > > i looked at the named pipe code: i believe that you are thinking
> > > of 'unix' named pipes, which are a totally diffrerent beast
> > > from nt named pipes.
> >
> > Right. I don't recall all the bits about creating named pipes in Windows,
> > but it may be feasible to use the "filename" in the
> > apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> > other words, it may be a simple matter of writing the function for Windows.
> > But then again, the function params may not be rich enough for what is
> > needed.
>
> See above, this has worked in the past, all we need to do is copy the code
> back into place, and possibly tweak it.
I really doubt it. Time to start from scratch, for the reasons I cited above.
> > > - a means to set up a server and have other programs (not
> > > remote programs) connect to and communicate with that server.
> > >
> > > the semantics must be identical to those of unix-domain-sockets,
> > > namely a listen, bind, accept, read, write and close.
> >
> > I would think you could use named pipes, TCP sockets, or Unix sockets.
> >
> > If the server spawns the programs, then you can also use regular pipes.
>
> I have been thinking of creating apr/rpc/... to handle stuff like this.
> However, right now, we have named pipes. They need to be implemented on
> more platforms, and that may require changing the API a bit, but please
> let's stick with what we have already.
>
> The only thing we can't do with named pipes today is communicate with
> different machines. IMHO, calling any cross-machine communication medium
> a named pipe is just going to confuse any unix programmer. Give it a
> different name.
No, it effectively is a named pipe. They *really* don't differ all that much.
The differences are in the naming rules, and Win9x compatibility.
Implementors are just going to have to accept that 9x aren't Operating Systems
in today's sense, but consumer appliances/gaming consoles, so they have
significant drawbacks. This does ***NOT*** imply I'm against getting httpd up
and running on Win9x! It just means that the sort of advanced features that
cross-machine dce would implement can't be effectively targetted to Win9x, or
they must be coded around.
Bill
Re: More migration of code from httpd to apr-util
Posted by "William A. Rowe, Jr." <ad...@rowe-clan.net>.
From: <rb...@covalent.net>
Sent: Monday, June 11, 2001 10:04 AM
> On Mon, 11 Jun 2001, Greg Stein wrote:
>
> > On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> > > On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
> > >...
> > > > BTW, APR already has named pipes. Just what are we trying to solve here?
> >
> > We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> > simply doesn't have an implementation for it.
>
> So, the solution is to implement them on other platforms, not to re-invent
> them. For historical completeness however, allow me to explain how we got
> where we are with named pipes. Unix has them becaues they are easy to
> implement. Windows doesn't have them because only certain versions of
> Windows supports named pipes. Namely, NT and 2000 have named pipes, 9x
> doesn't. Originally, APR supported named pipes on Windows, they were
> removed in revision 1.11. From looking at the log, I don't understand
> why.
Ok. Here are the not-so-subtle differences.
First, I understand 9x actually supports reading and writing to a named pipe
created on an NT/2000 machine, but they don't support them internally. And they
are always blocking.
Second, the naming conventions are entirely different. On win32, the name is
\\.\pipe\somepipename while on unix (if I understand it) any directory can 'host'
a named pipe. On unix, my pipe can be apache2.0/connectors/foo, while on win32
it is all within that 'pipe' pseudofolder. On Win32, we can actually open the
\\somemachine\pipe\somepipename from either 9x or WinNT, but the blocking features
change.
All this implies a common support for 'flat' entries (no directory tree, or the
caviat that the 'path' will be ignored on win32.) It implies one of; an alternate
file open call, a 'open pipe' flag bit, or returning the handles on open.
Which is also a requirement. When we call namedpipe_create, we have to RETURN
SOMETHING! Win32 will create a pipe handle (not the same as the read/write file
handle.) Every (NT/2000) machine could Create or Connect to get that pipe handle.
But once that pipe handle is closed, the pipe evaporates, they are not persistant.
So some semantics change. We don't 'rm' a pipe when we are finished with it, we just
close the last handle. And pipes on win32 carry a 'suggested size', but the default
is a paultry 8kb.
If this is what we want to use, let's start talking about what api we want. I'm happy
to propose the changes to the _namedpipe_ functions that will get this working in the
next day or two on Win32.
> > > i looked at the named pipe code: i believe that you are thinking
> > > of 'unix' named pipes, which are a totally diffrerent beast
> > > from nt named pipes.
> >
> > Right. I don't recall all the bits about creating named pipes in Windows,
> > but it may be feasible to use the "filename" in the
> > apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> > other words, it may be a simple matter of writing the function for Windows.
> > But then again, the function params may not be rich enough for what is
> > needed.
>
> See above, this has worked in the past, all we need to do is copy the code
> back into place, and possibly tweak it.
I really doubt it. Time to start from scratch, for the reasons I cited above.
> > > - a means to set up a server and have other programs (not
> > > remote programs) connect to and communicate with that server.
> > >
> > > the semantics must be identical to those of unix-domain-sockets,
> > > namely a listen, bind, accept, read, write and close.
> >
> > I would think you could use named pipes, TCP sockets, or Unix sockets.
> >
> > If the server spawns the programs, then you can also use regular pipes.
>
> I have been thinking of creating apr/rpc/... to handle stuff like this.
> However, right now, we have named pipes. They need to be implemented on
> more platforms, and that may require changing the API a bit, but please
> let's stick with what we have already.
>
> The only thing we can't do with named pipes today is communicate with
> different machines. IMHO, calling any cross-machine communication medium
> a named pipe is just going to confuse any unix programmer. Give it a
> different name.
No, it effectively is a named pipe. They *really* don't differ all that much.
The differences are in the naming rules, and Win9x compatibility.
Implementors are just going to have to accept that 9x aren't Operating Systems
in today's sense, but consumer appliances/gaming consoles, so they have
significant drawbacks. This does ***NOT*** imply I'm against getting httpd up
and running on Win9x! It just means that the sort of advanced features that
cross-machine dce would implement can't be effectively targetted to Win9x, or
they must be coded around.
Bill
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
On Mon, 11 Jun 2001, Greg Stein wrote:
> On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> > On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
> >...
> > > BTW, APR already has named pipes. Just what are we trying to solve here?
>
> We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> simply doesn't have an implementation for it.
So, the solution is to implement them on other platforms, not to re-invent
them. For historical completeness however, allow me to explain how we got
where we are with named pipes. Unix has them becaues they are easy to
implement. Windows doesn't have them because only certain versions of
Windows supports named pipes. Namely, NT and 2000 have named pipes, 9x
doesn't. Originally, APR supported named pipes on Windows, they were
removed in revision 1.11. From looking at the log, I don't understand
why.
> > i looked at the named pipe code: i believe that you are thinking
> > of 'unix' named pipes, which are a totally diffrerent beast
> > from nt named pipes.
>
> Right. I don't recall all the bits about creating named pipes in Windows,
> but it may be feasible to use the "filename" in the
> apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> other words, it may be a simple matter of writing the function for Windows.
> But then again, the function params may not be rich enough for what is
> needed.
See above, this has worked in the past, all we need to do is copy the code
back into place, and possibly tweak it.
> >...
> > - a means to set up a server and have other programs (not
> > remote programs) connect to and communicate with that server.
> >
> > the semantics must be identical to those of unix-domain-sockets,
> > namely a listen, bind, accept, read, write and close.
>
> I would think you could use named pipes, TCP sockets, or Unix sockets.
>
> If the server spawns the programs, then you can also use regular pipes.
I have been thinking of creating apr/rpc/... to handle stuff like this.
However, right now, we have named pipes. They need to be implemented on
more platforms, and that may require changing the API a bit, but please
let's stick with what we have already.
The only thing we can't do with named pipes today is communicate with
different machines. IMHO, calling any cross-machine communication medium
a named pipe is just going to confuse any unix programmer. Give it a
different name.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
On Mon, 11 Jun 2001, Greg Stein wrote:
> On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> > On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
> >...
> > > BTW, APR already has named pipes. Just what are we trying to solve here?
>
> We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
> simply doesn't have an implementation for it.
So, the solution is to implement them on other platforms, not to re-invent
them. For historical completeness however, allow me to explain how we got
where we are with named pipes. Unix has them becaues they are easy to
implement. Windows doesn't have them because only certain versions of
Windows supports named pipes. Namely, NT and 2000 have named pipes, 9x
doesn't. Originally, APR supported named pipes on Windows, they were
removed in revision 1.11. From looking at the log, I don't understand
why.
> > i looked at the named pipe code: i believe that you are thinking
> > of 'unix' named pipes, which are a totally diffrerent beast
> > from nt named pipes.
>
> Right. I don't recall all the bits about creating named pipes in Windows,
> but it may be feasible to use the "filename" in the
> apr_file_namedpipe_create function for the host/name of the pipe in NT. In
> other words, it may be a simple matter of writing the function for Windows.
> But then again, the function params may not be rich enough for what is
> needed.
See above, this has worked in the past, all we need to do is copy the code
back into place, and possibly tweak it.
> >...
> > - a means to set up a server and have other programs (not
> > remote programs) connect to and communicate with that server.
> >
> > the semantics must be identical to those of unix-domain-sockets,
> > namely a listen, bind, accept, read, write and close.
>
> I would think you could use named pipes, TCP sockets, or Unix sockets.
>
> If the server spawns the programs, then you can also use regular pipes.
I have been thinking of creating apr/rpc/... to handle stuff like this.
However, right now, we have named pipes. They need to be implemented on
more platforms, and that may require changing the API a bit, but please
let's stick with what we have already.
The only thing we can't do with named pipes today is communicate with
different machines. IMHO, calling any cross-machine communication medium
a named pipe is just going to confuse any unix programmer. Give it a
different name.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by Greg Stein <gs...@lyra.org>.
On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
>...
> > BTW, APR already has named pipes. Just what are we trying to solve here?
We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
simply doesn't have an implementation for it.
> i looked at the named pipe code: i believe that you are thinking
> of 'unix' named pipes, which are a totally diffrerent beast
> from nt named pipes.
Right. I don't recall all the bits about creating named pipes in Windows,
but it may be feasible to use the "filename" in the
apr_file_namedpipe_create function for the host/name of the pipe in NT. In
other words, it may be a simple matter of writing the function for Windows.
But then again, the function params may not be rich enough for what is
needed.
>...
> - a means to set up a server and have other programs (not
> remote programs) connect to and communicate with that server.
>
> the semantics must be identical to those of unix-domain-sockets,
> namely a listen, bind, accept, read, write and close.
I would think you could use named pipes, TCP sockets, or Unix sockets.
If the server spawns the programs, then you can also use regular pipes.
> - a means to transfer security context between
> the server and the 'other programs'. i.e. if the
> client program is running as 'foo', then it must be
> possible for the server to be 'told' this - _if_ it
> needs to know.
I do not believe there is a way in any OS to say what *program* you are
running (in an authenticated fashion). Reading man pages, it appears Linux
2.2 added a way to send another process the PID, UID, and GID. But that
would obviously be non-portable.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: More migration of code from httpd to apr-util
Posted by Greg Stein <gs...@lyra.org>.
On Mon, Jun 11, 2001 at 12:38:23AM +0200, Luke Kenneth Casson Leighton wrote:
> On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
>...
> > BTW, APR already has named pipes. Just what are we trying to solve here?
We only have them implemented for Unix. OS/2 returns an ENOTIMPL and Windows
simply doesn't have an implementation for it.
> i looked at the named pipe code: i believe that you are thinking
> of 'unix' named pipes, which are a totally diffrerent beast
> from nt named pipes.
Right. I don't recall all the bits about creating named pipes in Windows,
but it may be feasible to use the "filename" in the
apr_file_namedpipe_create function for the host/name of the pipe in NT. In
other words, it may be a simple matter of writing the function for Windows.
But then again, the function params may not be rich enough for what is
needed.
>...
> - a means to set up a server and have other programs (not
> remote programs) connect to and communicate with that server.
>
> the semantics must be identical to those of unix-domain-sockets,
> namely a listen, bind, accept, read, write and close.
I would think you could use named pipes, TCP sockets, or Unix sockets.
If the server spawns the programs, then you can also use regular pipes.
> - a means to transfer security context between
> the server and the 'other programs'. i.e. if the
> client program is running as 'foo', then it must be
> possible for the server to be 'told' this - _if_ it
> needs to know.
I do not believe there is a way in any OS to say what *program* you are
running (in an authenticated fashion). Reading man pages, it appears Linux
2.2 added a way to send another process the PID, UID, and GID. But that
would obviously be non-portable.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
>
> mod_cgid should NOT use a named pipe. We started with a named pipe, and
> we moved away from that model for some very good reasons. Secondly,
> mod_cgid does NOT work on windows, and it shouldn't. The reason for
> mod_cgid, is that Unix's tend to not know how to fork threaded programs.
> Windows doesn't know how to fork any program, so using mod_cgid on Windows
> makes zero sense.
ack.
> BTW, APR already has named pipes. Just what are we trying to solve here?
i looked at the named pipe code: i believe that you are thinking
of 'unix' named pipes, which are a totally diffrerent beast
from nt named pipes.
... but i could - and am likely to - be wrong.
time to write requirements:
i am looking for, in apr:
- a means to set up a server and have other programs (not
remote programs) connect to and communicate with that server.
the semantics must be identical to those of unix-domain-sockets,
namely a listen, bind, accept, read, write and close.
- a means to transfer security context between
the server and the 'other programs'. i.e. if the
client program is running as 'foo', then it must be
possible for the server to be 'told' this - _if_ it
needs to know.
so, the implementation of this _tends_ to suggest
unix-domain-sockets on unix, and NamedPipes (_not_
unix named pipes) on NT.
however, if unix named pipes do exactly the same
job as is listed in re3quirements above, please
let me know and i'll shut up and code :)
btw, i notice that there is no NT version of
apr_named_pipe_create().
... yet?
luke
> Ryan
>
> On Sat, 9 Jun 2001, Luke Kenneth Casson Leighton wrote:
>
> > btw, this message goes primarily to the developers of
> > mod_cgid, but is related to the apr_create_named_pipe
> > proposal.
> >
> > my question is, does mod_cgid compile under win32, and
> > would you like it to?
> >
> > so far i count 2 immediate projects that need
> > an apr_create_named_pipe (xvl and mod_cgid) and
> > two long-term ones (tng and dce/rpc).
> >
> > an apr_wait_named_pipe_handle_state() which
> > emulates .h.. WaitNamedPipeHandle will also
> > be needed.
> >
> > i know how to do this in unix-domain-socket land,
> > i'm eager to hear if anyone wants to do the win32 version.[btw plare
> > please remember i'm not subscribed to new-httpd, thx!]
> >
> > luke
> >
> >
>
>
> _______________________________________________________________________________
> Ryan Bloom rbb@apache.org
> 406 29th St.
> San Francisco, CA 94131
> -------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Sat, Jun 09, 2001 at 08:30:23AM -0700, rbb@covalent.net wrote:
>
> mod_cgid should NOT use a named pipe. We started with a named pipe, and
> we moved away from that model for some very good reasons. Secondly,
> mod_cgid does NOT work on windows, and it shouldn't. The reason for
> mod_cgid, is that Unix's tend to not know how to fork threaded programs.
> Windows doesn't know how to fork any program, so using mod_cgid on Windows
> makes zero sense.
ack.
> BTW, APR already has named pipes. Just what are we trying to solve here?
i looked at the named pipe code: i believe that you are thinking
of 'unix' named pipes, which are a totally diffrerent beast
from nt named pipes.
... but i could - and am likely to - be wrong.
time to write requirements:
i am looking for, in apr:
- a means to set up a server and have other programs (not
remote programs) connect to and communicate with that server.
the semantics must be identical to those of unix-domain-sockets,
namely a listen, bind, accept, read, write and close.
- a means to transfer security context between
the server and the 'other programs'. i.e. if the
client program is running as 'foo', then it must be
possible for the server to be 'told' this - _if_ it
needs to know.
so, the implementation of this _tends_ to suggest
unix-domain-sockets on unix, and NamedPipes (_not_
unix named pipes) on NT.
however, if unix named pipes do exactly the same
job as is listed in re3quirements above, please
let me know and i'll shut up and code :)
btw, i notice that there is no NT version of
apr_named_pipe_create().
... yet?
luke
> Ryan
>
> On Sat, 9 Jun 2001, Luke Kenneth Casson Leighton wrote:
>
> > btw, this message goes primarily to the developers of
> > mod_cgid, but is related to the apr_create_named_pipe
> > proposal.
> >
> > my question is, does mod_cgid compile under win32, and
> > would you like it to?
> >
> > so far i count 2 immediate projects that need
> > an apr_create_named_pipe (xvl and mod_cgid) and
> > two long-term ones (tng and dce/rpc).
> >
> > an apr_wait_named_pipe_handle_state() which
> > emulates .h.. WaitNamedPipeHandle will also
> > be needed.
> >
> > i know how to do this in unix-domain-socket land,
> > i'm eager to hear if anyone wants to do the win32 version.[btw plare
> > please remember i'm not subscribed to new-httpd, thx!]
> >
> > luke
> >
> >
>
>
> _______________________________________________________________________________
> Ryan Bloom rbb@apache.org
> 406 29th St.
> San Francisco, CA 94131
> -------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
mod_cgid should NOT use a named pipe. We started with a named pipe, and
we moved away from that model for some very good reasons. Secondly,
mod_cgid does NOT work on windows, and it shouldn't. The reason for
mod_cgid, is that Unix's tend to not know how to fork threaded programs.
Windows doesn't know how to fork any program, so using mod_cgid on Windows
makes zero sense.
BTW, APR already has named pipes. Just what are we trying to solve here?
Ryan
On Sat, 9 Jun 2001, Luke Kenneth Casson Leighton wrote:
> btw, this message goes primarily to the developers of
> mod_cgid, but is related to the apr_create_named_pipe
> proposal.
>
> my question is, does mod_cgid compile under win32, and
> would you like it to?
>
> so far i count 2 immediate projects that need
> an apr_create_named_pipe (xvl and mod_cgid) and
> two long-term ones (tng and dce/rpc).
>
> an apr_wait_named_pipe_handle_state() which
> emulates .h.. WaitNamedPipeHandle will also
> be needed.
>
> i know how to do this in unix-domain-socket land,
> i'm eager to hear if anyone wants to do the win32 version.[btw plare
> please remember i'm not subscribed to new-httpd, thx!]
>
> luke
>
>
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
mod_cgid should NOT use a named pipe. We started with a named pipe, and
we moved away from that model for some very good reasons. Secondly,
mod_cgid does NOT work on windows, and it shouldn't. The reason for
mod_cgid, is that Unix's tend to not know how to fork threaded programs.
Windows doesn't know how to fork any program, so using mod_cgid on Windows
makes zero sense.
BTW, APR already has named pipes. Just what are we trying to solve here?
Ryan
On Sat, 9 Jun 2001, Luke Kenneth Casson Leighton wrote:
> btw, this message goes primarily to the developers of
> mod_cgid, but is related to the apr_create_named_pipe
> proposal.
>
> my question is, does mod_cgid compile under win32, and
> would you like it to?
>
> so far i count 2 immediate projects that need
> an apr_create_named_pipe (xvl and mod_cgid) and
> two long-term ones (tng and dce/rpc).
>
> an apr_wait_named_pipe_handle_state() which
> emulates .h.. WaitNamedPipeHandle will also
> be needed.
>
> i know how to do this in unix-domain-socket land,
> i'm eager to hear if anyone wants to do the win32 version.[btw plare
> please remember i'm not subscribed to new-httpd, thx!]
>
> luke
>
>
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
btw, this message goes primarily to the developers of
mod_cgid, but is related to the apr_create_named_pipe
proposal.
my question is, does mod_cgid compile under win32, and
would you like it to?
so far i count 2 immediate projects that need
an apr_create_named_pipe (xvl and mod_cgid) and
two long-term ones (tng and dce/rpc).
an apr_wait_named_pipe_handle_state() which
emulates .h.. WaitNamedPipeHandle will also
be needed.
i know how to do this in unix-domain-socket land,
i'm eager to hear if anyone wants to do the win32 version.[btw plare
please remember i'm not subscribed to new-httpd, thx!]
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Fri, Jun 08, 2001 at 11:37:46AM -0700, Greg Stein wrote:
> On Fri, Jun 08, 2001 at 10:52:34AM -0700, Justin Erenkrantz wrote:
> > On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
> > > On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> > > > what i perhaps should recommend is that a series of independent
> > > > libraries be created.
> > > >
> > > > e.g. libaprhttputil (i know it's a bit long).
> > >
> > > why?
because i forgot about te extract only the needed bits thing :)
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Fri, Jun 08, 2001 at 11:37:46AM -0700, Greg Stein wrote:
> On Fri, Jun 08, 2001 at 10:52:34AM -0700, Justin Erenkrantz wrote:
> > On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
> > > On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> > > > what i perhaps should recommend is that a series of independent
> > > > libraries be created.
> > > >
> > > > e.g. libaprhttputil (i know it's a bit long).
> > >
> > > why?
because i forgot about te extract only the needed bits thing :)
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
btw, this message goes primarily to the developers of
mod_cgid, but is related to the apr_create_named_pipe
proposal.
my question is, does mod_cgid compile under win32, and
would you like it to?
so far i count 2 immediate projects that need
an apr_create_named_pipe (xvl and mod_cgid) and
two long-term ones (tng and dce/rpc).
an apr_wait_named_pipe_handle_state() which
emulates .h.. WaitNamedPipeHandle will also
be needed.
i know how to do this in unix-domain-socket land,
i'm eager to hear if anyone wants to do the win32 version.[btw plare
please remember i'm not subscribed to new-httpd, thx!]
luke
Re: More migration of code from httpd to apr-util
Posted by Greg Stein <gs...@lyra.org>.
On Fri, Jun 08, 2001 at 10:52:34AM -0700, Justin Erenkrantz wrote:
> On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
> > On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> > > what i perhaps should recommend is that a series of independent
> > > libraries be created.
> > >
> > > e.g. libaprhttputil (i know it's a bit long).
> >
> > why?
> >
> > we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
> > ...
> >
> > the point of a library is that when you link against it, the linker will
> > extract only the portions which are used.
>
> So, then we should go ahead and add stuff like HTTP client support to
> apr-util? Where's the boundary line? Is there a limit to what can go
> in apr or apr-util? -- justin
IMO, the boundary line is where a specific chunk of functionality is *huge*,
it should go in its own library. HTTP client code is a very large task (to
do it right).
Otherwise, I'm fine with widely useful functionality going into APRUTIL.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: More migration of code from httpd to apr-util
Posted by Greg Stein <gs...@lyra.org>.
On Fri, Jun 08, 2001 at 10:52:34AM -0700, Justin Erenkrantz wrote:
> On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
> > On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> > > what i perhaps should recommend is that a series of independent
> > > libraries be created.
> > >
> > > e.g. libaprhttputil (i know it's a bit long).
> >
> > why?
> >
> > we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
> > ...
> >
> > the point of a library is that when you link against it, the linker will
> > extract only the portions which are used.
>
> So, then we should go ahead and add stuff like HTTP client support to
> apr-util? Where's the boundary line? Is there a limit to what can go
> in apr or apr-util? -- justin
IMO, the boundary line is where a specific chunk of functionality is *huge*,
it should go in its own library. HTTP client code is a very large task (to
do it right).
Otherwise, I'm fine with widely useful functionality going into APRUTIL.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: More migration of code from httpd to apr-util
Posted by dean gaudet <dg...@arctic.org>.
On Fri, 8 Jun 2001, Justin Erenkrantz wrote:
> On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
> >
> >
> > On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> >
> > > what i perhaps should recommend is that a series of independent
> > > libraries be created.
> > >
> > > e.g. libaprhttputil (i know it's a bit long).
> >
> > why?
> >
> > we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
> > ...
> >
> > the point of a library is that when you link against it, the linker will
> > extract only the portions which are used.
>
> So, then we should go ahead and add stuff like HTTP client support to
> apr-util? Where's the boundary line? Is there a limit to what can go
> in apr or apr-util? -- justin
i dunno, i just didn't want to see another dozen projects spring up with
one or two functions each :)
-dean
Re: More migration of code from httpd to apr-util
Posted by dean gaudet <dg...@arctic.org>.
On Fri, 8 Jun 2001, Justin Erenkrantz wrote:
> On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
> >
> >
> > On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> >
> > > what i perhaps should recommend is that a series of independent
> > > libraries be created.
> > >
> > > e.g. libaprhttputil (i know it's a bit long).
> >
> > why?
> >
> > we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
> > ...
> >
> > the point of a library is that when you link against it, the linker will
> > extract only the portions which are used.
>
> So, then we should go ahead and add stuff like HTTP client support to
> apr-util? Where's the boundary line? Is there a limit to what can go
> in apr or apr-util? -- justin
i dunno, i just didn't want to see another dozen projects spring up with
one or two functions each :)
-dean
Re: More migration of code from httpd to apr-util
Posted by Justin Erenkrantz <je...@ebuilt.com>.
On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
>
>
> On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
>
> > what i perhaps should recommend is that a series of independent
> > libraries be created.
> >
> > e.g. libaprhttputil (i know it's a bit long).
>
> why?
>
> we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
> ...
>
> the point of a library is that when you link against it, the linker will
> extract only the portions which are used.
So, then we should go ahead and add stuff like HTTP client support to
apr-util? Where's the boundary line? Is there a limit to what can go
in apr or apr-util? -- justin
Re: More migration of code from httpd to apr-util
Posted by Justin Erenkrantz <je...@ebuilt.com>.
On Fri, Jun 08, 2001 at 10:47:10AM -0700, dean gaudet wrote:
>
>
> On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
>
> > what i perhaps should recommend is that a series of independent
> > libraries be created.
> >
> > e.g. libaprhttputil (i know it's a bit long).
>
> why?
>
> we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
> ...
>
> the point of a library is that when you link against it, the linker will
> extract only the portions which are used.
So, then we should go ahead and add stuff like HTTP client support to
apr-util? Where's the boundary line? Is there a limit to what can go
in apr or apr-util? -- justin
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Fri, Jun 08, 2001 at 09:48:08AM -0700, rbb@covalent.net wrote:
>
> > > - date/time manipulation.
> > >
> > > at present, all i need to be able to do is get a
> > > time_t and display it in a nice localtime format.
> > > ripping code from httpd to do this is... uh.. silly?
> >
> > Submit a patch and I'll commit it into apr. I think date and time stuff
> > should be yanked out into apr-util. I've already yanked out date.
> > Submit something for time and I'll commit. We can probably find some
> > helpful dude with httpd commit access to remove those files from httpd
> > entirely.
>
> Ummmmm..... apr_ctime takes an apr_time_t, and returns a formatted
> string. If you need more, apr_strftime allows you to define the format
> for the string.
>
ack. thx ryan. that's another function down. only 12 or so
more to go :)
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Fri, Jun 08, 2001 at 09:48:08AM -0700, rbb@covalent.net wrote:
>
> > > - date/time manipulation.
> > >
> > > at present, all i need to be able to do is get a
> > > time_t and display it in a nice localtime format.
> > > ripping code from httpd to do this is... uh.. silly?
> >
> > Submit a patch and I'll commit it into apr. I think date and time stuff
> > should be yanked out into apr-util. I've already yanked out date.
> > Submit something for time and I'll commit. We can probably find some
> > helpful dude with httpd commit access to remove those files from httpd
> > entirely.
>
> Ummmmm..... apr_ctime takes an apr_time_t, and returns a formatted
> string. If you need more, apr_strftime allows you to define the format
> for the string.
>
ack. thx ryan. that's another function down. only 12 or so
more to go :)
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
> > my preference is to do the unix-implementation of the
> > apr_named_pipe API, if there's anyone else willing to
> > help parallel-develop the unix implementation, using
> > xvl as a test to prove the case.
>
> I'd certainly look over the code, but I'm not terribly certain
> if I'd use it. I'd just have to see and evaluate if named pipes
> make sense in my cases. If it does in yours, great. -- justin
ack. se my other post/question: do you want mod_cgid to
work on win32? :) :)
all best,
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
> > my preference is to do the unix-implementation of the
> > apr_named_pipe API, if there's anyone else willing to
> > help parallel-develop the unix implementation, using
> > xvl as a test to prove the case.
>
> I'd certainly look over the code, but I'm not terribly certain
> if I'd use it. I'd just have to see and evaluate if named pipes
> make sense in my cases. If it does in yours, great. -- justin
ack. se my other post/question: do you want mod_cgid to
work on win32? :) :)
all best,
luke
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
> > - date/time manipulation.
> >
> > at present, all i need to be able to do is get a
> > time_t and display it in a nice localtime format.
> > ripping code from httpd to do this is... uh.. silly?
>
> Submit a patch and I'll commit it into apr. I think date and time stuff
> should be yanked out into apr-util. I've already yanked out date.
> Submit something for time and I'll commit. We can probably find some
> helpful dude with httpd commit access to remove those files from httpd
> entirely.
Ummmmm..... apr_ctime takes an apr_time_t, and returns a formatted
string. If you need more, apr_strftime allows you to define the format
for the string.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by rb...@covalent.net.
> > - date/time manipulation.
> >
> > at present, all i need to be able to do is get a
> > time_t and display it in a nice localtime format.
> > ripping code from httpd to do this is... uh.. silly?
>
> Submit a patch and I'll commit it into apr. I think date and time stuff
> should be yanked out into apr-util. I've already yanked out date.
> Submit something for time and I'll commit. We can probably find some
> helpful dude with httpd commit access to remove those files from httpd
> entirely.
Ummmmm..... apr_ctime takes an apr_time_t, and returns a formatted
string. If you need more, apr_strftime allows you to define the format
for the string.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: More migration of code from httpd to apr-util
Posted by Justin Erenkrantz <je...@ebuilt.com>.
On Fri, Jun 08, 2001 at 02:45:09PM +0200, Luke Kenneth Casson Leighton wrote:
> what i perhaps should recommend is that a series of independent
> libraries be created.
>
> e.g. libaprhttputil (i know it's a bit long).
Aaron Bannert and I were discussing that as well yesterday. Yeah, we
kind of get that feeling too over here. But, again, I'm not sure what
the process would be for doing something like this.
> - string/path/cgi manipulation.
>
> this i need to be able to parse HTTP requests, to parse
> directory components. i know httpd does, too.
Depending upon how general it is (i.e. non-HTTP specific), this might
make sense to place in apr-util.
> - date/time manipulation.
>
> at present, all i need to be able to do is get a
> time_t and display it in a nice localtime format.
> ripping code from httpd to do this is... uh.. silly?
Submit a patch and I'll commit it into apr. I think date and time stuff
should be yanked out into apr-util. I've already yanked out date.
Submit something for time and I'll commit. We can probably find some
helpful dude with httpd commit access to remove those files from httpd
entirely.
> - an HTTP client.
>
> i need to be able to make http GETs / POSTs, and so
> originally i ripped entire bits of mod_proxy out into
> a separate file.
>
> i can believe that these two programs, xvl and mod_proxy
> are not the _only_ places where HTTP client GETs/POSTS
> are needed. for example, what about martin pool's
> work on mod_rproxy? [that's a bit like mod_proxy except
> it uses the rsync algorithm to sync up the file being
> requested with the last version obtained, etc., very cool
> work :)]
>
> EITHER :
>
> - an HTTP server-parser
Yeah, well, again, we're going to need it here at eBuilt as well (we're
*supposed* to be writing a load tester). So, I think it might make
sense to pool our resources and come up with an HTTP client API that
works on top of APR. I'm going to be writing this functionality
anyway. We could start with what is in mod_proxy (although I haven't
looked at this code yet). Again, this doesn't belong in APR or
APR-util, but I do believe it belongs somewhere. We keep writing this
over and over again.
And, I'll probably be writing a SAX API for apr-util as well. Hmm, that
may depend on how nice Xerces is.
> > I think the question is whether or not there is a predefined limit to
> > what goes in apr-util. We've hit this recently with the crypto/ stuff.
>
> so, create some new libraries. maybe apr-util isn't the
> best place for some of this stuff, but an apr-http-util is.
> or whatever. i don't mind, or really care [well, i do,
> but not enough to lose sleep over it if it doesn't happen :)]
>
> [and then think carefully about where and how to use them:
> you can't distribute anything that uses openssl / any crypto
> library, everywhere]
Yup. See above.
> i am trying to keep xvl's codebase down to an absolute, absolute
> minimum. the more LOCs i can take _out_, the happier i will be.
> duplicating or copying effort is a no-no in my book.
> :)
Those LOCs got to end up somewhere else though. They just don't
disappear.
> btw, who was it who recommended mod_cgid? [was it you,
> justin?]
I haven't been around *that* long. =)
> my preference is to do the unix-implementation of the
> apr_named_pipe API, if there's anyone else willing to
> help parallel-develop the unix implementation, using
> xvl as a test to prove the case.
I'd certainly look over the code, but I'm not terribly certain
if I'd use it. I'd just have to see and evaluate if named pipes
make sense in my cases. If it does in yours, great. -- justin
Re: More migration of code from httpd to apr-util
Posted by Justin Erenkrantz <je...@ebuilt.com>.
On Fri, Jun 08, 2001 at 02:45:09PM +0200, Luke Kenneth Casson Leighton wrote:
> what i perhaps should recommend is that a series of independent
> libraries be created.
>
> e.g. libaprhttputil (i know it's a bit long).
Aaron Bannert and I were discussing that as well yesterday. Yeah, we
kind of get that feeling too over here. But, again, I'm not sure what
the process would be for doing something like this.
> - string/path/cgi manipulation.
>
> this i need to be able to parse HTTP requests, to parse
> directory components. i know httpd does, too.
Depending upon how general it is (i.e. non-HTTP specific), this might
make sense to place in apr-util.
> - date/time manipulation.
>
> at present, all i need to be able to do is get a
> time_t and display it in a nice localtime format.
> ripping code from httpd to do this is... uh.. silly?
Submit a patch and I'll commit it into apr. I think date and time stuff
should be yanked out into apr-util. I've already yanked out date.
Submit something for time and I'll commit. We can probably find some
helpful dude with httpd commit access to remove those files from httpd
entirely.
> - an HTTP client.
>
> i need to be able to make http GETs / POSTs, and so
> originally i ripped entire bits of mod_proxy out into
> a separate file.
>
> i can believe that these two programs, xvl and mod_proxy
> are not the _only_ places where HTTP client GETs/POSTS
> are needed. for example, what about martin pool's
> work on mod_rproxy? [that's a bit like mod_proxy except
> it uses the rsync algorithm to sync up the file being
> requested with the last version obtained, etc., very cool
> work :)]
>
> EITHER :
>
> - an HTTP server-parser
Yeah, well, again, we're going to need it here at eBuilt as well (we're
*supposed* to be writing a load tester). So, I think it might make
sense to pool our resources and come up with an HTTP client API that
works on top of APR. I'm going to be writing this functionality
anyway. We could start with what is in mod_proxy (although I haven't
looked at this code yet). Again, this doesn't belong in APR or
APR-util, but I do believe it belongs somewhere. We keep writing this
over and over again.
And, I'll probably be writing a SAX API for apr-util as well. Hmm, that
may depend on how nice Xerces is.
> > I think the question is whether or not there is a predefined limit to
> > what goes in apr-util. We've hit this recently with the crypto/ stuff.
>
> so, create some new libraries. maybe apr-util isn't the
> best place for some of this stuff, but an apr-http-util is.
> or whatever. i don't mind, or really care [well, i do,
> but not enough to lose sleep over it if it doesn't happen :)]
>
> [and then think carefully about where and how to use them:
> you can't distribute anything that uses openssl / any crypto
> library, everywhere]
Yup. See above.
> i am trying to keep xvl's codebase down to an absolute, absolute
> minimum. the more LOCs i can take _out_, the happier i will be.
> duplicating or copying effort is a no-no in my book.
> :)
Those LOCs got to end up somewhere else though. They just don't
disappear.
> btw, who was it who recommended mod_cgid? [was it you,
> justin?]
I haven't been around *that* long. =)
> my preference is to do the unix-implementation of the
> apr_named_pipe API, if there's anyone else willing to
> help parallel-develop the unix implementation, using
> xvl as a test to prove the case.
I'd certainly look over the code, but I'm not terribly certain
if I'd use it. I'd just have to see and evaluate if named pipes
make sense in my cases. If it does in yours, great. -- justin
Re: More migration of code from httpd to apr-util
Posted by dean gaudet <dg...@arctic.org>.
On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> what i perhaps should recommend is that a series of independent
> libraries be created.
>
> e.g. libaprhttputil (i know it's a bit long).
why?
we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
...
the point of a library is that when you link against it, the linker will
extract only the portions which are used.
-dean
Re: More migration of code from httpd to apr-util
Posted by dean gaudet <dg...@arctic.org>.
On Fri, 8 Jun 2001, Luke Kenneth Casson Leighton wrote:
> what i perhaps should recommend is that a series of independent
> libraries be created.
>
> e.g. libaprhttputil (i know it's a bit long).
why?
we don't link against libcstdio libcstring libcsyscalls libcsortnsearch
...
the point of a library is that when you link against it, the linker will
extract only the portions which are used.
-dean
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Thu, Jun 07, 2001 at 12:10:15PM -0700, Justin Erenkrantz wrote:
> On Thu, Jun 07, 2001 at 01:33:15PM +0200, Luke Kenneth Casson Leighton wrote:
> > ack. if i make a [rather short] list of functions i am literally
> > duplicating line-for-line, would that suffice as motivation to,
> > say, add them to aprutil?
>
> Your list of potential functions looks mostly good to me (haven't sat
> down and thought too hard about it though).
:) btw, i deleted apr_to_strlower() because there is an apr_tolower()
so that's off the list.
what i perhaps should recommend is that a series of independent
libraries be created.
e.g. libaprhttputil (i know it's a bit long).
just for xvl, i need the following bits of code / functionality,
and i _know_ they're already partially in httpd because i ripped
entire sections out for use in xvl once before, already:
- string/path/cgi manipulation.
this i need to be able to parse HTTP requests, to parse
directory components. i know httpd does, too.
- date/time manipulation.
at present, all i need to be able to do is get a
time_t and display it in a nice localtime format.
ripping code from httpd to do this is... uh.. silly?
- an HTTP client.
i need to be able to make http GETs / POSTs, and so
originally i ripped entire bits of mod_proxy out into
a separate file.
i can believe that these two programs, xvl and mod_proxy
are not the _only_ places where HTTP client GETs/POSTS
are needed. for example, what about martin pool's
work on mod_rproxy? [that's a bit like mod_proxy except
it uses the rsync algorithm to sync up the file being
requested with the last version obtained, etc., very cool
work :)]
EITHER :
- an HTTP server-parser
yes, that's right: i need to be able to parse HTML requests
and the prospect of ripping and having to maintain entire
sections of HTTP parsing code is not one i will take on
lightly.
OR:
- a programmatic means to plug-in PROGRAMS - not cgi-bin
executables - DIRECTLY into httpd.
now, on unix, the obvious way to do this is unix domain
sockets, ala mysql, nssswitch, winbindd, tng domain architecture
X-windows and probably a whole boat-load of others.
on NT, it spells 'Named Pipes'.
dunno about Beos and OS/2, OS/2 probly supports both,
i suspect (?)
basically, the mod_xxx is all very well and good, until
you start wanting to plug in thing like... oh, i dunno,
8 MILLION lines of existing code [yes, there is such
a project].
... and you want to turn all these services, some of
which are #ifdef KERNEL based, into mod_xxxes? ha!
don't make me laugh :)
so, having an inter-process communication mechanism,
there, becomes essential, allowing for separate
programs to plug their data directly in as HTTP
requests.
i.e. httpd can become, effectively, a 'transport' layer
component.
[for those of you familiar with dce/rpc, look up
ncacn_http :) :) ]
> I think the question is whether or not there is a predefined limit to
> what goes in apr-util. We've hit this recently with the crypto/ stuff.
so, create some new libraries. maybe apr-util isn't the
best place for some of this stuff, but an apr-http-util is.
or whatever. i don't mind, or really care [well, i do,
but not enough to lose sleep over it if it doesn't happen :)]
[and then think carefully about where and how to use them:
you can't distribute anything that uses openssl / any crypto
library, everywhere]
> I think it probably means that some of the committers need to work on
> things outside of httpd itself. When I wrote the mod_mbox stuff and
> had to write a standalone indexer, I needed the date and uri functions
> from httpd. I initially copied them over, but then, since I now have
> commit access and no one screamed too loudly, I moved them into
> apr-util (with some help on the httpd side) and deleted my private
> copy. But, I think this will have to be done on a case-by-case basis
> with lots of thought for each one.
ack.
> There also needs to be a guiding vision for apr-util as well. I'm not
> sure what that is exactly. =) I think we might be able to define that
> though...
>
> I'd be curious to look at subversion to see if it is duplicating any
> code, but the domains are so different, I doubt there is a lot of
> overlap.
well, surely they have string / file / directory manipulation.
surely they do timestamps!
> The more users of apr-util, the more we can identify things
> that might be worth migrating from httpd into apr-util.
ack.
> But, you
> are always free to copy the code if we don't move it into apr-util.
i am trying to keep xvl's codebase down to an absolute, absolute
minimum. the more LOCs i can take _out_, the happier i will be.
duplicating or copying effort is a no-no in my book.
:)
btw, who was it who recommended mod_cgid? [was it you,
justin?]
i took a look at it, and... well, okay, what i _meant_ was,
as described above, i need a platform-independent inter-process-
communication mechanism, where process includes separate
programs
and i think that doing Named Pipes, ala NT, is the best way
to go about this, emulating the same functionality under
Unix, not the other way round. and i'm saying that from
the viewpoint of having developed Named Pipes in Samba TNG to
emulate the NP functionality, for unix, already.
sander and i discussed this another time [3 months ago]
related to transport APIs, on APR. the topic was inter-related
with buckets and filters etc, and was... well, a bit too
abstract, ambitious and not well understood or explained.
[best, therefore, i think, to go in small stages, using
xvl to work to that end.]
iow, i need to make a decision to remain with the hook into
httpd via a ux-dom-sock-XML-wrapped implementation of mod_proxy,
help you guys write an apr_named_pipe API, or to cut out
the middle man and shove in entire sections of httpd so
xvl can be a self-serving http server.
my preference is to do the unix-implementation of the
apr_named_pipe API, if there's anyone else willing to
help parallel-develop the unix implementation, using
xvl as a test to prove the case.
takers, anyone?
luke
Re: More migration of code from httpd to apr-util
Posted by Luke Kenneth Casson Leighton <lk...@samba-tng.org>.
On Thu, Jun 07, 2001 at 12:10:15PM -0700, Justin Erenkrantz wrote:
> On Thu, Jun 07, 2001 at 01:33:15PM +0200, Luke Kenneth Casson Leighton wrote:
> > ack. if i make a [rather short] list of functions i am literally
> > duplicating line-for-line, would that suffice as motivation to,
> > say, add them to aprutil?
>
> Your list of potential functions looks mostly good to me (haven't sat
> down and thought too hard about it though).
:) btw, i deleted apr_to_strlower() because there is an apr_tolower()
so that's off the list.
what i perhaps should recommend is that a series of independent
libraries be created.
e.g. libaprhttputil (i know it's a bit long).
just for xvl, i need the following bits of code / functionality,
and i _know_ they're already partially in httpd because i ripped
entire sections out for use in xvl once before, already:
- string/path/cgi manipulation.
this i need to be able to parse HTTP requests, to parse
directory components. i know httpd does, too.
- date/time manipulation.
at present, all i need to be able to do is get a
time_t and display it in a nice localtime format.
ripping code from httpd to do this is... uh.. silly?
- an HTTP client.
i need to be able to make http GETs / POSTs, and so
originally i ripped entire bits of mod_proxy out into
a separate file.
i can believe that these two programs, xvl and mod_proxy
are not the _only_ places where HTTP client GETs/POSTS
are needed. for example, what about martin pool's
work on mod_rproxy? [that's a bit like mod_proxy except
it uses the rsync algorithm to sync up the file being
requested with the last version obtained, etc., very cool
work :)]
EITHER :
- an HTTP server-parser
yes, that's right: i need to be able to parse HTML requests
and the prospect of ripping and having to maintain entire
sections of HTTP parsing code is not one i will take on
lightly.
OR:
- a programmatic means to plug-in PROGRAMS - not cgi-bin
executables - DIRECTLY into httpd.
now, on unix, the obvious way to do this is unix domain
sockets, ala mysql, nssswitch, winbindd, tng domain architecture
X-windows and probably a whole boat-load of others.
on NT, it spells 'Named Pipes'.
dunno about Beos and OS/2, OS/2 probly supports both,
i suspect (?)
basically, the mod_xxx is all very well and good, until
you start wanting to plug in thing like... oh, i dunno,
8 MILLION lines of existing code [yes, there is such
a project].
... and you want to turn all these services, some of
which are #ifdef KERNEL based, into mod_xxxes? ha!
don't make me laugh :)
so, having an inter-process communication mechanism,
there, becomes essential, allowing for separate
programs to plug their data directly in as HTTP
requests.
i.e. httpd can become, effectively, a 'transport' layer
component.
[for those of you familiar with dce/rpc, look up
ncacn_http :) :) ]
> I think the question is whether or not there is a predefined limit to
> what goes in apr-util. We've hit this recently with the crypto/ stuff.
so, create some new libraries. maybe apr-util isn't the
best place for some of this stuff, but an apr-http-util is.
or whatever. i don't mind, or really care [well, i do,
but not enough to lose sleep over it if it doesn't happen :)]
[and then think carefully about where and how to use them:
you can't distribute anything that uses openssl / any crypto
library, everywhere]
> I think it probably means that some of the committers need to work on
> things outside of httpd itself. When I wrote the mod_mbox stuff and
> had to write a standalone indexer, I needed the date and uri functions
> from httpd. I initially copied them over, but then, since I now have
> commit access and no one screamed too loudly, I moved them into
> apr-util (with some help on the httpd side) and deleted my private
> copy. But, I think this will have to be done on a case-by-case basis
> with lots of thought for each one.
ack.
> There also needs to be a guiding vision for apr-util as well. I'm not
> sure what that is exactly. =) I think we might be able to define that
> though...
>
> I'd be curious to look at subversion to see if it is duplicating any
> code, but the domains are so different, I doubt there is a lot of
> overlap.
well, surely they have string / file / directory manipulation.
surely they do timestamps!
> The more users of apr-util, the more we can identify things
> that might be worth migrating from httpd into apr-util.
ack.
> But, you
> are always free to copy the code if we don't move it into apr-util.
i am trying to keep xvl's codebase down to an absolute, absolute
minimum. the more LOCs i can take _out_, the happier i will be.
duplicating or copying effort is a no-no in my book.
:)
btw, who was it who recommended mod_cgid? [was it you,
justin?]
i took a look at it, and... well, okay, what i _meant_ was,
as described above, i need a platform-independent inter-process-
communication mechanism, where process includes separate
programs
and i think that doing Named Pipes, ala NT, is the best way
to go about this, emulating the same functionality under
Unix, not the other way round. and i'm saying that from
the viewpoint of having developed Named Pipes in Samba TNG to
emulate the NP functionality, for unix, already.
sander and i discussed this another time [3 months ago]
related to transport APIs, on APR. the topic was inter-related
with buckets and filters etc, and was... well, a bit too
abstract, ambitious and not well understood or explained.
[best, therefore, i think, to go in small stages, using
xvl to work to that end.]
iow, i need to make a decision to remain with the hook into
httpd via a ux-dom-sock-XML-wrapped implementation of mod_proxy,
help you guys write an apr_named_pipe API, or to cut out
the middle man and shove in entire sections of httpd so
xvl can be a self-serving http server.
my preference is to do the unix-implementation of the
apr_named_pipe API, if there's anyone else willing to
help parallel-develop the unix implementation, using
xvl as a test to prove the case.
takers, anyone?
luke