Whither Windows (Was: Re: Intent to revert commit r1332643)

[Jim Jagielski <ji...@jaguNET.com>]

Maybe the real question is where exactly do we stand with
Windows right now...

We haven't had (complimentary) binary builds for Windows in
quite awhile and, afaict, there are really no people focusing
on Windows compatibility anymore.

For me, I wouldn't want to stunt httpd development for "every
other platform we care about" simply because it breaks
Windows. But it's not just my decision, 'natch.

On May 17, 2013, at 10:36 AM, Guenter Knauf <fu...@apache.org> wrote:

> Hi all,
> I will revert the changes done with:
> http://svn.apache.org/viewvc?view=revision&revision=1332643
> after 72 hours if nobody is going to fix the stuff properly for Windows since I'm tired of always copying mod_ssl over from 2.4.x branch in order to get a working mod_ssl with trunk.
> 
> Reasons:
> 1) within last 12 months there was no attempt made to fix the issues which wrowe mentioned in this thread [1] - instead discussion died
> 2) a suggestion to fix the issue [2] was not applied due to the concerns wrowe brought up, and to which I agree.
> 3) the same issue also causes a stalled backport in 2.2.x STATUS [3] for the last 12 months
> 
> [1] http://mail-archives.apache.org/mod_mbox/httpd-dev/201302.mbox/%3C20130205115224.33547872@hub%3E
> [2] http://mail-archives.apache.org/mod_mbox/httpd-dev/201302.mbox/%3C510D8293.8010103@gknw.net%3E
> [3] http://svn.apache.org/viewvc?view=revision&revision=1333501
> 
> I believe that one year in trunk without further review is long enough, and if someone wants to continue working on it its easy enough to checkout the last revision before removal.
> 
> Gün.
> 
> 

Re: Symbol Resolution (Was: Whither Windows (Was: Re: Intent to revert commit r1332643))

["William A. Rowe Jr." <wr...@rowe-clan.net>]

On Fri, 24 May 2013 21:53:50 +0200
Guenter Knauf <fu...@apache.org> wrote:

> On 24.05.2013 21:37, Ben Reser wrote:
> > The build system should be able to compile with the major tool
> > chains, nobody expects to know how to work around weird autoconf,
> > make, gcc, etc quirks on Linux.  I don't say this to be dismissive
> > of anyones contributions but just to point out that producing
> > Windows builds with a modern toolchain is not simple.

> yeah ...; and from what I see our project files are already broken
> even when not converted and used directly with MSVC6, f.e. when doing
> a release build a bunch of files land in the debug folder, and
> finally at linking stage it breaks ...
> probably we should think about moving either to plain Nmakefiles (as 
> Pierre Joy also suggested), or add a Cmake build system; for me SCon
> is no alternative since I saw it too often already fail on modern
> Linux boxes (with other projects), so I have no hope that it works
> any better on Windows ...

The concensus seems to be forming around cmake, and it is certainly
my preference from working with pcre and other libraries and projects.

But in the walk-before-you-run department, it's probably best to swap
the scons for cmake over at the apr project, and that will take our
most significant obstacle out of the way.  The httpd cmake will still
be quite complex, but not obnoxiously so, and our apr experience would
be beneficial to setting out a well thought out plan.

Re: Symbol Resolution (Was: Whither Windows (Was: Re: Intent to revert commit r1332643))

[Gregg Smith <gl...@gknw.net>]

On 5/24/2013 12:53 PM, Guenter Knauf wrote:
> yeah ...; and from what I see our project files are already broken 
> even when not converted and used directly with MSVC6, f.e. when doing 
> a release build a bunch of files land in the debug folder, and finally 
> at linking stage it breaks ... 
Vice-versa, I never have a problem with release builds which probably 
explains why I do so few debug builds.

It ends up being all objects land in /Release but then the linker looks 
in /Debug for them.

libhttpd is the culpret here for me, I do not think I have had the 
problem with anything else. It's now been two or three weeks since I 
fought the dreaded debug build so who knows as I have done too many 
release builds since :)  But libhttpd stands out above all and I have no 
idea why it's happening on this project, I'm just not seeing it.

Gregg

Re: Symbol Resolution (Was: Whither Windows (Was: Re: Intent to revert commit r1332643))

[Guenter Knauf <fu...@apache.org>]

On 24.05.2013 21:37, Ben Reser wrote:
> The build system should be able to compile with the major tool chains,
> nobody expects to know how to work around weird autoconf, make, gcc,
> etc quirks on Linux.  I don't say this to be dismissive of anyones
> contributions but just to point out that producing Windows builds with
> a modern toolchain is not simple.
yeah ...; and from what I see our project files are already broken even 
when not converted and used directly with MSVC6, f.e. when doing a 
release build a bunch of files land in the debug folder, and finally at 
linking stage it breaks ...
probably we should think about moving either to plain Nmakefiles (as 
Pierre Joy also suggested), or add a Cmake build system; for me SCon is 
no alternative since I saw it too often already fail on modern Linux 
boxes (with other projects), so I have no hope that it works any better 
on Windows ...

and regarding trunk: AFAICT there are no improvements (what I mentioned 
above was with trunk) ...

Gün.

Re: Symbol Resolution (Was: Whither Windows (Was: Re: Intent to revert commit r1332643))

[Ben Reser <be...@reser.org>]

On Fri, May 24, 2013 at 8:13 AM, William A. Rowe Jr.
<wr...@rowe-clan.net> wrote:
> That fortunately is documented, with some pretty good notes in
> the wiki as well that aught to percolate into the docs.  That
> said, documenting every Microsoft-version-quirk seems out of
> scope for a general purpose 'compiling' doc.

The build system should be able to compile with the major tool chains,
nobody expects to know how to work around weird autoconf, make, gcc,
etc quirks on Linux.  I don't say this to be dismissive of anyones
contributions but just to point out that producing Windows builds with
a modern toolchain is not simple.

I did a bunch of work on scripting building the dependencies for
Subversion on Windows that's located here:
http://svn.apache.org/repos/asf/subversion/trunk/tools/dev/build-svn-deps-win.pl

By far httpd was the biggest pain of any of the dependencies to get to work.

If your only interest is building httpd on Windows with Visual Studio
2012, taking a look at build_httpd() in that file should give a good
starting point.

Sometime when I find the time I want to fix the problems that I had to
work around the right way and not the hackish way I did in that script
and submit them back.  But I haven't gotten to it.

I'll admit that I haven't tried to build httpd-trunk on Windows so
maybe improvements have been made that haven't made their way over to
the 2.4 releases, Subversion certainly has similar issues with our 1.7
release branch.

Re: Symbol Resolution (Was: Whither Windows (Was: Re: Intent to revert commit r1332643))

["William A. Rowe Jr." <wr...@rowe-clan.net>]

On Fri, 24 May 2013 09:26:34 -0400
Jim Jagielski <ji...@jaguNET.com> wrote:

> 
> On May 24, 2013, at 9:08 AM, Jeff Trawick <tr...@gmail.com> wrote:
> > 
> > Lots of us are employees of or otherwise manage to siphon money
> > from these companies.  Make a pitch...  (And some of us are happy
> > to freelance ;) )
> 
> I'll be honest: I don't even know to to *build* for Windows,
> at least with any Visual Studio release from this century.

That fortunately is documented, with some pretty good notes in
the wiki as well that aught to percolate into the docs.  That
said, documenting every Microsoft-version-quirk seems out of
scope for a general purpose 'compiling' doc.

Symbol resolution on Linux is rather loosey-goosey, and very
relaxed.  Resolution in OS/X 2-level namespace mode is quite
strict and a very good mirror of the world of Windows, z/OS and
many others[1].  It should be relatively easy to enforce explicit
exports on some other platforms which more of the developers
use in day-to-day practice.

The patch to 'fix' this is trivial.  The well-documented issues
with that 'fix' is the reason this patch should be reverted.

httpd has a reasonably straightforward pattern to register
and consume optional functions, hook functions etc.  This has
existed since close to the dawn of 2.0 and /had/ resolved almost
all module load-ordering issues in the server software.  It was
close to world-class in behavior and quite foolproof when the
return codes were observed.  To see how simple these are in
practice, see my backport of the Mutex behavior for 2.4-based 
modules to borrow when compiled under 2.2;
  http://people.apache.org/~wrowe/httpd-2.2-ports/util_mutex.h
  http://people.apache.org/~wrowe/httpd-2.2-ports/mod_mutex.c

Optional hooks were introduced to allow for the registration of 
a sometimes-present, sometimes absent hook provider because not 
every hook was relevant to the core httpd server and not every
hook should always exist.  Both proxy and dav are good examples
of modules which must introduce new hooks themselves.

This patch ignored that, and introduced a hook provider that now
exists in a sometimes-loaded module which the core is looking
to register in.  It breaks the pattern because that hook is not
an optional hook.  The implementation is a mess and it has been
documented for a year what would fix it to follow httpd convention.

The much greater problem is that the current proxy provider stack
is riddled with load-time linkage between proxy modules, rather
than run-time optional functions and hooks.  This means that the
entire LoadModule mod_proxy schema is broken and fails to follow
the design principals of httpd 2.0.

How can developers without Windows possibly cope with identifying
these problems?  Magically, all it takes is to re-sort your
LoadModule list in inverse order and see what breaks.  There are
very few "Windows-only" behaviors in the server that can't be
documented with some trivial tests even on Linux.

So unless we really no longer care about this 'enhancement' to
the Apache 2.0 family, it seems past time to start re-factoring
such newly-reintroduced fragile behaviors and come back to that
design principal.  Or perhaps chuck it and go back to hard module
linkages and document the required load orders?  That would save
a few ms at startup.

It seems that we would all benefit from working out the libtool
and exports.c logic so that we can have some explicit-export model 
on Linux that will approximate windows and demonstrate this logic 
to those who don't (and shouldn't have to) build on Windows, WDYT?

p.s. I think your prior thread is rather dismissive of Gregg,
and folks like Steffan who go out of their way to follow these
docs and regularly report problems on Windows.  It's dismissive
of folks like Jeff and the rest who untangled the thorny change
in mod_ssl that broke the windows MPM preloaded socket data
approach so badly.  The fact that reports and objections are 
ignored does not mean the platform is ignored.  This is the only 
major issue I am aware of, nobody seems to have problems generating 
builds outside of the ASF, and we haven't seemed to have a large 
issue of following the Subversion model of binaries.

p.p.s. We have delivered 2.0 and 2.2 binaries and aught to update
those, now that my VM's are healed with the necessary crazy MSVCRT
based compilation environment the 'final' 2.0 binary can be built
and a refresh of 2.2 can be provided.  (Mladen's hybrid solution 
rocks, but there are some insane to workaround quirks for httpd 
with his solution that weren't present in building apr, -util etc).

p.p.p.s. I'll build a demonstration 2.4 package over the holiday
weekend, but don't consider it 'prime time' just yet.  We have a 
ways to go before all the Windows' players concerns can be resolved,
I'll put up a scratchpad in the wiki for us to discuss.  Lua et al
introduce more 3rd-party build questions than they answer.

p.p.p.p.s. Note that the Lua project makes life difficult for ALL
packagers, leaving shared lua libs entirely up to the packager and
offering no build support at all for that model.  Compare this to
httpd today which simply builds, with some initial file conversion
or ./configure'ation right out of the tarball with shared module
support.

[1] Note OS/X 2-level namespace allows 2 different versions of a
    similar library, e.g. openldap and netscape ldap to coexist in
    memory at the same time, much as the MS Windows DLL behavior
    demonstrates. I don't advocate that we need to enforce that
    level of complexity.  Exporting only explicit symbols in a
    single-level namespace based on _DECLARE macros would be
    perfectly sufficient to catch 99% of issues.

Re: Whither Windows (Was: Re: Intent to revert commit r1332643)

[Jim Jagielski <ji...@jaguNET.com>]

On May 24, 2013, at 9:08 AM, Jeff Trawick <tr...@gmail.com> wrote:
> 
> Lots of us are employees of or otherwise manage to siphon money from these companies.  Make a pitch...  (And some of us are happy to freelance ;) )
> 

I'll be honest: I don't even know to to *build* for Windows,
at least with any Visual Studio release from this century.

Re: Whither Windows (Was: Re: Intent to revert commit r1332643)

[Jeff Trawick <tr...@gmail.com>]

On Fri, May 24, 2013 at 8:52 AM, Jim Jagielski <ji...@jagunet.com> wrote:

> Maybe the real question is where exactly do we stand with
> Windows right now...
>
> We haven't had (complimentary) binary builds for Windows in
> quite awhile


IMO this is irrelevant now...  (tm)-lounge binaries are fine.


> and, afaict, there are really no people focusing
> on Windows compatibility anymore.
>

Alternatively, there are no people spending "enough time" on Windows
compatibility...


>
> For me, I wouldn't want to stunt httpd development for "every
> other platform we care about" simply because it breaks
> Windows. But it's not just my decision, 'natch.
>

FWLIW, Windows stands in for z/OS to some extent w.r.t. symbol resolution...

The unfortunate fact is that while developers *generally* aren't very
interested in Windows (and much less so z/OS or other niche platforms),
product managers at a slew of multinationals and smaller companies have
incredible lust for having that checkmark in the big table yet aren't
willing to spend a dime to make it happen.

Lots of us are employees of or otherwise manage to siphon money from these
companies.  Make a pitch...  (And some of us are happy to freelance ;) )




>
> On May 17, 2013, at 10:36 AM, Guenter Knauf <fu...@apache.org> wrote:
>
> > Hi all,
> > I will revert the changes done with:
> > http://svn.apache.org/viewvc?view=revision&revision=1332643
> > after 72 hours if nobody is going to fix the stuff properly for Windows
> since I'm tired of always copying mod_ssl over from 2.4.x branch in order
> to get a working mod_ssl with trunk.
> >
> > Reasons:
> > 1) within last 12 months there was no attempt made to fix the issues
> which wrowe mentioned in this thread [1] - instead discussion died
> > 2) a suggestion to fix the issue [2] was not applied due to the concerns
> wrowe brought up, and to which I agree.
> > 3) the same issue also causes a stalled backport in 2.2.x STATUS [3] for
> the last 12 months
> >
> > [1]
> http://mail-archives.apache.org/mod_mbox/httpd-dev/201302.mbox/%3C20130205115224.33547872@hub%3E
> > [2]
> http://mail-archives.apache.org/mod_mbox/httpd-dev/201302.mbox/%3C510D8293.8010103@gknw.net%3E
> > [3] http://svn.apache.org/viewvc?view=revision&revision=1333501
> >
> > I believe that one year in trunk without further review is long enough,
> and if someone wants to continue working on it its easy enough to checkout
> the last revision before removal.
> >
> > Gün.
> >
> >
>
>


-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: Whither Windows (Was: Re: Intent to revert commit r1332643)

["William A. Rowe Jr." <wr...@rowe-clan.net>]

On Fri, 24 May 2013 12:43:23 -0700
Ben Reser <be...@reser.org> wrote:

> On Fri, May 24, 2013 at 8:23 AM, William A. Rowe Jr.
> <wr...@rowe-clan.net> wrote:
> 
> > Also wondering where the OS/X download lives?  It will build on any
> > OS/X box with a deployed toolchain, but I imagine many OS/X users
> > don't install that toolchain and live with the Apple provided
> > flavors, and would guess that 2.4.x is not part of that Apple OS
> > distributions so far.
> 
> I've never bothered to try to download a httpd binary build, it's easy
> enough to build that I don't feel the need.
> 
> 10.7 still had httpd 2.2, not sure what 2.4 has.

As far as we are aware, no commercial distributions and so far,
no released free stable distributions incorporate 2.4.  A couple
have started integrating it, but the single biggest obstacle that
is reported by the packagers community is that mod_perl had not yet
supported 2.4 and that is an important package dependency to them.

Re: Whither Windows (Was: Re: Intent to revert commit r1332643)

[Ben Reser <be...@reser.org>]

On Fri, May 24, 2013 at 8:23 AM, William A. Rowe Jr.
<wr...@rowe-clan.net> wrote:
> Another question is where exactly do we stand with OS/X right now?
>
> Apple HFS+ is still not supported, there exists a forced lower-case
> canonicalization hack authored by Apple, but AFAICT still no progress
> on retrieving the true name of a file on a case-insensitive OS/X
> volume which I suspect are still in common use on most OS/X boxen.
> Of course, all the BSD-based file systems are strict case sensitive
> and don't have security bypass issues when running 'vanilla' httpd.

Can't speak to this, it's not important to my use since I avoid the
case sensitivity issues on OS X.

I've had no problems building httpd on OS/X on 10.7 (I haven't
bothered to upgrade to 10.8).  I do a fair amount of work on OS X
directly and often build httpd-trunk and releases with debugging.
There may be computability issues like you pointed out above, but I
quite frankly have never noticed them.  Granted my use of httpd on OS
X is purely developmental and I likely wouldn't run into the types of
issues that someone using httpd for production servers on OS X would.

> Also wondering where the OS/X download lives?  It will build on any
> OS/X box with a deployed toolchain, but I imagine many OS/X users
> don't install that toolchain and live with the Apple provided
> flavors, and would guess that 2.4.x is not part of that Apple OS
> distributions so far.

I've never bothered to try to download a httpd binary build, it's easy
enough to build that I don't feel the need.

10.7 still had httpd 2.2, not sure what 2.4 has.

Re: Whither Windows (Was: Re: Intent to revert commit r1332643)

["William A. Rowe Jr." <wr...@rowe-clan.net>]

On Fri, 24 May 2013 08:52:05 -0400
Jim Jagielski <ji...@jaguNET.com> wrote:

> Maybe the real question is where exactly do we stand with
> Windows right now...
> 
> We haven't had (complimentary) binary builds for Windows in
> quite awhile and, afaict, there are really no people focusing
> on Windows compatibility anymore.

Thanks you just reminded me...

Another question is where exactly do we stand with OS/X right now?

Apple HFS+ is still not supported, there exists a forced lower-case
canonicalization hack authored by Apple, but AFAICT still no progress
on retrieving the true name of a file on a case-insensitive OS/X
volume which I suspect are still in common use on most OS/X boxen.
Of course, all the BSD-based file systems are strict case sensitive
and don't have security bypass issues when running 'vanilla' httpd.

Unfortunately I don't run an OS/X box anymore so it's awfully hard
for me to complete research on such a fix (and isn't so personally
interesting since my ppc laptop was retired for lack of OS updates).

Also wondering where the OS/X download lives?  It will build on any
OS/X box with a deployed toolchain, but I imagine many OS/X users
don't install that toolchain and live with the Apple provided
flavors, and would guess that 2.4.x is not part of that Apple OS
distributions so far.

Re: Whither Windows (Was: Re: Intent to revert commit r1332643)

[Guenter Knauf <fu...@apache.org>]

Hi Jim,
On 24.05.2013 14:52, Jim Jagielski wrote:
> For me, I wouldn't want to stunt httpd development for "every
> other platform we care about" simply because it breaks
> Windows. But it's not just my decision, 'natch.
well, for me its no reason to just accept every code as long as it 
compiles on *nix platforms regardless of design flaws; also we could 
easily fix the stuff so that Windows would compile again even with the 
design flaw, but that's not what Bill would accept IIUC, and not what I 
would like to do ...

finally what I was mainly after was kicking on discussion again about 
how to fix it correctly, and thats now in progress ... ;-)

Gün.