Usability testing for httpd

[Jacob Champion <ch...@gmail.com>]

Hi all,

I attended the GNOME project's LAS [1] in Portland this past week, and 
one of things that stuck with me was a presentation on usability testing 
by Jim Hall. The usability of our configuration language (and the web 
server in general) has been something I've been mulling over for a while 
now, especially after having spent time on our #httpd channel.

I realize the notion of "usability testing" for an expert-level, 
text-based language might be a little odd/controversial. Rather than tie 
the discussion to some vague definition of "usability", it's probably 
easier to describe it the way Jim did:

Given a group of volunteer test subjects with varying degrees of httpd 
expertise, and a set of tasks that (we hope) all httpd users should be 
able to accomplish -- can the volunteers actually do the tasks? If so, 
how well and how quickly did they do it? If not, what areas tripped them up?

Since using httpd involves manipulation of a text file and not a GUI, 
some of the concepts in his presentation don't fully translate, but I 
still think knowing the answers to these questions would be incredibly 
useful for us. Are there others who would be interested in this sort of 
thing? Have we done something like it before?

--Jacob

[1] http://las.gnome.org/
[2] http://www.freedos.org/jhall/uploads/las-gnome-jhall-usability-16x9.pdf

Re: Usability testing for httpd

[Jacob Champion <ch...@gmail.com>]

On 09/26/2016 04:07 PM, William A Rowe Jr wrote:
> What has happened is that between 2.0 -> 2.2 -> 2.4 has been an ongoing
> process of reducing 'must configure' choices in the server.  The stated goal
> some 10-12 years ago was that every module/configuration option has some
> sensible default, to an end goal of being able to start httpd with an empty
> config file. That might be overly optimistic, but if you look at the modern
> httpd.conf file, it is significantly shorter than what used to ship with
> 1.3,
> and the other oddball and illustrative items all moved to conf/extra/.
>
> It would be nice as we look beyond 2.4 to not only simplify this further as
> we see those opportunities, but look at chances to avoid the 'multiple
> slider' scenario where two or three directives are required to achieve just
> a single benefit. Good example of this are the interrelated mpm tuning
> parameters; edit one and you've likely committed yourself to editing three
> or four values for a desired end result. Timeouts spread between core and
> proxy pose a similar challenge.
>
> There are limits to our solving this puzzle; we have an infinitely
> extensible
> modular architecture and with third party modules and in-process extension
> by lua or perl, configuration will always pose some complexity IMO. But
> patches to simplify things in trunk/ are always welcome.

Sure. I think I agree with all of those points:

- minimize the boilerplate needed in a minimal configuration
- reduce interdependencies between directives
- recognize that raw power will require some minimum level of complexity

What I'm asking about here, though, is something more basic: given the 
set of things that most entry-level users want to do (run a PHP 
application, or a static file server, or whatever), how easy is it for a 
typical user to *actually* do those things, starting from a default 
configuration? That's a much more pressing issue for me than manual 
performance tuning or the use of programmatic modules, which generally 
involve a more advanced user.

(To be clear: I think making it easier for advanced users to do advanced 
things has significant value. It's just not my focus here.)

--Jacob

Re: Usability testing for httpd

[William A Rowe Jr <wr...@rowe-clan.net>]

On Mon, Sep 26, 2016 at 2:57 PM, Jacob Champion <ch...@gmail.com>
wrote:

> Hi all,
>
> I attended the GNOME project's LAS [1] in Portland this past week, and one
> of things that stuck with me was a presentation on usability testing by Jim
> Hall. The usability of our configuration language (and the web server in
> general) has been something I've been mulling over for a while now,
> especially after having spent time on our #httpd channel.
>
> I realize the notion of "usability testing" for an expert-level,
> text-based language might be a little odd/controversial. Rather than tie
> the discussion to some vague definition of "usability", it's probably
> easier to describe it the way Jim did:
>
> Given a group of volunteer test subjects with varying degrees of httpd
> expertise, and a set of tasks that (we hope) all httpd users should be able
> to accomplish -- can the volunteers actually do the tasks? If so, how well
> and how quickly did they do it? If not, what areas tripped them up?
>
> Since using httpd involves manipulation of a text file and not a GUI, some
> of the concepts in his presentation don't fully translate, but I still
> think knowing the answers to these questions would be incredibly useful for
> us. Are there others who would be interested in this sort of thing? Have we
> done something like it before?


Not that I'm aware of.

What has happened is that between 2.0 -> 2.2 -> 2.4 has been an ongoing
process of reducing 'must configure' choices in the server.  The stated goal
some 10-12 years ago was that every module/configuration option has some
sensible default, to an end goal of being able to start httpd with an empty
config file. That might be overly optimistic, but if you look at the modern
httpd.conf file, it is significantly shorter than what used to ship with
1.3,
and the other oddball and illustrative items all moved to conf/extra/.

It would be nice as we look beyond 2.4 to not only simplify this further as
we see those opportunities, but look at chances to avoid the 'multiple
slider' scenario where two or three directives are required to achieve just
a single benefit. Good example of this are the interrelated mpm tuning
parameters; edit one and you've likely committed yourself to editing three
or four values for a desired end result. Timeouts spread between core and
proxy pose a similar challenge.

There are limits to our solving this puzzle; we have an infinitely
extensible
modular architecture and with third party modules and in-process extension
by lua or perl, configuration will always pose some complexity IMO. But
patches to simplify things in trunk/ are always welcome.