util.c hole and speed of security patch release

[Jennifer Myers <jm...@marigold.eecs.nwu.edu>]

Hi,

I hope you don't mind that I've subscribed to this mailing list.  It
takes *forever* to download from ftp.hyperreal.com, and given that it
is publically available there, I figured that the list membership
would be open as well.  I'll just lurk for the most part, but I wanted
to add my two cents on the newline/util.c issue.

I hope you guys remember that I told you about the phf/util.c hole
back in February:

  Subject: Security problem in phf
  To: jbrowne@uiuc.edu, apache-bugs@apache.org, httpd@hoohoo.ncsa.uiuc.edu
  Date: Mon, 5 Feb 1996 16:51:05 -0600 (CST)
  From: Jennifer Myers <jm...@marigold.eecs.nwu.edu>

    Hello,

  Both the NCSA httpd and Apache httpd distributions include a CGI
  program "phf" which is a forms-based interface to Ph servers written
  by Jim Browne <jb...@uiuc.edu>.

  "phf" can be made to execute commands by inserting "%0A" (newline) into the
  URL of a phf query, followed by whatever command you wish to execute.

  [..removed for brevity..]

    The fix is to add \x0A to the characters which escape_shell_cmd() in
  util.c will escape:

  -        if(ind("&;`'\"|*?~<>^()[]{}$\\",cmd[x]) != -1){
  +        if(ind("&;`'\"|*?~<>^()[]{}$\\\x0A",cmd[x]) != -1){


I went to investigate some other widely-used CGI programs, and found
the same type of error being repeated: a shell was being invoked on
user-supplied input without the newline character being stripped or
escaped.  Most of these programs tried to sanitize their input by
escaping a list of shell meta-characters, but those lists of nasty
characters, as provided by several CGI security FAQs did not include
the newline character.

When I saw that this error was widespread in public-domain CGI
scripts, despite the author's attempts at being secure, I posted an
advisory titled "CGI security vulnerability: %0A (newlines) in
user-supplied data" later that day (Feb 5th) to
comp.infosystems.www.authoring.cgi, comp.infosystems.www.servers.unix,
comp.security.unix and www-security and bugtraq with an example
exploit, but without information on which CGI programs specifically
are vulnerable.

That posting is available at
<http://www.eecs.nwu.edu/~jmyers/bugtraq/0515.html>.

I received an acknowledgment from Rob Hartill on Feb 8:

  From hartill@hyperreal.com  Thu Feb  8 22:46:13 1996
  From: Rob Hartill <ha...@hyperreal.com>
  Message-Id: <19...@taz.hyperreal.com>
  Subject: Re: Security problem in phf
  To: jmyers@marigold.eecs.nwu.edu (Jennifer Myers)
  Date: Thu, 8 Feb 1996 20:46:05 -0800 (PST)
  Cc: jbrowne@uiuc.edu, apache-bugs@apache.org, httpd@hoohoo.ncsa.uiuc.edu
  In-Reply-To: <19...@marigold.eecs.nwu.edu> from "Jennifer Myers" at Feb 5, 96 04:51:05 pm
  Reply-To: apache-bugs@hyperreal.com
  
  thanks for the warning.

  cheers,
  rob
  [...]

(And no reply from NCSA).

Curiously, a version of Apache was released after Feb 8th without the
problem fixed.  Not until IBM-ERS, CERT, CIAC, etc. picked up the
advisory in March did it seem that there was any interest in fixing
the bug!  Meanwhile, I have received reports that the bug is being
actively exploited, because a good majority of sites have installed
phf by way of installing all of the sample CGI code which comes with
Apache and NCSA httpd.  CERT has also indicated that the bug is being
actively exploited.

I understand that the bug only results from insecure programming
practice (interestingly, one of the first versions of phf in early
1994 did not escape *any* characters before passing them to popen()),
but I'm curious why action was not taken.

You (members of the list) complain that IBM-ERS or CIAC did not 
inform you of the bug in src/util.c prior to releasing their advisories.

Well, I quietly informed you of the bug in cgi-src/util.c on February
5th, right when I discovered it.  Better yet, I have never advertised
publically of the bug in phf (though lots of people mailed me to say
that they had found the problem in phf from my description) -- I
figured I ought to give you a chance to deal with it first -- and
instead, I released an "advisory" on the nature of the bug, which is
present in many poorly written CGI programs, and the security FAQs at
the time did not advise against, in not including the newline
character in their list of shell meta-characters.

Is this not what you asked of IBM-ERS and CIAC?  It seems you forget
that you were advised of this bug by me a month before IBM-ERS first
contacted you.

This has been my first experience of releasing an "advisory" to the
public.  It is disheartening that action is only taken after the bug
is publicized on zillions of security advisories rather than at the
time that discreet mail was sent to the author.

And you say you would prefer the discreet mail.

Sorry for all the negativity in this message. I think Apache is a
wonderful product and I've just put 1.1b1 on my 80,000 hit/day server.
Soon I want to learn the Apache API, which is why I have joined this
list.

--
Jennifer Myers				http://www.eecs.nwu.edu/~jmyers/

Re: util.c hole and speed of security patch release

[Brian Behlendorf <br...@organic.com>]

On Mon, 22 Apr 1996, Jennifer Myers wrote:
> I hope you don't mind that I've subscribed to this mailing list.  It
> takes *forever* to download from ftp.hyperreal.com, and given that it
> is publically available there, I figured that the list membership
> would be open as well. 

I figure if people can handle the volume, then it doesn't hurt to have them
here.  :) There's no formal procedure for addition. 

>  I'll just lurk for the most part, but I wanted
> to add my two cents on the newline/util.c issue.
> 
> I hope you guys remember that I told you about the phf/util.c hole
165q> back in February:
> 
>   Subject: Security problem in phf
>   To: jbrowne@uiuc.edu, apache-bugs@apache.org, httpd@hoohoo.ncsa.uiuc.edu
>   Date: Mon, 5 Feb 1996 16:51:05 -0600 (CST)
>   From: Jennifer Myers <jm...@marigold.eecs.nwu.edu>
> 
>     Hello,
> 
>   Both the NCSA httpd and Apache httpd distributions include a CGI
>   program "phf" which is a forms-based interface to Ph servers written
>   by Jim Browne <jb...@uiuc.edu>.
> 
>   "phf" can be made to execute commands by inserting "%0A" (newline) into the
>   URL of a phf query, followed by whatever command you wish to execute.
>
>   [..removed for brevity..]
> 
>     The fix is to add \x0A to the characters which escape_shell_cmd() in
>   util.c will escape:
> 
>   -        if(ind("&;`'\"|*?~<>^()[]{}$\\",cmd[x]) != -1){
>   +        if(ind("&;`'\"|*?~<>^()[]{}$\\\x0A",cmd[x]) != -1){

(first - "\x0A" equals "%0A"? huh? now...)

HOLD IT RIGHT THERE, PARTNER.  I would veto such a change to the code.  Why? 
Because I have written applications which required the passing around of
newline characters - for example, a TEXTAREA field in some browsers delineate
lines in the field using just \n.  Thus, I *need* that passed through to 
the CGI environment.  Looking at the draft CGI specification at 
http://www.ast.cam.ac.uk/~drtr/draft-robinson-www-interface-01.html,
there is *no* language about "%0A" not being allowed in 
"application/x-www-url-encoded" data.  I would also presume that there 
would be no such language at NCSA, either.  Thus, building applications 
which rely upon being able to get "%0A" has got to be preserved.  

The fact that user-supplied strings are often passed to a shell environment
by a CGI script unchecked is not the fault of the server, but rather the CGI
programmer.  It is a reason why I forever forsook CGI programming in shell
scripts (though yes, you can hang yourself similarly in other languages, just
not as insanely easily.)  Every security chapter of every book on CGI 
I've seen mentioned checking your data before supplying it to the shell.  
The problem is that with the advent of the web you have a lot of novice 
programmers who never had to think about security publishing "guestbook" 
scripts that millions of other people try to set up on their own.  

Checking for '\n' in the applications/x-www-url-encoded text chunk was 
alright by me because it would never be an acceptible character in that 
stream.  %0A is.

> escaped.  Most of these programs tried to sanitize their input by
> escaping a list of shell meta-characters, but those lists of nasty
> characters, as provided by several CGI security FAQs did not include
> the newline character.

So issue a security warning about the security FAQ's, not the server 
software.  Perhaps we should issue a warning about the OS as well, 
perhaps the CPU and disk hardware....

> Curiously, a version of Apache was released after Feb 8th without the
> problem fixed.  Not until IBM-ERS, CERT, CIAC, etc. picked up the
> advisory in March did it seem that there was any interest in fixing
> the bug!  Meanwhile, I have received reports that the bug is being
> actively exploited, because a good majority of sites have installed
> phf by way of installing all of the sample CGI code which comes with
> Apache and NCSA httpd.  CERT has also indicated that the bug is being
> actively exploited.

I guess we all considered CGI source extreme baggage from historical 
practice.  I will post a separate message to the list suggesting a 
solution, but yes, we are at fault for not declaring our lack of support 
for the cgi-src and cgi-bin software.  I think we can also be faulted for 
setting up the defaults in the config files such that the cgi-bin 
directory, with the default shell scripts, is used, rather than forcing 
people to create their own cgi-bin directory from scratch.  

> You (members of the list) complain that IBM-ERS or CIAC did not 
> inform you of the bug in src/util.c prior to releasing their advisories.
> 
> Well, I quietly informed you of the bug in cgi-src/util.c on February
> 5th, right when I discovered it.  Better yet, I have never advertised
> publically of the bug in phf (though lots of people mailed me to say
> that they had found the problem in phf from my description) -- I
> figured I ought to give you a chance to deal with it first -- and
> instead, I released an "advisory" on the nature of the bug, which is
> present in many poorly written CGI programs, and the security FAQs at
> the time did not advise against, in not including the newline
> character in their list of shell meta-characters.
> 
> Is this not what you asked of IBM-ERS and CIAC?  It seems you forget
> that you were advised of this bug by me a month before IBM-ERS first
> contacted you.

As others have noted, you contacted us about the first, but not about the 
second.  The first was in a piece of software we were not actively 
supporting - think of it like the contrib directory in most OS's - and 
thus did not see a need to make a big fuss about it.  

> This has been my first experience of releasing an "advisory" to the
> public.  It is disheartening that action is only taken after the bug
> is publicized on zillions of security advisories rather than at the
> time that discreet mail was sent to the author.
>
> And you say you would prefer the discreet mail.

I think we just hit a major disconnect.  We talked about it here (the 
archives of this list are also public) and decided it was not worth 
worrying about, since we didn't consider that code supported.  We 
goofed.  It's not a matter of not having listened to you or your 
message. 

I apologize if this message is brusque.  Welcome to new-httpd!  :)

	Brian


--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
brian@organic.com  |  We're hiring!  http://www.organic.com/Home/Info/Jobs/