Re: Subversion security needs to improve.

[kf...@collab.net]

"Alex Holst" <a...@mongers.org> writes:
> I have articulated some of these reasons in a document partly aimed at
> Subversion developers and partly aimed at Subversion users. The part
> aimed at the developers is mostly complete, and I'd like you all to read
> it so we can talk about how to best carry some of these suggestings into
> the project. I think it is badly needed.
> 
>         Subversion security - http://mongers.org/svnsec
> 
> I will continue to work on the document over the coming days.

I appreciate the thought put into the document, but find it less than
constructive.

   > Virtually every security bug found in software either stems from
   > a design or implementation flaw. Hence, making secure software
   > require careful adherence to rules about avoiding programming
   > language and OS pitfalls. Additionally, security code inspection
   > and testing can be used for increased assurance (not guarantee)
   > that security flaws are not present. In some situations, a
   > particular approach to software design can help reduce the
   > exposure, should a flaw sneak past security code review and
   > testing.

Except for the part about inspection, these are all vague wishes, not
concrete advice.  You mention "rules", but do not describe them.
Everyone who writes an API in Subversion thinks about the security
implications.  However, sometimes we don't think of them all.  If you
have an improved process for thinking of them, this would be the place
to describe it...

   > The Subversion team do none of these things. They have a webpage
   > with a contact address for reporting security bugs, and this is a
   > good thing. They even react to security problems with the usual
   > speed one has come to expect from non-commercial
   > vendors. However, it would be better if the entire team actively
   > started to audit existing code for bugs and implemented security
   > review of new code. One or two people cannot do this alone. The
   > entire team needs to participate.

Yes, that would be nice, yet no one has volunteered to do it, because
they find other things to be higher priority.

You are willing to assert that a security audit should be a higher
priority than whatever it was a given developer decided to do instead
of auditing.  But you're not volunteering to do the audit yourself,
nor pay for others to do it.  Security audits are mind-numbingly hard,
incredibly time-consuming.  You're saying "The team should do X"
without addressing the costs and difficulties of X.

What *specifically* is the team doing wrong now, or failing to do
right?  If you want to change their behavior, what will the costs be?

   > When this author briefly confronted Greg Stein (of Subversion and
   > Apache Software Foundation) about the serious lack of a security
   > stance long before 1.0 was released, Stein retorted that any
   > security bugs would be in Apache 2, not Subversion itself. There
   > was no need to worry too much about the security properties of
   > the Subversion source code.

[Was it necessary to use the word "confronted" instead of "asked"?
You want to get people on your side here, right? :-) ]

One developer's optimistic prediction from a long time ago does not
represent the group's opinion, then or now.  I'm not really sure why
you included that paragraph.

   > Karl Fogel announced to the Subversion developer list in June
   > 2004 that he was starting a security code audit and then followed
   > up a month later by stating that he would not be able to finish
   > the audit.

That's right.  It was taking a *huge* amount of time.  (And I still
didn't spot the path-based-authz problems that were later discovered!
So all that time was wasted.)

The costs of the audit turned out to far outweigh the benefits, i.e.,
it was costing a lot and not benefiting us much anyway.  If someone
with more skill at a security audit (you?) did it, it might cost less
and benefit Subversion more.  Then the tradeoff would be worth it.

   > To date, 4 of 9 releases (in the 1.0.x branch) since 1.0 have
   > been released to fix security bugs found by someone outside the
   > Subversion project. 4 security bugs in 6 months is worrying to
   > any user who puts much value into either the confidentiality,
   > integrity or availability of their source code. There will be
   > more bugs found. Hence, the improvement process must start now.

This is overly simplistic -- you're counting all security problems as
equally severe.  An exploit that gives someone shell access is far
worse than a problem that leaks, say, the names (but not contents) of
paths that were changed beneath an authz-protected dir.  We've had
both kinds of problems, and stuff in between.

You can't just say "4 security bugs in 6 months".  You have to
differentiate on the basis of severity.

In the same spirity, you later write:

   > The Subversion team is very serious about repository
   > integrity. They work hard to avoid bugs that could lead to
   > corrupt repositories. That same effort should be applied to
   > avoiding bugs that could lead to security compromises.

That's a fallacy -- the efforts are not tradeable, for all sorts of
reasons.

I'm saying all this only partly defend the Subversion team (of course
I too wish for fewer security releases!).  I'm also saying it because
whatever you propose must be based on an realistic assessment of the
costs and benefits, and describe -- concretely -- how to get the work
done.  You haven't done any of that, and that's why your document
isn't much help, I'm afraid.

-Karl

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Subversion security needs to improve.

[kf...@collab.net]

"Alex Holst" <a...@mongers.org> writes:
> > Yes, that would be nice, yet no one has volunteered to do it, because
> > they find other things to be higher priority.
> 
> This conflicts with the essence of a point Ben Reser made, doesn't it?
> He spends time looking for security problems and goes so far as to say
> the effort made toward avoiding corruption bugs also go towards avoiding
> security problems.

No, not at all.

I was talking about security audits.  Ben Reser was talking about
reviewing code commits with (among other things), security in mind.
Many people review commits -- you can too, if you want.  I review
every single C code commit, and security implications are one of the
things I'm looking at.  

Commit-by-commit review is not the same thing as a security audit.

> I'll try to be more specific. As mentioned on IRC, in my experience
> people seem to be more willing to follow along with an idea if they can
> see the overall point, instead of suddenly implementing a security
> counter-measure for no real reason. svn seems to be the exception to
> this rule :)

Huh?  

This doesn't really make sense.  Anyone, on being given a specific
recommendation for how to avoid certain kinds of security bugs, would
be happy to follow the recommendation, assuming its not too onerous.
The "overall point" of doing so would be obvious.  Who would look at a
"security counter-measure" and think it's done "for no real reason"?

-Karl

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Subversion security needs to improve.

[Ben Reser <be...@reser.org>]

On Thu, Oct 21, 2004 at 02:34:20AM +0200, Alex Holst wrote:
> >    > The Subversion team is very serious about repository
> >    > integrity. They work hard to avoid bugs that could lead to
> >    > corrupt repositories. That same effort should be applied to
> >    > avoiding bugs that could lead to security compromises.
> > 
> > That's a fallacy -- the efforts are not tradeable, for all sorts of
> > reasons.
> 
> Again, I got the sense from Ben Reser that the approach I praised to
> avoid corruption bugs went towards avoiding security bugs. You don't
> seem to think so. It would be a great first step if all/most developers
> had the same understanding of the level of effort that goes into
> avoiding security bugs.

I believe Karl was thinking more in the context of an audit, going back
over all the code that's already committed with an eye to finding
security problems.  I'm sure Karl would agree that our efforts to review
code as it is committed help prevent security bugs.  I just think Karl
and I were talking about different things.

That said, there has been some degree of efforts to audit some code.
For instance I made an effort to look into the string buffering code.
The only thing I ended up finding was a single, bug highly improbable,
way to put the code into an infinite loop.  Karl and I fixed it.

But the effort to audit code like that is completely different from
reviewing code as it is committed.  One is looking for existing errors
that are already comitted.  The other is looking to see if a change does
anything it shouldn't.  Both are valueable.

-- 
Ben Reser <be...@reser.org>
http://ben.reser.org

"Conscience is the inner voice which warns us somebody may be looking."
- H.L. Mencken

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Subversion security needs to improve.

[John Szakmeister <jo...@szakmeister.net>]

On Wednesday 20 October 2004 20:34, Alex Holst wrote:
> Quoting kfogel@collab.net (kfogel@collab.net):
[snip]
> > Except for the part about inspection, these are all vague wishes, not
> > concrete advice.  You mention "rules", but do not describe them.
> > Everyone who writes an API in Subversion thinks about the security
> > implications.  However, sometimes we don't think of them all.  If you
> > have an improved process for thinking of them, this would be the
> > place to describe it...
>
> I did this recently. When I suggested you read the excellent Microsoft
> Press book "Threat Modeling" by Swiderski and Snyder, your comment was
> something along the line that you probably couldn't learn anything
> about security from Microsoft. :)

I'm sure that was a sarcastic remark, and a funny one at that. :-)

> As for rules, I was deliberately trying to avoid specific suggestions
> as they might not be 100% compatible with all the activities in this
> project. I thought it would be best to narrow them down together. I am
> not yet disillusioned enough to believe that a single person can define
> how this project should go about improving this.

Avoiding suggestions helps no one.  I see words on a piece of paper that 
really don't add up to much.  You seem to suggest that we aren't aware of 
security issues, yet you don't site specific behaviors that you feel are 
contrary to developing more secure code, with the exception of auditing 
code.  But even then, you made a remark about not reviewing code and had 
no real evidence to back up that claim.

I don't email the list every time I review a piece of code.  I look at 
every single commit that happens, and scrutinize it for errors, style, 
and security.  If you've really been lurking since 2001, then you should 
know that the team does this, and does it well.  I didn't come into this 
project with that kind of style, the Subversion team molded me into it.  
There *is* an atmosphere for developing high quality code here, and that 
includes watching out for security implications.

Also, no one is trying to suggest that a single person review all of this 
code.  What is being suggested is that if you feel this is so important, 
why don't you invest your time?  We get a lot of suggestions on this list 
on how we should do things, but when asked to step up to the plate to fix 
it... well, it's all of a sudden less concerning to them.  In coding 
speak, "patches are welcome." :-)

> I'm just sorry I picked a wrong word and got the number of bugs wrong.
>
> >    > The Subversion team do none of these things. They have a webpage
> >    > with a contact address for reporting security bugs, and this is
> >    > a good thing. They even react to security problems with the
> >    > usual speed one has come to expect from non-commercial
> >    > vendors. However, it would be better if the entire team actively
> >    > started to audit existing code for bugs and implemented security
> >    > review of new code. One or two people cannot do this alone. The
> >    > entire team needs to participate.
> >
> > Yes, that would be nice, yet no one has volunteered to do it, because
> > they find other things to be higher priority.
>
> This conflicts with the essence of a point Ben Reser made, doesn't it?
> He spends time looking for security problems and goes so far as to say
> the effort made toward avoiding corruption bugs also go towards
> avoiding security problems.

No it doesn't.  Just because we don't sit down a pick a day to audit code 
for security flaws, doesn't mean we aren't concerned with it.  FWIW, we 
don't pick a day to sit down and examine the backends from top to bottom 
either.  We count on us reviewing each others commits, and when faced 
with a particularly difficult situation, it's pushed to the list first.

Also, there are a number of features in the Subversion code that are 
specifically designed to make programming easier, and to avoid common 
security pitfalls.

> > What *specifically* is the team doing wrong now, or failing to do
> > right?  If you want to change their behavior, what will the costs be?
>
> I've tried to expand on the specifics and I'll continue tomorrow. It's
> 2.30am here.

Where?  I saw one, make svnserve chroot().  Do you have others?

> The costs depend on how aggressive you want to be, but it really boils
> down to some other work not getting done. I can't qoute a number as the
> cost for the subversion project of postponing features, nor the cost to
> the project of being subject to a particular number of vulnerabilities.
>
> Can you elaborate on what "costs" mean in this context? Maybe I can
> answer the question better.

How about time and energy?  If we aren't doing enough, tell me how much 
time we should be spending on this?  How much code a day should we be 
auditing?  How much of an impact are the new more security-aware habits 
going to have in terms of development?  Let's face it, a person *could* 
sit down and stare at code all day, but then nothing new ever gets 
implemented.

> >    > Karl Fogel announced to the Subversion developer list in June
> >    > 2004 that he was starting a security code audit and then
> >    > followed up a month later by stating that he would not be able
> >    > to finish the audit.
> >
> > That's right.  It was taking a *huge* amount of time.  (And I still
> > didn't spot the path-based-authz problems that were later discovered!
> > So all that time was wasted.)
>
> I don't agree it was a waste of time. It "just" takes more effort and
> more people actively looking over the code. If most developers were
> able to spend 30 minutes per week looking for security bugs, they would
> slowly get better at it and catch more things without a serious
> disruption to the development progress.

I spend more than 30 minutes a week looking over commits, and I'm sure 
that most other Subversion developers do too.  That said, some of the 
security bugs found were pretty obscure, and most wouldn't have caught 
them.

> > The costs of the audit turned out to far outweigh the benefits, i.e.,
> > it was costing a lot and not benefiting us much anyway.  If someone
> > with more skill at a security audit (you?) did it, it might cost less
> > and benefit Subversion more.  Then the tradeoff would be worth it.
>
> Please see my point to Ben Reser about the value of building a culture
> of security-aware developers as opposed to a single individual who
> looks for these things. Do you still feel a single individual finding
> problems is the best approach?

Again, what about our culture isn't "security aware"?  It's suggestions 
like these that make me think you haven't been watching the list much 
since 2001. :-)

> >    > The Subversion team is very serious about repository
> >    > integrity. They work hard to avoid bugs that could lead to
> >    > corrupt repositories. That same effort should be applied to
> >    > avoiding bugs that could lead to security compromises.
> >
> > That's a fallacy -- the efforts are not tradeable, for all sorts of
> > reasons.
>
> Again, I got the sense from Ben Reser that the approach I praised to
> avoid corruption bugs went towards avoiding security bugs. You don't
> seem to think so. It would be a great first step if all/most developers
> had the same understanding of the level of effort that goes into
> avoiding security bugs.

I think Karl was talking about this from an implementation sort of view.  
We have tests in place to help detect when we do something wrong with the 
backend.  How do you do the same thing for security?  You said yourself 
that static tools aren't really up to the task, and require a great 
effort to get configured correctly.  It seems to me that good programming 
practices, and peer review is the way to go... and I believe we do that 
already.

> > I'm saying all this only partly defend the Subversion team (of course
> > I too wish for fewer security releases!).  I'm also saying it because
> > whatever you propose must be based on an realistic assessment of the
> > costs and benefits, and describe -- concretely -- how to get the work
> > done.  You haven't done any of that, and that's why your document
> > isn't much help, I'm afraid.
>
> I'll try to be more specific. As mentioned on IRC, in my experience
> people seem to be more willing to follow along with an idea if they can
> see the overall point, instead of suddenly implementing a security
> counter-measure for no real reason. svn seems to be the exception to
> this rule :)

Not at all.  Again, we gets lots of suggestions about how we should spend 
our resources with no real commitment from the person suggesting it.  If 
you commit to it, site specific examples and behaviors with suggestions 
on how to do it better, I'm willing to adopt them as long as they make 
sense.

> I've revised the document again based on this email exchange. I'll
> continue tomorrow. If I left anything out, please tell me.

You left out the "specifics." :-)

Please don't take offense to anything that I've said.  I'm really just 
trying to understand what you think are the problems.  You think we don't 
have a "security aware" culture, and I think we have that and more.

-John

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Subversion security needs to improve.

[Alex Holst <a...@mongers.org>]

Quoting kfogel@collab.net (kfogel@collab.net):
> I appreciate the thought put into the document, but find it less than
> constructive.

As mentioned, this was not intended. After all, I'm not esr :)

>    > Virtually every security bug found in software either stems from
>    > a design or implementation flaw. Hence, making secure software
>    > require careful adherence to rules about avoiding programming
[..]
> 
> Except for the part about inspection, these are all vague wishes, not
> concrete advice.  You mention "rules", but do not describe them.
> Everyone who writes an API in Subversion thinks about the security
> implications.  However, sometimes we don't think of them all.  If you
> have an improved process for thinking of them, this would be the place
> to describe it...

I did this recently. When I suggested you read the excellent Microsoft
Press book "Threat Modeling" by Swiderski and Snyder, your comment was
something along the line that you probably couldn't learn anything about
security from Microsoft. :)

As for rules, I was deliberately trying to avoid specific suggestions as
they might not be 100% compatible with all the activities in this
project. I thought it would be best to narrow them down together. I am
not yet disillusioned enough to believe that a single person can define
how this project should go about improving this.

I'm just sorry I picked a wrong word and got the number of bugs wrong.

>    > The Subversion team do none of these things. They have a webpage
>    > with a contact address for reporting security bugs, and this is a
>    > good thing. They even react to security problems with the usual
>    > speed one has come to expect from non-commercial
>    > vendors. However, it would be better if the entire team actively
>    > started to audit existing code for bugs and implemented security
>    > review of new code. One or two people cannot do this alone. The
>    > entire team needs to participate.
> 
> Yes, that would be nice, yet no one has volunteered to do it, because
> they find other things to be higher priority.

This conflicts with the essence of a point Ben Reser made, doesn't it?
He spends time looking for security problems and goes so far as to say
the effort made toward avoiding corruption bugs also go towards avoiding
security problems.

> What *specifically* is the team doing wrong now, or failing to do
> right?  If you want to change their behavior, what will the costs be?

I've tried to expand on the specifics and I'll continue tomorrow. It's
2.30am here.

The costs depend on how aggressive you want to be, but it really boils
down to some other work not getting done. I can't qoute a number as the
cost for the subversion project of postponing features, nor the cost to
the project of being subject to a particular number of vulnerabilities.

Can you elaborate on what "costs" mean in this context? Maybe I can answer the
question better.

>    > Karl Fogel announced to the Subversion developer list in June
>    > 2004 that he was starting a security code audit and then followed
>    > up a month later by stating that he would not be able to finish
>    > the audit.
> 
> That's right.  It was taking a *huge* amount of time.  (And I still
> didn't spot the path-based-authz problems that were later discovered!
> So all that time was wasted.)

I don't agree it was a waste of time. It "just" takes more effort and
more people actively looking over the code. If most developers were able
to spend 30 minutes per week looking for security bugs, they would slowly
get better at it and catch more things without a serious disruption to
the development progress. 

> The costs of the audit turned out to far outweigh the benefits, i.e.,
> it was costing a lot and not benefiting us much anyway.  If someone
> with more skill at a security audit (you?) did it, it might cost less
> and benefit Subversion more.  Then the tradeoff would be worth it.

Please see my point to Ben Reser about the value of building a culture
of security-aware developers as opposed to a single individual who looks
for these things. Do you still feel a single individual finding problems
is the best approach?

>    > The Subversion team is very serious about repository
>    > integrity. They work hard to avoid bugs that could lead to
>    > corrupt repositories. That same effort should be applied to
>    > avoiding bugs that could lead to security compromises.
> 
> That's a fallacy -- the efforts are not tradeable, for all sorts of
> reasons.

Again, I got the sense from Ben Reser that the approach I praised to
avoid corruption bugs went towards avoiding security bugs. You don't
seem to think so. It would be a great first step if all/most developers
had the same understanding of the level of effort that goes into
avoiding security bugs.

> I'm saying all this only partly defend the Subversion team (of course
> I too wish for fewer security releases!).  I'm also saying it because
> whatever you propose must be based on an realistic assessment of the
> costs and benefits, and describe -- concretely -- how to get the work
> done.  You haven't done any of that, and that's why your document
> isn't much help, I'm afraid.

I'll try to be more specific. As mentioned on IRC, in my experience
people seem to be more willing to follow along with an idea if they can
see the overall point, instead of suddenly implementing a security
counter-measure for no real reason. svn seems to be the exception to
this rule :)

I've revised the document again based on this email exchange. I'll
continue tomorrow. If I left anything out, please tell me.

-- 
I prefer the dark of the night, after midnight and before four-thirty,
when it's more bare, more hollow.                http://a.mongers.org 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Subversion security needs to improve.

[Florian Weimer <fw...@deneb.enyo.de>]

> What *specifically* is the team doing wrong now, or failing to do
> right?