Re: svnadmin create and not being method agnostic

[Nico Kadel-Garcia <nk...@gmail.com>]

On Thu, Dec 30, 2010 at 9:29 AM, Stefan Sperling <st...@elego.de> wrote:

> What if the user later decides to use svnserve instead of apache?
> How would the principle of least astonishment be applied then?
>
> Do we tell those users to copy svnserve.conf from another repository?
> Do we add a new option --create-svnserve-config to svnadmin as well?
> What if the user now wants to prevent apache httpd from accessing one
> particular repository, but still access some others?

I personally covered this one. Make it part of the next release, push
svnserve.conf.tmpl instead of svnserve.conf in default configurations,
and make svnserve choke if there is not an svnserve.conf for the
particular repository.

This makes new setups disabled by default, but previous release
configurations still work unmodified and after "svnadmin hotcopy". It
changes the security model moving forward to a safer one. And yes, it
needs to be documentd. The existing "svnserve operates wide open
without an existing svnserve.conf" is an undocumented misfeature, and
a surprisingly dangerous one for repositories that may have sensitive
information in them.

> Because of this, in your case, I would prefer if you simply wrote a
> simple wrapper script to create your repositories, instead of us adding
> a new feature to svnadmin:

Until he goes to a new setup and has to send someone else his tool.
Better to perform saner security and disable services by default than
force manual deletion by every admin.

>  create-svn-repos.sh:
>  #!/bin/sh
>  svnadmin create $1
>  rm -f $1/conf/svnserve.conf

And *this* is why it should be automated, rather than expecting admins
to do it themselves. Because this script is.... Well, it's
historically typical of Subversion internals. It ignores the results
of the "svnadmin create" directive, and doesn't sanitize the inputs to
prevent scubbing an svnserve.conf from an already existing directive,
nor is it safe against misparsing a reponame with spaces or control
characters in it. If I knew such a script was run automatically by the
Subversion admin, I could trivially hand them a repository name that
would "rm -f \-r\ /\ repoanme".

You've just made my point that it should be automated upstream. But if
you really must do this, try something like the script below, although
it might need a bit more thought.

#!/bin/sh
# create-svn-repos.sh - Create Subversion repos with svnserve config eliminated
# Note that it does not handle any svnadmin options
#
progname="`basename $0`"
if [ $# -ne 1 ]; then
   echo "Usage: $progname reponame"
   exit 1
fi
svnadmin create "$1" && \
    rm -f "$1"/conf/svnserve.conf

>> Admins today have to be competent in operating a few hundred different
>> subsystems.  Let's cut them some slack.
>
> IMO automation is a very good way of capturing this complexity.

*Good* automation is helpful. Bad automation, and replacing automation
with locally developed ritual, is not.

This is absolutely correct. It should be automatic in svnadmin default
behavior, not in 300 hand-written add-on wrappers for 300 different
admins.

> The script above is one way of automating repository creation to
> address your desire.

This is clearly wrong. I've just reverified it. As you'd mentioned
previously, a missing svnserve.conf allows full read-access, without
restriction. It doesn't accept write commands, but read can be too
much in many configurations.

> I would go as far as recommending to take a look at puppet, a flexible large
> scale automation system for system administrators, to control your
> Subversion setup and create your repositories. This way you can take
> care of a lot of special requirements in an automated fashion. Everyone
> has special requirements, which is of course fine, but everyone also
> has *different* special requirements.

Potentially useful, but why should he and every configuration tool
have to replace something that can be addressed more cleanly upstream?

> I may be biased but I don't think a core Subversion setup is particularly
> complex to set up.  It gets a lot more complex if you integrate
> Subversion with existing infrastructure and other tools. But there is not
> much Subversion's developers can do to help people with this, other than
> making sure that Subversion's solutions are as general, flexible, and
> scriptable as possible.

To set up the first time for testing? No. To set up securely? Youch.
It's paide me some very remunerative consulting wages, becuase it took
someone as paranoid as me to clean it up. It's quite painful due to
lack of documentation of necessary single-user configurations, the
multiplicity of access technologies each with entirely different
access control tools, and the expectation that each admin will *of
course write their own little wrappers for standard, sensible
behavior.

You just gave an *excellent* example of.the risks of.this with that
dangerous shell script. I really have to thank you for that unintended
blessing in this discussion.

>> >The common approach *is* to use an unprivileged user. But see above:
>> >the locking out of non-designated users from read or write access to
>> >Subversion repositories is under-documented.
>> >
>> >Setting up repositories to do this for two users, such as "apache" and
>> >"svn", is even more fun and creates its own security overlap issues.
>> >Coupled with the "svnadmin hotcopy" lack of preservation of group
>> >permissions or sgid bits, and it's even more adventuresome.
>>
>> Yes, exactly.  This is what I'm concerned about: lack of more
>> visible/obvious separation between what enables Apache and what
>> enables svnserve.
>
> I agree that the book could be improved here.
> One issue is that the book is supposed to be about Subversion and not
> UNIX administration. However, putting hints into the book about what
> to do on UNIX systems is fine, preferably with references to other
> literature that describes these issues in detail.

See, that's where you're making another dangerous assumption. This is
not a "UNIX" suggestion. This is suggestion inherent to every ACL for
filesystems in the world, whether UNIX, Linux, MacOS, Windows, or even
some weird upstream network file system based setup.

> Note that the book, just like Subversion itself, is an open source
> project and that contributions to the book are not only welcome
> but needed (see http://svnbook.org).
>
>> Agreed.  If you have the opportunity to make the software easier to
>> use, why not do it?
>
> We're trying to improve usability, all the time.
> But there are many ways to make the software easier to use, and we
> need to pick those that most of our users will benefit from.
>
> Stefan

And that's fair. Here's a very lightweight, testable, and reverse
compatible change that would notably improve security models going
forward. Enjoy.

RE: svnadmin create and not being method agnostic

[Tony Sweeney <ts...@omnifone.com>]

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@gmail.com] 
Sent: 02 January 2011 12:55
To: Philip Prindeville
Cc: Ryan Schmidt; users@subversion.apache.org
Subject: Re: svnadmin create and not being method agnostic

>On Sun, Jan 2, 2011 at 2:49 AM, Philip Prindeville
><ph...@redfish-solutions.com> wrote:
>> On 1/1/11 8:29 PM, Nico Kadel-Garcia wrote:

>>> To set up the first time for testing? No. To set up securely? Youch.
>>> It's paide me some very remunerative consulting wages, becuase it took
>>> someone as paranoid as me to clean it up. It's quite painful due to
>>> lack of documentation of necessary single-user configurations, the
>>> multiplicity of access technologies each with entirely different
>>> access control tools, and the expectation that each admin will *of
>>> course write their own little wrappers for standard, sensible
>>> behavior.
>>
>> So you're ok being made redundant and slaughtering this cash-cow?  :-)
>>
>> -Philip
>>

>It's not cow. Subversion security is *goat*. Inexpensive to buy the
>unprepared meat, but it;'s fairly gamey, risky for inexperienced
>chefs, and raises suspicious eyebrows if anyone sees you with the big
>hammer you need to tenderize it. But if the chef's time costs less
>than the raw materials, some customers want it.

Ahem.  Subversion security is not goat.  Goat is fine eating, from the Caribbean to the middle east and central and southern Asia.  Subversion security is *roadkill*.  At the top end, Apache security is venison; a delicacy that many would be happy to pay for or indeed to hunt themselves.  At the low end, svnserve security is possum; hey, it's free, and it does the job so long as you hold your nose while you swallow.  I'll leave it to others to fill in the intermediate tiers/tieren*.

>If we can get the goat meat tenderized before it lands in our
>kitchens, that saves me time to make sure the busboys (of whatever
>gender) aren't drinking all the table wine and writing the init
>scripts in Perl.

Don't get me started on perl: "it was a hungry man that ate the first oyster".

Tony.
[*] A nod to our German correspondents.

______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
______________________________________________________________________

Re: svnadmin create and not being method agnostic

[Nico Kadel-Garcia <nk...@gmail.com>]

On Sun, Jan 2, 2011 at 2:49 AM, Philip Prindeville
<ph...@redfish-solutions.com> wrote:
> On 1/1/11 8:29 PM, Nico Kadel-Garcia wrote:

>> To set up the first time for testing? No. To set up securely? Youch.
>> It's paide me some very remunerative consulting wages, becuase it took
>> someone as paranoid as me to clean it up. It's quite painful due to
>> lack of documentation of necessary single-user configurations, the
>> multiplicity of access technologies each with entirely different
>> access control tools, and the expectation that each admin will *of
>> course write their own little wrappers for standard, sensible
>> behavior.
>
> So you're ok being made redundant and slaughtering this cash-cow?  :-)
>
> -Philip
>

It's not cow. Subversion security is *goat*. Inexpensive to buy the
unprepared meat, but it;'s fairly gamey, risky for inexperienced
chefs, and raises suspicious eyebrows if anyone sees you with the big
hammer you need to tenderize it. But if the chef's time costs less
than the raw materials, some customers want it.

If we can get the goat meat tenderized before it lands in our
kitchens, that saves me time to make sure the busboys (of whatever
gender) aren't drinking all the table wine and writing the init
scripts in Perl.

Re: svnadmin create and not being method agnostic

[Philip Prindeville <ph...@redfish-solutions.com>]

On 1/1/11 8:29 PM, Nico Kadel-Garcia wrote:
> On Thu, Dec 30, 2010 at 9:29 AM, Stefan Sperling<st...@elego.de>  wrote:
>
>> What if the user later decides to use svnserve instead of apache?
>> How would the principle of least astonishment be applied then?
>>
>> Do we tell those users to copy svnserve.conf from another repository?
>> Do we add a new option --create-svnserve-config to svnadmin as well?
>> What if the user now wants to prevent apache httpd from accessing one
>> particular repository, but still access some others?
> I personally covered this one. Make it part of the next release, push
> svnserve.conf.tmpl instead of svnserve.conf in default configurations,
> and make svnserve choke if there is not an svnserve.conf for the
> particular repository.
>
> This makes new setups disabled by default, but previous release
> configurations still work unmodified and after "svnadmin hotcopy". It
> changes the security model moving forward to a safer one. And yes, it
> needs to be documentd. The existing "svnserve operates wide open
> without an existing svnserve.conf" is an undocumented misfeature, and
> a surprisingly dangerous one for repositories that may have sensitive
> information in them.

Agreed.  Can someone fill a bug to track this and put me as the buddy?

>> Because of this, in your case, I would prefer if you simply wrote a
>> simple wrapper script to create your repositories, instead of us adding
>> a new feature to svnadmin:
> Until he goes to a new setup and has to send someone else his tool.
> Better to perform saner security and disable services by default than
> force manual deletion by every admin.

Again, agreed.

>>   create-svn-repos.sh:
>>   #!/bin/sh
>>   svnadmin create $1
>>   rm -f $1/conf/svnserve.conf
> And *this* is why it should be automated, rather than expecting admins
> to do it themselves. Because this script is.... Well, it's
> historically typical of Subversion internals. It ignores the results
> of the "svnadmin create" directive, and doesn't sanitize the inputs to
> prevent scubbing an svnserve.conf from an already existing directive,
> nor is it safe against misparsing a reponame with spaces or control
> characters in it. If I knew such a script was run automatically by the
> Subversion admin, I could trivially hand them a repository name that
> would "rm -f \-r\ /\ repoanme".
>
> You've just made my point that it should be automated upstream. But if
> you really must do this, try something like the script below, although
> it might need a bit more thought.
>
> #!/bin/sh
> # create-svn-repos.sh - Create Subversion repos with svnserve config eliminated
> # Note that it does not handle any svnadmin options
> #
> progname="`basename $0`"
> if [ $# -ne 1 ]; then
>     echo "Usage: $progname reponame"
>     exit 1
> fi
> svnadmin create "$1"&&  \
>      rm -f "$1"/conf/svnserve.conf

Hmm... "$@" to handle options?

>>> Admins today have to be competent in operating a few hundred different
>>> subsystems.  Let's cut them some slack.
>> IMO automation is a very good way of capturing this complexity.
> *Good* automation is helpful. Bad automation, and replacing automation
> with locally developed ritual, is not.
>
> This is absolutely correct. It should be automatic in svnadmin default
> behavior, not in 300 hand-written add-on wrappers for 300 different
> admins.

True.  Not to mention the lost time of all 300 different admins having to figure this out and reinvent the wheel.



>> The script above is one way of automating repository creation to
>> address your desire.
> This is clearly wrong. I've just reverified it. As you'd mentioned
> previously, a missing svnserve.conf allows full read-access, without
> restriction. It doesn't accept write commands, but read can be too
> much in many configurations.

Yup.  In our case, we have IP that needs to be protected.

>> I would go as far as recommending to take a look at puppet, a flexible large
>> scale automation system for system administrators, to control your
>> Subversion setup and create your repositories. This way you can take
>> care of a lot of special requirements in an automated fashion. Everyone
>> has special requirements, which is of course fine, but everyone also
>> has *different* special requirements.
> Potentially useful, but why should he and every configuration tool
> have to replace something that can be addressed more cleanly upstream?

Agreed.  No need to reinvent the wheel.

>> I may be biased but I don't think a core Subversion setup is particularly
>> complex to set up.  It gets a lot more complex if you integrate
>> Subversion with existing infrastructure and other tools. But there is not
>> much Subversion's developers can do to help people with this, other than
>> making sure that Subversion's solutions are as general, flexible, and
>> scriptable as possible.
> To set up the first time for testing? No. To set up securely? Youch.
> It's paide me some very remunerative consulting wages, becuase it took
> someone as paranoid as me to clean it up. It's quite painful due to
> lack of documentation of necessary single-user configurations, the
> multiplicity of access technologies each with entirely different
> access control tools, and the expectation that each admin will *of
> course write their own little wrappers for standard, sensible
> behavior.

So you're ok being made redundant and slaughtering this cash-cow?  :-)

-Philip

> You just gave an *excellent* example of.the risks of.this with that
> dangerous shell script. I really have to thank you for that unintended
> blessing in this discussion.
>
>>>> The common approach *is* to use an unprivileged user. But see above:
>>>> the locking out of non-designated users from read or write access to
>>>> Subversion repositories is under-documented.
>>>>
>>>> Setting up repositories to do this for two users, such as "apache" and
>>>> "svn", is even more fun and creates its own security overlap issues.
>>>> Coupled with the "svnadmin hotcopy" lack of preservation of group
>>>> permissions or sgid bits, and it's even more adventuresome.
>>> Yes, exactly.  This is what I'm concerned about: lack of more
>>> visible/obvious separation between what enables Apache and what
>>> enables svnserve.
>> I agree that the book could be improved here.
>> One issue is that the book is supposed to be about Subversion and not
>> UNIX administration. However, putting hints into the book about what
>> to do on UNIX systems is fine, preferably with references to other
>> literature that describes these issues in detail.
> See, that's where you're making another dangerous assumption. This is
> not a "UNIX" suggestion. This is suggestion inherent to every ACL for
> filesystems in the world, whether UNIX, Linux, MacOS, Windows, or even
> some weird upstream network file system based setup.
>
>> Note that the book, just like Subversion itself, is an open source
>> project and that contributions to the book are not only welcome
>> but needed (see http://svnbook.org).
>>
>>> Agreed.  If you have the opportunity to make the software easier to
>>> use, why not do it?
>> We're trying to improve usability, all the time.
>> But there are many ways to make the software easier to use, and we
>> need to pick those that most of our users will benefit from.
>>
>> Stefan
> And that's fair. Here's a very lightweight, testable, and reverse
> compatible change that would notably improve security models going
> forward. Enjoy.

Re: svnadmin create and not being method agnostic

[Stefan Sperling <st...@elego.de>]

On Sat, Jan 01, 2011 at 10:29:22PM -0500, Nico Kadel-Garcia wrote:
> You've just made my point that it should be automated upstream.

Instead of riding on the obvious shortcomings of a two-line shell
script I was using to badly illustrate an idea, why don't you spend
time writing up a "Setting up Subversion securely" article?
I think you do have the knowledge to do that.
And it would benefit users a lot because we could improve the
documentation based on it.

> And that's fair. Here's a very lightweight, testable, and reverse
> compatible change that would notably improve security models going
> forward. Enjoy.

You have been glossing over one open issue I see with your proposal,
which I have mentioned before:

What if users run svnserve like this:
  svnserve --config-file /etc/svnserve.conf
Should we then still require an svnserve.conf file to be present
in each repository? Even if this svnserve.conf is not used?
Note that per-repos configs do not override the global ones (and we
can't change that), so there's potential for confusion about where
settings actually come from.

I'd rather use a separate marker file (like the git-daemon-export-ok file
in git), but we cannot use a new marker file because of backwards compat.