You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Matt <lm...@gmail.com> on 2011/10/10 18:45:29 UTC
[users@httpd] Allowing Robots.txt
In .htaccess I have something like this:
order allow,deny
allow from 192.168.x.x/24
<Files robots.txt>
allow from all
</Files>
RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteCond %{REQUEST_URI} /
RewriteRule ^(.*)$ https://mydomain49459302.com/$1 [R,L]
AuthName "Restricted Area"
AuthType Basic
AuthUserFile /var/www/.htpasswd
AuthGroupFile /dev/null
require valid-user
I want to restrict http access to the server to certain subnets,
require SSL and a username and password. The exception is the
robots.txt file. I want to allow anyone access to that. How do I
tell it not to enforce a password or SSL only on robots.txt?
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Allowing Robots.txt
Posted by Mark Montague <ma...@catseye.org>.
On October 10, 2011 15:04 , Matt <lm...@gmail.com> wrote:
> Is there a way to do something like this?
>
> <Files / !=/robots.txt !=/downloads/>
> AuthName "Restricted Area"
> AuthType Basic
> AuthUserFile /var/www/.htpasswd
> AuthGroupFile /dev/null
> require valid-user
> </Files>
Yes, but not using that syntax.
> I basically want to require a password everywhere but on robots.txt
> and the downloads folder. I want to still require encryption and
> limit access by subnet on the downloads though just no password.
> robots.txt needs to be open to the world.
# Require user authentication for everything:
AuthName "Restricted Area"
AuthType Basic
AuthUserFile/var/www/.htpasswd
AuthGroupFile /dev/null
require valid-user
# Except, do not require user authentication for robots.txt
# Allow robots.txt to be accessed from everywhere:
<Files robots.txt>
Order allow,deny
Allow from all
Satisfy any
</Files>
# note that the above actually allows access to ANY file named
# robots.txt anywhere beneath the document root directory.
# If you really need to restrict everything except /robots.txt
# that's possible, but requires a more sophisticated solution.
# Also, do not require user authentication for the /downloads
# directory, but limit access to it by subnet:
<Directory /downloads>
Order allow,deny
Allow from 192.168.0.0/16 172.16.0.0/12
Satisfy any
</Directory>
--
Mark Montague
mark@catseye.org
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Allowing Robots.txt
Posted by Matt <lm...@gmail.com>.
Is there a way to do something like this?
<Files / !=/robots.txt !=/downloads/>
AuthName "Restricted Area"
AuthType Basic
AuthUserFile /var/www/.htpasswd
AuthGroupFile /dev/null
require valid-user
</Files>
I basically want to require a password everywhere but on robots.txt
and the downloads folder. I want to still require encryption and
limit access by subnet on the downloads though just no password.
robots.txt needs to be open to the world. Confusing even myself but
learning as I go.
>> One other thing though. Suppose I want to exempt certain directories
>> from requiring a password but still leave all remaining restrictions.
>> I have this there:
>>
>> AuthName "Restricted Area"
>> AuthType Basic
>> AuthUserFile /var/www/.htpasswd
>> AuthGroupFile /dev/null
>> require valid-user
>>
>> Is there a way to exempt say /downloads/ directory?
>
> <Directory /path/to/downloads>
> Allow from all
> Satisfy any
> </Directory>
>
> This will make a username and password optional rather than required for the
> downloads directory and all files and subdirectories within it. Assuming
> that you have not specified "Satisfy any" at any higher level, all other
> directories at and below the scope of the "AuthType Basic" directive will
> require a username and password.
>
> Since this is the same answer as in my previous response, I suspect I am not
> understanding your question correctly... can you elaborate?
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Allowing Robots.txt
Posted by Mark Montague <ma...@catseye.org>.
On October 10, 2011 13:35 , Matt <lm...@gmail.com> wrote:
> One other thing though. Suppose I want to exempt certain directories
> from requiring a password but still leave all remaining restrictions.
> I have this there:
>
> AuthName "Restricted Area"
> AuthType Basic
> AuthUserFile /var/www/.htpasswd
> AuthGroupFile /dev/null
> require valid-user
>
> Is there a way to exempt say /downloads/ directory?
<Directory /path/to/downloads>
Allow from all
Satisfy any
</Directory>
This will make a username and password optional rather than required for
the downloads directory and all files and subdirectories within it.
Assuming that you have not specified "Satisfy any" at any higher level,
all other directories at and below the scope of the "AuthType Basic"
directive will require a username and password.
Since this is the same answer as in my previous response, I suspect I am
not understanding your question correctly... can you elaborate?
--
Mark Montague
mark@catseye.org
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Allowing Robots.txt
Posted by Matt <lm...@gmail.com>.
> On October 10, 2011 12:45 , Matt <lm...@gmail.com> wrote:
>>
>> I want to restrict http access to the server to certain subnets,
>> require SSL and a username and password. The exception is the
>> robots.txt file. I want to allow anyone access to that. How do I
>> tell it not to enforce a password or SSL only on robots.txt?
>
> Use the "Satisfy any" directive so that httpd will accept EITHER the
> host-based access control ("Allow from all") OR the user authentication
> ("Require valid-user") instead of requiring both of them as it does by
> default ("Satisfy all"). See the example at
> https://httpd.apache.org/docs/2.2/mod/core.html#satisfy
That worked, thanks. I also had to add "RewriteCond %{REQUEST_URI}
!=/robots.txt" to exempt it from SSL.
One other thing though. Suppose I want to exempt certain directories
from requiring a password but still leave all remaining restrictions.
I have this there:
AuthName "Restricted Area"
AuthType Basic
AuthUserFile /var/www/.htpasswd
AuthGroupFile /dev/null
require valid-user
Is there a way to exempt say /downloads/ directory?
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Allowing Robots.txt
Posted by Mark Montague <ma...@catseye.org>.
On October 10, 2011 12:45 , Matt <lm...@gmail.com> wrote:
> I want to restrict http access to the server to certain subnets,
> require SSL and a username and password. The exception is the
> robots.txt file. I want to allow anyone access to that. How do I
> tell it not to enforce a password or SSL only on robots.txt?
Use the "Satisfy any" directive so that httpd will accept EITHER the
host-based access control ("Allow from all") OR the user authentication
("Require valid-user") instead of requiring both of them as it does by
default ("Satisfy all"). See the example at
https://httpd.apache.org/docs/2.2/mod/core.html#satisfy
--
Mark Montague
mark@catseye.org
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org