You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Marc Girod <gi...@shire.ntc.nokia.com> on 2003/01/02 10:37:31 UTC
[users@httpd] Re: patch to avoid loops in htdig indexing
Hello,
I have still the same problem as last November now with 1.3.27, i.e.
htdif when indexing my web site will loop forever when processing
pages generated by mod_autoindex, because these will follow soft
links.
I offered a patch to fix the problem by modifying the hard-coded
preamble emitted by the module, so that it would contain a "robots"
meta-tag telling htdig to ignore such generated pages.
I asked for comments and got some:
>>>>> "JS" == Joshua Slive <jo...@slive.ca> writes:
JS> Some easier ways to deal with this problem:
JS> 1. Use an Alias or a Redirect rather than a symlink.
JS> 2. Put the exclusions in /robots.txt
JS> 3. Use IndexOptions SuppressHTMLPreamble in combination with HeaderName
JS> to change the html code before the index:
1) is an html only solution, and thus inapplicable.
3) would require a special file per directory, and thus unmaintainable
2) would require a new crontab job for generating the robots.txt file
based on "find -type l" and would be a significant penalty.
What about making the emitted preamble configurable?
Would this be an acceptable fix?
--
Marc Girod P.O. Box 323 Voice: +358-71 80 25581
Nokia NBI 00045 NOKIA Group Mobile: +358-50 38 78415
Takomo 1 / 4c27 Finland Fax: +358-71 80 61604
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] Re: HeaderName
Posted by Marc Girod <gi...@shire.ntc.nokia.com>.
>>>>> "MG" == Marc Girod <gi...@shire.ntc.nokia.com> writes:
MG> OK. Not for some time. I got lazy and decided that it was enough to
MG> get a fixed title and header for all the pages.
Whereas my HeaderName setting seems to work as expected for
interactive uses, the indexing with htdig seems to by-pass it
completely, and goes back to looping forever.
I did remove the /var/lib/htdig directory before statring the
indexation.
--
Marc Girod P.O. Box 323 Voice: +358-71 80 25581
Nokia NBI 00045 NOKIA Group Mobile: +358-50 38 78415
Takomo 1 / 4c27 Finland Fax: +358-71 80 61604
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] Re: HeaderName (was: patch to avoid loops in htdig indexing)
Posted by Marc Girod <gi...@shire.ntc.nokia.com>.
>>>>> "MG" == Marc Girod <gi...@shire.ntc.nokia.com> writes:
MG> I.e. I may again ask for help.
OK. Not for some time. I got lazy and decided that it was enough to
get a fixed title and header for all the pages.
- The '+' syntax in "IndexOptions +SuppressHTMLPreamble" didn't work
- I had to comment away the default: HeaderName HEADER before the
global /autoindex-header started to apply.
- I *had* to use MultiViews Option
--
Marc Girod P.O. Box 323 Voice: +358-71 80 25581
Nokia NBI 00045 NOKIA Group Mobile: +358-50 38 78415
Takomo 1 / 4c27 Finland Fax: +358-71 80 61604
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] Re: HeaderName (was: patch to avoid loops in htdig indexing)
Posted by Marc Girod <gi...@shire.ntc.nokia.com>.
Thanks again, and thanks to John.
>>>>> "JS" == Joshua Slive <jo...@slive.ca> writes:
JS> In recent Apache versions, this is not the case. You must
JS> explictly name HeaderName Header.html
Not yet in 1.3.27 as it seems, but this is valuable information for me
for the future.
JS> or you must turn on Multiviews in the directory's "Options".
This is what I was missing. I am sorry but I failed to understand this
from the docs (mod_autoindex.html#headername)
Now, I got my first local header included, after <body>, which mean
that I'll have to use the SupressHTMLPreamble you mentioned before, as
well as a CGI (or server-side include... I try to grasp the paragraph
on Content negociation) to produce the expected title from a global
setting.
I.e. I may again ask for help.
--
Marc Girod P.O. Box 323 Voice: +358-71 80 25581
Nokia NBI 00045 NOKIA Group Mobile: +358-50 38 78415
Takomo 1 / 4c27 Finland Fax: +358-71 80 61604
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] HeaderName (was: patch to avoid loops in htdig
indexing)
Posted by Joshua Slive <jo...@slive.ca>.
On Tue, 7 Jan 2003, Marc Girod wrote:
>
> I understand that the default and global "HeaderName HEADER" should
> actually find a local HEADER.html (or HEADER) file and insert it. It
> doesn't in my case.
In recent Apache versions, this is not the case. You must explictly name
HeaderName Header.html
or you must turn on Multiviews in the directory's "Options".
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] HeaderName (was: patch to avoid loops in htdig
indexing)
Posted by "John P. Dodge" <do...@cruciate.ca.boeing.com>.
Did you try:
HeaderName HEADER.html
See:
http://httpd.apache.org/docs/mod/mod_autoindex.html#headername
I believe that the action you describe is only applicable to pre v1.3.6
versions and that in post v1.3.6 versions the <filename> argument is a
relative URI.
On 7 Jan 2003, Marc Girod wrote:
> I am still unable to get the HeaderName directive to work as I
> understand it should.
>
> I was able to comment away the FancyIndexing, and to have this apply
> by restarting my httpd service, but I still fail to insert my header
> lines into the emitted preamble.
>
> I understand that the default and global "HeaderName HEADER" should
> actually find a local HEADER.html (or HEADER) file and insert it. It
> doesn't in my case. Could somebody give me an example of a suitable
> contents? Or tell me what is wrong with:
>
> <!-- foo bar -->
> <p>Foo bar
>
> Thanks.
>
> --
> Marc Girod P.O. Box 323 Voice: +358-71 80 25581
> Nokia NBI 00045 NOKIA Group Mobile: +358-50 38 78415
> Takomo 1 / 4c27 Finland Fax: +358-71 80 61604
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
----------------------------------------
"Mon aƩroglisseur est plein d'anguilles"
John P. Dodge
Boeing Shared Services
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] HeaderName (was: patch to avoid loops in htdig indexing)
Posted by Marc Girod <gi...@shire.ntc.nokia.com>.
I am still unable to get the HeaderName directive to work as I
understand it should.
I was able to comment away the FancyIndexing, and to have this apply
by restarting my httpd service, but I still fail to insert my header
lines into the emitted preamble.
I understand that the default and global "HeaderName HEADER" should
actually find a local HEADER.html (or HEADER) file and insert it. It
doesn't in my case. Could somebody give me an example of a suitable
contents? Or tell me what is wrong with:
<!-- foo bar -->
<p>Foo bar
Thanks.
--
Marc Girod P.O. Box 323 Voice: +358-71 80 25581
Nokia NBI 00045 NOKIA Group Mobile: +358-50 38 78415
Takomo 1 / 4c27 Finland Fax: +358-71 80 61604
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] Re: patch to avoid loops in htdig indexing
Posted by Marc Girod <gi...@shire.ntc.nokia.com>.
>>>>> "MG" == Marc Girod <gi...@shire.ntc.nokia.com> writes:
MG> What do I do wrong?
I would add that within the same <IfModule ...> </IfModule> brackets,
I have the(default?):
IndexOptions FancyIndexing NameWidth=*
IndexOptions.
The two files I set as headers (global and local) are both world
readable. They end in a new line.
--
Marc Girod P.O. Box 323 Voice: +358-71 80 25581
Nokia NBI 00045 NOKIA Group Mobile: +358-50 38 78415
Takomo 1 / 4c27 Finland Fax: +358-71 80 61604
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] Re: patch to avoid loops in htdig indexing
Posted by Marc Girod <gi...@shire.ntc.nokia.com>.
>>>>> "JS" == Joshua Slive <jo...@slive.ca> writes:
JS> No, this is not correct. HeaderName can take an absolute-url-path,
JS> as in Headername /globalheader.html
Thanks for correcting me.
I tried again, but failed so far.
I have (by default?) in httpd.conf:
<IfModule mod_autoindex.c>
[...]
ReadmeName README
HeaderName HEADER
[...]
IndexIgnore .??* *~ *# HEADER* README* RCS CVS *,v *,t
[...]
</IfModule>
I added:
HeaderName /autoindex-header.html
Under my DirectoryRoot, I added an autoindex-header.html file with a
comment as contents:
<!-- foo bar -->
Actually, in a directory without an index, I added a HEADER file with
a different contents.
Now, looking at the source of the generated Index, I can find neither
of them.
What do I do wrong?
--
Marc Girod P.O. Box 323 Voice: +358-71 80 25581
Nokia NBI 00045 NOKIA Group Mobile: +358-50 38 78415
Takomo 1 / 4c27 Finland Fax: +358-71 80 61604
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Re: patch to avoid loops in htdig indexing
Posted by Joshua Slive <jo...@slive.ca>.
On Thu, 2 Jan 2003, Marc Girod wrote:
> I have still the same problem as last November now with 1.3.27, i.e.
> htdif when indexing my web site will loop forever when processing
> pages generated by mod_autoindex, because these will follow soft
> links.
>
> I offered a patch to fix the problem by modifying the hard-coded
> preamble emitted by the module, so that it would contain a "robots"
> meta-tag telling htdig to ignore such generated pages.
>
> JS> 3. Use IndexOptions SuppressHTMLPreamble in combination with HeaderName
> JS> to change the html code before the index:
>
> 3) would require a special file per directory, and thus unmaintainable
No, this is not correct. HeaderName can take an absolute-url-path, as in
Headername /globalheader.html
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org