You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Per Jessen <pe...@computer.org> on 2011/01/27 08:45:06 UTC
eval:html_tag_balance - short tags not accepted?
I've just been looking at a mail that got a hit on
HTML_TAG_BALANCE_HEAD due to this:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/1999/REC-html401-19991224/loose.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head/>
<body style="width: 800px">
I can't quite figure out whether the short tag syntax is allowed - the HTML
above was generated by XSLT based on this input:
<head></head>
Other "popular" short tags: <br/> <div/> <p/> - I don't think we should
be judging those to be unbalanced HTML tags.
/Per Jessen, Zürich
Re: eval:html_tag_balance - short tags not accepted?
Posted by "Lawrence @ Rogers" <la...@nl.rogers.com>.
On 28/01/2011 5:28 AM, Per Jessen wrote:
> <script type=""/>
> <style type=""/>
> <fieldset/>
> <legend/>
Sounds like it doesn't care whether the actual tag can be shorthand or
not. It just looks at the structure and decides they're valid, even
though HTML and XHTML specifications say otherwise.
- Lawrence
Re: eval:html_tag_balance - short tags not accepted?
Posted by Per Jessen <pe...@computer.org>.
Lawrence @ Rogers wrote:
> OT: I am curious to know why the W3C Validator considers <p/> to be
> valid, when it goes against every bit of documentation from them I've
> ever read.
It also thinks these are fine:
<div/>
<a/>
<li/>
<td/>
<script type=""/>
<style type=""/>
<fieldset/>
<legend/>
See http://www.jessen.ch/shorttagstest.html
/Per Jessen, Zürich
Re: eval:html_tag_balance - short tags not accepted?
Posted by "Lawrence @ Rogers" <la...@nl.rogers.com>.
On 28/01/2011 4:13 AM, Per Jessen wrote:
>> letely valid
>> > don't you think? It's invalid HTML and contains no content.
> Yes, I agree.
>
All HTML/XHTML tags are required to close (XHTML is supposed to be more
strict, as it was intended to follow XML structure moreso), but only the
ones I mentioned earlier are allowed to self-close with a /. The others
require an explicit closing in order to be considered valid.
Example: <p></p>
OT: I am curious to know why the W3C Validator considers <p/> to be
valid, when it goes against every bit of documentation from them I've
ever read.
I agree with John Hardin though. <head/> is more likely to appear in
spam than ham (ham being legit e-mail that the sender wants to be
readable by even the most broken of HTML renderers, like Outlook 2010).
- Lawrence
Re: eval:html_tag_balance - short tags not accepted?
Posted by Per Jessen <pe...@computer.org>.
Lawrence @ Rogers wrote:
> On 27/01/2011 5:36 PM, Per Jessen wrote:
>>
>>> I believe that the behavior of HTML_TAG_BALANCE_HEAD is valid in
>>> this case, as<head/> is invalid HTML (despite what the validator
>>> says) and should not be used by anyone.
>>
>> True, but html_eval_tag() will fire on _any_ short tag.
>>
>>
>> /Per Jessen, Zürich
>>
>>
>
> If it's firing on <head/> with no content, that's completely valid
> don't you think? It's invalid HTML and contains no content.
Yes, I agree.
> Could you provide an example of a site using <div/> or <p/> shorthand
> tags? I've never seen them before anywhere.
I could point you to one of mine :-), but otherwise I don't know of any
off hand. I don't do much website programming, I'm more focused on
filtering out spam.
> Previously, my understanding has always been that shorthanded closing
> was only allowed for tags that didn't have a closing tag before (such
> as <meta>). The HTML recommendations support this.
In XML, they all have a closing tag though? XSLT will certainly complain
about a missing tag:
XSLT warning: Fatal Error at (file notify-email.html.xsl, line 54, column 51):
Expected end of tag 'meta' (notify-email.html.xsl, line 54, column 51)
SAXParseException: Expected end of tag 'meta' (notify-email.html.xsl, line 54, column 51)
(I asked for XHTML Transitional output).
> Perhaps there is further work to be done in SA regarding handling HTML
> balancing, but <head/> is pointless to test for as it has no reason or
> possible use in the real world.
Agree.
> If html_eval_tag() is firing on any short tag, and not just the
> invalid example code, that would signal a possible bug and
> investigation.
Actually no - whilst html_eval_tag() does fire on any short tag, you
have specify which tag you want to check for. I hadn't noticed that
yesterday. The 3.2.5 ruleset only checks on body and head. Problem
solved. Thanks for the chat anyway.
/Per Jessen, Zürich
Re: eval:html_tag_balance - short tags not accepted?
Posted by "Lawrence @ Rogers" <la...@nl.rogers.com>.
On 27/01/2011 5:36 PM, Per Jessen wrote:
> Lawrence @ Rogers wrote:
>
>> <div/> and<p/> may pass the validator, but that is most certainly a
>> bug. A quick look through the XHTML 1.0 DTD's reveals only ten tags
>> that may be closed using the short form, and I am unable to find any
>> documentation on the W3C web site to support anything otherwise.
>>
>> <area />
>> <base />
>> <br />
>> <col />
>> <hr />
>> <img />
>> <input />
>> <link />
>> <meta />
>> <param />
>>
>> Using any other shorthandled elements would result in HTML rendering
>> engines choking and giving unpredictable results.
> I'm not so sure - I think relatively modern renderers are quite capable
> of dealing with both<div/> and<p/> without causing any problems.
> <p/> instead of<p></p> is not unusual.
>
>> What I was suggesting is that your belief is flawed because your test
>> was flawed itself. No e-mail will ever be just<head></head>. Ignoring
>> the fact that a<title> tag is required as a minimum (although many
>> e-mails probably omit it), the<head/> form is invalid as well.
> Accepted, but it doesn't change the problem in html_eval_tag() - the
> code doesn't attempt to validate html, it just does a simple regex
> check for a balanced tag, but doesn't accept or ignore the short tag
> version with no content.
>
>> I believe that the behavior of HTML_TAG_BALANCE_HEAD is valid in this
>> case, as<head/> is invalid HTML (despite what the validator says) and
>> should not be used by anyone.
> True, but html_eval_tag() will fire on _any_ short tag.
>
>
> /Per Jessen, Zürich
>
>
The problem is that, the majority of HTML e-mails out there are being
handled by older HTML engines (like IE7 or worse)
If it's firing on <head/> with no content, that's completely valid don't
you think? It's invalid HTML and contains no content. That throws the
balance off.
Any HTML coder will never assume that people are using programs with
modern HTML capabilities. Most of the world is only now finally letting
IE6 die.
Could you provide an example of a site using <div/> or <p/> shorthand
tags? I've never seen them before anywhere.
Previously, my understanding has always been that shorthanded closing
was only allowed for tags that didn't have a closing tag before (such as
<meta>). The HTML recommendations support this.
Perhaps there is further work to be done in SA regarding handling HTML
balancing, but <head/> is pointless to test for as it has no reason or
possible use in the real world.
If html_eval_tag() is firing on any short tag, and not just the invalid
example code, that would signal a possible bug and investigation.
Cheers,
Lawrence
Re: eval:html_tag_balance - short tags not accepted?
Posted by Per Jessen <pe...@computer.org>.
Lawrence @ Rogers wrote:
> <div/> and <p/> may pass the validator, but that is most certainly a
> bug. A quick look through the XHTML 1.0 DTD's reveals only ten tags
> that may be closed using the short form, and I am unable to find any
> documentation on the W3C web site to support anything otherwise.
>
> <area />
> <base />
> <br />
> <col />
> <hr />
> <img />
> <input />
> <link />
> <meta />
> <param />
>
> Using any other shorthandled elements would result in HTML rendering
> engines choking and giving unpredictable results.
I'm not so sure - I think relatively modern renderers are quite capable
of dealing with both <div/> and <p/> without causing any problems.
<p/> instead of <p></p> is not unusual.
> What I was suggesting is that your belief is flawed because your test
> was flawed itself. No e-mail will ever be just <head></head>. Ignoring
> the fact that a <title> tag is required as a minimum (although many
> e-mails probably omit it), the <head/> form is invalid as well.
Accepted, but it doesn't change the problem in html_eval_tag() - the
code doesn't attempt to validate html, it just does a simple regex
check for a balanced tag, but doesn't accept or ignore the short tag
version with no content.
> I believe that the behavior of HTML_TAG_BALANCE_HEAD is valid in this
> case, as <head/> is invalid HTML (despite what the validator says) and
> should not be used by anyone.
True, but html_eval_tag() will fire on _any_ short tag.
/Per Jessen, Zürich
Re: eval:html_tag_balance - short tags not accepted?
Posted by "Lawrence @ Rogers" <la...@nl.rogers.com>.
On 27/01/2011 4:43 PM, Per Jessen wrote:
> Lawrence @ Rogers wrote:
>
>> On 27/01/2011 4:15 AM, Per Jessen wrote:
>>> I've just been looking at a mail that got a hit on
>>> HTML_TAG_BALANCE_HEAD due to this:
>>>
>>> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
>>> "http://www.w3.org/TR/1999/REC-html401-19991224/loose.dtd"> <html
>>> xmlns="http://www.w3.org/1999/xhtml"> <head/>
>>> <body style="width: 800px">
>>>
>>> I can't quite figure out whether the short tag syntax is allowed -
>>> the HTML above was generated by XSLT based on this input:
>>>
>>> <head></head>
>>>
>>> Other "popular" short tags:<br/> <div/> <p/> - I don't think we
>>> should be judging those to be unbalanced HTML tags.
>>>
>>>
>>> /Per Jessen, Zürich
>>>
>>>
>> As a person who writes HTML/XHTML every single day, there are several
>> flaws in your argument:
>>
>> -<head/> is not valid HTML or XHTML (in any version)
> Ah, because it needs at least<title>. Okay.
>
>> - HTML 4.01 Transitional doesn't allow for an XHTML xmlns attribute,
>> nor does it permit "short tags"
> Irrelevant for this issue. Spamassassin doesn't care about the DTD when
> it's evaluating for unbalanced tags. Use your imagtion and put any
> suitable DTD instead.
>
>> - The only "valid" short tag that you mentioned is<br />.<div/> and
>> <p/> are not
> They're certainly all valid in XHTML. (the validator at w3c says ok for
> both).
>
>> - Using a short tag without a space between the name and the / is also
>> not recommended as it causes problems for older browsers and poorly
>> written HTML parsers.
> Irrelevant for this issue.
>
>> You appear to have made a flawed statement based upon a flawed study
> Gee, what's with the hostility? I never made an argument, I asked a
> simple question.
>
>> (no HTML e-mail will ever be just a<head></head> combination)
> I didn't suggest that.
>
>
> /Per Jessen, Zürich
>
>
Hi Per,
I did not intend for my message to be hostile in any way. My apologies
if my terse tone came across that way.
<div/> and <p/> may pass the validator, but that is most certainly a
bug. A quick look through the XHTML 1.0 DTD's reveals only ten tags that
may be closed using the short form, and I am unable to find any
documentation on the W3C web site to support anything otherwise.
<area />
<base />
<br />
<col />
<hr />
<img />
<input />
<link />
<meta />
<param />
Using any other shorthandled elements would result in HTML rendering
engines choking and giving unpredictable results.
What I was suggesting is that your belief is flawed because your test
was flawed itself. No e-mail will ever be just <head></head>. Ignoring
the fact that a <title> tag is required as a minimum (although many
e-mails probably omit it), the <head/> form is invalid as well.
SpamAssassin may not care about DTDs and the like, but HTML rendering
engines such as the one used in Internet Explorer (where people may be
using webmail clients) and Outlook (which recently reverted from IE's
engine to a crappy one used in Microsoft Word) do care. Programs who
send HTML e-mails are going to do at least the bare minimum to ensure
their messages are displayed and readable, and they will know that
Internet Explorer's HTML rendering engine is what will most likely be
parsing the HTML they supply. This almost ensures that a HTML message
will be at least like this
<html>
<head></head>
<body>
Some content here
</body>
</html>
Even spammers know that using anything less than the above runs a very
real risk of the message being unable to be displayed, which would make
the e-mail completely pointless.
I believe that the behavior of HTML_TAG_BALANCE_HEAD is valid in this
case, as <head/> is invalid HTML (despite what the validator says) and
should not be used by anyone.
Regards,
Lawrence
(For what it's worth, <div/> and <p/> are not "popular". I've never seen
them used on any legit site)
Re: eval:html_tag_balance - short tags not accepted?
Posted by John Hardin <jh...@impsec.org>.
On Thu, 27 Jan 2011, Per Jessen wrote:
> Lawrence @ Rogers wrote:
>
>> On 27/01/2011 4:15 AM, Per Jessen wrote:
>>>
>>> Other "popular" short tags:<br/> <div/> <p/> - I don't think we
>>> should be judging those to be unbalanced HTML tags.
>
>> - Using a short tag without a space between the name and the / is also
>> not recommended as it causes problems for older browsers and poorly
>> written HTML parsers.
>
> Irrelevant for this issue.
Totally relevant if it appears more in spam than in ham. We're not an
XHTML validation service.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Efficiency can magnify good, but it magnifies evil just as well.
So, we should not be surprised to find that modern electronic
communication magnifies stupidity as *efficiently* as it magnifies
intelligence. -- Robert A. Matern
-----------------------------------------------------------------------
Today: the 44th anniversary of the loss of Apollo 1
Re: eval:html_tag_balance - short tags not accepted?
Posted by Per Jessen <pe...@computer.org>.
Lawrence @ Rogers wrote:
> On 27/01/2011 4:15 AM, Per Jessen wrote:
>> I've just been looking at a mail that got a hit on
>> HTML_TAG_BALANCE_HEAD due to this:
>>
>> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
>> "http://www.w3.org/TR/1999/REC-html401-19991224/loose.dtd"> <html
>> xmlns="http://www.w3.org/1999/xhtml"> <head/>
>> <body style="width: 800px">
>>
>> I can't quite figure out whether the short tag syntax is allowed -
>> the HTML above was generated by XSLT based on this input:
>>
>> <head></head>
>>
>> Other "popular" short tags:<br/> <div/> <p/> - I don't think we
>> should be judging those to be unbalanced HTML tags.
>>
>>
>> /Per Jessen, Zürich
>>
>>
> As a person who writes HTML/XHTML every single day, there are several
> flaws in your argument:
>
> - <head/> is not valid HTML or XHTML (in any version)
Ah, because it needs at least <title>. Okay.
> - HTML 4.01 Transitional doesn't allow for an XHTML xmlns attribute,
> nor does it permit "short tags"
Irrelevant for this issue. Spamassassin doesn't care about the DTD when
it's evaluating for unbalanced tags. Use your imagtion and put any
suitable DTD instead.
> - The only "valid" short tag that you mentioned is <br />. <div/> and
> <p/> are not
They're certainly all valid in XHTML. (the validator at w3c says ok for
both).
> - Using a short tag without a space between the name and the / is also
> not recommended as it causes problems for older browsers and poorly
> written HTML parsers.
Irrelevant for this issue.
> You appear to have made a flawed statement based upon a flawed study
Gee, what's with the hostility? I never made an argument, I asked a
simple question.
> (no HTML e-mail will ever be just a <head></head> combination)
I didn't suggest that.
/Per Jessen, Zürich
Re: eval:html_tag_balance - short tags not accepted?
Posted by "Lawrence @ Rogers" <la...@nl.rogers.com>.
On 27/01/2011 4:15 AM, Per Jessen wrote:
> I've just been looking at a mail that got a hit on
> HTML_TAG_BALANCE_HEAD due to this:
>
> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/1999/REC-html401-19991224/loose.dtd">
> <html xmlns="http://www.w3.org/1999/xhtml">
> <head/>
> <body style="width: 800px">
>
> I can't quite figure out whether the short tag syntax is allowed - the HTML
> above was generated by XSLT based on this input:
>
> <head></head>
>
> Other "popular" short tags:<br/> <div/> <p/> - I don't think we should
> be judging those to be unbalanced HTML tags.
>
>
> /Per Jessen, Zürich
>
>
As a person who writes HTML/XHTML every single day, there are several
flaws in your argument:
- <head/> is not valid HTML or XHTML (in any version)
- HTML 4.01 Transitional doesn't allow for an XHTML xmlns attribute, nor
does it permit "short tags"
- The only "valid" short tag that you mentioned is <br />. <div/> and
<p/> are not
- Using a short tag without a space between the name and the / is also
not recommended as it causes problems for older browsers and poorly
written HTML parsers.
You appear to have made a flawed statement based upon a flawed study (no
HTML e-mail will ever be just a <head></head> combination)
Regards,
Lawrence