You are viewing a plain text version of this content. The canonical link for it is here.
Posted to j-users@xerces.apache.org by Martin Jericho <ma...@yahoo.com.au> on 2002/08/20 09:23:27 UTC

Getting the position of a node in the input stream (using Neko)

This is a newbie question so I hope it's not too dumb.

I want to parse an HTML document using Neko, and all I want to find out is the character position of particular nodes in the input stream.  I saw the XMLLocator interface, which I presume allows me to find out the line number and the column within the line, but it doesn't include the position from the start of the stream.

Is there an example somewhere of doing this with Neko?  I would really appreciate it if someone could help me with this, as I have nearly spent the whole day trying to figure it out from the source code.

Thanks in advance
Martin

Re: Default decoding in Neko

Posted by Andy Clark <an...@apache.org>.
Takumi Fujiwara wrote:
> I know I can change it if I want, I just want to understand why 
> "Windows-1252" is choosen instead of UTF-8?

If the default were UTF-8, the reader would throw
an exception on many pages. Any page that contains
an ISO Latin 1 character (above the typical ASCII
range), would make the UTF-8 reader die.

ISO Latin 1 (or Windows-1252) are safe defaults
because every possible byte is acceptable. But, if
the high bit is set on a byte read by the UTF-8
reader, then it assumes that it matches the proper
UTF-8 sequence and when it doesn't, it throws an
exception.

As long as the page specifies its encoding using
the http-equiv meta tag, then NekoHTML will change
to the correct reader and everything will be fine.
But, if it does *not*, then we need to use a "safe"
encoding. Therefore, I chose Windows-1252.

-- 
Andy Clark * andyc@apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org


Default decoding in Neko

Posted by Takumi Fujiwara <tr...@yahoo.com>.
Hi, 
Could someone pleasea tell me why the default decoding in Neko is Windows-1252 instead of UTF-8? I want to parse pages like yahoo.jp, yahoo.co.jp, dk.yahoo.com, it.yahoo.com, hk.yahoo.com
I know I can change it if I want, I just want to understand why "Windows-1252" is choosen instead of UTF-8?
Thank you.
Sam



---------------------------------
Do You Yahoo!?
Yahoo! Finance - Get real-time stock quotes

RE: Problem: White space is required between the version and the encoding declaration.

Posted by Tom Wang <to...@panscopic.com>.
Will,

Encoding is optional for xml decl.  I am as confused as you're on this.
Anyone else has input on this?

Tom Wang
Panscopic Corporation
Web Reporting, Just Add Data
http://www.panscopic.com/


> -----Original Message-----
> From: Will Hartung [mailto:willh@msoft.com]
> Sent: Friday, August 23, 2002 2:41 PM
> To: xerces-j-user@xml.apache.org; tomw@panscopic.com
> Subject: Re: Problem: White space is required between the version and
> the encoding declaration.
>
>
> So, is the 'encoding' portion required? I thought it was
> optional. Or is it
> only required for DTDs? If I simply add the 'encoding' clause to the DTD,
> then it works, even though the actual XML file has the same, simple <?xml
> version="1.0"?> header.
>
> Confused and peplexed.
>
> Regards,
>
> Will Hartung
> (willh@msoft.com)
>
> ----- Original Message -----
> From: "Tom Wang" <to...@panscopic.com>
> To: <xe...@xml.apache.org>
> Sent: Friday, August 23, 2002 12:40 PM
> Subject: RE: Problem: White space is required between the version and the
> encoding declaration.
>
>
> > Will,
> >
> > You can remove the xml decl or force adding the encoding info:
> >
> > <?xml version="1.0" encoding="UTF-8" ?>
> >
> > Tom Wang
> > Panscopic Corporation
> > Web Reporting, Just Add Data
> > http://www.panscopic.com/
> >
> >
> > > -----Original Message-----
> > > From: Will Hartung [mailto:willh@msoft.com]
> > > Sent: Thursday, August 22, 2002 6:13 PM
> > > To: xerces-j-user@xml.apache.org
> > > Subject: Problem: White space is required between the version and the
> > > encoding declaration.
> > >
> > >
> > >
> > > I'm getting this error while I'm trying to parse a simple XML
> document.
> > >
> > > SAXParseException: White space is required between the version and the
> > > encoding declaration.
> > >
> > > The error appears to be in the DTD that it doesn't like the <?xml
> > > version="1.0"?> line, and I don't understand what the issue is. Can
> anyone
> > > provide any hints? I get this error on both 1.4.3 and 1.4.4.
> > >
> > > XML I'm trying to parse:
> > > <?xml version="1.0"?>
> > >
> > > <!DOCTYPE Basic SYSTEM "file:basic.dtd">
> > >
> > > <Basic>
> > >     <anInt>1</anInt>
> > >     <aDouble>2.4</aDouble>
> > >     <aString>This is a string</aString>
> > >     <anotherString>ANother string</anotherString>
> > > </Basic>
> > >
> > > The DTD (in basic.dtd):
> > > <?xml version="1.0"?>
> > > <!ELEMENT Basic (anInt?, aDouble?, aString?, anotherString?)>
> > >
> > > <!ELEMENT anInt (#PCDATA)>
> > > <!ELEMENT aDouble (#PCDATA)>
> > > <!ELEMENT aString (#PCDATA)>
> > > <!ELEMENT anotherString (#PCDATA)>
> > >
> > > Thanx!
> > >
> > > Best Regards,
> > >
> > > Will Hartung
> > > (willh@msoft.com)
> > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
> > > For additional commands, e-mail: xerces-j-user-help@xml.apache.org
> > >
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
> > For additional commands, e-mail: xerces-j-user-help@xml.apache.org
> >
> >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-j-user-help@xml.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org


Re: Problem: White space is required between the version and the encoding declaration.

Posted by Will Hartung <wi...@msoft.com>.
So, is the 'encoding' portion required? I thought it was optional. Or is it
only required for DTDs? If I simply add the 'encoding' clause to the DTD,
then it works, even though the actual XML file has the same, simple <?xml
version="1.0"?> header.

Confused and peplexed.

Regards,

Will Hartung
(willh@msoft.com)

----- Original Message -----
From: "Tom Wang" <to...@panscopic.com>
To: <xe...@xml.apache.org>
Sent: Friday, August 23, 2002 12:40 PM
Subject: RE: Problem: White space is required between the version and the
encoding declaration.


> Will,
>
> You can remove the xml decl or force adding the encoding info:
>
> <?xml version="1.0" encoding="UTF-8" ?>
>
> Tom Wang
> Panscopic Corporation
> Web Reporting, Just Add Data
> http://www.panscopic.com/
>
>
> > -----Original Message-----
> > From: Will Hartung [mailto:willh@msoft.com]
> > Sent: Thursday, August 22, 2002 6:13 PM
> > To: xerces-j-user@xml.apache.org
> > Subject: Problem: White space is required between the version and the
> > encoding declaration.
> >
> >
> >
> > I'm getting this error while I'm trying to parse a simple XML document.
> >
> > SAXParseException: White space is required between the version and the
> > encoding declaration.
> >
> > The error appears to be in the DTD that it doesn't like the <?xml
> > version="1.0"?> line, and I don't understand what the issue is. Can
anyone
> > provide any hints? I get this error on both 1.4.3 and 1.4.4.
> >
> > XML I'm trying to parse:
> > <?xml version="1.0"?>
> >
> > <!DOCTYPE Basic SYSTEM "file:basic.dtd">
> >
> > <Basic>
> >     <anInt>1</anInt>
> >     <aDouble>2.4</aDouble>
> >     <aString>This is a string</aString>
> >     <anotherString>ANother string</anotherString>
> > </Basic>
> >
> > The DTD (in basic.dtd):
> > <?xml version="1.0"?>
> > <!ELEMENT Basic (anInt?, aDouble?, aString?, anotherString?)>
> >
> > <!ELEMENT anInt (#PCDATA)>
> > <!ELEMENT aDouble (#PCDATA)>
> > <!ELEMENT aString (#PCDATA)>
> > <!ELEMENT anotherString (#PCDATA)>
> >
> > Thanx!
> >
> > Best Regards,
> >
> > Will Hartung
> > (willh@msoft.com)
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
> > For additional commands, e-mail: xerces-j-user-help@xml.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-j-user-help@xml.apache.org
>
>



---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org


RE: Problem: White space is required between the version and the encoding declaration.

Posted by Tom Wang <to...@panscopic.com>.
Will,

You can remove the xml decl or force adding the encoding info:

<?xml version="1.0" encoding="UTF-8" ?>

Tom Wang
Panscopic Corporation
Web Reporting, Just Add Data
http://www.panscopic.com/


> -----Original Message-----
> From: Will Hartung [mailto:willh@msoft.com]
> Sent: Thursday, August 22, 2002 6:13 PM
> To: xerces-j-user@xml.apache.org
> Subject: Problem: White space is required between the version and the
> encoding declaration.
>
>
>
> I'm getting this error while I'm trying to parse a simple XML document.
>
> SAXParseException: White space is required between the version and the
> encoding declaration.
>
> The error appears to be in the DTD that it doesn't like the <?xml
> version="1.0"?> line, and I don't understand what the issue is. Can anyone
> provide any hints? I get this error on both 1.4.3 and 1.4.4.
>
> XML I'm trying to parse:
> <?xml version="1.0"?>
>
> <!DOCTYPE Basic SYSTEM "file:basic.dtd">
>
> <Basic>
>     <anInt>1</anInt>
>     <aDouble>2.4</aDouble>
>     <aString>This is a string</aString>
>     <anotherString>ANother string</anotherString>
> </Basic>
>
> The DTD (in basic.dtd):
> <?xml version="1.0"?>
> <!ELEMENT Basic (anInt?, aDouble?, aString?, anotherString?)>
>
> <!ELEMENT anInt (#PCDATA)>
> <!ELEMENT aDouble (#PCDATA)>
> <!ELEMENT aString (#PCDATA)>
> <!ELEMENT anotherString (#PCDATA)>
>
> Thanx!
>
> Best Regards,
>
> Will Hartung
> (willh@msoft.com)
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-j-user-help@xml.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org


Problem: White space is required between the version and the encoding declaration.

Posted by Will Hartung <wi...@msoft.com>.
I'm getting this error while I'm trying to parse a simple XML document.

SAXParseException: White space is required between the version and the
encoding declaration.

The error appears to be in the DTD that it doesn't like the <?xml
version="1.0"?> line, and I don't understand what the issue is. Can anyone
provide any hints? I get this error on both 1.4.3 and 1.4.4.

XML I'm trying to parse:
<?xml version="1.0"?>

<!DOCTYPE Basic SYSTEM "file:basic.dtd">

<Basic>
    <anInt>1</anInt>
    <aDouble>2.4</aDouble>
    <aString>This is a string</aString>
    <anotherString>ANother string</anotherString>
</Basic>

The DTD (in basic.dtd):
<?xml version="1.0"?>
<!ELEMENT Basic (anInt?, aDouble?, aString?, anotherString?)>

<!ELEMENT anInt (#PCDATA)>
<!ELEMENT aDouble (#PCDATA)>
<!ELEMENT aString (#PCDATA)>
<!ELEMENT anotherString (#PCDATA)>

Thanx!

Best Regards,

Will Hartung
(willh@msoft.com)




---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org


Re: Getting the position of a node in the input stream (using Neko)

Posted by Andy Clark <an...@apache.org>.
Martin Jericho wrote:
>  > to implement the same thing in NekoHTML. But in neither
>  > case do we track "character offsets", which I think has
>  > limited usefulness but others disagree.
> Hopefully my arguments below will help to convince you of their usefulness.

I can understand the cases in which people would like to
be able to do this but I also realize what it would take
to implement it. ;)

The "limited usefulness" that I was referring to was the
fact that reporting character offsets only works if the
parsed source is already a character stream. If it's
anything else (say a byte stream in UTF8 or Shift_JIS)
then the application can't map those offsets back to the
source without re-reading the file.

>  > Because "no-change" has the potential of producing XML
>  > that is not well-formed. And the whole purpose of Neko-
>  > HTML is to parse HTML and make it appear as XML.
> So Neko has to do this because otherwise the underlying xerces parser 
> would not be able to parse it.  Is that right?  This would not be of any 

I don't use the Xerces2 parser to implement NekoHTML --
I only use the XNI framework and some utility classes.
The NekoHTML scanner is written completely from scratch
to be able to handle HTML

> concern to me anyway if character positions were reported, I was just 
> using it as an example to demonstrate that you can't get Neko to output 
> the orginal source unchanged.

Ugh. ;)

>  > Please let me know more detail about these bugs so
>  > that I can fix them. Minimal sample files would be
>  > preferable.
>  
> I have attached the relevant files.

Thanks for attaching those! I fixed the problem and
have included your sample input file (albeit a little
bit more stripped down) in my set of regression tests.
So if I ever break it, I'll know right away. :)

I changed the behavior of <COL> to *not* automatically
insert a <COLGROUP> as its parent. Is this the behavior
you were expecting? Also, as a general question, do
you think that NekoHTML should insert a <TBODY> parent
for <TR> elements? I notice that Mozilla inserts one.
<aside>The DOM Inspector rules!</aside>

> I get the feeling that this would have to be implemented in the XNI 
> framework rather than as a Neko improvement.  I would love to get 

If it were to be added, the place would be in the
XNI interfaces which would then be implemented in
NekoHTML.

> buffer, not the original source.  In fact the only HTML parser I have 
> found which does it is the one in the swing package, although I still 
> haven't tested it properly.  If that doesn't do what I want, I might 
> even have to write my own from scratch.

Well, whatever you do, take advantage of all of the
code that's available. I hope NekoHTML can be of use
but if not then that's ok, too.

I'm trying to get a new version of NekoHTML posted
"real soon now". I will make an announcement when
it's ready.

-- 
Andy Clark * andyc@apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org


Validating Sample Data for xerces 2.1.0 does not work!

Posted by Richard Heintze <si...@yahoo.com>.
I just upgraded to xerces 2.1.0 and I'm getting the
same error from my data file as I am from the sample
data. This is also the same error I was getting from
2.0.1.

Previously, when I posted this error message, I was
using bash and had a lot of double backslashes and
someone kindly responded and suggested I use the
default shell for Windows 2000. So here it is.

  What I am doing wrong?

          Thanks,
                 Siegfried

Microsoft(R) Windows NT(TM)
(C) Copyright 1985-1996 Microsoft Corp.

g:\xerces-2_1_0\data>java -cp
.;g:\xerces-2_1_0\xercesSamples.jar;g:\xerces-2_1_0\xercesImpl.jar;g:\xerces-2_1_0\xmlParserAPIs.jar
sax.Counter -s -S -v personal-schema.xml
[Error] personal-schema.xml:8:50: Document root
element "personnel", must match DOCTYPE root "null".
[Error] personal-schema.xml:8:50: Document is invalid:
no grammar found.
personal-schema.xml: 461 ms (37 elems, 13 attrs, 0
spaces, 268 chars)

g:\xerces-2_1_0\data>

__________________________________________________
Do You Yahoo!?
Yahoo! Finance - Get real-time stock quotes
http://finance.yahoo.com

---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org


Re: Getting the position of a node in the input stream (using Neko)

Posted by Martin Jericho <ma...@yahoo.com.au>.
----- Original Message -----
From: "Andy Clark" <an...@apache.org>
To: <xe...@xml.apache.org>
Sent: Thursday, August 22, 2002 2:41 AM
Subject: Re: Getting the position of a node in the input stream (using Neko)


> Martin Jericho wrote:
> > Thanks for your quick reply Andy.  I was half expecting it to be this
sort
> > of problem, but I was then puzzled that you can track the line and
column
> > number.
>
> That's standard locator information provided by the SAX
> interfaces. So we implement that in Xerces and I decided
> to implement the same thing in NekoHTML. But in neither
> case do we track "character offsets", which I think has
> limited usefulness but others disagree.

Hopefully my arguments below will help to convince you of their usefulness.

>
> > 1.  Insert some code at particular points, but it is imperative that the
> > rest of the html remains EXACTLY the same.  This is not possible using
the
> > Writer filter, as some properties, such as
> > http://cyberneko.org/html/properties/names/elems, do not have a
"no-change"
> > option.  Even if this option were present, there are still some changes
made
>
> Because "no-change" has the potential of producing XML
> that is not well-formed. And the whole purpose of Neko-
> HTML is to parse HTML and make it appear as XML.

So Neko has to do this because otherwise the underlying xerces parser would not be able to parse it.  Is that right?  This would not be of any concern to me anyway if character positions were reported, I was just using it as an example to demonstrate that you can't get Neko to output the orginal source unchanged.

> Take
> the following instance:
>
>    <tAbLe> ... </TaBlE>
>
> How do you handle this and still make it well formed
> in an XML sense? NekoHTML lets you transform these to
> uppercase, lowercase, or just to match the end tag w/
> whatever the start tag is. The latter option will
> produce the following:
>
>    <tAbLe> ... </tAbLe>
>
> > to the output.  One example is that it inserts a <COLGROUP> element, and
> > </col> tags in the wrong places. (sorry I haven't had time to report
these
>
> Please let me know more detail about these bugs so
> that I can fix them. Minimal sample files would be
> preferable.

I have attached the relevant files.

>
> > bugs).  The point is that I don't want to have to worry about Neko being
> > able to regenerate the original source verbatim, all I want is the
character
> > positions so I can insert the code myself.
>
> Is your HTML string generated? Or serialized into a
> String object?

The HTML is created by an end user, probably using some kind of GUI tool.  I want to do a one-off parse of it to insert some velocity tags and store it in a database for later generation of dynamic content.  The reason it is so important that the HTML remains exactly the same (apart from the deliberate changes), is that the GUI HTML editors are fine-tuned for all the little idiosyncracies of any client.  The end user may want the page to display in Netscape 4.5, but if Neko goes and makes even the slightest of changes, we know all to well how easy Netscape is to break.

Neko is an excellent HTML parser, but its usefulness is limited by the fact that the developer doesn't have full control over the output.  Having the original character offsets in an Augmentations object (?) would provide the ultimate flexibility that would make it useful in many more applications.

I get the feeling that this would have to be implemented in the XNI framework rather than as a Neko improvement.  I would love to get involved myself but like most other people have too much work on my plate already.

>
> > Is there any way of doing this, assuming a unicode character set?
>
> Not unless you've stored the offsets for each separate
> line within the document String. Then you could use the
> line/column information.
>
> > By the way, congratulations and thanks for your efforts so far.  I can
see
> > Neko being useful to me in future projects, but for my current problem I
may
> > have to use JTidy.
>
> By all means. JTidy is a really nice tool.

I have since found out that JTidy doesn't do this either.  The fields I thought were character offsets were just in relation to an internal buffer, not the original source.  In fact the only HTML parser I have found which does it is the one in the swing package, although I still haven't tested it properly.  If that doesn't do what I want, I might even have to write my own from scratch.

> > Also, I very nearly didn't find Neko during my search for HTML parsers.
Two
> > reasons for this:
> > - You have to dig pretty deep into the Xerces documentation to find out
that
> > it is capable of parsing HTML.  Even the FAQ says it is not possible!  I
>
> Xerces is *not* capable of parsing HTML because HTML is
> not XML. However, we do mention that an HTML parser
> configuration is possible with the XNI framework. And
> perhaps NekoHTML will make it into the Xerces download
> or at least as a kind of sub-project that's explicitly
> mentioned in the Xerces pages.
>
> > - The Neko home page
(http://www.apache.org/~andyc/neko/doc/html/index.html)
> > does not contain any meta keywords or anything to make it easy for
search
> > engines to find it. Have you registered it with any search engines?
>
> Not really. It's rather quiet 'cause I concentrate on
> Xerces developers. However, I do announce on Freshmeat
> so that make sa lot of people aware of its existence.
>
> > Something else I found to be quite ironic, considering one of the prime
uses
> > of Neko, is that it's homepage isn't even valid HTML!
>
> Ummm... that's sort of the point. To show that NekoHTML
> can even parse its own sloppy documentation. :)
>
> > I found it when google came up with one of your posts on a mailing list
> > archive.  It would be a pity if people start using inferior products
simply
> > because they don't know Neko exists.
>
> True. I should work harder to let people know that it
> is available.
>
> Thanks for the insight and I'd really like to hear more
> about the bugs you're experiencing. Type at you later...
>
> --
> Andy Clark * andyc@apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-j-user-help@xml.apache.org


Re: Getting the position of a node in the input stream (using Neko)

Posted by Andy Clark <an...@apache.org>.
Martin Jericho wrote:
> Thanks for your quick reply Andy.  I was half expecting it to be this sort
> of problem, but I was then puzzled that you can track the line and column
> number.

That's standard locator information provided by the SAX
interfaces. So we implement that in Xerces and I decided
to implement the same thing in NekoHTML. But in neither
case do we track "character offsets", which I think has
limited usefulness but others disagree.

> 1.  Insert some code at particular points, but it is imperative that the
> rest of the html remains EXACTLY the same.  This is not possible using the
> Writer filter, as some properties, such as
> http://cyberneko.org/html/properties/names/elems, do not have a "no-change"
> option.  Even if this option were present, there are still some changes made

Because "no-change" has the potential of producing XML
that is not well-formed. And the whole purpose of Neko-
HTML is to parse HTML and make it appear as XML. Take
the following instance:

   <tAbLe> ... </TaBlE>

How do you handle this and still make it well formed
in an XML sense? NekoHTML lets you transform these to
uppercase, lowercase, or just to match the end tag w/
whatever the start tag is. The latter option will
produce the following:

   <tAbLe> ... </tAbLe>

> to the output.  One example is that it inserts a <COLGROUP> element, and
> </col> tags in the wrong places. (sorry I haven't had time to report these

Please let me know more detail about these bugs so
that I can fix them. Minimal sample files would be
preferable.

> bugs).  The point is that I don't want to have to worry about Neko being
> able to regenerate the original source verbatim, all I want is the character
> positions so I can insert the code myself.

Is your HTML string generated? Or serialized into a
String object?

> Is there any way of doing this, assuming a unicode character set?

Not unless you've stored the offsets for each separate
line within the document String. Then you could use the
line/column information.

> By the way, congratulations and thanks for your efforts so far.  I can see
> Neko being useful to me in future projects, but for my current problem I may
> have to use JTidy.

By all means. JTidy is a really nice tool.

> Also, I very nearly didn't find Neko during my search for HTML parsers.  Two
> reasons for this:
> - You have to dig pretty deep into the Xerces documentation to find out that
> it is capable of parsing HTML.  Even the FAQ says it is not possible!  I

Xerces is *not* capable of parsing HTML because HTML is
not XML. However, we do mention that an HTML parser
configuration is possible with the XNI framework. And
perhaps NekoHTML will make it into the Xerces download
or at least as a kind of sub-project that's explicitly
mentioned in the Xerces pages.

> - The Neko home page (http://www.apache.org/~andyc/neko/doc/html/index.html)
> does not contain any meta keywords or anything to make it easy for search
> engines to find it. Have you registered it with any search engines?

Not really. It's rather quiet 'cause I concentrate on
Xerces developers. However, I do announce on Freshmeat
so that make sa lot of people aware of its existence.

> Something else I found to be quite ironic, considering one of the prime uses
> of Neko, is that it's homepage isn't even valid HTML!

Ummm... that's sort of the point. To show that NekoHTML
can even parse its own sloppy documentation. :)

> I found it when google came up with one of your posts on a mailing list
> archive.  It would be a pity if people start using inferior products simply
> because they don't know Neko exists.

True. I should work harder to let people know that it
is available.

Thanks for the insight and I'd really like to hear more
about the bugs you're experiencing. Type at you later...

-- 
Andy Clark * andyc@apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org


Re: Getting the position of a node in the input stream (using Neko)

Posted by Martin Jericho <ma...@yahoo.com.au>.
Thanks for your quick reply Andy.  I was half expecting it to be this sort
of problem, but I was then puzzled that you can track the line and column
number.

My source is always going to be unicode as I will be parsing a String.  The
reason I want to find out the character positions is because I want to do
the following things:

1.  Insert some code at particular points, but it is imperative that the
rest of the html remains EXACTLY the same.  This is not possible using the
Writer filter, as some properties, such as
http://cyberneko.org/html/properties/names/elems, do not have a "no-change"
option.  Even if this option were present, there are still some changes made
to the output.  One example is that it inserts a <COLGROUP> element, and
</col> tags in the wrong places. (sorry I haven't had time to report these
bugs).  The point is that I don't want to have to worry about Neko being
able to regenerate the original source verbatim, all I want is the character
positions so I can insert the code myself.

2.  I only want to parse the file once, but make some modifications at a
later stage.  I would prefer to persist only a few integer character
positions rather than the whole document node tree.

Is there any way of doing this, assuming a unicode character set?

By the way, congratulations and thanks for your efforts so far.  I can see
Neko being useful to me in future projects, but for my current problem I may
have to use JTidy.

Also, I very nearly didn't find Neko during my search for HTML parsers.  Two
reasons for this:
- You have to dig pretty deep into the Xerces documentation to find out that
it is capable of parsing HTML.  Even the FAQ says it is not possible!  I
think you should advertise the fact on the front page.
- The Neko home page (http://www.apache.org/~andyc/neko/doc/html/index.html)
does not contain any meta keywords or anything to make it easy for search
engines to find it. Have you registered it with any search engines?
Something else I found to be quite ironic, considering one of the prime uses
of Neko, is that it's homepage isn't even valid HTML!

I found it when google came up with one of your posts on a mailing list
archive.  It would be a pity if people start using inferior products simply
because they don't know Neko exists.


----- Original Message -----
From: "Andy Clark" <an...@apache.org>
To: <xe...@xml.apache.org>
Sent: Wednesday, August 21, 2002 4:20 AM
Subject: Re: Getting the position of a node in the input stream (using Neko)


> Martin Jericho wrote:
> > I want to parse an HTML document using Neko, and all I want to find out
> > is the character position of particular nodes in the input stream.  I
>
> You have to make a distinction between "character position"
> and "byte offset" into the source file. They are not equivalent
> and can vary greatly depending on the character encoding of
> the file.
>
> > saw the XMLLocator interface, which I presume allows me to find out the
> > line number and the column within the line, but it doesn't include the
> > position from the start of the stream.
>
> This is because it is very difficult to map back to the
> original byte offset of the source file. Unless I wanted to
> re-implement all of the character decoders, that is...
>
> > Is there an example somewhere of doing this with Neko?  I would really
> > appreciate it if someone could help me with this, as I have nearly spent
> > the whole day trying to figure it out from the source code.
>
> Because I use the standard Java character decoders, I have
> no way of knowing the original byte offsets that correspond
> to the resulting Unicode characters.
>
> Could you explain in more detail exactly what information
> you are trying to retrieve?
>
>
> --
> Andy Clark * andyc@apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
> For additional commands, e-mail: xerces-j-user-help@xml.apache.org

http://digital.yahoo.com.au - Yahoo! Digital How To
- Get the best out of your PC!

---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org


Re: Getting the position of a node in the input stream (using Neko)

Posted by Andy Clark <an...@apache.org>.
Martin Jericho wrote:
> I want to parse an HTML document using Neko, and all I want to find out 
> is the character position of particular nodes in the input stream.  I 

You have to make a distinction between "character position"
and "byte offset" into the source file. They are not equivalent
and can vary greatly depending on the character encoding of
the file.

> saw the XMLLocator interface, which I presume allows me to find out the 
> line number and the column within the line, but it doesn't include the 
> position from the start of the stream.

This is because it is very difficult to map back to the
original byte offset of the source file. Unless I wanted to
re-implement all of the character decoders, that is...

> Is there an example somewhere of doing this with Neko?  I would really 
> appreciate it if someone could help me with this, as I have nearly spent 
> the whole day trying to figure it out from the source code.

Because I use the standard Java character decoders, I have
no way of knowing the original byte offsets that correspond
to the resulting Unicode characters.

Could you explain in more detail exactly what information
you are trying to retrieve?


-- 
Andy Clark * andyc@apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: xerces-j-user-unsubscribe@xml.apache.org
For additional commands, e-mail: xerces-j-user-help@xml.apache.org