You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Stephan Michels <st...@vern.chem.tu-berlin.de> on 2002/02/13 10:37:04 UTC
Re: text parser (was: schedule/batch feature inside cocoon)
On Wed, 13 Feb 2002, Bertrand Delacretaz wrote:
> On Wednesday 13 February 2002 08:23, Lars Steiger wrote:
> >. . .
> > simple xml. the parser is a generator. i call it
> > SimpleTextParserGenerator. it needs an xml configuration für parsing
> > a text stream. the xml configuration defines which text fragment goes
> > under which xml node.
> >. . .
> Changing topics, but are you parsing "structured text" a la PHPwiki or
> something? I've been thinking for a while (but didn't find time to do
> it yet) that this would be a nice addition to Cocoon, being able to use
> structured text as input.
Hi,
I had the same idea. I used the time to write a LALR(1)
Parser like yacc and Regex Scanner like lex. This project is near to be
finished.
This parser used xml for all input(grammar) and output(parser table
generation process)
One thing that keep me off to publish this project under sf, is that I
havn't a name for this project.
A text parser generator and a fragment text parser generator s also
written.
If anyone has a suggestion for name? Please help me!
Thanx, Stephan Michels.
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: text parser
Posted by Stephan Michels <st...@vern.chem.tu-berlin.de>.
On Wed, 13 Feb 2002, Bertrand Delacretaz wrote:
> On Wednesday 13 February 2002 10:37, Stephan Michels wrote:
> > I had the same idea. I used the time to write a LALR(1)
> > Parser like yacc and Regex Scanner like lex. This project is near to
> > be finished.
> > This parser used xml for all input(grammar) and output(parser table
> > generation process)
>
> > One thing that keep me off to publish this project under sf, is that
> > I havn't a name for this project.
>
> Here are a few ideas, didn't do any checks on them:
> styxml or stixml (Structured Text to XML)
> stex (Structured Text to X)
> xrules (XML parsing rules)
Hmm. I my first choice was jxc(java xml compiler)
> But maybe it's too restrictive, if your parser is a general one?
> For the Cocoon-specific parts of your project (you mentioned
> Generators), I think donating the code to Cocoon would help the
> visibility of your project.
I will do. I must frist install a cvs server on one of our computers
outside the firewall, but this will be obsolete if I have a name, so
I can put this on sf.net.
> If you keep some parts outside of Cocoon, I'd suggest selecting a
> license (ASF-compatible, I don't know the exact requirements) that
> allows your library to be included with the standard Cocoon
> distribution.
Yes, I will use the apache license.
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: text parser
Posted by Bertrand Delacretaz <bd...@codeconsult.ch>.
On Wednesday 13 February 2002 10:37, Stephan Michels wrote:
> I had the same idea. I used the time to write a LALR(1)
> Parser like yacc and Regex Scanner like lex. This project is near to
> be finished.
> This parser used xml for all input(grammar) and output(parser table
> generation process)
> One thing that keep me off to publish this project under sf, is that
> I havn't a name for this project.
Here are a few ideas, didn't do any checks on them:
styxml or stixml (Structured Text to XML)
stex (Structured Text to X)
xrules (XML parsing rules)
But maybe it's too restrictive, if your parser is a general one?
For the Cocoon-specific parts of your project (you mentioned
Generators), I think donating the code to Cocoon would help the
visibility of your project.
If you keep some parts outside of Cocoon, I'd suggest selecting a
license (ASF-compatible, I don't know the exact requirements) that
allows your library to be included with the standard Cocoon
distribution.
--
-- Bertrand Delacrétaz, www.codeconsult.ch
-- web technologies consultant - OO, Java, XML, C++
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: text parser
Posted by Bertrand Delacretaz <bd...@codeconsult.ch>.
On Wednesday 13 February 2002 12:18, Stephan Michels wrote:
>. . .
> > Also look at the APTConvert
> >
> > (http://www.xmlmind.com/aptconvert/distrib/docs/userguidetoc.html),
> > may be this tool can help you.
>
> I think my project could help you.
This is what I meant initially by "structured text a la PHPwiki" - if
your parser can parse such structured text as used by APTConvert -
great!
If could be a first set towards a "very poor man's CMS" using Cocoon...
- Bertrand
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re[2]: text parser
Posted by Stephan Michels <st...@vern.chem.tu-berlin.de>.
On Wed, 13 Feb 2002, Andrew Answer wrote:
> > Finding a name isn't so easy as I think. :(
>
> Hmmm.... why you reject project name if domain-name are reserved?
> Number of projects more than number of domains :)
No, but most of them had a copyright of their name, i think...
> Well-driven engine! It's look like XML parser...
> Suggestions:
> I'm worked with byacc/flex, but already forget his syntax. May be
> better to make DTD of your grammar more readable?
Before I used a text form of a grammer like this
%token Identifier [A-Za-z][A-Za-z0-9_]*;
%token Number [+\-]? ( ([0-9]+ \.[0-9]*)?(E[+\-]?[0-9]+)? |
[0-9]*\.[0-9]+(E[+\-]?[0-9]+)? );
%token String \" ( [^\\\"] | \\[^u] |
\\u[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F] )* \";
%token Boolean TRUE|FALSE;
%comment #([^\n]*);
%whitespace [\n\r\t\u0020]+;
%start File;
%%
Tupel : (String | Number) (String | Number)+;
Field : "[" (String | Number | Tupel) ("," (String | Number |
Tupel) )* "]";
Declaration : Identifier (String | Number | Tupel | Field);
Node : Identifier "{" (Node | Declaration)* "}";
File : (Node | Declaration)+;
> Then, you can even write stylesheet for converting byacc grammar
> into your grammar. And use it with your parser - it's a good test.
This I have already done ;-)
> How about whitespaces? Unlike XML, text files need to recognize one
> or two CR/LF and apply different rules, etc...
Whats about
<token tsymbol="eol">
<alt>
<concat>
<string content="\r"\><string minOccurs="0" content="\n"\>
</concat>
<string content="\n"\>
</alt>
</token>
> May be you can to produce one text from another (line formatting,
> adjusting, lists formatting, etc)?
I think yes, with a little work, you could do something like
syntaxhighlighting.
> And later you can transmute it into Generator/Transformer (but you must
> produce SAX stream for right work, i think)...
The (fragment) text parser generator produce a SAX stream, which could be
transformed in a pipeline.
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re[2]: text parser
Posted by Andrew Answer <A....@ftc.ru>.
Hello Stephan,
>*************Original message*************
> From: Stephan Michels <st...@vern.chem.tu-berlin.de>
> To: Stephan Michels <co...@xml.apache.org>
> Date: Wednesday, February 13, 2002, 5:18:34 PM
> Subject: text parser
> On Wed, 13 Feb 2002, Andrew Answer wrote:
>> Hello Stephan,
>>
>> is a good idea! Now i converting many text documents to XML by using
>> PHP scripts offline...
>> Some names for your parser: txt2xml (simply and clear),
> There exists already a project this this name:
> http://xml.gsfc.nasa.gov/ingest_demo/txt2XML.html
>> JTF (Java Text Formatter),
> Look an JTF.org:Jewish Task Force ;-)
>> JTC (Java Text Converter).
> http://www.jtc.com/ is also given
> Finding a name isn't so easy as I think. :(
Hmmm.... why you reject project name if domain-name are reserved?
Number of projects more than number of domains :)
>> Also look at the APTConvert
>> (http://www.xmlmind.com/aptconvert/distrib/docs/userguidetoc.html),
>> may be this tool can help you.
> I think my project could help you.
> A example grammar looks like:
> <grammar>
> <tokens>
> <token tsymbol="id">
> <concat>
> <cc><ci min="A" max="Z"/><ci min="a" max="z"/></cc>
> <cc minOccurs="0" maxOccurs="*">
> <ci min="A" max="Z"/><ci min="a" max="z"/><ci min="0" max="9"/>
> <cs content="_"/>
> </cc>
> </concat>
> </token>
> <token tsymbol="mult" assoc="right">
> <string content="*"/>
> </token>
> <token tsymbol="plus" assoc="left">
> <string content="+"/>
> </token>
> <token tsymbol="dopen">
> <string content="("/>
> </token>
> <token tsymbol="dclose">
> <string content=")"/>
> </token>
> </tokens>
> <whitespace>
> <cc maxOccurs="*"><cs content=" 	 "/></cc>
> </whitespace>
> <productions>
> <production ntsymbol="E">
> <ntsymbol name="E"/><tsymbol name="plus"/><ntsymbol name="E"/>
> </production>
> <production ntsymbol="E">
> <ntsymbol name="E"/><tsymbol name="mult"/><ntsymbol name="E"/>
> </production>
> <production ntsymbol="E">
> <tsymbol name="dopen"/><ntsymbol name="E"/><tsymbol name="dclose"/>
> </production>
> <production ntsymbol="E">
> <tsymbol name="id"/>
> </production>
> </productions>
> <ssymbol ntsymbol="E"/>
> </grammar>
> This grammar converts the string "A*b+c*D+(e+F)*G" to
> <E>
> <E>
> <E>
> <E>
> <id>A</id>
> </E>
> <mult>*</mult>
> <E>
> <id>b</id>
> </E>
> </E>
> <plus>+</plus>
> <E>
> <E>
> <id>c</id>
> </E>
> <mult>*</mult>
> <E>
> <id>D</id>
> </E>
> </E>
> </E>
> <plus>+</plus>
> <E>
> <E>
> <dopen>(</dopen>
> <E>
> <E>
> <id>e</id>
> </E>
> <plus>+</plus>
> <E>
> <id>F</id>
> </E>
> </E>
> <dclose>)</dclose>
> </E>
> <mult>*</mult>
> <E>
> <id>G</id>
> </E>
> </E>
> </E>
Well-driven engine! It's look like XML parser...
Suggestions:
I'm worked with byacc/flex, but already forget his syntax. May be
better to make DTD of your grammar more readable?
Then, you can even write stylesheet for converting byacc grammar
into your grammar. And use it with your parser - it's a good test.
How about whitespaces? Unlike XML, text files need to recognize one
or two CR/LF and apply different rules, etc...
May be you can to produce one text from another (line formatting,
adjusting, lists formatting, etc)?
And later you can transmute it into Generator/Transformer (but you must
produce SAX stream for right work, i think)...
Happy hacking!
Best regards,
Andrew Answer A.Nuzhdov@ftc.ru
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: text parser
Posted by Stephan Michels <st...@vern.chem.tu-berlin.de>.
On Wed, 13 Feb 2002, Andrew Answer wrote:
> Hello Stephan,
>
> is a good idea! Now i converting many text documents to XML by using
> PHP scripts offline...
> Some names for your parser: txt2xml (simply and clear),
There exists already a project this this name:
http://xml.gsfc.nasa.gov/ingest_demo/txt2XML.html
> JTF (Java Text Formatter),
Look an JTF.org:Jewish Task Force ;-)
> JTC (Java Text Converter).
http://www.jtc.com/ is also given
Finding a name isn't so easy as I think. :(
> Also look at the APTConvert
> (http://www.xmlmind.com/aptconvert/distrib/docs/userguidetoc.html),
> may be this tool can help you.
I think my project could help you.
A example grammar looks like:
<grammar>
<tokens>
<token tsymbol="id">
<concat>
<cc><ci min="A" max="Z"/><ci min="a" max="z"/></cc>
<cc minOccurs="0" maxOccurs="*">
<ci min="A" max="Z"/><ci min="a" max="z"/><ci min="0" max="9"/>
<cs content="_"/>
</cc>
</concat>
</token>
<token tsymbol="mult" assoc="right">
<string content="*"/>
</token>
<token tsymbol="plus" assoc="left">
<string content="+"/>
</token>
<token tsymbol="dopen">
<string content="("/>
</token>
<token tsymbol="dclose">
<string content=")"/>
</token>
</tokens>
<whitespace>
<cc maxOccurs="*"><cs content=" 	 "/></cc>
</whitespace>
<productions>
<production ntsymbol="E">
<ntsymbol name="E"/><tsymbol name="plus"/><ntsymbol name="E"/>
</production>
<production ntsymbol="E">
<ntsymbol name="E"/><tsymbol name="mult"/><ntsymbol name="E"/>
</production>
<production ntsymbol="E">
<tsymbol name="dopen"/><ntsymbol name="E"/><tsymbol name="dclose"/>
</production>
<production ntsymbol="E">
<tsymbol name="id"/>
</production>
</productions>
<ssymbol ntsymbol="E"/>
</grammar>
This grammar converts the string "A*b+c*D+(e+F)*G" to
<E>
<E>
<E>
<E>
<id>A</id>
</E>
<mult>*</mult>
<E>
<id>b</id>
</E>
</E>
<plus>+</plus>
<E>
<E>
<id>c</id>
</E>
<mult>*</mult>
<E>
<id>D</id>
</E>
</E>
</E>
<plus>+</plus>
<E>
<E>
<dopen>(</dopen>
<E>
<E>
<id>e</id>
</E>
<plus>+</plus>
<E>
<id>F</id>
</E>
</E>
<dclose>)</dclose>
</E>
<mult>*</mult>
<E>
<id>G</id>
</E>
</E>
</E>
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: text parser
Posted by Andrew Answer <A....@ftc.ru>.
Hello Stephan,
is a good idea! Now i converting many text documents to XML by using
PHP scripts offline...
Some names for your parser: txt2xml (simply and clear),
JTF (Java Text Formatter), JTC (Java Text Converter).
Also look at the APTConvert
(http://www.xmlmind.com/aptconvert/distrib/docs/userguidetoc.html),
may be this tool can help you.
I think what your parser can be used for converting virtually ANY types of text
files by using some modification of XSL: i.e. your parser can use
XML-based stylesheets for transforming text files into XML.
Examples of text:
[
First paragraph. First paragraph. First paragraph. First paragraph.
First paragraph. First paragraph. First paragraph. First paragraph.
First paragraph. First paragraph.
Second paragraph.
] OR [
First paragraph. First paragraph. First paragraph. First paragraph.
First paragraph. First paragraph. First paragraph. First paragraph.
First paragraph. First paragraph.
Second paragraph.
]
Examples of convert rules:
[
<xsl:template match="enter()+5*space()">
<p><xsl:value-of select="."/></p>
</xsl:template>
] OR [
<xsl:template match="2*enter()">
<p><xsl:value-of select="."/></p>
</xsl:template>
]
Probably, it's can't work, but i will think deeply about this next time...
May be, in stylesheets parser must use even SGML instead of XML...
What is your opinion?
P.S. Anybody use PHPGenerator under C2? Please help me to switch this
feature...
>*************Original message*************
> From: Stephan Michels <st...@vern.chem.tu-berlin.de>
> To: cocoon-dev@xml.apache.org <co...@xml.apache.org>
> Date: Wednesday, February 13, 2002, 3:37:04 PM
> Subject: text parser (was: schedule/batch feature inside cocoon)
> On Wed, 13 Feb 2002, Bertrand Delacretaz wrote:
>> On Wednesday 13 February 2002 08:23, Lars Steiger wrote:
>> >. . .
>> > simple xml. the parser is a generator. i call it
>> > SimpleTextParserGenerator. it needs an xml configuration fьr parsing
>> > a text stream. the xml configuration defines which text fragment goes
>> > under which xml node.
>> >. . .
>> Changing topics, but are you parsing "structured text" a la PHPwiki or
>> something? I've been thinking for a while (but didn't find time to do
>> it yet) that this would be a nice addition to Cocoon, being able to use
>> structured text as input.
> Hi,
> I had the same idea. I used the time to write a LALR(1)
> Parser like yacc and Regex Scanner like lex. This project is near to be
> finished.
> This parser used xml for all input(grammar) and output(parser table
> generation process)
> One thing that keep me off to publish this project under sf, is that I
> havn't a name for this project.
> A text parser generator and a fragment text parser generator s also
> written.
> If anyone has a suggestion for name? Please help me!
> Thanx, Stephan Michels.
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
> For additional commands, email: cocoon-dev-help@xml.apache.org
>*************Original message*************
Best regards,
Andrew Answer A.Nuzhdov@ftc.ru
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org