You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@daffodil.apache.org by "Costello, Roger L." <co...@mitre.org> on 2019/05/28 14:07:18 UTC
DFDL versus ANTLR
Hello DFDL community,
I created some slides comparing DFDL to ANTLR. See below. Have I accurately and fairly captured the differences? Anything you would add or delete? All suggestions are greatly appreciated. /Roger
Slide #1:
[cid:image001.png@01D5153D.1E1049F0]
Slide #2:
[cid:image002.png@01D5153D.1E1049F0]
Slide #3:
[cid:image003.png@01D5153D.1E1049F0]
Slide #4:
[cid:image004.png@01D5153D.1E1049F0]
Slide #5:
[cid:image005.png@01D5153D.1E1049F0]
Slide #6:
[cid:image006.png@01D5153D.1E1049F0]
Slide #7:
[cid:image007.png@01D5153D.1E1049F0]
Slide #8:
[cid:image008.png@01D5153D.1E1049F0]
Slide #9:
[cid:image009.png@01D5153D.1E1049F0]
Re: DFDL versus ANTLR
Posted by "Wilson, Art" <aw...@tresys.com>.
Good work, Roger, as always!
A few more typos on slide 9:
Open Grid _Forum_ standards organization in the next to last bullet.
ANTLR is mis-typed as ANTRL in the last two bullets.
Art
________________________________
From: Sloane, Brandon <bs...@tresys.com>
Sent: Tuesday, May 28, 2019 4:44 PM
To: users@daffodil.apache.org
Subject: Re: DFDL versus ANTLR
Roger,
Between slides 7 and 8, you switch from "description of the input file" to "DFDL Schema". It is not clear that these boxes are intended to be the same thing.
Julian,
> As ANTLR has a code-gen phase where it generates the parsing code Daffodil does that internally at runtime (so its kind of like thrift vs avro).
Daffodil does not have a code-gen phase, even internally. It compiles the schema to an internal data-structure, which is then interperated by the runtime.
It is, however, possible for the compile phase to be done ahead-of-time. I tend to view this as an optimization for complex schema, so it is probably good to leave it omitted from these slides.
________________________________
From: Costello, Roger L. <co...@mitre.org>
Sent: Tuesday, May 28, 2019 11:51:40 AM
To: users@daffodil.apache.org
Subject: RE: DFDL versus ANTLR
Hello DFDL community,
I fixed a few typos and added a mention of JSON. See updated slides below. /Roger
Slide #1:
[cid:image010.png@01D5154B.B3B287D0]
Slide #2:
[cid:image011.png@01D5154B.B3B287D0]
Slide #3:
[cid:image012.png@01D5154B.B3B287D0]
Slide #4:
[cid:image013.png@01D5154B.B3B287D0]
Slide #5:
[cid:image014.png@01D5154B.B3B287D0]
Slide #6:
[cid:image015.png@01D5154B.B3B287D0]
Slide #7:
[cid:image016.png@01D5154B.B3B287D0]
Slide #8:
[cid:image017.png@01D5154B.B3B287D0]
Slide #9:
[cid:image018.png@01D5154B.B3B287D0]
Re: DFDL versus ANTLR
Posted by "Sloane, Brandon" <bs...@tresys.com>.
Roger,
Between slides 7 and 8, you switch from "description of the input file" to "DFDL Schema". It is not clear that these boxes are intended to be the same thing.
Julian,
> As ANTLR has a code-gen phase where it generates the parsing code Daffodil does that internally at runtime (so its kind of like thrift vs avro).
Daffodil does not have a code-gen phase, even internally. It compiles the schema to an internal data-structure, which is then interperated by the runtime.
It is, however, possible for the compile phase to be done ahead-of-time. I tend to view this as an optimization for complex schema, so it is probably good to leave it omitted from these slides.
________________________________
From: Costello, Roger L. <co...@mitre.org>
Sent: Tuesday, May 28, 2019 11:51:40 AM
To: users@daffodil.apache.org
Subject: RE: DFDL versus ANTLR
Hello DFDL community,
I fixed a few typos and added a mention of JSON. See updated slides below. /Roger
Slide #1:
[cid:image010.png@01D5154B.B3B287D0]
Slide #2:
[cid:image011.png@01D5154B.B3B287D0]
Slide #3:
[cid:image012.png@01D5154B.B3B287D0]
Slide #4:
[cid:image013.png@01D5154B.B3B287D0]
Slide #5:
[cid:image014.png@01D5154B.B3B287D0]
Slide #6:
[cid:image015.png@01D5154B.B3B287D0]
Slide #7:
[cid:image016.png@01D5154B.B3B287D0]
Slide #8:
[cid:image017.png@01D5154B.B3B287D0]
Slide #9:
[cid:image018.png@01D5154B.B3B287D0]
Re: DFDL versus ANTLR
Posted by "Sloane, Brandon" <bs...@tresys.com>.
Now that I think about it, I am not sure "higher level of abstraction" is accurate.
At its core, DFDL is just defining a grammar. And it parses said grammar using a naive backtracking algorithm (unless the schema provides specific instructions for a given dispatch point). When dealing with a text base format, you often end up effectively writing your lexer rules inline by using regular expressions to determine the length of string fields.
I would say that DFDL is a more powerful or richer abstraction, in that it can annotate the grammar in ways that are either impossible or non-trivial in a traditional grammar.
Much of this power comes from the DFDL expressions. I would probably give an example showing choiceDispatchKey, or occursCountKind=expression. In particular, DFDL can make parsing decisions based on the content of the data, not just the catagory of the data.
Arguably, DFDL does not even provide the core abstraction that ANTLR does. ANTLR allows you to just express the grammar, from which it will generate an efficient LL* parser. DFDL does not provide this abstraction. The parser you end up with for an "unannotated" grammar will be a naive backtracking based parser.
There are certainly some parts of the description langaguage that are more high-level, such as specifying separators instead of the corresponding production rule, but I am not sure how compelling that would be.
Maybe someone else could come up with a better statement?
________________________________
From: Costello, Roger L. <co...@mitre.org>
Sent: Tuesday, May 28, 2019 5:30:27 PM
To: users@daffodil.apache.org
Subject: RE: DFDL versus ANTLR
Hi Folks,
Thank you so much for your excellent feedback!
The last bullet on the last slide has this statement:
* DFDL is at a higher level of abstraction; for example, you don’t have to write lexer rules. ANTLR is at a lower level of abstraction.
As I reread that, it occurs to me that it is pretty vague. Can the statement be made less vague? How about this:
* DFDL is at a higher level of abstraction; for example, you can use built-in datatypes such as “integer” rather than creating an equivalent regular expression.
That statement seems more concrete, but also less compelling.
Can you give a concrete, compelling statement of how DFDL provides the user a higher level of abstraction than ANTLR?
/Roger
From: Costello, Roger L. <co...@mitre.org>
Sent: Tuesday, May 28, 2019 11:52 AM
To: users@daffodil.apache.org
Subject: RE: DFDL versus ANTLR
Hello DFDL community,
I fixed a few typos and added a mention of JSON. See updated slides below. /Roger
Slide #1:
[cid:image001.png@01D5157B.075DB4C0]
Slide #2:
[cid:image002.png@01D5157B.075DB4C0]
Slide #3:
[cid:image003.png@01D5157B.075DB4C0]
Slide #4:
[cid:image004.png@01D5157B.075DB4C0]
Slide #5:
[cid:image005.png@01D5157B.075DB4C0]
Slide #6:
[cid:image006.png@01D5157B.075DB4C0]
Slide #7:
[cid:image007.png@01D5157B.075DB4C0]
Slide #8:
[cid:image008.png@01D5157B.075DB4C0]
Slide #9:
[cid:image009.png@01D5157B.075DB4C0]
RE: DFDL versus ANTLR
Posted by "Costello, Roger L." <co...@mitre.org>.
Hi Folks,
Thank you so much for your excellent feedback!
The last bullet on the last slide has this statement:
* DFDL is at a higher level of abstraction; for example, you don't have to write lexer rules. ANTLR is at a lower level of abstraction.
As I reread that, it occurs to me that it is pretty vague. Can the statement be made less vague? How about this:
* DFDL is at a higher level of abstraction; for example, you can use built-in datatypes such as "integer" rather than creating an equivalent regular expression.
That statement seems more concrete, but also less compelling.
Can you give a concrete, compelling statement of how DFDL provides the user a higher level of abstraction than ANTLR?
/Roger
From: Costello, Roger L. <co...@mitre.org>
Sent: Tuesday, May 28, 2019 11:52 AM
To: users@daffodil.apache.org
Subject: RE: DFDL versus ANTLR
Hello DFDL community,
I fixed a few typos and added a mention of JSON. See updated slides below. /Roger
Slide #1:
[cid:image001.png@01D5157B.075DB4C0]
Slide #2:
[cid:image002.png@01D5157B.075DB4C0]
Slide #3:
[cid:image003.png@01D5157B.075DB4C0]
Slide #4:
[cid:image004.png@01D5157B.075DB4C0]
Slide #5:
[cid:image005.png@01D5157B.075DB4C0]
Slide #6:
[cid:image006.png@01D5157B.075DB4C0]
Slide #7:
[cid:image007.png@01D5157B.075DB4C0]
Slide #8:
[cid:image008.png@01D5157B.075DB4C0]
Slide #9:
[cid:image009.png@01D5157B.075DB4C0]
RE: DFDL versus ANTLR
Posted by "Costello, Roger L." <co...@mitre.org>.
Hello DFDL community,
I fixed a few typos and added a mention of JSON. See updated slides below. /Roger
Slide #1:
[cid:image010.png@01D5154B.B3B287D0]
Slide #2:
[cid:image011.png@01D5154B.B3B287D0]
Slide #3:
[cid:image012.png@01D5154B.B3B287D0]
Slide #4:
[cid:image013.png@01D5154B.B3B287D0]
Slide #5:
[cid:image014.png@01D5154B.B3B287D0]
Slide #6:
[cid:image015.png@01D5154B.B3B287D0]
Slide #7:
[cid:image016.png@01D5154B.B3B287D0]
Slide #8:
[cid:image017.png@01D5154B.B3B287D0]
Slide #9:
[cid:image018.png@01D5154B.B3B287D0]
Re: DFDL versus ANTLR
Posted by Julian Feinauer <j....@pragmaticminds.de>.
First, sorry for that... somehow tapped the wrong key.
I never thought of DFDL being somewhat similar to ANTLR but in fact it kind of is… conceptually.
I think it could be useful to show the different lifecycle of both tools.
As ANTLR has a code-gen phase where it generates the parsing code Daffodil does that internally at runtime (so its kind of like thrift vs avro).
But that’s just a suggestion.
Julian
Von: Julian Feinauer <j....@pragmaticminds.de>
Antworten an: "users@daffodil.apache.org" <us...@daffodil.apache.org>
Datum: Dienstag, 28. Mai 2019 um 17:46
An: "users@daffodil.apache.org" <us...@daffodil.apache.org>
Betreff: Re: DFDL versus ANTLR
Hi all,
I like the comparison and I never thought
Von: "Beckerle, Mike" <mb...@tresys.com>
Antworten an: "users@daffodil.apache.org" <us...@daffodil.apache.org>
Datum: Dienstag, 28. Mai 2019 um 17:09
An: "users@daffodil.apache.org" <us...@daffodil.apache.org>
Betreff: Re: DFDL versus ANTLR
Great Roger. These kinds of clarifications are really helpful to people trying to get a feel for how DFDL is different from things they've heard of, or used in the past. I think you've captured it pretty well.
I suggest mentioning the other popular Infoset representation, which is JSON.
My experience is that people have love/hate relationships with XML. Some people have sworn allegiance to JSON, etc.
________________________________
From: Costello, Roger L. <co...@mitre.org>
Sent: Tuesday, May 28, 2019 10:07:18 AM
To: users@daffodil.apache.org
Subject: DFDL versus ANTLR
Hello DFDL community,
I created some slides comparing DFDL to ANTLR. See below. Have I accurately and fairly captured the differences? Anything you would add or delete? All suggestions are greatly appreciated. /Roger
Slide #1:
[cid:image001.png@01D5153D.1E1049F0]
Slide #2:
[cid:image002.png@01D5153D.1E1049F0]
Slide #3:
[cid:image003.png@01D5153D.1E1049F0]
Slide #4:
[cid:image004.png@01D5153D.1E1049F0]
Slide #5:
[cid:image005.png@01D5153D.1E1049F0]
Slide #6:
[cid:image006.png@01D5153D.1E1049F0]
Slide #7:
[cid:image007.png@01D5153D.1E1049F0]
Slide #8:
[cid:image008.png@01D5153D.1E1049F0]
Slide #9:
[cid:image009.png@01D5153D.1E1049F0]
Re: DFDL versus ANTLR
Posted by Julian Feinauer <j....@pragmaticminds.de>.
Hi all,
I like the comparison and I never thought
Von: "Beckerle, Mike" <mb...@tresys.com>
Antworten an: "users@daffodil.apache.org" <us...@daffodil.apache.org>
Datum: Dienstag, 28. Mai 2019 um 17:09
An: "users@daffodil.apache.org" <us...@daffodil.apache.org>
Betreff: Re: DFDL versus ANTLR
Great Roger. These kinds of clarifications are really helpful to people trying to get a feel for how DFDL is different from things they've heard of, or used in the past. I think you've captured it pretty well.
I suggest mentioning the other popular Infoset representation, which is JSON.
My experience is that people have love/hate relationships with XML. Some people have sworn allegiance to JSON, etc.
________________________________
From: Costello, Roger L. <co...@mitre.org>
Sent: Tuesday, May 28, 2019 10:07:18 AM
To: users@daffodil.apache.org
Subject: DFDL versus ANTLR
Hello DFDL community,
I created some slides comparing DFDL to ANTLR. See below. Have I accurately and fairly captured the differences? Anything you would add or delete? All suggestions are greatly appreciated. /Roger
Slide #1:
[cid:image001.png@01D5153D.1E1049F0]
Slide #2:
[cid:image002.png@01D5153D.1E1049F0]
Slide #3:
[cid:image003.png@01D5153D.1E1049F0]
Slide #4:
[cid:image004.png@01D5153D.1E1049F0]
Slide #5:
[cid:image005.png@01D5153D.1E1049F0]
Slide #6:
[cid:image006.png@01D5153D.1E1049F0]
Slide #7:
[cid:image007.png@01D5153D.1E1049F0]
Slide #8:
[cid:image008.png@01D5153D.1E1049F0]
Slide #9:
[cid:image009.png@01D5153D.1E1049F0]
Re: DFDL versus ANTLR
Posted by "Beckerle, Mike" <mb...@tresys.com>.
Great Roger. These kinds of clarifications are really helpful to people trying to get a feel for how DFDL is different from things they've heard of, or used in the past. I think you've captured it pretty well.
I suggest mentioning the other popular Infoset representation, which is JSON.
My experience is that people have love/hate relationships with XML. Some people have sworn allegiance to JSON, etc.
________________________________
From: Costello, Roger L. <co...@mitre.org>
Sent: Tuesday, May 28, 2019 10:07:18 AM
To: users@daffodil.apache.org
Subject: DFDL versus ANTLR
Hello DFDL community,
I created some slides comparing DFDL to ANTLR. See below. Have I accurately and fairly captured the differences? Anything you would add or delete? All suggestions are greatly appreciated. /Roger
Slide #1:
[cid:image001.png@01D5153D.1E1049F0]
Slide #2:
[cid:image002.png@01D5153D.1E1049F0]
Slide #3:
[cid:image003.png@01D5153D.1E1049F0]
Slide #4:
[cid:image004.png@01D5153D.1E1049F0]
Slide #5:
[cid:image005.png@01D5153D.1E1049F0]
Slide #6:
[cid:image006.png@01D5153D.1E1049F0]
Slide #7:
[cid:image007.png@01D5153D.1E1049F0]
Slide #8:
[cid:image008.png@01D5153D.1E1049F0]
Slide #9:
[cid:image009.png@01D5153D.1E1049F0]