You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by GitBox <gi...@apache.org> on 2021/01/27 17:24:35 UTC
[GitHub] [incubator-daffodil-site] mbeckerle opened a new pull request #32: update doap.rdf to make less XML/JSON specific.
mbeckerle opened a new pull request #32:
URL: https://github.com/apache/incubator-daffodil-site/pull/32
I want people to see that Daffodil isn't about XML/JSON specifically, but is a general data format capability that can directly interface with other systems like NiFi or Spark or Flink, etc. etc. without the baggage/overhead of XML or JSON.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil-site] mbeckerle commented on a change in pull request #32: update doap.rdf to make less XML/JSON specific.
Posted by GitBox <gi...@apache.org>.
mbeckerle commented on a change in pull request #32:
URL: https://github.com/apache/incubator-daffodil-site/pull/32#discussion_r565583541
##########
File path: site/doap.rdf
##########
@@ -29,17 +29,19 @@
<asfext:pmc rdf:resource="https://incubator.apache.org" />
<shortdesc>Apache Daffodil is an open-source implementation of the
- Data Format Description Language to convert between fixed
- format data and XML/JSON.</shortdesc>
+ Data Format Description Language to convert between fixed format
+ data and XML, JSON, or other data structures.</shortdesc>
<description>Apache Daffodil is an open-source implementation of
the DFDL specification that uses DFDL data descriptions to parse
- fixed format data into an infoset, which is most commonly
- represented as either XML or JSON. This allows the use of
+ fixed format data into an infoset. This infoset is commonly converted
+ into XML or JSON to enable the use of
well-established XML or JSON technologies and libraries to
consume, inspect, and manipulate fixed format data in existing
- solutions. Daffodil is also capable of the reverse by
- serializing or "unparsing" an XML or JSON infoset back to the
- original data format.</description>
+ solutions. Daffodil is also capable of serializing or "unparsing"
+ data back to the original data format.
+ The DFDL infoset can also be converted directly to/from the
+ data structures carried by data processing frameworks so as to bypass any
+ XML/JSON overheads. </description>
Review comment:
Smooks is a non-Apache framework that integrated Daffodil.
The NiFi processor is not yet complete/polished. The Spark thing is just an example and uses XML, so doesn't illustrate what I'm getting at in this rephrasing, which is that daffodil is about parsing and unparsing data, and whether you want to convert it to JSON or XML or something else is a flexible/fungible open-ended aspect of it.
I'm trying to avoid being cast as "an XML tool" at the level of the concept of the project scope.
The C backend (runtime2) is a case in point. The XML aspects of that are about bridging it to/from our test infrastructure and TDML. Ultimately the DFDL Infoset is a standard C struct data structure being populated by parse. Applications that use this C backend I assume would have nothing to do with XML.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil-site] tuxji commented on a change in pull request #32: update doap.rdf to make less XML/JSON specific.
Posted by GitBox <gi...@apache.org>.
tuxji commented on a change in pull request #32:
URL: https://github.com/apache/incubator-daffodil-site/pull/32#discussion_r565508654
##########
File path: site/doap.rdf
##########
@@ -29,17 +29,19 @@
<asfext:pmc rdf:resource="https://incubator.apache.org" />
<shortdesc>Apache Daffodil is an open-source implementation of the
- Data Format Description Language to convert between fixed
- format data and XML/JSON.</shortdesc>
+ Data Format Description Language to convert between fixed format
+ data and XML, JSON, or other data structures.</shortdesc>
<description>Apache Daffodil is an open-source implementation of
the DFDL specification that uses DFDL data descriptions to parse
- fixed format data into an infoset, which is most commonly
- represented as either XML or JSON. This allows the use of
+ fixed format data into an infoset. This infoset is commonly converted
+ into XML or JSON to enable the use of
well-established XML or JSON technologies and libraries to
consume, inspect, and manipulate fixed format data in existing
- solutions. Daffodil is also capable of the reverse by
- serializing or "unparsing" an XML or JSON infoset back to the
- original data format.</description>
+ solutions. Daffodil is also capable of serializing or "unparsing"
+ data back to the original data format.
+ The DFDL infoset can also be converted directly to/from the
+ data structures carried by data processing frameworks so as to bypass any
+ XML/JSON overheads. </description>
Review comment:
The changes make this page clearer, although they leave me with a question. If I'm a new user looking at this page and I've just read that the infoset can be converted directly to data structures used by data processing frameworks, I'm going to want a link I can click on to find more details how to do that conversion. Is the conversion already automated or must it be programmed/scripted? Where is the conversion implemented, in Daffodil or in the data processing framework? Which frameworks (Spark, NiFi, anything else)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil-site] mbeckerle merged pull request #32: update doap.rdf to make less XML/JSON specific.
Posted by GitBox <gi...@apache.org>.
mbeckerle merged pull request #32:
URL: https://github.com/apache/incubator-daffodil-site/pull/32
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil-site] bsloane1650 commented on pull request #32: update doap.rdf to make less XML/JSON specific.
Posted by GitBox <gi...@apache.org>.
bsloane1650 commented on pull request #32:
URL: https://github.com/apache/incubator-daffodil-site/pull/32#issuecomment-768454649
+1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil-site] stevedlawrence commented on a change in pull request #32: update doap.rdf to make less XML/JSON specific.
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on a change in pull request #32:
URL: https://github.com/apache/incubator-daffodil-site/pull/32#discussion_r565519377
##########
File path: site/doap.rdf
##########
@@ -29,17 +29,19 @@
<asfext:pmc rdf:resource="https://incubator.apache.org" />
<shortdesc>Apache Daffodil is an open-source implementation of the
- Data Format Description Language to convert between fixed
- format data and XML/JSON.</shortdesc>
+ Data Format Description Language to convert between fixed format
+ data and XML, JSON, or other data structures.</shortdesc>
<description>Apache Daffodil is an open-source implementation of
the DFDL specification that uses DFDL data descriptions to parse
- fixed format data into an infoset, which is most commonly
- represented as either XML or JSON. This allows the use of
+ fixed format data into an infoset. This infoset is commonly converted
+ into XML or JSON to enable the use of
well-established XML or JSON technologies and libraries to
consume, inspect, and manipulate fixed format data in existing
- solutions. Daffodil is also capable of the reverse by
- serializing or "unparsing" an XML or JSON infoset back to the
- original data format.</description>
+ solutions. Daffodil is also capable of serializing or "unparsing"
+ data back to the original data format.
+ The DFDL infoset can also be converted directly to/from the
+ data structures carried by data processing frameworks so as to bypass any
+ XML/JSON overheads. </description>
Review comment:
The [getting started page](https://daffodil.apache.org/getting-started/) does mention NiFi and calabash, but that's it. I think there are Spark implementations thoug I'm not sure there are any that are public. Having that information on the main page certainly would be useful. I'm not sure if it makes sense to be the doap file though.
In fact, I would say our entire splash page isn't very friendly in general. You have to read this paragraph to grok what Daffodil is, and I'm still not sure it's entirely clear. I always figure we could distill this down to a couple bullet points with some graphics that could more succiently get the point across. For example, something like the OGF DFDL life cycle graphic:
https://www.ogf.org/ogf/lib/exe/fetch.php/standards/dfdl/dfdl_system.gif
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil-site] stevedlawrence commented on a change in pull request #32: update doap.rdf to make less XML/JSON specific.
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on a change in pull request #32:
URL: https://github.com/apache/incubator-daffodil-site/pull/32#discussion_r565514504
##########
File path: site/doap.rdf
##########
@@ -29,17 +29,19 @@
<asfext:pmc rdf:resource="https://incubator.apache.org" />
<shortdesc>Apache Daffodil is an open-source implementation of the
- Data Format Description Language to convert between fixed
- format data and XML/JSON.</shortdesc>
+ Data Format Description Language to convert between fixed format
+ data and XML, JSON, or other data structures.</shortdesc>
<description>Apache Daffodil is an open-source implementation of
the DFDL specification that uses DFDL data descriptions to parse
- fixed format data into an infoset, which is most commonly
Review comment:
A very similar blurb exists on the main page at daffodil.apache.org in the site/index.md page. We should probably update that to to match this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org