You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@daffodil.apache.org by "Thompson, Mark M [US] (DS)" <ma...@ngc.com> on 2022/05/15 18:11:06 UTC

RE: EXT :Re: Re: Re: Re: Extending JAPI for processing binary input with an XML Payload

Steve,

        Thank you for your quick response. We will have to give some thought as to how we
        can get the results needed for the XML payload validation. Maybe we can pull out the element with
        RuntimeProperties = “stringAsXML=true” attribute and validate it where we parse the overall binary
        Input.

        A few points:
1)      The exception generated is not from our code but from the call to
        validator.validate(new DOMSource(document));
2)      If, when envisioning the embedded XML feature, the validation of the payload would not take place in the InfosetOutputter, why does the example provided extend InfossetOutputter?
         What am I missing?

Thank you for your time,
  Mark T
                .

-----Original Message-----
From: Steve Lawrence <sl...@apache.org>
Sent: Friday, May 13, 2022 5:00 AM
To: users@daffodil.apache.org
Subject: EXT :Re: Re: Re: Re: Extending JAPI for processing binary input with an XML Payload

There isn't currently a way for InfosetOutputter's to add to the diagnostics. In order to do so, an outputter would need access to the PState (where diagnostics are stored), which would require an API change. I'm not sure we'd want to give direct access to the PState though--some discussions would be needed to figure out what that API would look like to get diagnostics out of a outputter and into the PState.

Also, note that throwing an Exception, whether it be a SAXException or something else, won't really give the behavior your want. That exception will just bubble up and lead to an "Abort" error about an unexpected exception being thrown.

The one exception you could throw is a SchemaDefinitionError and that will be added to diagnostics. But it will immediately stop processing and you will not get a complete infoset. This is different than normal validation that Daffodil does which happens at the end of processing and does not prevent the creation of an infoset.

When we first envisioned this embedded XML feature, we imagined that the validation of the payload would not take place in the InfosetOutputter, but instead would happen after the parse completed in some other process that is purely for validation. This process would use a schema that combines the DFDL schema with the schema for the embedded XML so that it would validate correctly.

If you really wanted to do it all in Daffodil, maybe another option would be to create a custom Validator. This Validator would need to find/extract/unescape the payload XML (the XML that the Validator gets is not the same as what the InfosetOutputter creates), and then you can do whatever validation is necessary. The Validator API does support returning errors/warnings from validation and adding them to the diagnostics.


On 5/12/22 9:55 PM, Thompson, Mark M [US] (DS) wrote:
> Hello again from XML Payload world,
>           - We have succeeded in implementing an extension of
> XMLInfosetOutputter that will parse and validation a binary file
>           with an XML payload. It will validate and report if the
> binary fields do not conform to the DFDL schema facet restrictions.
>            - We have also implemented a routine to validate the XML
> payload against a schema which will report if the payload is
>           malformed. An exception (SAXException) is generated in this case.
>           What is not clear is how one gets the Diagnostics to be
> populated with this info or, if they are populated, how to make
>           the Diagnostics available to the routine that called
> DataProcessor.Parse() in the first place. The Diagnostics are available
>           here for the validation of the binary inputs against the
> corresponding schema but not those reported for the XML payload.
>           Do I need to extend the ErrorHandler class and use the
> warning and/or error methods to populate Diagnostics as part of the
> Exception handling for code below?
>           The following is an excerpt of the code that seems to do
> what I want except for making the SAXException info available.
> document = db.parse(new InputSource (new StringReader (xmlPayload)));
>                    // load a WXS schema, represented by a Schema instance
>                    StreamSource schemaFile = new StreamSource(new
> File(schemaFileName));
>                    Schema schema = s_factory.newSchema(schemaFile);
>                    // create a Validator instance, which can be used
> to validate an instance document
>                    javax.xml.validation.Validator validator = schema.newValidator();
>                    System.out.println("**** validateXMLPayload - Try
> to validate XML Payload.");
>                    // validate the DOM tree
>                    validator.validate(new DOMSource(document));
>           If anyone can provide a clue as to what I’m missing or
> invalid assumptions, I would greatly appreciate it.
> Thank you for your time,
>     Mark T
> -----Original Message-----
> From: Steve Lawrence <sl...@apache.org>>
> Sent: Thursday, April 28, 2022 4:36 PM
> To: users@daffodil.apache.org<ma...@daffodil.apache.org>
> Subject: EXT :Re: Re: Re: Extending JAPI for processing binary input
> with an XML Payload It looks like our InfosetOutputter API isn't very
> Java friendly, at least once you start needing to get information out
> of the ERD. I'd open a ticket, but it looks like JIRA is down at the
> moment.
> I think the way to fix your issue is to just use straight reference
> equality comparison with NodeInfo.String(), e.g.:
>      if (simple.erd().optPrimType().get() == NodeInfo.String()) { ...
> } The instance that NodeInfo.String() returns is essentially a
> singleton to represent DFDL String primitives, which is also what is
> returned from optPrimtType().get().
> I'm not sure why our InfosetOutputter uses isInstanceOf[String.Kind]
> instead of just using reference equality with NodeInfo.String. Might
> have been old code that never got updated...
> On 4/28/22 1:24 PM, Thompson, Mark M [US] (DS) wrote:
>> Hello all,
>>
>>        We are attempting to generate a Java class for this XML payload by extending XMLInfosetOutputter using XMLTextInfosetOUtputter
>>        As a guide.
>>        We're having trouble coming up with a Java equivalent for the StartSimple() method code:
>>                  if
>> (simple.erd.optPrimType.get.isInstanceOf[NodeInfo.String.Kind]) {
>>
>>        We've tried the following with no luck:
>>                if (simple.erd().optPrimType().get() instanceof NodeInfo.String().Kind) {
>>        and
>>                       if
>> (simple.erd().optPrimType().get().isInstanceOf(NodeInfo.String().???)
>> )
>> {
>>
>>        Any input or suggestions are greatly appreciated.
>>
>> Thank you for your time,
>>    Mark T
>>
>> -----Original Message-----
>> From: Steve Lawrence <slawrence@apache.org
>> <ma...@apache.org>>
>> Sent: Thursday, April 21, 2022 4:31 AM
>> To: users@daffodil.apache.org<ma...@daffodil.apache.org> <ma...@daffodil.apache.org>
>> Subject: EXT :Re: Re: Extending JAPI for processing binary input with
>> an XML Payload
>>
>>   > Contributing to Daffodil is likely not an option for my company.
>>
>> That's sad to hear, and I hope you can try to persuade your company to change this position. Opensource works best with many contributors from many different organizations. And if your company depends on Daffodil, contributing back to it can help ensure  its success and continued growth.
>>
>>   > Based on my limited understanding, this seems to suggest that I
>> would have to add another infoset type to make use of the new class
>>
>> If you're using the CLI, this is correct. Infoset inputters and
>> outputters must currently be hardcoded in. If you're using the API to
>> invoke Daffodil this isn't a problem, but if you're using the CLI
>> then you must modify the CLI to support your new classes.  This
>> hasn't been an issue so far since anytime a new one is needed we just
> added it to Daffodil.
>>
>> Part of the reason for this is that creating InfosetInputters/Outputters and getting the result is very specific to the infoset representation.
>> It's probably possible that the API could be made more generic, and then we could use a ServiceLoader or something to support pluggable infoset inputters/outputters, but that hasn't be a priority so far.
>>
>> On 4/20/22 10:08 PM, Thompson, Mark M [US] (DS) wrote:
>>> Hi Steve,
>>>
>>>       I have been looking into your suggestions and researching the existing code and here
>>>       Is where I'm at:
>>>       - Contributing to Daffodil is likely not an option for my company.
>>>       - I've been looking into XMLTextInfosetInputter and ScalaXMLInfosetOutputter for insight.
>>>          What I don't understand (Java and Scala newby) is, having extended InfosetOutputter with a new class,
>>>          the Daffodil parse operation magically has access to this new class. I see, in main.scala, where
>>>          XMLTextInfosetOutputter and ScalaXMLInfosetOutputter are instantiated based on command line
>>>          Input for infoset type. Based on my limited understanding, this seems to suggest that I would have
>>>          To add another infoset type to make use of the new class.
>>>
>>>        Can someone clear up my misunderstanding or point to some info that will cause the scales to fall
>>>       From my eyes?
>>>
>>> Thank you for your time,
>>>      Mark T
>>>
>>> -----Original Message-----
>>> From: Steve Lawrence <slawrence@apache.org
>>> <ma...@apache.org>>
>>> Sent: Friday, April 8, 2022 6:31 AM
>>> To: users@daffodil.apache.org<ma...@daffodil.apache.org> <ma...@daffodil.apache.org>
>>> Subject: EXT :Re: Extending JAPI for processing binary input with an
>>> XML Payload
>>>
>>> I've put a decent amount of thought into this  issue, I'll add some additional info that might help out...
>>>
>>>    > for example, the startSimple() method. It would be helpful to have an idea what a given method does normally when there is no class extension.
>>>
>>> The InfosetOutputter class is an abstract class with no default implementations, so an implementation of all functions, including startSimple, is required--InfosetInputters don't do anything by default.
>>> The InfosetOutputters that are bundled with Daffodil implement all of these functions to create the kind of infoset that is output (e.g. XML text, Scala Nodes, JSON). If you want to take a look at how our bundled implementations handle their logic,  they are all in this folder:
>>>
>>> https://github.com/apache/daffodil/tree/main/daffodil-runtime1/src/m
>>> a
> <https://github.com/apache/daffodil/tree/main/daffodil-runtime1/src/ma
> >
>>> i
>>> n/scala/org/apache/daffodil/infoset
>>>
>>> If you plan to keep your InfosetOutputter outside of the Daffodil codebase, you'll need to implement all of the functions/logic yourself.
>>> Our current implementions are mostly private, so extending and overriding them to do you you want is likely not possible.
>>>
>>> A good place to start would be to pick one that that's close to what you want and modify the various functions as described in the example implementation in the runtime properties proposal:
>>>
>>> https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+Run
>>> t
> <https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+Runt
> >
>>> i
>>> me+Properties
>>>
>>> Alternatively, it might be easier to just add the support to our
>>> existing bundled InfosetOutputters to support this capability, and
>>> contribute that work to Daffodil. Not only will this probably be
>>> easier since you can reuse alot of the existing logic,  but our
>>> ability to provide support greatly increases when it's related to
> things being added to Daffodil. This feature is a request we've seen
> multiple times--it would be a very welcome addition and we'd be happy
> help get it added to Daffodil.
>>>
>>>
>>>
>>> On 4/8/22 8:42 AM, Mike Beckerle wrote:
>>>> Mark,
>>>>
>>>> There definitely are javadoc jars, and the javadoc is also online
>>>> here (for release 3.3.0)
>>>> https://daffodil.apache.org/docs/latest/javadoc/
> <https://daffodil.apache.org/docs/latest/javadoc/>
>>>> <https://daffodil.apache.org/docs/latest/javadoc/
> <https://daffodil.apache.org/docs/latest/javadoc/>>
>>>>
>>>> The source and javadoc jars are normally pulled down using maven or
>>>> sbt. But you can find them directly on maven central also.
>>>>
>>>> For example for the daffodil-runtime1 jar, all the jars, compiled
>>>> binary, sources, and the javadoc jar is at
>>>> https://repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtime
>>>> 1
> <https://repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtime1>
>>>> _
>>>> 2
>>>> .12/3.3.0/
>>>> <https://repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtim
>>>> e
>>>> 1
>>>> _
>>>> 2.12/3.3.0/>
>>>>
>>>>
>>>> On Thu, Apr 7, 2022 at 10:07 PM Thompson, Mark M [US] (DS)
>>>> <mark.m.thompson@ngc.com <mailto:mark.m.thompson@ngc.com
> <mailto:mark.m.thompson@ngc.com <ma...@ngc.com>>> wrote:
>>>>
>>>>        Greetings to All,____
>>>>
>>>>        __ __
>>>>
>>>>                       Caveat: I am new to Daffodil and DFDL. I’m familiar with the
>>>>        basic functionality and have successfully written____
>>>>
>>>>                                      DFDL schemas to handle text and binary inputs
>>>>        to the parse/unparse cycle (separate assignments). ____
>>>>
>>>>        __ __
>>>>
>>>>        I am working on a task that requires the processing of binary files
>>>>        containing an XML payload. The goal is to be____
>>>>
>>>>                       able to validate the raw (un-escapified) XML against an XML
>>>>        schema during the parsing phase. I have been researching____
>>>>
>>>>                       the Daffodil JAPI InfosetOutputter family of classes as a
>>>>        first step. I’ve also read Steve Lawrence’s write-up on the
>>>> ____
>>>>
>>>>                       RuntimePropertiies proposal. What I have yet to grasp or find
>>>>        is what InfosetOutputter functionality is to be extended
>>>> .____
>>>>
>>>>                       I’ve searched high and low but I have yet to find any
>>>>        detailed information on what basic functionality is provided
>>>> by,____
>>>>
>>>>        for example, the startSimple() method. It would be helpful to have an idea
>>>>        what a given method does normally when____
>>>>
>>>>        there is no class extension.____
>>>>
>>>>        __ __
>>>>
>>>>        Can anyone recommend a place to start or a source of info on the
>>>>        InfosetOutputter family of classes?____
>>>>
>>>>        __ __
>>>>
>>>>        I am working in NetBeans and every class I find seems to
>>>> indicate that:____
>>>>
>>>>        Javadoc not found. Either Javadoc documentation for this item does not exist
>>>>        or ____
>>>>
>>>>                                you have not added specified Javadoc in the Java
>>>>        Platform Manager or the Library ____
>>>>
>>>>                                Manager.____
>>>>
>>>>        __ __
>>>>
>>>>                       Are there Javadoc(s) available corresponding to the Daffodil
>>>>        JAPI classes?____
>>>>
>>>>        __ __
>>>>
>>>>                       It feels like I’m missing some basic configuration setup
>>>>        settings or files. Any guidance or suggestions will____
>>>>
>>>>                       be greatly appreciated.____
>>>>
>>>>        __ __
>>>>
>>>>        Thank you for your time,____
>>>>
>>>>           Mark M. Thompson____
>>>>
>>>>           Northrop Grumman Defense Systems____
>>>>
>>>>           Software Engineer____
>>>>
>>>>           (818) 712-7439____
>>>>
>>>>        __ __
>>>>


Re: EXT :Re: Re: Re: Re: Extending JAPI for processing binary input with an XML Payload

Posted by Steve Lawrence <sl...@apache.org>.
For 1, you would need to catch that exception and rethrow as an SDE 
exception, if you were to go that route.

For 2, if we did not have the InfosetOutputter do anything, then 
Daffodil would output the embedded XML string with escapes, so it might 
look something like this:

   <root>
     <payload>&lt;xml&gt;&lt;foo/&gt;&lt;bar/&gt;&lt;/xml&gt;</payload>
   </root>

That's hard to use and validate without some kind of post processing. So 
the idea was that the enhanced InfosetOutputter would only be 
responsible for outputting the payload as unescaped raw XML, to get 
something like this:

   <root>
     <payload><xml><foo/><bar/></xml></payload>
   </root>

The InfosetOutputter wouldn't actually care if the resulting XML was 
valid, and may not even care if it was well formed.

That resulting XML would then be fed to some other process outside of 
Daffodil to do the actual validation. Likely that would use a schema 
that combines the the DFDL schema and payload schema into a single 
schema, where the "payload" element is no longer an xs:string, but is a 
complex type with the payload schema as its child element.




On 5/15/22 2:11 PM, Thompson, Mark M [US] (DS) wrote:
> Steve,
>           Thank you for your quick response. We will have to give some thought as
> to how we
>           can get the results needed for the XML payload validation. Maybe we can
> pull out the element with
>           RuntimeProperties = “stringAsXML=true” attribute and validate it where
> we parse the overall binary
>           Input.
>           A few points:
> 
>   1. The exception generated is not from our code but from the call to
> 
> validator.validate(new DOMSource(document));
> 
>   2. If, when envisioning the embedded XML feature, the validation of the payload
>      would not take place in the InfosetOutputter, why does the example provided
>      extend InfossetOutputter?
> 
> What am I missing?
> Thank you for your time,
>     Mark T
>                   .
> -----Original Message-----
> From: Steve Lawrence <sl...@apache.org>
> Sent: Friday, May 13, 2022 5:00 AM
> To: users@daffodil.apache.org
> Subject: EXT :Re: Re: Re: Re: Extending JAPI for processing binary input with an
> XML Payload
> There isn't currently a way for InfosetOutputter's to add to the diagnostics. In
> order to do so, an outputter would need access to the PState (where diagnostics
> are stored), which would require an API change. I'm not sure we'd want to give
> direct access to the PState though--some discussions would be needed to figure
> out what that API would look like to get diagnostics out of a outputter and into
> the PState.
> Also, note that throwing an Exception, whether it be a SAXException or something
> else, won't really give the behavior your want. That exception will just bubble
> up and lead to an "Abort" error about an unexpected exception being thrown.
> The one exception you could throw is a SchemaDefinitionError and that will be
> added to diagnostics. But it will immediately stop processing and you will not
> get a complete infoset. This is different than normal validation that Daffodil
> does which happens at the end of processing and does not prevent the creation of
> an infoset.
> When we first envisioned this embedded XML feature, we imagined that the
> validation of the payload would not take place in the InfosetOutputter, but
> instead would happen after the parse completed in some other process that is
> purely for validation. This process would use a schema that combines the DFDL
> schema with the schema for the embedded XML so that it would validate correctly.
> If you really wanted to do it all in Daffodil, maybe another option would be to
> create a custom Validator. This Validator would need to find/extract/unescape
> the payload XML (the XML that the Validator gets is not the same as what the
> InfosetOutputter creates), and then you can do whatever validation is necessary.
> The Validator API does support returning errors/warnings from validation and
> adding them to the diagnostics.
> On 5/12/22 9:55 PM, Thompson, Mark M [US] (DS) wrote:
>> Hello again from XML Payload world,
>>           - We have succeeded in implementing an extension of 
>> XMLInfosetOutputter that will parse and validation a binary file
>>           with an XML payload. It will validate and report if the 
>> binary fields do not conform to the DFDL schema facet restrictions.
>>            - We have also implemented a routine to validate the XML 
>> payload against a schema which will report if the payload is
>>           malformed. An exception (SAXException) is generated in this case.
>>           What is not clear is how one gets the Diagnostics to be 
>> populated with this info or, if they are populated, how to make
>>           the Diagnostics available to the routine that called
>> DataProcessor.Parse() in the first place. The Diagnostics are available
>>           here for the validation of the binary inputs against the 
>> corresponding schema but not those reported for the XML payload.
>>           Do I need to extend the ErrorHandler class and use the 
>> warning and/or error methods to populate Diagnostics as part of the 
>> Exception handling for code below?
>>           The following is an excerpt of the code that seems to do 
>> what I want except for making the SAXException info available.
>> document = db.parse(new InputSource (new StringReader (xmlPayload)));
>>                    // load a WXS schema, represented by a Schema instance
>>                    StreamSource schemaFile = new StreamSource(new 
>> File(schemaFileName));
>>                    Schema schema = s_factory.newSchema(schemaFile);
>>                    // create a Validator instance, which can be used 
>> to validate an instance document
>>                    javax.xml.validation.Validator validator = schema.newValidator();
>>                    System.out.println("**** validateXMLPayload - Try 
>> to validate XML Payload.");
>>                    // validate the DOM tree
>>                    validator.validate(new DOMSource(document));
>>           If anyone can provide a clue as to what I’m missing or 
>> invalid assumptions, I would greatly appreciate it.
>> Thank you for your time,
>>     Mark T
>> -----Original Message-----
>> From: Steve Lawrence <slawrence@apache.org <ma...@apache.org>>
>> Sent: Thursday, April 28, 2022 4:36 PM
>> To: users@daffodil.apache.org <ma...@daffodil.apache.org>
>> Subject: EXT :Re: Re: Re: Extending JAPI for processing binary input 
>> with an XML Payload It looks like our InfosetOutputter API isn't very 
>> Java friendly, at least once you start needing to get information out 
>> of the ERD. I'd open a ticket, but it looks like JIRA is down at the 
>> moment.
>> I think the way to fix your issue is to just use straight reference 
>> equality comparison with NodeInfo.String(), e.g.:
>>      if (simple.erd().optPrimType().get() == NodeInfo.String()) { ... 
>> } The instance that NodeInfo.String() returns is essentially a 
>> singleton to represent DFDL String primitives, which is also what is 
>> returned from optPrimtType().get().
>> I'm not sure why our InfosetOutputter uses isInstanceOf[String.Kind] 
>> instead of just using reference equality with NodeInfo.String. Might 
>> have been old code that never got updated...
>> On 4/28/22 1:24 PM, Thompson, Mark M [US] (DS) wrote:
>>> Hello all,
>>> 
>>>        We are attempting to generate a Java class for this XML payload by extending XMLInfosetOutputter using XMLTextInfosetOUtputter
>>>        As a guide.
>>>        We're having trouble coming up with a Java equivalent for the StartSimple() method code:
>>>                  if
>>> (simple.erd.optPrimType.get.isInstanceOf[NodeInfo.String.Kind]) {
>>> 
>>>        We've tried the following with no luck:
>>>                if (simple.erd().optPrimType().get() instanceof NodeInfo.String().Kind) {
>>>        and
>>>                       if
>>> (simple.erd().optPrimType().get().isInstanceOf(NodeInfo.String().???)
>>> )
>>> {
>>> 
>>>        Any input or suggestions are greatly appreciated.
>>> 
>>> Thank you for your time,
>>>    Mark T
>>> 
>>> -----Original Message-----
>>> From: Steve Lawrence <slawrence@apache.org 
>>> <mailto:slawrence@apache.org <ma...@apache.org>>>
>>> Sent: Thursday, April 21, 2022 4:31 AM
>>> To: users@daffodil.apache.org <ma...@daffodil.apache.org> 
> <mailto:users@daffodil.apache.org <ma...@daffodil.apache.org>>
>>> Subject: EXT :Re: Re: Extending JAPI for processing binary input with 
>>> an XML Payload
>>> 
>>>   > Contributing to Daffodil is likely not an option for my company.
>>> 
>>> That's sad to hear, and I hope you can try to persuade your company to change this position. Opensource works best with many contributors from many different organizations. And if your company depends on Daffodil, contributing back to it can help ensure its success and continued growth.
>>> 
>>>   > Based on my limited understanding, this seems to suggest that I 
>>> would have to add another infoset type to make use of the new class
>>> 
>>> If you're using the CLI, this is correct. Infoset inputters and 
>>> outputters must currently be hardcoded in. If you're using the API to 
>>> invoke Daffodil this isn't a problem, but if you're using the CLI 
>>> then you must modify the CLI to support your new classes.  This 
>>> hasn't been an issue so far since anytime a new one is needed we just
>> added it to Daffodil.
>>> 
>>> Part of the reason for this is that creating InfosetInputters/Outputters and getting the result is very specific to the infoset representation.
>>> It's probably possible that the API could be made more generic, and then we could use a ServiceLoader or something to support pluggable infoset inputters/outputters, but that hasn't be a priority so far.
>>> 
>>> On 4/20/22 10:08 PM, Thompson, Mark M [US] (DS) wrote:
>>>> Hi Steve,
>>>>
>>>>       I have been looking into your suggestions and researching the existing code and here
>>>>       Is where I'm at:
>>>>       - Contributing to Daffodil is likely not an option for my company.
>>>>       - I've been looking into XMLTextInfosetInputter and ScalaXMLInfosetOutputter for insight.
>>>>          What I don't understand (Java and Scala newby) is, having extended InfosetOutputter with a new class,
>>>>          the Daffodil parse operation magically has access to this new class. I see, in main.scala, where
>>>>          XMLTextInfosetOutputter and ScalaXMLInfosetOutputter are instantiated based on command line
>>>>          Input for infoset type. Based on my limited understanding, this seems to suggest that I would have
>>>>          To add another infoset type to make use of the new class.
>>>>
>>>>        Can someone clear up my misunderstanding or point to some info that will cause the scales to fall
>>>>       From my eyes?
>>>>
>>>> Thank you for your time,
>>>>      Mark T
>>>>
>>>> -----Original Message-----
>>>> From: Steve Lawrence <slawrence@apache.org 
>>>> <mailto:slawrence@apache.org <ma...@apache.org>>>
>>>> Sent: Friday, April 8, 2022 6:31 AM
>>>> To: users@daffodil.apache.org <ma...@daffodil.apache.org> 
> <mailto:users@daffodil.apache.org <ma...@daffodil.apache.org>>
>>>> Subject: EXT :Re: Extending JAPI for processing binary input with an 
>>>> XML Payload
>>>>
>>>> I've put a decent amount of thought into this  issue, I'll add some additional info that might help out...
>>>>
>>>>    > for example, the startSimple() method. It would be helpful to have an idea what a given method does normally when there is no class extension.
>>>>
>>>> The InfosetOutputter class is an abstract class with no default implementations, so an implementation of all functions, including startSimple, is required--InfosetInputters don't do anything by default.
>>>> The InfosetOutputters that are bundled with Daffodil implement all of these functions to create the kind of infoset that is output (e.g. XML text, Scala Nodes, JSON). If you want to take a look at how our bundled implementations handle their logic, they are all in this folder:
>>>>
>>>> https://github.com/apache/daffodil/tree/main/daffodil-runtime1/src/m 
> <https://github.com/apache/daffodil/tree/main/daffodil-runtime1/src/m>
>>>> a
>> <https://github.com/apache/daffodil/tree/main/daffodil-runtime1/src/ma
>> >
>>>> i
>>>> n/scala/org/apache/daffodil/infoset
>>>>
>>>> If you plan to keep your InfosetOutputter outside of the Daffodil codebase, you'll need to implement all of the functions/logic yourself.
>>>> Our current implementions are mostly private, so extending and overriding them to do you you want is likely not possible.
>>>>
>>>> A good place to start would be to pick one that that's close to what you want and modify the various functions as described in the example implementation in the runtime properties proposal:
>>>>
>>>> https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+Run 
> <https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+Run>
>>>> t
>> <https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+Runt
>> >
>>>> i
>>>> me+Properties
>>>>
>>>> Alternatively, it might be easier to just add the support to our 
>>>> existing bundled InfosetOutputters to support this capability, and 
>>>> contribute that work to Daffodil. Not only will this probably be 
>>>> easier since you can reuse alot of the existing logic,  but our 
>>>> ability to provide support greatly increases when it's related to
>> things being added to Daffodil. This feature is a request we've seen 
>> multiple times--it would be a very welcome addition and we'd be happy 
>> help get it added to Daffodil.
>>>>
>>>>
>>>>
>>>> On 4/8/22 8:42 AM, Mike Beckerle wrote:
>>>>> Mark,
>>>>>
>>>>> There definitely are javadoc jars, and the javadoc is also online 
>>>>> here (for release 3.3.0) 
>>>>> https://daffodil.apache.org/docs/latest/javadoc/ 
> <https://daffodil.apache.org/docs/latest/javadoc/>
>> <https://daffodil.apache.org/docs/latest/javadoc/ 
> <https://daffodil.apache.org/docs/latest/javadoc/>>
>>>>> <https://daffodil.apache.org/docs/latest/javadoc/
>> <https://daffodil.apache.org/docs/latest/javadoc/ 
> <https://daffodil.apache.org/docs/latest/javadoc/>>>
>>>>>
>>>>> The source and javadoc jars are normally pulled down using maven or 
>>>>> sbt. But you can find them directly on maven central also.
>>>>>
>>>>> For example for the daffodil-runtime1 jar, all the jars, compiled 
>>>>> binary, sources, and the javadoc jar is at
>>>>> https://repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtime 
> <https://repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtime>
>>>>> 1
>> <https://repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtime1 
> <https://repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtime1>>
>>>>> _
>>>>> 2
>>>>> .12/3.3.0/
>>>>> <https://repo1.maven.org/maven2/org/apache/daffodil/daffodil-runtim
>>>>> e
>>>>> 1
>>>>> _
>>>>> 2.12/3.3.0/>
>>>>>
>>>>>
>>>>> On Thu, Apr 7, 2022 at 10:07 PM Thompson, Mark M [US] (DS) 
>>>>> <mark.m.thompson@ngc.com <mailto:mark.m.thompson@ngc.com
>> <mailto:mark.m.thompson@ngc.com <mailto:mark.m.thompson@ngc.com 
> <mailto:mark.m.thompson@ngc.com <ma...@ngc.com>>>> wrote:
>>>>>
>>>>>        Greetings to All,____
>>>>>
>>>>>        __ __
>>>>>
>>>>>                       Caveat: I am new to Daffodil and DFDL. I’m familiar with the
>>>>>        basic functionality and have successfully written____
>>>>>
>>>>>                                      DFDL schemas to handle text and binary inputs
>>>>>        to the parse/unparse cycle (separate assignments). ____
>>>>>
>>>>>        __ __
>>>>>
>>>>>        I am working on a task that requires the processing of binary files
>>>>>        containing an XML payload. The goal is to be____
>>>>>
>>>>>                       able to validate the raw (un-escapified) XML against an XML
>>>>>        schema during the parsing phase. I have been researching____
>>>>>
>>>>>                       the Daffodil JAPI InfosetOutputter family of classes as a
>>>>>        first step. I’ve also read Steve Lawrence’s write-up on the 
>>>>> ____
>>>>>
>>>>>                       RuntimePropertiies proposal. What I have yet to grasp or find
>>>>>        is what InfosetOutputter functionality is to be extended 
>>>>> .____
>>>>>
>>>>>                       I’ve searched high and low but I have yet to find any
>>>>>        detailed information on what basic functionality is provided 
>>>>> by,____
>>>>>
>>>>>        for example, the startSimple() method. It would be helpful to have an idea
>>>>>        what a given method does normally when____
>>>>>
>>>>>        there is no class extension.____
>>>>>
>>>>>        __ __
>>>>>
>>>>>        Can anyone recommend a place to start or a source of info on the
>>>>>        InfosetOutputter family of classes?____
>>>>>
>>>>>        __ __
>>>>>
>>>>>        I am working in NetBeans and every class I find seems to 
>>>>> indicate that:____
>>>>>
>>>>>        Javadoc not found. Either Javadoc documentation for this item does not exist
>>>>>        or ____
>>>>>
>>>>>                                you have not added specified Javadoc in the Java
>>>>>        Platform Manager or the Library ____
>>>>>
>>>>>                                Manager.____
>>>>>
>>>>>        __ __
>>>>>
>>>>>                       Are there Javadoc(s) available corresponding to the Daffodil
>>>>>        JAPI classes?____
>>>>>
>>>>>        __ __
>>>>>
>>>>>                       It feels like I’m missing some basic configuration setup
>>>>>        settings or files. Any guidance or suggestions will____
>>>>>
>>>>>                       be greatly appreciated.____
>>>>>
>>>>>        __ __
>>>>>
>>>>>        Thank you for your time,____
>>>>>
>>>>>           Mark M. Thompson____
>>>>>
>>>>>           Northrop Grumman Defense Systems____
>>>>>
>>>>>           Software Engineer____
>>>>>
>>>>>           (818) 712-7439____
>>>>>
>>>>>        __ __
>>>>>