You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by "Sloane, Brandon" <bs...@owlcyberdefense.com> on 2021/01/24 19:08:02 UTC
Possible road map for GUI debugger
Following up on the recent discussion about the Daffodil debugger, I would like to propose a roadmap for creating graphical debugger.
Organizationally, the GUI debugger is a separate application (maintained in the normal Daffodil repository) that may include any of the current Daffodil libraries as a dependency.
Phases 1 and 2 focus on developing an enhanced trace functionality; where Daffodil outputs a machine readable trace of the parse/unparse process. The GUI then allows a user to view this trace offline.
Phase 3 adds support for interactive debugging, by allowing the Daffodil-cli and Daffodil-gui-debugger to act in a client/server relationship
Phase 4 focuses on backend debugger enhancements.
In more detail:
Phase 1:
* Add a --machine-trace flag to Daffodil CLI, that is analogous to the current --trace option. Output is a stream of XML parse-state nodes, each one describing the current state of the parse, including:
* Current infoset
* Current bit position
* Current parser
* A UID of the current state
* A UID of the parent state (in event of backtracking, multiple states will have the same parent).
* Machine-trace output stream also include input-data nodes, that include:
* A region of (byte aligned) input data (hex encoded?)
* The byte position of the start of the region
* The relative location of the input data nodes is undefined, but given the complete trace, it should be possible to reconstruct the original dataset by stitching together the data in the nodes. The goal here is that we can just output data as Daffodil reads it in.
* --machine-trace take an optional argument of a file path. If provided, its output will be directed to a file at said location (otherwise stderr)
* Create a GUI program that can ingest the machine trace and display it in a 4-pane view:
* Pane 1 - trace navigator
* Represents the entire parse in a tree format. Each parse state corresponds to a node on the tree
* The user can select a node on the tree to load it into the other panes
* Pane 2 - Infoset viewer
* Shows the complete infoset corresponding to the node selected in pane 1.
* Indicates the "current" position of the parser within the infoset
* Highlights changes in the infoset relative to the parent node.
* Pane 3 - binary viewer
* Shows a complete hex-dump of the input data, indicating current location within the parse
* Pane 3 -- parser view
* Shows the current parser. Probably will be a dumping ground for additional information that does not warrant the addition of a new pane.
Phase 2:
* Support showing suspensions on unparse
* Display additional data about the parse/unparse state:
* Delimeter stack
* Variables
* Support additional formats for displaying input data
* Text (of selectable encoding)
* Binary (not hex; actual 1s and 0s)
* Support generating traces through the API as well.
Phase 2.5:
Gather user feedback and iterate on the trace viewer functionality
Phase 3:
* Define protocol for the debugger to talk with the CLI.
* Debugger can load schema, and input data.
* Debugger can step through the parse/unparse
* Viewer built in phases 1 and 2 continually updates as the parse proceeds
* User can send CLI debugger commands and get the textual response; but only for the current parse/unparse state.
* User can set breakpoints (possibly using textual commands). Debugger supports continue until breakpoint
Phase 3.5:
* Not debugger related, but as long as the Daffodil CLI can operate as a server, support server mode operations for non-debug use cases.
Phase 4:
* Support time travel debugging, with an interface in both the CLI and GUI debuggers
* Support associating a trace with schema source, and allowing a "go to source" functionality where the user can navigate directly to the location in source corresponding to the current parser
* Support dynamically editing/reloading schema and input data
Thoughts?
Brandon Sloane | Engineer
[cid:e3942e8c-91b8-46c9-946e-a7d444bda4a7]
bsloane@owlcyberdefense.com
Connect with us!
[cid:54cd2295-7dba-42bb-a70f-0a93aa44b072]<https://www.linkedin.com/company/owlcyberdefense/>[cid:7d97c978-b1c6-4c20-b747-570920dded63]<https://twitter.com/owlcyberdefense>
[cid:1e0d1441-a879-4f36-bd0a-7fc68bb1d58e]<https://owlcyberdefense.com/resources/events/>
The information contained in this transmission is for the personal and confidential use of the individual or entity to which it is addressed. If the reader is not the intended recipient, you are hereby notified that any review, dissemination, or copying of this communication is strictly prohibited. If you have received this transmission in error, please notify the sender immediately
Re: Possible road map for GUI debugger
Posted by Steve Lawrence <sl...@apache.org>.
This seems like a very reasonable roadmap for the future debugger.
Unfortunately, as great as all the feedback as been from everyone, I
think my main focus right now is going to be just getting the current
debugger to a more functional state, and if there is time after that, to
focus on making a more machine readable output and and wrapping that
around a new GUI. I absolutely think what you've described is where we
want to head towards and a reasonable approach to get there, but is
still likely more effort than I can put towards this at the moment.
On 1/24/21 2:08 PM, Sloane, Brandon wrote:
> Following up on the recent discussion about the Daffodil debugger, I would like
> to propose a roadmap for creating graphical debugger.
> Organizationally, the GUI debugger is a separate application (maintained in the
> normal Daffodil repository) that may include any of the current Daffodil
> libraries as a dependency.
>
> Phases 1 and 2 focus on developing an enhanced trace functionality; where
> Daffodil outputs a machine readable trace of the parse/unparse process. The GUI
> then allows a user to view this trace offline.
>
> Phase 3 adds support for interactive debugging, by allowing the Daffodil-cli and
> Daffodil-gui-debugger to act in a client/server relationship
>
> Phase 4 focuses on backend debugger enhancements.
>
> In more detail:
>
> Phase 1:
>
> * Add a --machine-trace flag to Daffodil CLI, that is analogous to the current
> --trace option. Output is a stream of XML parse-state nodes, each one
> describing the current state of the parse, including:
> o Current infoset
> o Current bit position
> o Current parser
> o A UID of the current state
> o A UID of the parent state (in event of backtracking, multiple states
> will have the same parent).
> * Machine-trace output stream also include input-data nodes, that include:
> o A region of (byte aligned) input data (hex encoded?)
> o The byte position of the start of the region
> o The relative location of the input data nodes is undefined, but given
> the complete trace, it should be possible to reconstruct the original
> dataset by stitching together the data in the nodes. The goal here is
> that we can just output data as Daffodil reads it in.
> * --machine-trace take an optional argument of a file path. If provided, its
> output will be directed to a file at said location (otherwise stderr)
> * Create a GUI program that can ingest the machine trace and display it in a
> 4-pane view:
> o Pane 1 - trace navigator
> + Represents the entire parse in a tree format. Each parse state
> corresponds to a node on the tree
> + The user can select a node on the tree to load it into the other panes
> o Pane 2 - Infoset viewer
> + Shows the complete infoset corresponding to the node selected in pane 1.
> + Indicates the "current" position of the parser within the infoset
> + Highlights changes in the infoset relative to the parent node.
> o Pane 3 - binary viewer
> + Shows a complete hex-dump of the input data, indicating current
> location within the parse
> o Pane 3 -- parser view
> + Shows the current parser. Probably will be a dumping ground for
> additional information that does not warrant the addition of a new pane.
>
>
> Phase 2:
>
> * Support showing suspensions on unparse
> * Display additional data about the parse/unparse state:
> o Delimeter stack
> o Variables
> * Support additional formats for displaying input data
> o Text (of selectable encoding)
> o Binary (not hex; actual 1s and 0s)
> * Support generating traces through the API as well.
>
> Phase 2.5:
>
> Gather user feedback and iterate on the trace viewer functionality
>
> Phase 3:
>
> * Define protocol for the debugger to talk with the CLI.
> * Debugger can load schema, and input data.
> * Debugger can step through the parse/unparse
> * Viewer built in phases 1 and 2 continually updates as the parse proceeds
> * User can send CLI debugger commands and get the textual response; but only
> for the current parse/unparse state.
> * User can set breakpoints (possibly using textual commands). Debugger
> supports continue until breakpoint
>
> Phase 3.5:
>
> * Not debugger related, but as long as the Daffodil CLI can operate as a
> server, support server mode operations for non-debug use cases.
>
>
> Phase 4:
>
> * Support time travel debugging, with an interface in both the CLI and GUI
> debuggers
> * Support associating a trace with schema source, and allowing a "go to
> source" functionality where the user can navigate directly to the location
> in source corresponding to the current parser
> * Support dynamically editing/reloading schema and input data
>
> Thoughts?
> Brandon Sloane | Engineer
>
> bsloane@owlcyberdefense.com
>
> Connect with us!
>
> <https://www.linkedin.com/company/owlcyberdefense/><https://twitter.com/owlcyberdefense>
>
> <https://owlcyberdefense.com/resources/events/>
>
> **
>
> The information contained in this transmission is for the personal and
> confidential use of the individual or entity to which it is addressed. If the
> reader is not the intended recipient, you are hereby notified that any review,
> dissemination, or copying of this communication is strictly prohibited. If you
> have received this transmission in error, please notify the sender immediately
>
Re: Possible road map for GUI debugger
Posted by "Sloane, Brandon" <bs...@owlcyberdefense.com>.
In my opinion, the nature of DFDL makes it much more amenable to a read-only time-travel style of debugging. We kind of already support this with the --trace functionality, but the UI there (a giant text dump) is less then ideal. The mininimal viable product for this would be a reader for such a trace dump). Everything beyond that point is enhancements. With all the enhancements we fully blur the line between glorified log viewer and full time-traveling debugger.
In addition to the debug environment being based around time travel; the Daffodil runtime itself is based around time travel (backtracing for parse, and suspensions for unsparse).
There are additional mismatches between an ideal DFDL debugging environment and a normal debugger.
With most programming langauges the central objects for debugging are variables and the stacktrace. For DFDL, the central objects are the infoset and datastream.
The main benefit I see with an IDE integration are easier integrations with a source code viewer/editor. This is no small thing, but is also our lowest priority and delegated to phase 4.
I also see this as an excersise in learning how a DFDL debug environment ought to look like. Since we know (or, at least I think) a traditional debug view does not match well, I would prefer that we avoid fighting with an IDEs notion before we know what it is we want.
What we might want to re-evaluate our plans at phase 3. This is when we will be adding the communication protocal an interactive debugger would need to use. At this point we can ask if it is easier to get the IDE to understand how we want to display information, or if it is easier to implement/integrate whatever IDE functionality we want into our own program.
Hopefully at that point, we have a better idea of what it is we are building.
________________________________
From: John Wass <jw...@gmail.com>
Sent: Tuesday, February 16, 2021 7:31 AM
To: Sloane, Brandon <bs...@owlcyberdefense.com>
Cc: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: Re: Possible road map for GUI debugger
> I am conceiving of it as a standalone utility.
I'm interested in the reasoning there. No strong opinions, I can see some advantages either way, but an IDE integration does seem to have a head start in several areas related to debugging.
On Mon, Feb 15, 2021 at 1:27 PM Sloane, Brandon <bs...@owlcyberdefense.com>> wrote:
We are at the very early stages. I am conceiving of it as a standalone utility.
Brandon Sloane | Engineer
[cid:177aacd632cba97c0f91]
bsloane@owlcyberdefense.com<ma...@owlcyberdefense.com>
Connect with us!
[cid:177aacd632cd46063f02]<https://www.linkedin.com/company/owlcyberdefense/>[cid:177aacd632c9d1c8fa93]<https://twitter.com/owlcyberdefense>
[cid:177aacd632cc519938a4]<https://owlcyberdefense.com/resources/events/>
The information contained in this transmission is for the personal and confidential use of the individual or entity to which it is addressed. If the reader is not the intended recipient, you are hereby notified that any review, dissemination, or copying of this communication is strictly prohibited. If you have received this transmission in error, please notify the sender immediately
________________________________
From: John Wass <jw...@gmail.com>>
Sent: Monday, February 15, 2021 7:39 AM
To: dev@daffodil.apache.org<ma...@daffodil.apache.org> <de...@daffodil.apache.org>>
Subject: Re: Possible road map for GUI debugger
Is the gui envisioned to be an ide plugin or a standalone application (ie
swing, fx, ...)?
On Sun, Jan 24, 2021 at 2:08 PM Sloane, Brandon <bs...@owlcyberdefense.com>>
wrote:
> Following up on the recent discussion about the Daffodil debugger, I would
> like to propose a roadmap for creating graphical debugger.
> Organizationally, the GUI debugger is a separate application (maintained
> in the normal Daffodil repository) that may include any of the current
> Daffodil libraries as a dependency.
>
> Phases 1 and 2 focus on developing an enhanced trace functionality; where
> Daffodil outputs a machine readable trace of the parse/unparse process. The
> GUI then allows a user to view this trace offline.
>
> Phase 3 adds support for interactive debugging, by allowing the
> Daffodil-cli and Daffodil-gui-debugger to act in a client/server
> relationship
>
> Phase 4 focuses on backend debugger enhancements.
>
> In more detail:
>
> Phase 1:
>
> - Add a --machine-trace flag to Daffodil CLI, that is analogous to the
> current --trace option. Output is a stream of XML parse-state nodes, each
> one describing the current state of the parse, including:
> - Current infoset
> - Current bit position
> - Current parser
> - A UID of the current state
> - A UID of the parent state (in event of backtracking, multiple
> states will have the same parent).
> - Machine-trace output stream also include input-data nodes, that
> include:
> - A region of (byte aligned) input data (hex encoded?)
> - The byte position of the start of the region
> - The relative location of the input data nodes is undefined, but
> given the complete trace, it should be possible to reconstruct the original
> dataset by stitching together the data in the nodes. The goal here is that
> we can just output data as Daffodil reads it in.
> - --machine-trace take an optional argument of a file path. If
> provided, its output will be directed to a file at said location (otherwise
> stderr)
> - Create a GUI program that can ingest the machine trace and display
> it in a 4-pane view:
> - Pane 1 - trace navigator
> - Represents the entire parse in a tree format. Each parse state
> corresponds to a node on the tree
> - The user can select a node on the tree to load it into the
> other panes
> - Pane 2 - Infoset viewer
> - Shows the complete infoset corresponding to the node selected
> in pane 1.
> - Indicates the "current" position of the parser within the
> infoset
> - Highlights changes in the infoset relative to the parent node.
> - Pane 3 - binary viewer
> - Shows a complete hex-dump of the input data, indicating
> current location within the parse
> - Pane 3 -- parser view
> - Shows the current parser. Probably will be a dumping ground
> for additional information that does not warrant the addition of a new pane.
>
>
> Phase 2:
>
> - Support showing suspensions on unparse
> - Display additional data about the parse/unparse state:
> - Delimeter stack
> - Variables
> - Support additional formats for displaying input data
> - Text (of selectable encoding)
> - Binary (not hex; actual 1s and 0s)
> - Support generating traces through the API as well.
>
> Phase 2.5:
>
> Gather user feedback and iterate on the trace viewer functionality
>
> Phase 3:
>
> - Define protocol for the debugger to talk with the CLI.
> - Debugger can load schema, and input data.
> - Debugger can step through the parse/unparse
> - Viewer built in phases 1 and 2 continually updates as the parse
> proceeds
> - User can send CLI debugger commands and get the textual response;
> but only for the current parse/unparse state.
> - User can set breakpoints (possibly using textual commands). Debugger
> supports continue until breakpoint
>
> Phase 3.5:
>
>
> - Not debugger related, but as long as the Daffodil CLI can operate as
> a server, support server mode operations for non-debug use cases.
>
>
> Phase 4:
>
> - Support time travel debugging, with an interface in both the CLI and
> GUI debuggers
> - Support associating a trace with schema source, and allowing a "go
> to source" functionality where the user can navigate directly to the
> location in source corresponding to the current parser
> - Support dynamically editing/reloading schema and input data
>
> Thoughts?
> Brandon Sloane | Engineer
>
> bsloane@owlcyberdefense.com<ma...@owlcyberdefense.com>
>
> Connect with us!
>
> <https://www.linkedin.com/company/owlcyberdefense/>
> <https://twitter.com/owlcyberdefense>
>
> <https://owlcyberdefense.com/resources/events/>
>
>
>
> The information contained in this transmission is for the personal and
> confidential use of the individual or entity to which it is addressed. If
> the reader is not the intended recipient, you are hereby notified that any
> review, dissemination, or copying of this communication is strictly
> prohibited. If you have received this transmission in error, please notify
> the sender immediately
>
>
Re: Possible road map for GUI debugger
Posted by John Wass <jw...@gmail.com>.
> I am conceiving of it as a standalone utility.
I'm interested in the reasoning there. No strong opinions, I can see some
advantages either way, but an IDE integration does seem to have a head
start in several areas related to debugging.
On Mon, Feb 15, 2021 at 1:27 PM Sloane, Brandon <bs...@owlcyberdefense.com>
wrote:
> We are at the very early stages. I am conceiving of it as a standalone
> utility.
>
>
>
> Brandon Sloane | Engineer
>
> bsloane@owlcyberdefense.com
>
> Connect with us!
>
> <https://www.linkedin.com/company/owlcyberdefense/>
> <https://twitter.com/owlcyberdefense>
>
> <https://owlcyberdefense.com/resources/events/>
>
>
>
> The information contained in this transmission is for the personal and
> confidential use of the individual or entity to which it is addressed. If
> the reader is not the intended recipient, you are hereby notified that any
> review, dissemination, or copying of this communication is strictly
> prohibited. If you have received this transmission in error, please notify
> the sender immediately
>
> ------------------------------
> *From:* John Wass <jw...@gmail.com>
> *Sent:* Monday, February 15, 2021 7:39 AM
> *To:* dev@daffodil.apache.org <de...@daffodil.apache.org>
> *Subject:* Re: Possible road map for GUI debugger
>
> Is the gui envisioned to be an ide plugin or a standalone application (ie
> swing, fx, ...)?
>
>
>
> On Sun, Jan 24, 2021 at 2:08 PM Sloane, Brandon <
> bsloane@owlcyberdefense.com>
> wrote:
>
> > Following up on the recent discussion about the Daffodil debugger, I
> would
> > like to propose a roadmap for creating graphical debugger.
> > Organizationally, the GUI debugger is a separate application (maintained
> > in the normal Daffodil repository) that may include any of the current
> > Daffodil libraries as a dependency.
> >
> > Phases 1 and 2 focus on developing an enhanced trace functionality; where
> > Daffodil outputs a machine readable trace of the parse/unparse process.
> The
> > GUI then allows a user to view this trace offline.
> >
> > Phase 3 adds support for interactive debugging, by allowing the
> > Daffodil-cli and Daffodil-gui-debugger to act in a client/server
> > relationship
> >
> > Phase 4 focuses on backend debugger enhancements.
> >
> > In more detail:
> >
> > Phase 1:
> >
> > - Add a --machine-trace flag to Daffodil CLI, that is analogous to the
> > current --trace option. Output is a stream of XML parse-state nodes,
> each
> > one describing the current state of the parse, including:
> > - Current infoset
> > - Current bit position
> > - Current parser
> > - A UID of the current state
> > - A UID of the parent state (in event of backtracking, multiple
> > states will have the same parent).
> > - Machine-trace output stream also include input-data nodes, that
> > include:
> > - A region of (byte aligned) input data (hex encoded?)
> > - The byte position of the start of the region
> > - The relative location of the input data nodes is undefined, but
> > given the complete trace, it should be possible to reconstruct the
> original
> > dataset by stitching together the data in the nodes. The goal here
> is that
> > we can just output data as Daffodil reads it in.
> > - --machine-trace take an optional argument of a file path. If
> > provided, its output will be directed to a file at said location
> (otherwise
> > stderr)
> > - Create a GUI program that can ingest the machine trace and display
> > it in a 4-pane view:
> > - Pane 1 - trace navigator
> > - Represents the entire parse in a tree format. Each parse state
> > corresponds to a node on the tree
> > - The user can select a node on the tree to load it into the
> > other panes
> > - Pane 2 - Infoset viewer
> > - Shows the complete infoset corresponding to the node selected
> > in pane 1.
> > - Indicates the "current" position of the parser within the
> > infoset
> > - Highlights changes in the infoset relative to the parent node.
> > - Pane 3 - binary viewer
> > - Shows a complete hex-dump of the input data, indicating
> > current location within the parse
> > - Pane 3 -- parser view
> > - Shows the current parser. Probably will be a dumping ground
> > for additional information that does not warrant the addition
> of a new pane.
> >
> >
> > Phase 2:
> >
> > - Support showing suspensions on unparse
> > - Display additional data about the parse/unparse state:
> > - Delimeter stack
> > - Variables
> > - Support additional formats for displaying input data
> > - Text (of selectable encoding)
> > - Binary (not hex; actual 1s and 0s)
> > - Support generating traces through the API as well.
> >
> > Phase 2.5:
> >
> > Gather user feedback and iterate on the trace viewer functionality
> >
> > Phase 3:
> >
> > - Define protocol for the debugger to talk with the CLI.
> > - Debugger can load schema, and input data.
> > - Debugger can step through the parse/unparse
> > - Viewer built in phases 1 and 2 continually updates as the parse
> > proceeds
> > - User can send CLI debugger commands and get the textual response;
> > but only for the current parse/unparse state.
> > - User can set breakpoints (possibly using textual commands). Debugger
> > supports continue until breakpoint
> >
> > Phase 3.5:
> >
> >
> > - Not debugger related, but as long as the Daffodil CLI can operate as
> > a server, support server mode operations for non-debug use cases.
> >
> >
> > Phase 4:
> >
> > - Support time travel debugging, with an interface in both the CLI and
> > GUI debuggers
> > - Support associating a trace with schema source, and allowing a "go
> > to source" functionality where the user can navigate directly to the
> > location in source corresponding to the current parser
> > - Support dynamically editing/reloading schema and input data
> >
> > Thoughts?
> > Brandon Sloane | Engineer
> >
> > bsloane@owlcyberdefense.com
> >
> > Connect with us!
> >
> > <https://www.linkedin.com/company/owlcyberdefense/>
> > <https://twitter.com/owlcyberdefense>
> >
> > <https://owlcyberdefense.com/resources/events/>
> >
> >
> >
> > The information contained in this transmission is for the personal and
> > confidential use of the individual or entity to which it is addressed. If
> > the reader is not the intended recipient, you are hereby notified that
> any
> > review, dissemination, or copying of this communication is strictly
> > prohibited. If you have received this transmission in error, please
> notify
> > the sender immediately
> >
> >
>
Re: Possible road map for GUI debugger
Posted by "Sloane, Brandon" <bs...@owlcyberdefense.com>.
We are at the very early stages. I am conceiving of it as a standalone utility.
Brandon Sloane | Engineer
[cid:3e406d2d-2881-4a74-a244-1b1b4029cd3f]
bsloane@owlcyberdefense.com
Connect with us!
[cid:30741052-3fa7-4bcb-ae04-ff10bde2357f]<https://www.linkedin.com/company/owlcyberdefense/>[cid:ffca3a30-c5c8-4b9c-a1b4-ccb4cb5bbbb3]<https://twitter.com/owlcyberdefense>
[cid:3d6d8c5d-675b-4741-89fc-59ef97665ded]<https://owlcyberdefense.com/resources/events/>
The information contained in this transmission is for the personal and confidential use of the individual or entity to which it is addressed. If the reader is not the intended recipient, you are hereby notified that any review, dissemination, or copying of this communication is strictly prohibited. If you have received this transmission in error, please notify the sender immediately
________________________________
From: John Wass <jw...@gmail.com>
Sent: Monday, February 15, 2021 7:39 AM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: Re: Possible road map for GUI debugger
Is the gui envisioned to be an ide plugin or a standalone application (ie
swing, fx, ...)?
On Sun, Jan 24, 2021 at 2:08 PM Sloane, Brandon <bs...@owlcyberdefense.com>
wrote:
> Following up on the recent discussion about the Daffodil debugger, I would
> like to propose a roadmap for creating graphical debugger.
> Organizationally, the GUI debugger is a separate application (maintained
> in the normal Daffodil repository) that may include any of the current
> Daffodil libraries as a dependency.
>
> Phases 1 and 2 focus on developing an enhanced trace functionality; where
> Daffodil outputs a machine readable trace of the parse/unparse process. The
> GUI then allows a user to view this trace offline.
>
> Phase 3 adds support for interactive debugging, by allowing the
> Daffodil-cli and Daffodil-gui-debugger to act in a client/server
> relationship
>
> Phase 4 focuses on backend debugger enhancements.
>
> In more detail:
>
> Phase 1:
>
> - Add a --machine-trace flag to Daffodil CLI, that is analogous to the
> current --trace option. Output is a stream of XML parse-state nodes, each
> one describing the current state of the parse, including:
> - Current infoset
> - Current bit position
> - Current parser
> - A UID of the current state
> - A UID of the parent state (in event of backtracking, multiple
> states will have the same parent).
> - Machine-trace output stream also include input-data nodes, that
> include:
> - A region of (byte aligned) input data (hex encoded?)
> - The byte position of the start of the region
> - The relative location of the input data nodes is undefined, but
> given the complete trace, it should be possible to reconstruct the original
> dataset by stitching together the data in the nodes. The goal here is that
> we can just output data as Daffodil reads it in.
> - --machine-trace take an optional argument of a file path. If
> provided, its output will be directed to a file at said location (otherwise
> stderr)
> - Create a GUI program that can ingest the machine trace and display
> it in a 4-pane view:
> - Pane 1 - trace navigator
> - Represents the entire parse in a tree format. Each parse state
> corresponds to a node on the tree
> - The user can select a node on the tree to load it into the
> other panes
> - Pane 2 - Infoset viewer
> - Shows the complete infoset corresponding to the node selected
> in pane 1.
> - Indicates the "current" position of the parser within the
> infoset
> - Highlights changes in the infoset relative to the parent node.
> - Pane 3 - binary viewer
> - Shows a complete hex-dump of the input data, indicating
> current location within the parse
> - Pane 3 -- parser view
> - Shows the current parser. Probably will be a dumping ground
> for additional information that does not warrant the addition of a new pane.
>
>
> Phase 2:
>
> - Support showing suspensions on unparse
> - Display additional data about the parse/unparse state:
> - Delimeter stack
> - Variables
> - Support additional formats for displaying input data
> - Text (of selectable encoding)
> - Binary (not hex; actual 1s and 0s)
> - Support generating traces through the API as well.
>
> Phase 2.5:
>
> Gather user feedback and iterate on the trace viewer functionality
>
> Phase 3:
>
> - Define protocol for the debugger to talk with the CLI.
> - Debugger can load schema, and input data.
> - Debugger can step through the parse/unparse
> - Viewer built in phases 1 and 2 continually updates as the parse
> proceeds
> - User can send CLI debugger commands and get the textual response;
> but only for the current parse/unparse state.
> - User can set breakpoints (possibly using textual commands). Debugger
> supports continue until breakpoint
>
> Phase 3.5:
>
>
> - Not debugger related, but as long as the Daffodil CLI can operate as
> a server, support server mode operations for non-debug use cases.
>
>
> Phase 4:
>
> - Support time travel debugging, with an interface in both the CLI and
> GUI debuggers
> - Support associating a trace with schema source, and allowing a "go
> to source" functionality where the user can navigate directly to the
> location in source corresponding to the current parser
> - Support dynamically editing/reloading schema and input data
>
> Thoughts?
> Brandon Sloane | Engineer
>
> bsloane@owlcyberdefense.com
>
> Connect with us!
>
> <https://www.linkedin.com/company/owlcyberdefense/>
> <https://twitter.com/owlcyberdefense>
>
> <https://owlcyberdefense.com/resources/events/>
>
>
>
> The information contained in this transmission is for the personal and
> confidential use of the individual or entity to which it is addressed. If
> the reader is not the intended recipient, you are hereby notified that any
> review, dissemination, or copying of this communication is strictly
> prohibited. If you have received this transmission in error, please notify
> the sender immediately
>
>
Re: Possible road map for GUI debugger
Posted by John Wass <jw...@gmail.com>.
Is the gui envisioned to be an ide plugin or a standalone application (ie
swing, fx, ...)?
On Sun, Jan 24, 2021 at 2:08 PM Sloane, Brandon <bs...@owlcyberdefense.com>
wrote:
> Following up on the recent discussion about the Daffodil debugger, I would
> like to propose a roadmap for creating graphical debugger.
> Organizationally, the GUI debugger is a separate application (maintained
> in the normal Daffodil repository) that may include any of the current
> Daffodil libraries as a dependency.
>
> Phases 1 and 2 focus on developing an enhanced trace functionality; where
> Daffodil outputs a machine readable trace of the parse/unparse process. The
> GUI then allows a user to view this trace offline.
>
> Phase 3 adds support for interactive debugging, by allowing the
> Daffodil-cli and Daffodil-gui-debugger to act in a client/server
> relationship
>
> Phase 4 focuses on backend debugger enhancements.
>
> In more detail:
>
> Phase 1:
>
> - Add a --machine-trace flag to Daffodil CLI, that is analogous to the
> current --trace option. Output is a stream of XML parse-state nodes, each
> one describing the current state of the parse, including:
> - Current infoset
> - Current bit position
> - Current parser
> - A UID of the current state
> - A UID of the parent state (in event of backtracking, multiple
> states will have the same parent).
> - Machine-trace output stream also include input-data nodes, that
> include:
> - A region of (byte aligned) input data (hex encoded?)
> - The byte position of the start of the region
> - The relative location of the input data nodes is undefined, but
> given the complete trace, it should be possible to reconstruct the original
> dataset by stitching together the data in the nodes. The goal here is that
> we can just output data as Daffodil reads it in.
> - --machine-trace take an optional argument of a file path. If
> provided, its output will be directed to a file at said location (otherwise
> stderr)
> - Create a GUI program that can ingest the machine trace and display
> it in a 4-pane view:
> - Pane 1 - trace navigator
> - Represents the entire parse in a tree format. Each parse state
> corresponds to a node on the tree
> - The user can select a node on the tree to load it into the
> other panes
> - Pane 2 - Infoset viewer
> - Shows the complete infoset corresponding to the node selected
> in pane 1.
> - Indicates the "current" position of the parser within the
> infoset
> - Highlights changes in the infoset relative to the parent node.
> - Pane 3 - binary viewer
> - Shows a complete hex-dump of the input data, indicating
> current location within the parse
> - Pane 3 -- parser view
> - Shows the current parser. Probably will be a dumping ground
> for additional information that does not warrant the addition of a new pane.
>
>
> Phase 2:
>
> - Support showing suspensions on unparse
> - Display additional data about the parse/unparse state:
> - Delimeter stack
> - Variables
> - Support additional formats for displaying input data
> - Text (of selectable encoding)
> - Binary (not hex; actual 1s and 0s)
> - Support generating traces through the API as well.
>
> Phase 2.5:
>
> Gather user feedback and iterate on the trace viewer functionality
>
> Phase 3:
>
> - Define protocol for the debugger to talk with the CLI.
> - Debugger can load schema, and input data.
> - Debugger can step through the parse/unparse
> - Viewer built in phases 1 and 2 continually updates as the parse
> proceeds
> - User can send CLI debugger commands and get the textual response;
> but only for the current parse/unparse state.
> - User can set breakpoints (possibly using textual commands). Debugger
> supports continue until breakpoint
>
> Phase 3.5:
>
>
> - Not debugger related, but as long as the Daffodil CLI can operate as
> a server, support server mode operations for non-debug use cases.
>
>
> Phase 4:
>
> - Support time travel debugging, with an interface in both the CLI and
> GUI debuggers
> - Support associating a trace with schema source, and allowing a "go
> to source" functionality where the user can navigate directly to the
> location in source corresponding to the current parser
> - Support dynamically editing/reloading schema and input data
>
> Thoughts?
> Brandon Sloane | Engineer
>
> bsloane@owlcyberdefense.com
>
> Connect with us!
>
> <https://www.linkedin.com/company/owlcyberdefense/>
> <https://twitter.com/owlcyberdefense>
>
> <https://owlcyberdefense.com/resources/events/>
>
>
>
> The information contained in this transmission is for the personal and
> confidential use of the individual or entity to which it is addressed. If
> the reader is not the intended recipient, you are hereby notified that any
> review, dissemination, or copying of this communication is strictly
> prohibited. If you have received this transmission in error, please notify
> the sender immediately
>
>