You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by Shane Dell <sh...@apache.org> on 2023/05/16 17:24:42 UTC
[VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Hello all,I'd like to call a vote to release Apache Daffodil VS Code 1.3.0-rc2.
All distribution packages, including signatures, digests, etc. can be
found at:
https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
This release has been signed with PGP key
86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
to shanedell@apache.org, which is included in the KEYS file here:
https://downloads.apache.org/daffodil/KEYS
The release candidate has been tagged in git with 1.3.0-rc2. The binaries
For reference, here is a list of all closed GitHub issues tagged with 1.3.0:
https://github.com/apache/daffodil-vscode/milestone/4?closed=1
Please review and vote. The vote will be open for at least 72 hours
(Friday, 19 May 2023, 1:30pm EST).
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)
Documentation for 1.3.0 can be found here
Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
<https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0>.
Please note one of the large features that was focused on in 1.3.0 was
improving the data editor. To open the data editor you will need to open the
command palette using:
- Ctrl + Shift + P (windows/linux)
- Command + Shift + P (mac)
Then typing "data.edit", make sure "Daffodil Debug: Data Editor" is selected
and then hitting enter. By default the data editor will run on port 9000 of
your machine. However, if you want to use a different port follow steps here
Data Editor Launch Settings
<https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings>
Thank you,
- Shane Dell
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by "Shearer, Davin" <sh...@ctc.com>.
Okay, so we’ll need to do a bit of research on this to see if it’s possible as a cross-platform solution. From the TypeScript (client) side we can use named FIFOs (for Windows), and UDS (Linux and macOS). Java/Scala probably have cross platform support for named FIFOs and UDS. For Ωedit (the Data Editor server), it’s using Apache Pekko. Several months ago, I tried using UDS with Akka and wasn’t successful with that. I can add this to the backlog, but I doubt we’ll be able to get it into v1.3.0. Hopefully you’ll find an environment where the TCP sockets will work.
From: Mike Beckerle <mb...@apache.org>
Date: Tuesday, May 23, 2023 at 5:29 PM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
A unix-domain socket would definitely be preferable.
On Tue, May 23, 2023 at 5:24 PM Shearer, Davin <sh...@ctc.com> wrote:
> Is it possible that the daffodil debugger server could connect to a domain
> socket instead of a TCP socket? I think that could solve a lot of these
> issues.
>
> From: Mike Beckerle <mb...@apache.org>
> Date: Tuesday, May 23, 2023 at 5:20 PM
> To: dev@daffodil.apache.org <de...@daffodil.apache.org>
> Subject: Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
> I really am trying to eval the RC2.
>
> Right now my VMWare Linux system has been broken by recent updates to the
> MS Windows host environment.
>
> I don't exactly understand this, but the VSCode front end can no longer
> connect to the server at localhost:4711. I get ECONNREFUSED. This started
> when a bunch of MS-Windows updates were done on the host OS. I run VSCode
> on the linux client machine, which lives behind a NAT firewall created by
> VMWare workstation - so I really don't get how these could be interacting
> this way, but... there it is.
>
> I will switch back to my standalone linux PC if this isn't resolved today.
>
>
> On Tue, May 16, 2023 at 1:24 PM Shane Dell <sh...@apache.org> wrote:
>
> > Hello all,I'd like to call a vote to release Apache Daffodil VS Code
> > 1.3.0-rc2.
> >
> > All distribution packages, including signatures, digests, etc. can be
> > found at:
> >
> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
> >
> > This release has been signed with PGP key
> > 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
> > to shanedell@apache.org, which is included in the KEYS file here:
> > https://downloads.apache.org/daffodil/KEYS
> >
> > The release candidate has been tagged in git with 1.3.0-rc2. The binaries
> >
> > For reference, here is a list of all closed GitHub issues tagged with
> > 1.3.0:
> > https://github.com/apache/daffodil-vscode/milestone/4?closed=1
> >
> > Please review and vote. The vote will be open for at least 72 hours
> > (Friday, 19 May 2023, 1:30pm EST).
> >
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove (and reason why)
> >
> > Documentation for 1.3.0 can be found here
> > Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
> > <
> >
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
> > >.
> >
> > Please note one of the large features that was focused on in 1.3.0 was
> > improving the data editor. To open the data editor you will need to open
> > the
> > command palette using:
> >
> > - Ctrl + Shift + P (windows/linux)
> > - Command + Shift + P (mac)
> >
> > Then typing "data.edit", make sure "Daffodil Debug: Data Editor" is
> > selected
> > and then hitting enter. By default the data editor will run on port 9000
> of
> > your machine. However, if you want to use a different port follow steps
> > here
> > Data Editor Launch Settings
> > <
> >
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
> > >
> >
> > Thank you,
> >
> > - Shane Dell
> >
>
>
> This message and any files transmitted within are intended solely for the
> addressee or its representative and may contain company sensitive
> information. If you are not the intended recipient, notify the sender
> immediately and delete this message. Publication, reproduction,
> forwarding, or content disclosure is prohibited without the consent of the
> original sender and may be unlawful.
>
> Concurrent Technologies Corporation and its Affiliates.
> www.ctc.com<http://www.ctc.com> 1-800-282-4392
>
>
This message and any files transmitted within are intended solely for the addressee or its representative and may contain company sensitive information. If you are not the intended recipient, notify the sender immediately and delete this message. Publication, reproduction, forwarding, or content disclosure is prohibited without the consent of the original sender and may be unlawful.
Concurrent Technologies Corporation and its Affiliates.
www.ctc.com 1-800-282-4392
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Mike Beckerle <mb...@apache.org>.
A unix-domain socket would definitely be preferable.
On Tue, May 23, 2023 at 5:24 PM Shearer, Davin <sh...@ctc.com> wrote:
> Is it possible that the daffodil debugger server could connect to a domain
> socket instead of a TCP socket? I think that could solve a lot of these
> issues.
>
> From: Mike Beckerle <mb...@apache.org>
> Date: Tuesday, May 23, 2023 at 5:20 PM
> To: dev@daffodil.apache.org <de...@daffodil.apache.org>
> Subject: Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
> I really am trying to eval the RC2.
>
> Right now my VMWare Linux system has been broken by recent updates to the
> MS Windows host environment.
>
> I don't exactly understand this, but the VSCode front end can no longer
> connect to the server at localhost:4711. I get ECONNREFUSED. This started
> when a bunch of MS-Windows updates were done on the host OS. I run VSCode
> on the linux client machine, which lives behind a NAT firewall created by
> VMWare workstation - so I really don't get how these could be interacting
> this way, but... there it is.
>
> I will switch back to my standalone linux PC if this isn't resolved today.
>
>
> On Tue, May 16, 2023 at 1:24 PM Shane Dell <sh...@apache.org> wrote:
>
> > Hello all,I'd like to call a vote to release Apache Daffodil VS Code
> > 1.3.0-rc2.
> >
> > All distribution packages, including signatures, digests, etc. can be
> > found at:
> >
> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
> >
> > This release has been signed with PGP key
> > 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
> > to shanedell@apache.org, which is included in the KEYS file here:
> > https://downloads.apache.org/daffodil/KEYS
> >
> > The release candidate has been tagged in git with 1.3.0-rc2. The binaries
> >
> > For reference, here is a list of all closed GitHub issues tagged with
> > 1.3.0:
> > https://github.com/apache/daffodil-vscode/milestone/4?closed=1
> >
> > Please review and vote. The vote will be open for at least 72 hours
> > (Friday, 19 May 2023, 1:30pm EST).
> >
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove (and reason why)
> >
> > Documentation for 1.3.0 can be found here
> > Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
> > <
> >
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
> > >.
> >
> > Please note one of the large features that was focused on in 1.3.0 was
> > improving the data editor. To open the data editor you will need to open
> > the
> > command palette using:
> >
> > - Ctrl + Shift + P (windows/linux)
> > - Command + Shift + P (mac)
> >
> > Then typing "data.edit", make sure "Daffodil Debug: Data Editor" is
> > selected
> > and then hitting enter. By default the data editor will run on port 9000
> of
> > your machine. However, if you want to use a different port follow steps
> > here
> > Data Editor Launch Settings
> > <
> >
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
> > >
> >
> > Thank you,
> >
> > - Shane Dell
> >
>
>
> This message and any files transmitted within are intended solely for the
> addressee or its representative and may contain company sensitive
> information. If you are not the intended recipient, notify the sender
> immediately and delete this message. Publication, reproduction,
> forwarding, or content disclosure is prohibited without the consent of the
> original sender and may be unlawful.
>
> Concurrent Technologies Corporation and its Affiliates.
> www.ctc.com 1-800-282-4392
>
>
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by "Shearer, Davin" <sh...@ctc.com>.
Is it possible that the daffodil debugger server could connect to a domain socket instead of a TCP socket? I think that could solve a lot of these issues.
From: Mike Beckerle <mb...@apache.org>
Date: Tuesday, May 23, 2023 at 5:20 PM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
I really am trying to eval the RC2.
Right now my VMWare Linux system has been broken by recent updates to the
MS Windows host environment.
I don't exactly understand this, but the VSCode front end can no longer
connect to the server at localhost:4711. I get ECONNREFUSED. This started
when a bunch of MS-Windows updates were done on the host OS. I run VSCode
on the linux client machine, which lives behind a NAT firewall created by
VMWare workstation - so I really don't get how these could be interacting
this way, but... there it is.
I will switch back to my standalone linux PC if this isn't resolved today.
On Tue, May 16, 2023 at 1:24 PM Shane Dell <sh...@apache.org> wrote:
> Hello all,I'd like to call a vote to release Apache Daffodil VS Code
> 1.3.0-rc2.
>
> All distribution packages, including signatures, digests, etc. can be
> found at:
> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
>
> This release has been signed with PGP key
> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
> to shanedell@apache.org, which is included in the KEYS file here:
> https://downloads.apache.org/daffodil/KEYS
>
> The release candidate has been tagged in git with 1.3.0-rc2. The binaries
>
> For reference, here is a list of all closed GitHub issues tagged with
> 1.3.0:
> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
>
> Please review and vote. The vote will be open for at least 72 hours
> (Friday, 19 May 2023, 1:30pm EST).
>
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>
> Documentation for 1.3.0 can be found here
> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
> <
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
> >.
>
> Please note one of the large features that was focused on in 1.3.0 was
> improving the data editor. To open the data editor you will need to open
> the
> command palette using:
>
> - Ctrl + Shift + P (windows/linux)
> - Command + Shift + P (mac)
>
> Then typing "data.edit", make sure "Daffodil Debug: Data Editor" is
> selected
> and then hitting enter. By default the data editor will run on port 9000 of
> your machine. However, if you want to use a different port follow steps
> here
> Data Editor Launch Settings
> <
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
> >
>
> Thank you,
>
> - Shane Dell
>
This message and any files transmitted within are intended solely for the addressee or its representative and may contain company sensitive information. If you are not the intended recipient, notify the sender immediately and delete this message. Publication, reproduction, forwarding, or content disclosure is prohibited without the consent of the original sender and may be unlawful.
Concurrent Technologies Corporation and its Affiliates.
www.ctc.com 1-800-282-4392
Fwd: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Davin Shearer <da...@apache.org>.
Excellent, I'm glad it's not a JRE version problem since we're on Java 8 to
maximize compatibility across our supported platforms. Also agree with
dropping the JRE bundling for 1.3.0 at least.
Environment isolation and configuration validation updates ought to be
straightforward PRs to review and test. Certainly much simpler than
bundling a JRE!
On Mon, Jun 5, 2023 at 5:20 PM Shane Dell <sh...@apache.org> wrote:
> Yeah I was wondering if it was documented for that before we back ported
> to support Java 8,
> will try find that and get it updated if so. But thats awesome that isn't
> a Java version issue. I
> agree with dropping the bundle a JVM for now, I think its a cool idea and
> would be interesting. I
> appreciate you jumping on that so quick. I plan on closing this vote here
> today since you -1ed
> with pretty good reasons I think we should do an rc3.
>
>
> On 2023/06/05 21:13:56 Mike Beckerle wrote:
> > I re-tested, and changing to Java 17 in my launch script (with very
> limited
> > set of env vars) works.
> >
> > So the environment interactions I was having are NOT due to the JVM being
> > Java 17.
> >
> > I suspect we can drop the issue of bundling Java with it. I doubt we can
> do
> > that realistically given the license of Java.
> >
> > Really the behavior needs to be that which JVM doesn't affect the
> behavior,
> > even when you have an erroneous configuration of launch.json.
> >
> > The configurations were faulty - daffodilDebugClasspath was incorrect at
> > various points, making it get a NPE when setting breakpoints, etc.
> Getting
> > that right, and realizing that I needed to have BOTH regular files and
> Jar
> > files on the daffodilDebugClasspath was hard-won knowledge.
> >
> > I do recall reading doc that said it requires Java 11, and that's part of
> > why I started creating this isolation environment for it, because it
> wasn't
> > working, and seemed at the time like it might have been a Java 11 vs. 17
> > incompatibility. Clearly it ended up NOT being that.
> >
> > -mikeb
> >
> > On Mon, Jun 5, 2023 at 4:53 PM Mike Beckerle <mb...@apache.org>
> wrote:
> >
> > > Well I had sufficient problems with it that I should re-verify if it
> works
> > > with Java 17. It may be that the java version thing was a non-issue
> that
> > > other things were interacting with it.
> > >
> > >
> > > On Mon, Jun 5, 2023 at 4:45 PM Shane Dell <sh...@apache.org>
> wrote:
> > >
> > >> The extension does not depend on Java 11 it should work fine with Java
> > >> 17. I remember back
> > >> porting to make sure we supported Java 8 also. Other than relying on
> the
> > >> system Java, I am not
> > >> sure how on else we can really on Java. I know Davin mentioned
> packaging
> > >> a JVM or JDK or
> > >> something like that but is that very common practice to do? The only
> time
> > >> I ever heard of doing
> > >> that was for a language server so was wondering if its necessary or
> might
> > >> be overkill. I agree
> > >> with the needing to fix the issue with the environment when it comes
> to
> > >> shell and think what you
> > >> posted will help us do that. However, when it comes to Java I think if
> > >> there is an issue with Java
> > >> 17 we should figure that out first. Then maybe put it out for a vote
> or
> > >> something about bundling
> > >> a Java version with the extension and then letting the user specify a
> > >> different path to Java or
> > >> something like that?
> > >>
> > >>
> > >> On 2023/05/31 23:02:03 Mike Beckerle wrote:
> > >> > So, I have the 1.3.0 RC2 vscode extension debugger working now for
> PCAP
> > >> w/
> > >> > ethernetIP and I can set a breakpoint in ethernetIP.dfdl.xsd and I
> no
> > >> > longer get an NPE.
> > >> >
> > >> > Configuring this is really far too difficult, and I have to have the
> > >> schema
> > >> > working first with sbt in order to even hope to configure it.
> > >> >
> > >> > But I'll get back to that later.
> > >> >
> > >> > More basic issue is that I am unable to debug even dead-simple
> errors
> > >> with
> > >> > it. I clobbered the magic number in the first bytes of an icmp1.cap
> > >> file.
> > >> > This fails an assert at line 97 of pcap.dfdl.xsd, which is the first
> > >> > element of simple type.
> > >> >
> > >> > I step through with the debugger, and nothing displays *anything*
> about
> > >> the
> > >> > assert failure. Execution ends. A little pop-up dialog says it
> wrote my
> > >> > infoset out, which it didn't. There's nothing showing that anything
> went
> > >> > wrong. All panels on the left are blank.
> > >> >
> > >> > If I modify the schema and make any mistake, then I get a blocking
> > >> dialog
> > >> > showing the SDEs. This is basically intolerable, and is why I have
> to go
> > >> > back to sbt to debug this. You can't even go through the SDEs as you
> > >> move
> > >> > the editor. It's a blocking dialog.
> > >> >
> > >> > So both for schema compilation, and runtime errors there simply has
> to
> > >> be a
> > >> > non-modal display of errors, probably on the left with variables,
> watch,
> > >> > call-stack, and breakpoints. But a quasi-edit-buffer containing them
> > >> would
> > >> > also be fine, or even just output them to the terminal window.
> > >> >
> > >> > The rest of this email is about the hoops I had to jump through to
> get
> > >> this
> > >> > to work at all. Arguably, these issues are about working with a
> > >> > multi-component schema, and we could define our objectives to
> simpler
> > >> > schemas than this, but that puts it in the realm of toy-only
> schemas.
> > >> The
> > >> > example is just PCAP + ethernetIP, which is not an extreme example.
> > >> >
> > >> > The existing shell environment of the terminal definitely causes
> > >> problems.
> > >> > But just exiting that so that the debugger starts a new terminal
> doesn't
> > >> > fix it, as that inherits the environment from where vscode was
> started.
> > >> It
> > >> > starts a new login shell unless you take steps to prevent that.
> > >> >
> > >> > A big trip-up in usability is that the wizard that creates the
> > >> launch.json,
> > >> > doesn't actually let you populate the daffodilDebugClaspath. You
> have to
> > >> > edit that by hand, and nothing verifies it, so any typo and you get
> > >> tedious
> > >> > failures that are hard to figure out. You just get an SDE - if you
> are
> > >> > lucky - via a blocking dialog.
> > >> >
> > >> > The wizard should help you put directories and corresponding jars
> on the
> > >> > daffodilDebugClasspath, and it should not let you put a non-existing
> > >> > directory or jar file on it. Ideally, at launch time the
> > >> > daffodilDebugClasspath should be checked, just in case something on
> it
> > >> no
> > >> > longer exists.
> > >> >
> > >> > If the debugger happens to find a file on the
> daffodilDebugClasspath in
> > >> a
> > >> > jar, vs. in a regular directory (maybe you have a typo in the
> regular
> > >> > directory path), so it passes that and hits the jar, then setting a
> > >> > breakpoint causes a NPE. Something requires a "hierarchical URL"
> which a
> > >> > jar-file URL is not. This is the bug that really has to be fixed so
> that
> > >> > you can set breakpoints regardless of whether the source DFDL schema
> > >> file
> > >> > was in a jar or not.
> > >> >
> > >> > Note: at Owl we have DFDL schemas where the starting root element
> of the
> > >> > schema isn't even part of the schema, it's part of a component
> schema
> > >> that
> > >> > is in a jar on the classpath. This happens when you have an
> > >> > envelope-payload idiom. The schema that combines the envelope and
> > >> payload
> > >> > together contains only the glue part that connects the two
> together. The
> > >> > root element is the envelope's root element, and that schema is in
> a jar
> > >> > file.
> > >> >
> > >> > I did not see any feature in launch.json for specifying the
> > >> name/namespace
> > >> > of the root element. There is just the "program" element, which
> must be
> > >> to
> > >> > a file in the current schema. That needs to be able to be specified
> > >> just as
> > >> > element name + namespace.
> > >> >
> > >> > The jars have to be on the daffodilDebugClasspath because of the
> plugins
> > >> > that daffodil must find in them. That's understandable, the problem
> is
> > >> > having to also have a regular file location for any schema you want
> to
> > >> be
> > >> > able to put a breakpoint in and it must be before the corresponding
> jar.
> > >> >
> > >> > That all said, here are the scripts and settings I used to get it to
> > >> work:
> > >> >
> > >> > First, I have this script I use to launch vscode in a minimal/empty
> > >> > environment. This technique is useful to
> > >> > determine if you are depending on the environment in any way:
> > >> >
> > >> > #! /bin/bash
> > >> > if [ "$1" = "emptyEnv" ] # is it the special first arg
> > >> > then
> > >> > shift # discard special first arg, and continue with the rest of
> the
> > >> > script
> > >> > else
> > >> > CMD=$0
> > >> > exec -c $CMD "emptyEnv" $* # exec with an empty environment,
> special
> > >> > first arg
> > >> > fi
> > >> > # the environment is empty when we get here
> > >> > # just add env vars we need to get vscode to work
> > >> > export LANG=en_US.UTF-8
> > >> > export HOME=/home/mbeckerle
> > >> > export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 # java 11
> required
> > >> > export PATH=$JAVA_HOME/bin:$PATH
> > >> > export DISPLAY=:0
> > >> > export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus #
> vscode
> > >> spits
> > >> > lots of errors without this
> > >> > exec /usr/bin/code
> > >> > # end
> > >> >
> > >> > Then, within ~/.config/Code/User/settings.json, I had to create
> these
> > >> > settings for terminal:
> > >> >
> > >> > "terminal.integrated.inheritEnv": false,
> > >> > "terminal.integrated.profiles.linux": {
> > >> > "Custom Init": {
> > >> > "path": "/bin/bash",
> > >> > "args": [
> > >> > "--norc"
> > >> > ]
> > >> > }
> > >> > },
> > >> > "terminal.integrated.defaultProfile.linux": "Custom Init"
> > >> >
> > >> > That instructs vscode not to create a login shell for the terminal.
> The
> > >> > --norc option means it does not run the .bashrc file so that the
> java
> > >> > version specified in the launch script sticks.
> > >> >
> > >> > I am not sure the inheritEnv setting actually does anything. That
> alone
> > >> > sounds like the right thing, but that alone didn't solve the
> problem.
> > >> > You still have everything in the login shell environment where
> vscode
> > >> was
> > >> > launched. Hence the need for the environment-emptying launch script.
> > >> >
> > >> > With those two things, a terminal created by vscode does not inherit
> > >> > everything my typical shell has. For example there is no daffodil
> on the
> > >> > path, no DAFFODIL_CLASSPATH env var, etc.
> > >> >
> > >> > Given the above if I just create a terminal with Terminal > new
> > >> Terminal:
> > >> > The environment is relatively simple:
> > >> >
> > >> > bash-5.0$ env
> > >> > COLORTERM=truecolor
> > >> > TERM_PROGRAM_VERSION=1.78.2
> > >> > JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
> > >> > NO_AT_BRIDGE=1
> > >> > PWD=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP
> > >> > VSCODE_GIT_ASKPASS_NODE=/usr/share/code/code
> > >> > HOME=/home/mbeckerle
> > >> > LANG=en_US.UTF-8
> > >> >
> GIT_ASKPASS=/usr/share/code/resources/app/extensions/git/dist/askpass.sh
> > >> > CHROME_DESKTOP=code-url-handler.desktop
> > >> > VSCODE_GIT_ASKPASS_EXTRA_ARGS=--ms-enable-electron-run-as-node
> > >> > TERM=xterm-256color
> > >> > VSCODE_GIT_IPC_HANDLE=/tmp/vscode-git-56a269c0f5.sock
> > >> > DISPLAY=:0
> > >> > SHLVL=1
> > >> >
> > >>
> VSCODE_GIT_ASKPASS_MAIN=/usr/share/code/resources/app/extensions/git/dist/askpass-main.js
> > >> > GDK_BACKEND=x11
> > >> >
> > >>
> PATH=/usr/lib/jvm/java-11-openjdk-amd64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
> > >> > ORIGINAL_XDG_CURRENT_DESKTOP=undefined
> > >> > DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
> > >> > TERM_PROGRAM=vscode
> > >> > _=/usr/bin/env
> > >> >
> > >> > If I run daffodil, then c-C in the terminal it creates, then the
> > >> > environment has this added environment variable only.
> > >> >
> > >> >
> > >>
> DAFFODIL_DEBUG_CLASSPATH=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/../ethernetIP/src/main/resources:/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/lib_managed/jars/com.owlcyberdefense/dfdl-ethernetip/dfdl-ethernetip-1.2.0.jar
> > >> >
> > >> > This works. My launch.json is attached.
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > On Fri, May 26, 2023 at 2:06 PM Shane Dell <sh...@apache.org>
> > >> wrote:
> > >> >
> > >> > > Mike,
> > >> > >
> > >> > > The VSCode Extension should be using the daffodil command that is
> > >> bundled
> > >> > > with the
> > >> > > extension. The issue might still be with that shell env variable
> you
> > >> have
> > >> > > for
> > >> > > DAFFODIL_CLASSPATH, but this might only be an issue with a
> terminal
> > >> that
> > >> > > is already opened
> > >> > > with the name "daffodil-debugger". If you close that terminal, the
> > >> next
> > >> > > you time you run it should
> > >> > > create a new terminal with the env set properly. If you could try
> > >> this and
> > >> > > see if it works how you
> > >> > > expect, that would help us single out this issue out to an already
> > >> active
> > >> > > terminal.
> > >> > >
> > >> > > Thank you,
> > >> > >
> > >> > > Shane Dell
> > >> > >
> > >> > >
> > >> > > On 2023/05/25 23:59:53 Mike Beckerle wrote:
> > >> > > > There are so many environment interactions....
> > >> > > >
> > >> > > > So I have discovered that the VSCode extension just seems to run
> > >> whatever
> > >> > > > daffodil command is on the path. It's not isolated at all from
> the
> > >> shell
> > >> > > > environment.
> > >> > > >
> > >> > > > I found this out because I have a customized daffodil shell
> script
> > >> that
> > >> > > > setups a bunch of DAFFODIL_CLASSPATH contents, and I couldn't
> > >> figure out
> > >> > > > why those were part of the VSCode environment.
> > >> > > >
> > >> > > > Shouldn't vscode run exactly the daffodil bundled with it,
> unless
> > >> you
> > >> > > > explicitly configure it not to?
> > >> > > >
> > >> > > > That's why it is finding the ethernetIP.dfdl.xsd file in the
> jar,
> > >> not in
> > >> > > > the file system. Because my daffodil command sets up lots of
> jars
> > >> on the
> > >> > > > DAFFODIL_CLASSPATH.
> > >> > > >
> > >> > > > It also depends on Java 11, breaks with Java 17. I also don't
> think
> > >> this
> > >> > > > should be inherited from the environment.
> > >> > > >
> > >> > > > I really have a hard time using, or even testing the VSCode
> > >> extension if
> > >> > > I
> > >> > > > have to scrub my shell environment of everything in it in order
> to
> > >> make
> > >> > > the
> > >> > > > vscode extension work.
> > >> > > >
> > >> > > > But it is this kind of problem that caused people to invent
> docker
> > >> > > > containers.
> > >> > > >
> > >> > > > I am going to make a launcher script for VSCode that empties the
> > >> > > > environment, then supplies only what is absolutely necessary,
> with
> > >> no
> > >> > > > outside environment assumptions.
> > >> > > >
> > >> > > > I will try to do this Friday if possible.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Wed, May 24, 2023 at 2:50 PM Shane Dell <
> shanedell@apache.org>
> > >> wrote:
> > >> > > >
> > >> > > > > Mike,
> > >> > > > >
> > >> > > > > I am not sure why you would be seeing this. I am able to set
> the
> > >> > > > > breakpoint where you
> > >> > > > > mentioned without any issues. Can you try removing the
> > >> daffodil-dap
> > >> > > > > folder? This should be
> > >> > > > > somewhere like:
> > >> > > > > /home/$USER/.local/share/daffodil-dap
> > >> > > > > The issue I am thinking is possibly a bug with the rc1
> debugger
> > >> that
> > >> > > may
> > >> > > > > not be present on rc2.
> > >> > > > > But if the folder daffodil-dap/daffodil-debugger-3.4.0-1.3.0/
> > >> already
> > >> > > > > exists from rc1 then rc2 will
> > >> > > > > not remake it. So in order to remake it you have to manually
> > >> remove
> > >> > > this
> > >> > > > > folder. Let me know if
> > >> > > > > this helps.
> > >> > > > >
> > >> > > > > Thank you,
> > >> > > > >
> > >> > > > > Shane Dell
> > >> > > > >
> > >> > > > > On 2023/05/24 15:15:33 Mike Beckerle wrote:
> > >> > > > > > I have been able to continue evaluating RC2.
> > >> > > > > >
> > >> > > > > > My vote is -1
> > >> > > > > >
> > >> > > > > > My testing steps listed below. I am trying to debug PCAP. I
> > >> have a
> > >> > > broken
> > >> > > > > > icmp packet (bad magic number), and also just step through a
> > >> correct
> > >> > > icmp
> > >> > > > > > packet.
> > >> > > > > >
> > >> > > > > > I am trying to debug icmp.badMagicNum.cap data file.
> > >> > > > > >
> > >> > > > > > I should get a failure of an assert because the magic
> number is
> > >> not
> > >> > > one
> > >> > > > > of
> > >> > > > > > the expected values.
> > >> > > > > >
> > >> > > > > > I'm just trying to single-step through it until it hits the
> > >> error.
> > >> > > > > >
> > >> > > > > > Once we get to the MagicNumber element, it doesn't position
> on
> > >> the
> > >> > > assert
> > >> > > > > > while evaluating that expression or even on the element
> > >> carrying the
> > >> > > > > assert.
> > >> > > > > > It jumps back to the top-level xs:schema position.
> > >> > > > > >
> > >> > > > > > I continue to click single step. Watching for the error to
> be
> > >> > > displayed
> > >> > > > > in
> > >> > > > > > the upper left under Variables > Parse > Diagnostics >
> Errors
> > >> which
> > >> > > is
> > >> > > > > > where I think it ought to go.
> > >> > > > > >
> > >> > > > > > At some point the failure is somehow detected as the debug
> > >> session
> > >> > > ends.
> > >> > > > > A
> > >> > > > > > dialog offers to open /tmp/infoset.xml for me, which I open
> and
> > >> it is
> > >> > > > > empty
> > >> > > > > > of course.
> > >> > > > > >
> > >> > > > > > Nothing displayed the error message from the assert, or even
> > >> > > indicated
> > >> > > > > what
> > >> > > > > > went wrong at all. Not the state display in the upper left,
> not
> > >> the
> > >> > > > > > terminal, nor debug console nor output.
> > >> > > > > >
> > >> > > > > > The upper left box with "Variables" is cleared at the end.
> It
> > >> should
> > >> > > > > > certainly not be cleared at the end of a run when final
> state
> > >> may be
> > >> > > of
> > >> > > > > > interest. At the start of a run, or manually, but not at the
> > >> end of a
> > >> > > > > run.
> > >> > > > > >
> > >> > > > > > Second attempt to use it:
> > >> > > > > >
> > >> > > > > > I create a second launch config this time for what should
> be a
> > >> > > successful
> > >> > > > > > parse of icmp1.cap
> > >> > > > > >
> > >> > > > > > As I single step, the position of the cursor on the schema
> is
> > >> jumping
> > >> > > > > > around all over the place back and forth between a
> containing
> > >> > > element and
> > >> > > > > > the child elements it contains.
> > >> > > > > > The infoset display does not get updated until long after
> the
> > >> > > display has
> > >> > > > > > moved on and is positioned on the next element.
> > >> > > > > >
> > >> > > > > > This jumping around gets completely out of hand once you
> step
> > >> into
> > >> > > the
> > >> > > > > > complexType "Ethernet" which is in the ethernetIP jar file.
> > >> > > > > >
> > >> > > > > > At that point, after each child of Ethernet, it jumps back
> to
> > >> the
> > >> > > > > Ethernet
> > >> > > > > > element in the pcap.dfdl.xsd file, and then back into the
> > >> > > > > > ethernetIP.dfdl.xsd for the next child, then back to the
> > >> > > pcap.dfdl.xsd
> > >> > > > > > file, etc.
> > >> > > > > >
> > >> > > > > > All this jumping around basically makes stepping useless.
> So I
> > >> > > decide to
> > >> > > > > > put a breakpoint on the Version element inside the
> IPv4Header
> > >> element
> > >> > > > > > inside ethernetIP.dfdl.xsd.
> > >> > > > > >
> > >> > > > > > Just adding the breakpoint causes the debug session to end
> with
> > >> an
> > >> > > NPE in
> > >> > > > > > the dap setBreakpoint.
> > >> > > > > >
> > >> > > > > > ERROR o.a.d.d.d.DAPodil - unhandled error
> > >> > > > > > java.lang.NullPointerException: null
> > >> > > > > > at
> > >> > > > >
> java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> > >> > > > > > at
> > >> java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> > >> > > > > > at
> > >> > > > > >
> > >> java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> > >> > > > > > at java.base/java.nio.file.Path.of(Path.java:147)
> > >> > > > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
> > >> > > > > > at
> > >> > > > > >
> > >> > > > >
> > >> > >
> > >>
> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> > >> > > > > > at get @
> fs2.internal.Scope.openScope(Scope.scala:281)
> > >> > > > > > at flatMap @
> > >> > > > > >
> > >> > > > >
> > >> > >
> > >>
> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > at handleErrorWith @
> > >> > > > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > at get @
> fs2.internal.Scope.openScope(Scope.scala:281)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> > >> > > > > > at get @
> fs2.internal.Scope.openScope(Scope.scala:281)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > at flatMap @
> > >> > > fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> > >> > > > > > at get @
> fs2.internal.Scope.openScope(Scope.scala:281)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > 2023-05-24 11:09:10,717 [io-compute-1] DEBUG
> o.a.d.d.d.DAPodil -
> > >> > > > > whenDone:
> > >> > > > > > completed
> > >> > > > > > java.lang.NullPointerException
> > >> > > > > > at
> > >> > > > >
> java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> > >> > > > > > at
> > >> java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> > >> > > > > > at
> > >> > > > > >
> > >> java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> > >> > > > > > at java.base/java.nio.file.Path.of(Path.java:147)
> > >> > > > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
> > >> > > > > > at
> > >> > > > > >
> > >> > > > >
> > >> > >
> > >>
> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> > >> > > > > > at get @
> fs2.internal.Scope.openScope(Scope.scala:281)
> > >> > > > > > at flatMap @
> > >> > > > > >
> > >> > > > >
> > >> > >
> > >>
> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > at handleErrorWith @
> > >> > > > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > at get @
> fs2.internal.Scope.openScope(Scope.scala:281)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> > >> > > > > > at get @
> fs2.internal.Scope.openScope(Scope.scala:281)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > > at flatMap @
> > >> > > fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> > >> > > > > > at get @
> fs2.internal.Scope.openScope(Scope.scala:281)
> > >> > > > > > at flatMap @
> > >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Tue, May 23, 2023 at 5:19 PM Mike Beckerle <
> > >> mbeckerle@apache.org>
> > >> > > > > wrote:
> > >> > > > > >
> > >> > > > > > > I really am trying to eval the RC2.
> > >> > > > > > >
> > >> > > > > > > Right now my VMWare Linux system has been broken by recent
> > >> updates
> > >> > > to
> > >> > > > > the
> > >> > > > > > > MS Windows host environment.
> > >> > > > > > >
> > >> > > > > > > I don't exactly understand this, but the VSCode front end
> can
> > >> no
> > >> > > longer
> > >> > > > > > > connect to the server at localhost:4711. I get
> ECONNREFUSED.
> > >> This
> > >> > > > > started
> > >> > > > > > > when a bunch of MS-Windows updates were done on the host
> OS.
> > >> I run
> > >> > > > > VSCode
> > >> > > > > > > on the linux client machine, which lives behind a NAT
> firewall
> > >> > > created
> > >> > > > > by
> > >> > > > > > > VMWare workstation - so I really don't get how these
> could be
> > >> > > > > interacting
> > >> > > > > > > this way, but... there it is.
> > >> > > > > > >
> > >> > > > > > > I will switch back to my standalone linux PC if this isn't
> > >> resolved
> > >> > > > > today.
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > On Tue, May 16, 2023 at 1:24 PM Shane Dell <
> > >> shanedell@apache.org>
> > >> > > > > wrote:
> > >> > > > > > >
> > >> > > > > > >> Hello all,I'd like to call a vote to release Apache
> Daffodil
> > >> VS
> > >> > > Code
> > >> > > > > > >> 1.3.0-rc2.
> > >> > > > > > >>
> > >> > > > > > >> All distribution packages, including signatures, digests,
> > >> etc.
> > >> > > can be
> > >> > > > > > >> found at:
> > >> > > > > > >>
> > >> > > > >
> > >> > >
> > >>
> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
> > >> > > > > > >>
> > >> > > > > > >> This release has been signed with PGP key
> > >> > > > > > >> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
> > >> > > > > > >> to shanedell@apache.org, which is included in the KEYS
> file
> > >> here:
> > >> > > > > > >> https://downloads.apache.org/daffodil/KEYS
> > >> > > > > > >>
> > >> > > > > > >> The release candidate has been tagged in git with
> 1.3.0-rc2.
> > >> The
> > >> > > > > binaries
> > >> > > > > > >>
> > >> > > > > > >> For reference, here is a list of all closed GitHub issues
> > >> tagged
> > >> > > with
> > >> > > > > > >> 1.3.0:
> > >> > > > > > >>
> > >> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
> > >> > > > > > >>
> > >> > > > > > >> Please review and vote. The vote will be open for at
> least 72
> > >> > > hours
> > >> > > > > > >> (Friday, 19 May 2023, 1:30pm EST).
> > >> > > > > > >>
> > >> > > > > > >> [ ] +1 approve
> > >> > > > > > >> [ ] +0 no opinion
> > >> > > > > > >> [ ] -1 disapprove (and reason why)
> > >> > > > > > >>
> > >> > > > > > >> Documentation for 1.3.0 can be found here
> > >> > > > > > >> Apache Daffodil Extension for Visual Studio Code v1.3.0
> Wiki
> > >> > > > > > >> <
> > >> > > > > > >>
> > >> > > > >
> > >> > >
> > >>
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
> > >> > > > > > >> >.
> > >> > > > > > >>
> > >> > > > > > >> Please note one of the large features that was focused
> on in
> > >> > > 1.3.0 was
> > >> > > > > > >> improving the data editor. To open the data editor you
> will
> > >> need
> > >> > > to
> > >> > > > > open
> > >> > > > > > >> the
> > >> > > > > > >> command palette using:
> > >> > > > > > >>
> > >> > > > > > >> - Ctrl + Shift + P (windows/linux)
> > >> > > > > > >> - Command + Shift + P (mac)
> > >> > > > > > >>
> > >> > > > > > >> Then typing "data.edit", make sure "Daffodil Debug: Data
> > >> Editor"
> > >> > > is
> > >> > > > > > >> selected
> > >> > > > > > >> and then hitting enter. By default the data editor will
> run
> > >> on
> > >> > > port
> > >> > > > > 9000
> > >> > > > > > >> of
> > >> > > > > > >> your machine. However, if you want to use a different
> port
> > >> follow
> > >> > > > > steps
> > >> > > > > > >> here
> > >> > > > > > >> Data Editor Launch Settings
> > >> > > > > > >> <
> > >> > > > > > >>
> > >> > > > >
> > >> > >
> > >>
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
> > >> > > > > > >> >
> > >> > > > > > >>
> > >> > > > > > >> Thank you,
> > >> > > > > > >>
> > >> > > > > > >> - Shane Dell
> > >> > > > > > >>
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Shane Dell <sh...@apache.org>.
Yeah I was wondering if it was documented for that before we back ported to support Java 8,
will try find that and get it updated if so. But thats awesome that isn't a Java version issue. I
agree with dropping the bundle a JVM for now, I think its a cool idea and would be interesting. I
appreciate you jumping on that so quick. I plan on closing this vote here today since you -1ed
with pretty good reasons I think we should do an rc3.
On 2023/06/05 21:13:56 Mike Beckerle wrote:
> I re-tested, and changing to Java 17 in my launch script (with very limited
> set of env vars) works.
>
> So the environment interactions I was having are NOT due to the JVM being
> Java 17.
>
> I suspect we can drop the issue of bundling Java with it. I doubt we can do
> that realistically given the license of Java.
>
> Really the behavior needs to be that which JVM doesn't affect the behavior,
> even when you have an erroneous configuration of launch.json.
>
> The configurations were faulty - daffodilDebugClasspath was incorrect at
> various points, making it get a NPE when setting breakpoints, etc. Getting
> that right, and realizing that I needed to have BOTH regular files and Jar
> files on the daffodilDebugClasspath was hard-won knowledge.
>
> I do recall reading doc that said it requires Java 11, and that's part of
> why I started creating this isolation environment for it, because it wasn't
> working, and seemed at the time like it might have been a Java 11 vs. 17
> incompatibility. Clearly it ended up NOT being that.
>
> -mikeb
>
> On Mon, Jun 5, 2023 at 4:53 PM Mike Beckerle <mb...@apache.org> wrote:
>
> > Well I had sufficient problems with it that I should re-verify if it works
> > with Java 17. It may be that the java version thing was a non-issue that
> > other things were interacting with it.
> >
> >
> > On Mon, Jun 5, 2023 at 4:45 PM Shane Dell <sh...@apache.org> wrote:
> >
> >> The extension does not depend on Java 11 it should work fine with Java
> >> 17. I remember back
> >> porting to make sure we supported Java 8 also. Other than relying on the
> >> system Java, I am not
> >> sure how on else we can really on Java. I know Davin mentioned packaging
> >> a JVM or JDK or
> >> something like that but is that very common practice to do? The only time
> >> I ever heard of doing
> >> that was for a language server so was wondering if its necessary or might
> >> be overkill. I agree
> >> with the needing to fix the issue with the environment when it comes to
> >> shell and think what you
> >> posted will help us do that. However, when it comes to Java I think if
> >> there is an issue with Java
> >> 17 we should figure that out first. Then maybe put it out for a vote or
> >> something about bundling
> >> a Java version with the extension and then letting the user specify a
> >> different path to Java or
> >> something like that?
> >>
> >>
> >> On 2023/05/31 23:02:03 Mike Beckerle wrote:
> >> > So, I have the 1.3.0 RC2 vscode extension debugger working now for PCAP
> >> w/
> >> > ethernetIP and I can set a breakpoint in ethernetIP.dfdl.xsd and I no
> >> > longer get an NPE.
> >> >
> >> > Configuring this is really far too difficult, and I have to have the
> >> schema
> >> > working first with sbt in order to even hope to configure it.
> >> >
> >> > But I'll get back to that later.
> >> >
> >> > More basic issue is that I am unable to debug even dead-simple errors
> >> with
> >> > it. I clobbered the magic number in the first bytes of an icmp1.cap
> >> file.
> >> > This fails an assert at line 97 of pcap.dfdl.xsd, which is the first
> >> > element of simple type.
> >> >
> >> > I step through with the debugger, and nothing displays *anything* about
> >> the
> >> > assert failure. Execution ends. A little pop-up dialog says it wrote my
> >> > infoset out, which it didn't. There's nothing showing that anything went
> >> > wrong. All panels on the left are blank.
> >> >
> >> > If I modify the schema and make any mistake, then I get a blocking
> >> dialog
> >> > showing the SDEs. This is basically intolerable, and is why I have to go
> >> > back to sbt to debug this. You can't even go through the SDEs as you
> >> move
> >> > the editor. It's a blocking dialog.
> >> >
> >> > So both for schema compilation, and runtime errors there simply has to
> >> be a
> >> > non-modal display of errors, probably on the left with variables, watch,
> >> > call-stack, and breakpoints. But a quasi-edit-buffer containing them
> >> would
> >> > also be fine, or even just output them to the terminal window.
> >> >
> >> > The rest of this email is about the hoops I had to jump through to get
> >> this
> >> > to work at all. Arguably, these issues are about working with a
> >> > multi-component schema, and we could define our objectives to simpler
> >> > schemas than this, but that puts it in the realm of toy-only schemas.
> >> The
> >> > example is just PCAP + ethernetIP, which is not an extreme example.
> >> >
> >> > The existing shell environment of the terminal definitely causes
> >> problems.
> >> > But just exiting that so that the debugger starts a new terminal doesn't
> >> > fix it, as that inherits the environment from where vscode was started.
> >> It
> >> > starts a new login shell unless you take steps to prevent that.
> >> >
> >> > A big trip-up in usability is that the wizard that creates the
> >> launch.json,
> >> > doesn't actually let you populate the daffodilDebugClaspath. You have to
> >> > edit that by hand, and nothing verifies it, so any typo and you get
> >> tedious
> >> > failures that are hard to figure out. You just get an SDE - if you are
> >> > lucky - via a blocking dialog.
> >> >
> >> > The wizard should help you put directories and corresponding jars on the
> >> > daffodilDebugClasspath, and it should not let you put a non-existing
> >> > directory or jar file on it. Ideally, at launch time the
> >> > daffodilDebugClasspath should be checked, just in case something on it
> >> no
> >> > longer exists.
> >> >
> >> > If the debugger happens to find a file on the daffodilDebugClasspath in
> >> a
> >> > jar, vs. in a regular directory (maybe you have a typo in the regular
> >> > directory path), so it passes that and hits the jar, then setting a
> >> > breakpoint causes a NPE. Something requires a "hierarchical URL" which a
> >> > jar-file URL is not. This is the bug that really has to be fixed so that
> >> > you can set breakpoints regardless of whether the source DFDL schema
> >> file
> >> > was in a jar or not.
> >> >
> >> > Note: at Owl we have DFDL schemas where the starting root element of the
> >> > schema isn't even part of the schema, it's part of a component schema
> >> that
> >> > is in a jar on the classpath. This happens when you have an
> >> > envelope-payload idiom. The schema that combines the envelope and
> >> payload
> >> > together contains only the glue part that connects the two together. The
> >> > root element is the envelope's root element, and that schema is in a jar
> >> > file.
> >> >
> >> > I did not see any feature in launch.json for specifying the
> >> name/namespace
> >> > of the root element. There is just the "program" element, which must be
> >> to
> >> > a file in the current schema. That needs to be able to be specified
> >> just as
> >> > element name + namespace.
> >> >
> >> > The jars have to be on the daffodilDebugClasspath because of the plugins
> >> > that daffodil must find in them. That's understandable, the problem is
> >> > having to also have a regular file location for any schema you want to
> >> be
> >> > able to put a breakpoint in and it must be before the corresponding jar.
> >> >
> >> > That all said, here are the scripts and settings I used to get it to
> >> work:
> >> >
> >> > First, I have this script I use to launch vscode in a minimal/empty
> >> > environment. This technique is useful to
> >> > determine if you are depending on the environment in any way:
> >> >
> >> > #! /bin/bash
> >> > if [ "$1" = "emptyEnv" ] # is it the special first arg
> >> > then
> >> > shift # discard special first arg, and continue with the rest of the
> >> > script
> >> > else
> >> > CMD=$0
> >> > exec -c $CMD "emptyEnv" $* # exec with an empty environment, special
> >> > first arg
> >> > fi
> >> > # the environment is empty when we get here
> >> > # just add env vars we need to get vscode to work
> >> > export LANG=en_US.UTF-8
> >> > export HOME=/home/mbeckerle
> >> > export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 # java 11 required
> >> > export PATH=$JAVA_HOME/bin:$PATH
> >> > export DISPLAY=:0
> >> > export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus # vscode
> >> spits
> >> > lots of errors without this
> >> > exec /usr/bin/code
> >> > # end
> >> >
> >> > Then, within ~/.config/Code/User/settings.json, I had to create these
> >> > settings for terminal:
> >> >
> >> > "terminal.integrated.inheritEnv": false,
> >> > "terminal.integrated.profiles.linux": {
> >> > "Custom Init": {
> >> > "path": "/bin/bash",
> >> > "args": [
> >> > "--norc"
> >> > ]
> >> > }
> >> > },
> >> > "terminal.integrated.defaultProfile.linux": "Custom Init"
> >> >
> >> > That instructs vscode not to create a login shell for the terminal. The
> >> > --norc option means it does not run the .bashrc file so that the java
> >> > version specified in the launch script sticks.
> >> >
> >> > I am not sure the inheritEnv setting actually does anything. That alone
> >> > sounds like the right thing, but that alone didn't solve the problem.
> >> > You still have everything in the login shell environment where vscode
> >> was
> >> > launched. Hence the need for the environment-emptying launch script.
> >> >
> >> > With those two things, a terminal created by vscode does not inherit
> >> > everything my typical shell has. For example there is no daffodil on the
> >> > path, no DAFFODIL_CLASSPATH env var, etc.
> >> >
> >> > Given the above if I just create a terminal with Terminal > new
> >> Terminal:
> >> > The environment is relatively simple:
> >> >
> >> > bash-5.0$ env
> >> > COLORTERM=truecolor
> >> > TERM_PROGRAM_VERSION=1.78.2
> >> > JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
> >> > NO_AT_BRIDGE=1
> >> > PWD=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP
> >> > VSCODE_GIT_ASKPASS_NODE=/usr/share/code/code
> >> > HOME=/home/mbeckerle
> >> > LANG=en_US.UTF-8
> >> > GIT_ASKPASS=/usr/share/code/resources/app/extensions/git/dist/askpass.sh
> >> > CHROME_DESKTOP=code-url-handler.desktop
> >> > VSCODE_GIT_ASKPASS_EXTRA_ARGS=--ms-enable-electron-run-as-node
> >> > TERM=xterm-256color
> >> > VSCODE_GIT_IPC_HANDLE=/tmp/vscode-git-56a269c0f5.sock
> >> > DISPLAY=:0
> >> > SHLVL=1
> >> >
> >> VSCODE_GIT_ASKPASS_MAIN=/usr/share/code/resources/app/extensions/git/dist/askpass-main.js
> >> > GDK_BACKEND=x11
> >> >
> >> PATH=/usr/lib/jvm/java-11-openjdk-amd64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
> >> > ORIGINAL_XDG_CURRENT_DESKTOP=undefined
> >> > DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
> >> > TERM_PROGRAM=vscode
> >> > _=/usr/bin/env
> >> >
> >> > If I run daffodil, then c-C in the terminal it creates, then the
> >> > environment has this added environment variable only.
> >> >
> >> >
> >> DAFFODIL_DEBUG_CLASSPATH=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/../ethernetIP/src/main/resources:/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/lib_managed/jars/com.owlcyberdefense/dfdl-ethernetip/dfdl-ethernetip-1.2.0.jar
> >> >
> >> > This works. My launch.json is attached.
> >> >
> >> >
> >> >
> >> >
> >> > On Fri, May 26, 2023 at 2:06 PM Shane Dell <sh...@apache.org>
> >> wrote:
> >> >
> >> > > Mike,
> >> > >
> >> > > The VSCode Extension should be using the daffodil command that is
> >> bundled
> >> > > with the
> >> > > extension. The issue might still be with that shell env variable you
> >> have
> >> > > for
> >> > > DAFFODIL_CLASSPATH, but this might only be an issue with a terminal
> >> that
> >> > > is already opened
> >> > > with the name "daffodil-debugger". If you close that terminal, the
> >> next
> >> > > you time you run it should
> >> > > create a new terminal with the env set properly. If you could try
> >> this and
> >> > > see if it works how you
> >> > > expect, that would help us single out this issue out to an already
> >> active
> >> > > terminal.
> >> > >
> >> > > Thank you,
> >> > >
> >> > > Shane Dell
> >> > >
> >> > >
> >> > > On 2023/05/25 23:59:53 Mike Beckerle wrote:
> >> > > > There are so many environment interactions....
> >> > > >
> >> > > > So I have discovered that the VSCode extension just seems to run
> >> whatever
> >> > > > daffodil command is on the path. It's not isolated at all from the
> >> shell
> >> > > > environment.
> >> > > >
> >> > > > I found this out because I have a customized daffodil shell script
> >> that
> >> > > > setups a bunch of DAFFODIL_CLASSPATH contents, and I couldn't
> >> figure out
> >> > > > why those were part of the VSCode environment.
> >> > > >
> >> > > > Shouldn't vscode run exactly the daffodil bundled with it, unless
> >> you
> >> > > > explicitly configure it not to?
> >> > > >
> >> > > > That's why it is finding the ethernetIP.dfdl.xsd file in the jar,
> >> not in
> >> > > > the file system. Because my daffodil command sets up lots of jars
> >> on the
> >> > > > DAFFODIL_CLASSPATH.
> >> > > >
> >> > > > It also depends on Java 11, breaks with Java 17. I also don't think
> >> this
> >> > > > should be inherited from the environment.
> >> > > >
> >> > > > I really have a hard time using, or even testing the VSCode
> >> extension if
> >> > > I
> >> > > > have to scrub my shell environment of everything in it in order to
> >> make
> >> > > the
> >> > > > vscode extension work.
> >> > > >
> >> > > > But it is this kind of problem that caused people to invent docker
> >> > > > containers.
> >> > > >
> >> > > > I am going to make a launcher script for VSCode that empties the
> >> > > > environment, then supplies only what is absolutely necessary, with
> >> no
> >> > > > outside environment assumptions.
> >> > > >
> >> > > > I will try to do this Friday if possible.
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Wed, May 24, 2023 at 2:50 PM Shane Dell <sh...@apache.org>
> >> wrote:
> >> > > >
> >> > > > > Mike,
> >> > > > >
> >> > > > > I am not sure why you would be seeing this. I am able to set the
> >> > > > > breakpoint where you
> >> > > > > mentioned without any issues. Can you try removing the
> >> daffodil-dap
> >> > > > > folder? This should be
> >> > > > > somewhere like:
> >> > > > > /home/$USER/.local/share/daffodil-dap
> >> > > > > The issue I am thinking is possibly a bug with the rc1 debugger
> >> that
> >> > > may
> >> > > > > not be present on rc2.
> >> > > > > But if the folder daffodil-dap/daffodil-debugger-3.4.0-1.3.0/
> >> already
> >> > > > > exists from rc1 then rc2 will
> >> > > > > not remake it. So in order to remake it you have to manually
> >> remove
> >> > > this
> >> > > > > folder. Let me know if
> >> > > > > this helps.
> >> > > > >
> >> > > > > Thank you,
> >> > > > >
> >> > > > > Shane Dell
> >> > > > >
> >> > > > > On 2023/05/24 15:15:33 Mike Beckerle wrote:
> >> > > > > > I have been able to continue evaluating RC2.
> >> > > > > >
> >> > > > > > My vote is -1
> >> > > > > >
> >> > > > > > My testing steps listed below. I am trying to debug PCAP. I
> >> have a
> >> > > broken
> >> > > > > > icmp packet (bad magic number), and also just step through a
> >> correct
> >> > > icmp
> >> > > > > > packet.
> >> > > > > >
> >> > > > > > I am trying to debug icmp.badMagicNum.cap data file.
> >> > > > > >
> >> > > > > > I should get a failure of an assert because the magic number is
> >> not
> >> > > one
> >> > > > > of
> >> > > > > > the expected values.
> >> > > > > >
> >> > > > > > I'm just trying to single-step through it until it hits the
> >> error.
> >> > > > > >
> >> > > > > > Once we get to the MagicNumber element, it doesn't position on
> >> the
> >> > > assert
> >> > > > > > while evaluating that expression or even on the element
> >> carrying the
> >> > > > > assert.
> >> > > > > > It jumps back to the top-level xs:schema position.
> >> > > > > >
> >> > > > > > I continue to click single step. Watching for the error to be
> >> > > displayed
> >> > > > > in
> >> > > > > > the upper left under Variables > Parse > Diagnostics > Errors
> >> which
> >> > > is
> >> > > > > > where I think it ought to go.
> >> > > > > >
> >> > > > > > At some point the failure is somehow detected as the debug
> >> session
> >> > > ends.
> >> > > > > A
> >> > > > > > dialog offers to open /tmp/infoset.xml for me, which I open and
> >> it is
> >> > > > > empty
> >> > > > > > of course.
> >> > > > > >
> >> > > > > > Nothing displayed the error message from the assert, or even
> >> > > indicated
> >> > > > > what
> >> > > > > > went wrong at all. Not the state display in the upper left, not
> >> the
> >> > > > > > terminal, nor debug console nor output.
> >> > > > > >
> >> > > > > > The upper left box with "Variables" is cleared at the end. It
> >> should
> >> > > > > > certainly not be cleared at the end of a run when final state
> >> may be
> >> > > of
> >> > > > > > interest. At the start of a run, or manually, but not at the
> >> end of a
> >> > > > > run.
> >> > > > > >
> >> > > > > > Second attempt to use it:
> >> > > > > >
> >> > > > > > I create a second launch config this time for what should be a
> >> > > successful
> >> > > > > > parse of icmp1.cap
> >> > > > > >
> >> > > > > > As I single step, the position of the cursor on the schema is
> >> jumping
> >> > > > > > around all over the place back and forth between a containing
> >> > > element and
> >> > > > > > the child elements it contains.
> >> > > > > > The infoset display does not get updated until long after the
> >> > > display has
> >> > > > > > moved on and is positioned on the next element.
> >> > > > > >
> >> > > > > > This jumping around gets completely out of hand once you step
> >> into
> >> > > the
> >> > > > > > complexType "Ethernet" which is in the ethernetIP jar file.
> >> > > > > >
> >> > > > > > At that point, after each child of Ethernet, it jumps back to
> >> the
> >> > > > > Ethernet
> >> > > > > > element in the pcap.dfdl.xsd file, and then back into the
> >> > > > > > ethernetIP.dfdl.xsd for the next child, then back to the
> >> > > pcap.dfdl.xsd
> >> > > > > > file, etc.
> >> > > > > >
> >> > > > > > All this jumping around basically makes stepping useless. So I
> >> > > decide to
> >> > > > > > put a breakpoint on the Version element inside the IPv4Header
> >> element
> >> > > > > > inside ethernetIP.dfdl.xsd.
> >> > > > > >
> >> > > > > > Just adding the breakpoint causes the debug session to end with
> >> an
> >> > > NPE in
> >> > > > > > the dap setBreakpoint.
> >> > > > > >
> >> > > > > > ERROR o.a.d.d.d.DAPodil - unhandled error
> >> > > > > > java.lang.NullPointerException: null
> >> > > > > > at
> >> > > > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> >> > > > > > at
> >> java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> >> > > > > > at
> >> > > > > >
> >> java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> >> > > > > > at java.base/java.nio.file.Path.of(Path.java:147)
> >> > > > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
> >> > > > > > at
> >> > > > > >
> >> > > > >
> >> > >
> >> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> >> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > > > at flatMap @
> >> > > > > >
> >> > > > >
> >> > >
> >> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > at handleErrorWith @
> >> > > > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> >> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > at flatMap @
> >> > > fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> >> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > 2023-05-24 11:09:10,717 [io-compute-1] DEBUG o.a.d.d.d.DAPodil -
> >> > > > > whenDone:
> >> > > > > > completed
> >> > > > > > java.lang.NullPointerException
> >> > > > > > at
> >> > > > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> >> > > > > > at
> >> java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> >> > > > > > at
> >> > > > > >
> >> java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> >> > > > > > at java.base/java.nio.file.Path.of(Path.java:147)
> >> > > > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
> >> > > > > > at
> >> > > > > >
> >> > > > >
> >> > >
> >> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> >> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > > > at flatMap @
> >> > > > > >
> >> > > > >
> >> > >
> >> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > at handleErrorWith @
> >> > > > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> >> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > > at flatMap @
> >> > > fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> >> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > > > at flatMap @
> >> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > On Tue, May 23, 2023 at 5:19 PM Mike Beckerle <
> >> mbeckerle@apache.org>
> >> > > > > wrote:
> >> > > > > >
> >> > > > > > > I really am trying to eval the RC2.
> >> > > > > > >
> >> > > > > > > Right now my VMWare Linux system has been broken by recent
> >> updates
> >> > > to
> >> > > > > the
> >> > > > > > > MS Windows host environment.
> >> > > > > > >
> >> > > > > > > I don't exactly understand this, but the VSCode front end can
> >> no
> >> > > longer
> >> > > > > > > connect to the server at localhost:4711. I get ECONNREFUSED.
> >> This
> >> > > > > started
> >> > > > > > > when a bunch of MS-Windows updates were done on the host OS.
> >> I run
> >> > > > > VSCode
> >> > > > > > > on the linux client machine, which lives behind a NAT firewall
> >> > > created
> >> > > > > by
> >> > > > > > > VMWare workstation - so I really don't get how these could be
> >> > > > > interacting
> >> > > > > > > this way, but... there it is.
> >> > > > > > >
> >> > > > > > > I will switch back to my standalone linux PC if this isn't
> >> resolved
> >> > > > > today.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Tue, May 16, 2023 at 1:24 PM Shane Dell <
> >> shanedell@apache.org>
> >> > > > > wrote:
> >> > > > > > >
> >> > > > > > >> Hello all,I'd like to call a vote to release Apache Daffodil
> >> VS
> >> > > Code
> >> > > > > > >> 1.3.0-rc2.
> >> > > > > > >>
> >> > > > > > >> All distribution packages, including signatures, digests,
> >> etc.
> >> > > can be
> >> > > > > > >> found at:
> >> > > > > > >>
> >> > > > >
> >> > >
> >> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
> >> > > > > > >>
> >> > > > > > >> This release has been signed with PGP key
> >> > > > > > >> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
> >> > > > > > >> to shanedell@apache.org, which is included in the KEYS file
> >> here:
> >> > > > > > >> https://downloads.apache.org/daffodil/KEYS
> >> > > > > > >>
> >> > > > > > >> The release candidate has been tagged in git with 1.3.0-rc2.
> >> The
> >> > > > > binaries
> >> > > > > > >>
> >> > > > > > >> For reference, here is a list of all closed GitHub issues
> >> tagged
> >> > > with
> >> > > > > > >> 1.3.0:
> >> > > > > > >>
> >> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
> >> > > > > > >>
> >> > > > > > >> Please review and vote. The vote will be open for at least 72
> >> > > hours
> >> > > > > > >> (Friday, 19 May 2023, 1:30pm EST).
> >> > > > > > >>
> >> > > > > > >> [ ] +1 approve
> >> > > > > > >> [ ] +0 no opinion
> >> > > > > > >> [ ] -1 disapprove (and reason why)
> >> > > > > > >>
> >> > > > > > >> Documentation for 1.3.0 can be found here
> >> > > > > > >> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
> >> > > > > > >> <
> >> > > > > > >>
> >> > > > >
> >> > >
> >> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
> >> > > > > > >> >.
> >> > > > > > >>
> >> > > > > > >> Please note one of the large features that was focused on in
> >> > > 1.3.0 was
> >> > > > > > >> improving the data editor. To open the data editor you will
> >> need
> >> > > to
> >> > > > > open
> >> > > > > > >> the
> >> > > > > > >> command palette using:
> >> > > > > > >>
> >> > > > > > >> - Ctrl + Shift + P (windows/linux)
> >> > > > > > >> - Command + Shift + P (mac)
> >> > > > > > >>
> >> > > > > > >> Then typing "data.edit", make sure "Daffodil Debug: Data
> >> Editor"
> >> > > is
> >> > > > > > >> selected
> >> > > > > > >> and then hitting enter. By default the data editor will run
> >> on
> >> > > port
> >> > > > > 9000
> >> > > > > > >> of
> >> > > > > > >> your machine. However, if you want to use a different port
> >> follow
> >> > > > > steps
> >> > > > > > >> here
> >> > > > > > >> Data Editor Launch Settings
> >> > > > > > >> <
> >> > > > > > >>
> >> > > > >
> >> > >
> >> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
> >> > > > > > >> >
> >> > > > > > >>
> >> > > > > > >> Thank you,
> >> > > > > > >>
> >> > > > > > >> - Shane Dell
> >> > > > > > >>
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Mike Beckerle <mb...@apache.org>.
I re-tested, and changing to Java 17 in my launch script (with very limited
set of env vars) works.
So the environment interactions I was having are NOT due to the JVM being
Java 17.
I suspect we can drop the issue of bundling Java with it. I doubt we can do
that realistically given the license of Java.
Really the behavior needs to be that which JVM doesn't affect the behavior,
even when you have an erroneous configuration of launch.json.
The configurations were faulty - daffodilDebugClasspath was incorrect at
various points, making it get a NPE when setting breakpoints, etc. Getting
that right, and realizing that I needed to have BOTH regular files and Jar
files on the daffodilDebugClasspath was hard-won knowledge.
I do recall reading doc that said it requires Java 11, and that's part of
why I started creating this isolation environment for it, because it wasn't
working, and seemed at the time like it might have been a Java 11 vs. 17
incompatibility. Clearly it ended up NOT being that.
-mikeb
On Mon, Jun 5, 2023 at 4:53 PM Mike Beckerle <mb...@apache.org> wrote:
> Well I had sufficient problems with it that I should re-verify if it works
> with Java 17. It may be that the java version thing was a non-issue that
> other things were interacting with it.
>
>
> On Mon, Jun 5, 2023 at 4:45 PM Shane Dell <sh...@apache.org> wrote:
>
>> The extension does not depend on Java 11 it should work fine with Java
>> 17. I remember back
>> porting to make sure we supported Java 8 also. Other than relying on the
>> system Java, I am not
>> sure how on else we can really on Java. I know Davin mentioned packaging
>> a JVM or JDK or
>> something like that but is that very common practice to do? The only time
>> I ever heard of doing
>> that was for a language server so was wondering if its necessary or might
>> be overkill. I agree
>> with the needing to fix the issue with the environment when it comes to
>> shell and think what you
>> posted will help us do that. However, when it comes to Java I think if
>> there is an issue with Java
>> 17 we should figure that out first. Then maybe put it out for a vote or
>> something about bundling
>> a Java version with the extension and then letting the user specify a
>> different path to Java or
>> something like that?
>>
>>
>> On 2023/05/31 23:02:03 Mike Beckerle wrote:
>> > So, I have the 1.3.0 RC2 vscode extension debugger working now for PCAP
>> w/
>> > ethernetIP and I can set a breakpoint in ethernetIP.dfdl.xsd and I no
>> > longer get an NPE.
>> >
>> > Configuring this is really far too difficult, and I have to have the
>> schema
>> > working first with sbt in order to even hope to configure it.
>> >
>> > But I'll get back to that later.
>> >
>> > More basic issue is that I am unable to debug even dead-simple errors
>> with
>> > it. I clobbered the magic number in the first bytes of an icmp1.cap
>> file.
>> > This fails an assert at line 97 of pcap.dfdl.xsd, which is the first
>> > element of simple type.
>> >
>> > I step through with the debugger, and nothing displays *anything* about
>> the
>> > assert failure. Execution ends. A little pop-up dialog says it wrote my
>> > infoset out, which it didn't. There's nothing showing that anything went
>> > wrong. All panels on the left are blank.
>> >
>> > If I modify the schema and make any mistake, then I get a blocking
>> dialog
>> > showing the SDEs. This is basically intolerable, and is why I have to go
>> > back to sbt to debug this. You can't even go through the SDEs as you
>> move
>> > the editor. It's a blocking dialog.
>> >
>> > So both for schema compilation, and runtime errors there simply has to
>> be a
>> > non-modal display of errors, probably on the left with variables, watch,
>> > call-stack, and breakpoints. But a quasi-edit-buffer containing them
>> would
>> > also be fine, or even just output them to the terminal window.
>> >
>> > The rest of this email is about the hoops I had to jump through to get
>> this
>> > to work at all. Arguably, these issues are about working with a
>> > multi-component schema, and we could define our objectives to simpler
>> > schemas than this, but that puts it in the realm of toy-only schemas.
>> The
>> > example is just PCAP + ethernetIP, which is not an extreme example.
>> >
>> > The existing shell environment of the terminal definitely causes
>> problems.
>> > But just exiting that so that the debugger starts a new terminal doesn't
>> > fix it, as that inherits the environment from where vscode was started.
>> It
>> > starts a new login shell unless you take steps to prevent that.
>> >
>> > A big trip-up in usability is that the wizard that creates the
>> launch.json,
>> > doesn't actually let you populate the daffodilDebugClaspath. You have to
>> > edit that by hand, and nothing verifies it, so any typo and you get
>> tedious
>> > failures that are hard to figure out. You just get an SDE - if you are
>> > lucky - via a blocking dialog.
>> >
>> > The wizard should help you put directories and corresponding jars on the
>> > daffodilDebugClasspath, and it should not let you put a non-existing
>> > directory or jar file on it. Ideally, at launch time the
>> > daffodilDebugClasspath should be checked, just in case something on it
>> no
>> > longer exists.
>> >
>> > If the debugger happens to find a file on the daffodilDebugClasspath in
>> a
>> > jar, vs. in a regular directory (maybe you have a typo in the regular
>> > directory path), so it passes that and hits the jar, then setting a
>> > breakpoint causes a NPE. Something requires a "hierarchical URL" which a
>> > jar-file URL is not. This is the bug that really has to be fixed so that
>> > you can set breakpoints regardless of whether the source DFDL schema
>> file
>> > was in a jar or not.
>> >
>> > Note: at Owl we have DFDL schemas where the starting root element of the
>> > schema isn't even part of the schema, it's part of a component schema
>> that
>> > is in a jar on the classpath. This happens when you have an
>> > envelope-payload idiom. The schema that combines the envelope and
>> payload
>> > together contains only the glue part that connects the two together. The
>> > root element is the envelope's root element, and that schema is in a jar
>> > file.
>> >
>> > I did not see any feature in launch.json for specifying the
>> name/namespace
>> > of the root element. There is just the "program" element, which must be
>> to
>> > a file in the current schema. That needs to be able to be specified
>> just as
>> > element name + namespace.
>> >
>> > The jars have to be on the daffodilDebugClasspath because of the plugins
>> > that daffodil must find in them. That's understandable, the problem is
>> > having to also have a regular file location for any schema you want to
>> be
>> > able to put a breakpoint in and it must be before the corresponding jar.
>> >
>> > That all said, here are the scripts and settings I used to get it to
>> work:
>> >
>> > First, I have this script I use to launch vscode in a minimal/empty
>> > environment. This technique is useful to
>> > determine if you are depending on the environment in any way:
>> >
>> > #! /bin/bash
>> > if [ "$1" = "emptyEnv" ] # is it the special first arg
>> > then
>> > shift # discard special first arg, and continue with the rest of the
>> > script
>> > else
>> > CMD=$0
>> > exec -c $CMD "emptyEnv" $* # exec with an empty environment, special
>> > first arg
>> > fi
>> > # the environment is empty when we get here
>> > # just add env vars we need to get vscode to work
>> > export LANG=en_US.UTF-8
>> > export HOME=/home/mbeckerle
>> > export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 # java 11 required
>> > export PATH=$JAVA_HOME/bin:$PATH
>> > export DISPLAY=:0
>> > export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus # vscode
>> spits
>> > lots of errors without this
>> > exec /usr/bin/code
>> > # end
>> >
>> > Then, within ~/.config/Code/User/settings.json, I had to create these
>> > settings for terminal:
>> >
>> > "terminal.integrated.inheritEnv": false,
>> > "terminal.integrated.profiles.linux": {
>> > "Custom Init": {
>> > "path": "/bin/bash",
>> > "args": [
>> > "--norc"
>> > ]
>> > }
>> > },
>> > "terminal.integrated.defaultProfile.linux": "Custom Init"
>> >
>> > That instructs vscode not to create a login shell for the terminal. The
>> > --norc option means it does not run the .bashrc file so that the java
>> > version specified in the launch script sticks.
>> >
>> > I am not sure the inheritEnv setting actually does anything. That alone
>> > sounds like the right thing, but that alone didn't solve the problem.
>> > You still have everything in the login shell environment where vscode
>> was
>> > launched. Hence the need for the environment-emptying launch script.
>> >
>> > With those two things, a terminal created by vscode does not inherit
>> > everything my typical shell has. For example there is no daffodil on the
>> > path, no DAFFODIL_CLASSPATH env var, etc.
>> >
>> > Given the above if I just create a terminal with Terminal > new
>> Terminal:
>> > The environment is relatively simple:
>> >
>> > bash-5.0$ env
>> > COLORTERM=truecolor
>> > TERM_PROGRAM_VERSION=1.78.2
>> > JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
>> > NO_AT_BRIDGE=1
>> > PWD=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP
>> > VSCODE_GIT_ASKPASS_NODE=/usr/share/code/code
>> > HOME=/home/mbeckerle
>> > LANG=en_US.UTF-8
>> > GIT_ASKPASS=/usr/share/code/resources/app/extensions/git/dist/askpass.sh
>> > CHROME_DESKTOP=code-url-handler.desktop
>> > VSCODE_GIT_ASKPASS_EXTRA_ARGS=--ms-enable-electron-run-as-node
>> > TERM=xterm-256color
>> > VSCODE_GIT_IPC_HANDLE=/tmp/vscode-git-56a269c0f5.sock
>> > DISPLAY=:0
>> > SHLVL=1
>> >
>> VSCODE_GIT_ASKPASS_MAIN=/usr/share/code/resources/app/extensions/git/dist/askpass-main.js
>> > GDK_BACKEND=x11
>> >
>> PATH=/usr/lib/jvm/java-11-openjdk-amd64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
>> > ORIGINAL_XDG_CURRENT_DESKTOP=undefined
>> > DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
>> > TERM_PROGRAM=vscode
>> > _=/usr/bin/env
>> >
>> > If I run daffodil, then c-C in the terminal it creates, then the
>> > environment has this added environment variable only.
>> >
>> >
>> DAFFODIL_DEBUG_CLASSPATH=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/../ethernetIP/src/main/resources:/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/lib_managed/jars/com.owlcyberdefense/dfdl-ethernetip/dfdl-ethernetip-1.2.0.jar
>> >
>> > This works. My launch.json is attached.
>> >
>> >
>> >
>> >
>> > On Fri, May 26, 2023 at 2:06 PM Shane Dell <sh...@apache.org>
>> wrote:
>> >
>> > > Mike,
>> > >
>> > > The VSCode Extension should be using the daffodil command that is
>> bundled
>> > > with the
>> > > extension. The issue might still be with that shell env variable you
>> have
>> > > for
>> > > DAFFODIL_CLASSPATH, but this might only be an issue with a terminal
>> that
>> > > is already opened
>> > > with the name "daffodil-debugger". If you close that terminal, the
>> next
>> > > you time you run it should
>> > > create a new terminal with the env set properly. If you could try
>> this and
>> > > see if it works how you
>> > > expect, that would help us single out this issue out to an already
>> active
>> > > terminal.
>> > >
>> > > Thank you,
>> > >
>> > > Shane Dell
>> > >
>> > >
>> > > On 2023/05/25 23:59:53 Mike Beckerle wrote:
>> > > > There are so many environment interactions....
>> > > >
>> > > > So I have discovered that the VSCode extension just seems to run
>> whatever
>> > > > daffodil command is on the path. It's not isolated at all from the
>> shell
>> > > > environment.
>> > > >
>> > > > I found this out because I have a customized daffodil shell script
>> that
>> > > > setups a bunch of DAFFODIL_CLASSPATH contents, and I couldn't
>> figure out
>> > > > why those were part of the VSCode environment.
>> > > >
>> > > > Shouldn't vscode run exactly the daffodil bundled with it, unless
>> you
>> > > > explicitly configure it not to?
>> > > >
>> > > > That's why it is finding the ethernetIP.dfdl.xsd file in the jar,
>> not in
>> > > > the file system. Because my daffodil command sets up lots of jars
>> on the
>> > > > DAFFODIL_CLASSPATH.
>> > > >
>> > > > It also depends on Java 11, breaks with Java 17. I also don't think
>> this
>> > > > should be inherited from the environment.
>> > > >
>> > > > I really have a hard time using, or even testing the VSCode
>> extension if
>> > > I
>> > > > have to scrub my shell environment of everything in it in order to
>> make
>> > > the
>> > > > vscode extension work.
>> > > >
>> > > > But it is this kind of problem that caused people to invent docker
>> > > > containers.
>> > > >
>> > > > I am going to make a launcher script for VSCode that empties the
>> > > > environment, then supplies only what is absolutely necessary, with
>> no
>> > > > outside environment assumptions.
>> > > >
>> > > > I will try to do this Friday if possible.
>> > > >
>> > > >
>> > > >
>> > > > On Wed, May 24, 2023 at 2:50 PM Shane Dell <sh...@apache.org>
>> wrote:
>> > > >
>> > > > > Mike,
>> > > > >
>> > > > > I am not sure why you would be seeing this. I am able to set the
>> > > > > breakpoint where you
>> > > > > mentioned without any issues. Can you try removing the
>> daffodil-dap
>> > > > > folder? This should be
>> > > > > somewhere like:
>> > > > > /home/$USER/.local/share/daffodil-dap
>> > > > > The issue I am thinking is possibly a bug with the rc1 debugger
>> that
>> > > may
>> > > > > not be present on rc2.
>> > > > > But if the folder daffodil-dap/daffodil-debugger-3.4.0-1.3.0/
>> already
>> > > > > exists from rc1 then rc2 will
>> > > > > not remake it. So in order to remake it you have to manually
>> remove
>> > > this
>> > > > > folder. Let me know if
>> > > > > this helps.
>> > > > >
>> > > > > Thank you,
>> > > > >
>> > > > > Shane Dell
>> > > > >
>> > > > > On 2023/05/24 15:15:33 Mike Beckerle wrote:
>> > > > > > I have been able to continue evaluating RC2.
>> > > > > >
>> > > > > > My vote is -1
>> > > > > >
>> > > > > > My testing steps listed below. I am trying to debug PCAP. I
>> have a
>> > > broken
>> > > > > > icmp packet (bad magic number), and also just step through a
>> correct
>> > > icmp
>> > > > > > packet.
>> > > > > >
>> > > > > > I am trying to debug icmp.badMagicNum.cap data file.
>> > > > > >
>> > > > > > I should get a failure of an assert because the magic number is
>> not
>> > > one
>> > > > > of
>> > > > > > the expected values.
>> > > > > >
>> > > > > > I'm just trying to single-step through it until it hits the
>> error.
>> > > > > >
>> > > > > > Once we get to the MagicNumber element, it doesn't position on
>> the
>> > > assert
>> > > > > > while evaluating that expression or even on the element
>> carrying the
>> > > > > assert.
>> > > > > > It jumps back to the top-level xs:schema position.
>> > > > > >
>> > > > > > I continue to click single step. Watching for the error to be
>> > > displayed
>> > > > > in
>> > > > > > the upper left under Variables > Parse > Diagnostics > Errors
>> which
>> > > is
>> > > > > > where I think it ought to go.
>> > > > > >
>> > > > > > At some point the failure is somehow detected as the debug
>> session
>> > > ends.
>> > > > > A
>> > > > > > dialog offers to open /tmp/infoset.xml for me, which I open and
>> it is
>> > > > > empty
>> > > > > > of course.
>> > > > > >
>> > > > > > Nothing displayed the error message from the assert, or even
>> > > indicated
>> > > > > what
>> > > > > > went wrong at all. Not the state display in the upper left, not
>> the
>> > > > > > terminal, nor debug console nor output.
>> > > > > >
>> > > > > > The upper left box with "Variables" is cleared at the end. It
>> should
>> > > > > > certainly not be cleared at the end of a run when final state
>> may be
>> > > of
>> > > > > > interest. At the start of a run, or manually, but not at the
>> end of a
>> > > > > run.
>> > > > > >
>> > > > > > Second attempt to use it:
>> > > > > >
>> > > > > > I create a second launch config this time for what should be a
>> > > successful
>> > > > > > parse of icmp1.cap
>> > > > > >
>> > > > > > As I single step, the position of the cursor on the schema is
>> jumping
>> > > > > > around all over the place back and forth between a containing
>> > > element and
>> > > > > > the child elements it contains.
>> > > > > > The infoset display does not get updated until long after the
>> > > display has
>> > > > > > moved on and is positioned on the next element.
>> > > > > >
>> > > > > > This jumping around gets completely out of hand once you step
>> into
>> > > the
>> > > > > > complexType "Ethernet" which is in the ethernetIP jar file.
>> > > > > >
>> > > > > > At that point, after each child of Ethernet, it jumps back to
>> the
>> > > > > Ethernet
>> > > > > > element in the pcap.dfdl.xsd file, and then back into the
>> > > > > > ethernetIP.dfdl.xsd for the next child, then back to the
>> > > pcap.dfdl.xsd
>> > > > > > file, etc.
>> > > > > >
>> > > > > > All this jumping around basically makes stepping useless. So I
>> > > decide to
>> > > > > > put a breakpoint on the Version element inside the IPv4Header
>> element
>> > > > > > inside ethernetIP.dfdl.xsd.
>> > > > > >
>> > > > > > Just adding the breakpoint causes the debug session to end with
>> an
>> > > NPE in
>> > > > > > the dap setBreakpoint.
>> > > > > >
>> > > > > > ERROR o.a.d.d.d.DAPodil - unhandled error
>> > > > > > java.lang.NullPointerException: null
>> > > > > > at
>> > > > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
>> > > > > > at
>> java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
>> > > > > > at
>> > > > > >
>> java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
>> > > > > > at java.base/java.nio.file.Path.of(Path.java:147)
>> > > > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
>> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > > > at flatMap @
>> > > > > >
>> > > > >
>> > >
>> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > at handleErrorWith @
>> > > > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
>> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > at flatMap @
>> > > fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
>> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > 2023-05-24 11:09:10,717 [io-compute-1] DEBUG o.a.d.d.d.DAPodil -
>> > > > > whenDone:
>> > > > > > completed
>> > > > > > java.lang.NullPointerException
>> > > > > > at
>> > > > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
>> > > > > > at
>> java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
>> > > > > > at
>> > > > > >
>> java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
>> > > > > > at java.base/java.nio.file.Path.of(Path.java:147)
>> > > > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
>> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > > > at flatMap @
>> > > > > >
>> > > > >
>> > >
>> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > at handleErrorWith @
>> > > > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
>> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > > at flatMap @
>> > > fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
>> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > > > at flatMap @
>> fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Tue, May 23, 2023 at 5:19 PM Mike Beckerle <
>> mbeckerle@apache.org>
>> > > > > wrote:
>> > > > > >
>> > > > > > > I really am trying to eval the RC2.
>> > > > > > >
>> > > > > > > Right now my VMWare Linux system has been broken by recent
>> updates
>> > > to
>> > > > > the
>> > > > > > > MS Windows host environment.
>> > > > > > >
>> > > > > > > I don't exactly understand this, but the VSCode front end can
>> no
>> > > longer
>> > > > > > > connect to the server at localhost:4711. I get ECONNREFUSED.
>> This
>> > > > > started
>> > > > > > > when a bunch of MS-Windows updates were done on the host OS.
>> I run
>> > > > > VSCode
>> > > > > > > on the linux client machine, which lives behind a NAT firewall
>> > > created
>> > > > > by
>> > > > > > > VMWare workstation - so I really don't get how these could be
>> > > > > interacting
>> > > > > > > this way, but... there it is.
>> > > > > > >
>> > > > > > > I will switch back to my standalone linux PC if this isn't
>> resolved
>> > > > > today.
>> > > > > > >
>> > > > > > >
>> > > > > > > On Tue, May 16, 2023 at 1:24 PM Shane Dell <
>> shanedell@apache.org>
>> > > > > wrote:
>> > > > > > >
>> > > > > > >> Hello all,I'd like to call a vote to release Apache Daffodil
>> VS
>> > > Code
>> > > > > > >> 1.3.0-rc2.
>> > > > > > >>
>> > > > > > >> All distribution packages, including signatures, digests,
>> etc.
>> > > can be
>> > > > > > >> found at:
>> > > > > > >>
>> > > > >
>> > >
>> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
>> > > > > > >>
>> > > > > > >> This release has been signed with PGP key
>> > > > > > >> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
>> > > > > > >> to shanedell@apache.org, which is included in the KEYS file
>> here:
>> > > > > > >> https://downloads.apache.org/daffodil/KEYS
>> > > > > > >>
>> > > > > > >> The release candidate has been tagged in git with 1.3.0-rc2.
>> The
>> > > > > binaries
>> > > > > > >>
>> > > > > > >> For reference, here is a list of all closed GitHub issues
>> tagged
>> > > with
>> > > > > > >> 1.3.0:
>> > > > > > >>
>> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
>> > > > > > >>
>> > > > > > >> Please review and vote. The vote will be open for at least 72
>> > > hours
>> > > > > > >> (Friday, 19 May 2023, 1:30pm EST).
>> > > > > > >>
>> > > > > > >> [ ] +1 approve
>> > > > > > >> [ ] +0 no opinion
>> > > > > > >> [ ] -1 disapprove (and reason why)
>> > > > > > >>
>> > > > > > >> Documentation for 1.3.0 can be found here
>> > > > > > >> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
>> > > > > > >> <
>> > > > > > >>
>> > > > >
>> > >
>> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
>> > > > > > >> >.
>> > > > > > >>
>> > > > > > >> Please note one of the large features that was focused on in
>> > > 1.3.0 was
>> > > > > > >> improving the data editor. To open the data editor you will
>> need
>> > > to
>> > > > > open
>> > > > > > >> the
>> > > > > > >> command palette using:
>> > > > > > >>
>> > > > > > >> - Ctrl + Shift + P (windows/linux)
>> > > > > > >> - Command + Shift + P (mac)
>> > > > > > >>
>> > > > > > >> Then typing "data.edit", make sure "Daffodil Debug: Data
>> Editor"
>> > > is
>> > > > > > >> selected
>> > > > > > >> and then hitting enter. By default the data editor will run
>> on
>> > > port
>> > > > > 9000
>> > > > > > >> of
>> > > > > > >> your machine. However, if you want to use a different port
>> follow
>> > > > > steps
>> > > > > > >> here
>> > > > > > >> Data Editor Launch Settings
>> > > > > > >> <
>> > > > > > >>
>> > > > >
>> > >
>> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
>> > > > > > >> >
>> > > > > > >>
>> > > > > > >> Thank you,
>> > > > > > >>
>> > > > > > >> - Shane Dell
>> > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Mike Beckerle <mb...@apache.org>.
Well I had sufficient problems with it that I should re-verify if it works
with Java 17. It may be that the java version thing was a non-issue that
other things were interacting with it.
On Mon, Jun 5, 2023 at 4:45 PM Shane Dell <sh...@apache.org> wrote:
> The extension does not depend on Java 11 it should work fine with Java 17.
> I remember back
> porting to make sure we supported Java 8 also. Other than relying on the
> system Java, I am not
> sure how on else we can really on Java. I know Davin mentioned packaging a
> JVM or JDK or
> something like that but is that very common practice to do? The only time
> I ever heard of doing
> that was for a language server so was wondering if its necessary or might
> be overkill. I agree
> with the needing to fix the issue with the environment when it comes to
> shell and think what you
> posted will help us do that. However, when it comes to Java I think if
> there is an issue with Java
> 17 we should figure that out first. Then maybe put it out for a vote or
> something about bundling
> a Java version with the extension and then letting the user specify a
> different path to Java or
> something like that?
>
>
> On 2023/05/31 23:02:03 Mike Beckerle wrote:
> > So, I have the 1.3.0 RC2 vscode extension debugger working now for PCAP
> w/
> > ethernetIP and I can set a breakpoint in ethernetIP.dfdl.xsd and I no
> > longer get an NPE.
> >
> > Configuring this is really far too difficult, and I have to have the
> schema
> > working first with sbt in order to even hope to configure it.
> >
> > But I'll get back to that later.
> >
> > More basic issue is that I am unable to debug even dead-simple errors
> with
> > it. I clobbered the magic number in the first bytes of an icmp1.cap file.
> > This fails an assert at line 97 of pcap.dfdl.xsd, which is the first
> > element of simple type.
> >
> > I step through with the debugger, and nothing displays *anything* about
> the
> > assert failure. Execution ends. A little pop-up dialog says it wrote my
> > infoset out, which it didn't. There's nothing showing that anything went
> > wrong. All panels on the left are blank.
> >
> > If I modify the schema and make any mistake, then I get a blocking dialog
> > showing the SDEs. This is basically intolerable, and is why I have to go
> > back to sbt to debug this. You can't even go through the SDEs as you move
> > the editor. It's a blocking dialog.
> >
> > So both for schema compilation, and runtime errors there simply has to
> be a
> > non-modal display of errors, probably on the left with variables, watch,
> > call-stack, and breakpoints. But a quasi-edit-buffer containing them
> would
> > also be fine, or even just output them to the terminal window.
> >
> > The rest of this email is about the hoops I had to jump through to get
> this
> > to work at all. Arguably, these issues are about working with a
> > multi-component schema, and we could define our objectives to simpler
> > schemas than this, but that puts it in the realm of toy-only schemas. The
> > example is just PCAP + ethernetIP, which is not an extreme example.
> >
> > The existing shell environment of the terminal definitely causes
> problems.
> > But just exiting that so that the debugger starts a new terminal doesn't
> > fix it, as that inherits the environment from where vscode was started.
> It
> > starts a new login shell unless you take steps to prevent that.
> >
> > A big trip-up in usability is that the wizard that creates the
> launch.json,
> > doesn't actually let you populate the daffodilDebugClaspath. You have to
> > edit that by hand, and nothing verifies it, so any typo and you get
> tedious
> > failures that are hard to figure out. You just get an SDE - if you are
> > lucky - via a blocking dialog.
> >
> > The wizard should help you put directories and corresponding jars on the
> > daffodilDebugClasspath, and it should not let you put a non-existing
> > directory or jar file on it. Ideally, at launch time the
> > daffodilDebugClasspath should be checked, just in case something on it no
> > longer exists.
> >
> > If the debugger happens to find a file on the daffodilDebugClasspath in a
> > jar, vs. in a regular directory (maybe you have a typo in the regular
> > directory path), so it passes that and hits the jar, then setting a
> > breakpoint causes a NPE. Something requires a "hierarchical URL" which a
> > jar-file URL is not. This is the bug that really has to be fixed so that
> > you can set breakpoints regardless of whether the source DFDL schema file
> > was in a jar or not.
> >
> > Note: at Owl we have DFDL schemas where the starting root element of the
> > schema isn't even part of the schema, it's part of a component schema
> that
> > is in a jar on the classpath. This happens when you have an
> > envelope-payload idiom. The schema that combines the envelope and payload
> > together contains only the glue part that connects the two together. The
> > root element is the envelope's root element, and that schema is in a jar
> > file.
> >
> > I did not see any feature in launch.json for specifying the
> name/namespace
> > of the root element. There is just the "program" element, which must be
> to
> > a file in the current schema. That needs to be able to be specified just
> as
> > element name + namespace.
> >
> > The jars have to be on the daffodilDebugClasspath because of the plugins
> > that daffodil must find in them. That's understandable, the problem is
> > having to also have a regular file location for any schema you want to be
> > able to put a breakpoint in and it must be before the corresponding jar.
> >
> > That all said, here are the scripts and settings I used to get it to
> work:
> >
> > First, I have this script I use to launch vscode in a minimal/empty
> > environment. This technique is useful to
> > determine if you are depending on the environment in any way:
> >
> > #! /bin/bash
> > if [ "$1" = "emptyEnv" ] # is it the special first arg
> > then
> > shift # discard special first arg, and continue with the rest of the
> > script
> > else
> > CMD=$0
> > exec -c $CMD "emptyEnv" $* # exec with an empty environment, special
> > first arg
> > fi
> > # the environment is empty when we get here
> > # just add env vars we need to get vscode to work
> > export LANG=en_US.UTF-8
> > export HOME=/home/mbeckerle
> > export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 # java 11 required
> > export PATH=$JAVA_HOME/bin:$PATH
> > export DISPLAY=:0
> > export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus # vscode
> spits
> > lots of errors without this
> > exec /usr/bin/code
> > # end
> >
> > Then, within ~/.config/Code/User/settings.json, I had to create these
> > settings for terminal:
> >
> > "terminal.integrated.inheritEnv": false,
> > "terminal.integrated.profiles.linux": {
> > "Custom Init": {
> > "path": "/bin/bash",
> > "args": [
> > "--norc"
> > ]
> > }
> > },
> > "terminal.integrated.defaultProfile.linux": "Custom Init"
> >
> > That instructs vscode not to create a login shell for the terminal. The
> > --norc option means it does not run the .bashrc file so that the java
> > version specified in the launch script sticks.
> >
> > I am not sure the inheritEnv setting actually does anything. That alone
> > sounds like the right thing, but that alone didn't solve the problem.
> > You still have everything in the login shell environment where vscode was
> > launched. Hence the need for the environment-emptying launch script.
> >
> > With those two things, a terminal created by vscode does not inherit
> > everything my typical shell has. For example there is no daffodil on the
> > path, no DAFFODIL_CLASSPATH env var, etc.
> >
> > Given the above if I just create a terminal with Terminal > new Terminal:
> > The environment is relatively simple:
> >
> > bash-5.0$ env
> > COLORTERM=truecolor
> > TERM_PROGRAM_VERSION=1.78.2
> > JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
> > NO_AT_BRIDGE=1
> > PWD=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP
> > VSCODE_GIT_ASKPASS_NODE=/usr/share/code/code
> > HOME=/home/mbeckerle
> > LANG=en_US.UTF-8
> > GIT_ASKPASS=/usr/share/code/resources/app/extensions/git/dist/askpass.sh
> > CHROME_DESKTOP=code-url-handler.desktop
> > VSCODE_GIT_ASKPASS_EXTRA_ARGS=--ms-enable-electron-run-as-node
> > TERM=xterm-256color
> > VSCODE_GIT_IPC_HANDLE=/tmp/vscode-git-56a269c0f5.sock
> > DISPLAY=:0
> > SHLVL=1
> >
> VSCODE_GIT_ASKPASS_MAIN=/usr/share/code/resources/app/extensions/git/dist/askpass-main.js
> > GDK_BACKEND=x11
> >
> PATH=/usr/lib/jvm/java-11-openjdk-amd64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
> > ORIGINAL_XDG_CURRENT_DESKTOP=undefined
> > DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
> > TERM_PROGRAM=vscode
> > _=/usr/bin/env
> >
> > If I run daffodil, then c-C in the terminal it creates, then the
> > environment has this added environment variable only.
> >
> >
> DAFFODIL_DEBUG_CLASSPATH=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/../ethernetIP/src/main/resources:/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/lib_managed/jars/com.owlcyberdefense/dfdl-ethernetip/dfdl-ethernetip-1.2.0.jar
> >
> > This works. My launch.json is attached.
> >
> >
> >
> >
> > On Fri, May 26, 2023 at 2:06 PM Shane Dell <sh...@apache.org> wrote:
> >
> > > Mike,
> > >
> > > The VSCode Extension should be using the daffodil command that is
> bundled
> > > with the
> > > extension. The issue might still be with that shell env variable you
> have
> > > for
> > > DAFFODIL_CLASSPATH, but this might only be an issue with a terminal
> that
> > > is already opened
> > > with the name "daffodil-debugger". If you close that terminal, the next
> > > you time you run it should
> > > create a new terminal with the env set properly. If you could try this
> and
> > > see if it works how you
> > > expect, that would help us single out this issue out to an already
> active
> > > terminal.
> > >
> > > Thank you,
> > >
> > > Shane Dell
> > >
> > >
> > > On 2023/05/25 23:59:53 Mike Beckerle wrote:
> > > > There are so many environment interactions....
> > > >
> > > > So I have discovered that the VSCode extension just seems to run
> whatever
> > > > daffodil command is on the path. It's not isolated at all from the
> shell
> > > > environment.
> > > >
> > > > I found this out because I have a customized daffodil shell script
> that
> > > > setups a bunch of DAFFODIL_CLASSPATH contents, and I couldn't figure
> out
> > > > why those were part of the VSCode environment.
> > > >
> > > > Shouldn't vscode run exactly the daffodil bundled with it, unless you
> > > > explicitly configure it not to?
> > > >
> > > > That's why it is finding the ethernetIP.dfdl.xsd file in the jar,
> not in
> > > > the file system. Because my daffodil command sets up lots of jars on
> the
> > > > DAFFODIL_CLASSPATH.
> > > >
> > > > It also depends on Java 11, breaks with Java 17. I also don't think
> this
> > > > should be inherited from the environment.
> > > >
> > > > I really have a hard time using, or even testing the VSCode
> extension if
> > > I
> > > > have to scrub my shell environment of everything in it in order to
> make
> > > the
> > > > vscode extension work.
> > > >
> > > > But it is this kind of problem that caused people to invent docker
> > > > containers.
> > > >
> > > > I am going to make a launcher script for VSCode that empties the
> > > > environment, then supplies only what is absolutely necessary, with no
> > > > outside environment assumptions.
> > > >
> > > > I will try to do this Friday if possible.
> > > >
> > > >
> > > >
> > > > On Wed, May 24, 2023 at 2:50 PM Shane Dell <sh...@apache.org>
> wrote:
> > > >
> > > > > Mike,
> > > > >
> > > > > I am not sure why you would be seeing this. I am able to set the
> > > > > breakpoint where you
> > > > > mentioned without any issues. Can you try removing the daffodil-dap
> > > > > folder? This should be
> > > > > somewhere like:
> > > > > /home/$USER/.local/share/daffodil-dap
> > > > > The issue I am thinking is possibly a bug with the rc1 debugger
> that
> > > may
> > > > > not be present on rc2.
> > > > > But if the folder daffodil-dap/daffodil-debugger-3.4.0-1.3.0/
> already
> > > > > exists from rc1 then rc2 will
> > > > > not remake it. So in order to remake it you have to manually remove
> > > this
> > > > > folder. Let me know if
> > > > > this helps.
> > > > >
> > > > > Thank you,
> > > > >
> > > > > Shane Dell
> > > > >
> > > > > On 2023/05/24 15:15:33 Mike Beckerle wrote:
> > > > > > I have been able to continue evaluating RC2.
> > > > > >
> > > > > > My vote is -1
> > > > > >
> > > > > > My testing steps listed below. I am trying to debug PCAP. I have
> a
> > > broken
> > > > > > icmp packet (bad magic number), and also just step through a
> correct
> > > icmp
> > > > > > packet.
> > > > > >
> > > > > > I am trying to debug icmp.badMagicNum.cap data file.
> > > > > >
> > > > > > I should get a failure of an assert because the magic number is
> not
> > > one
> > > > > of
> > > > > > the expected values.
> > > > > >
> > > > > > I'm just trying to single-step through it until it hits the
> error.
> > > > > >
> > > > > > Once we get to the MagicNumber element, it doesn't position on
> the
> > > assert
> > > > > > while evaluating that expression or even on the element carrying
> the
> > > > > assert.
> > > > > > It jumps back to the top-level xs:schema position.
> > > > > >
> > > > > > I continue to click single step. Watching for the error to be
> > > displayed
> > > > > in
> > > > > > the upper left under Variables > Parse > Diagnostics > Errors
> which
> > > is
> > > > > > where I think it ought to go.
> > > > > >
> > > > > > At some point the failure is somehow detected as the debug
> session
> > > ends.
> > > > > A
> > > > > > dialog offers to open /tmp/infoset.xml for me, which I open and
> it is
> > > > > empty
> > > > > > of course.
> > > > > >
> > > > > > Nothing displayed the error message from the assert, or even
> > > indicated
> > > > > what
> > > > > > went wrong at all. Not the state display in the upper left, not
> the
> > > > > > terminal, nor debug console nor output.
> > > > > >
> > > > > > The upper left box with "Variables" is cleared at the end. It
> should
> > > > > > certainly not be cleared at the end of a run when final state
> may be
> > > of
> > > > > > interest. At the start of a run, or manually, but not at the end
> of a
> > > > > run.
> > > > > >
> > > > > > Second attempt to use it:
> > > > > >
> > > > > > I create a second launch config this time for what should be a
> > > successful
> > > > > > parse of icmp1.cap
> > > > > >
> > > > > > As I single step, the position of the cursor on the schema is
> jumping
> > > > > > around all over the place back and forth between a containing
> > > element and
> > > > > > the child elements it contains.
> > > > > > The infoset display does not get updated until long after the
> > > display has
> > > > > > moved on and is positioned on the next element.
> > > > > >
> > > > > > This jumping around gets completely out of hand once you step
> into
> > > the
> > > > > > complexType "Ethernet" which is in the ethernetIP jar file.
> > > > > >
> > > > > > At that point, after each child of Ethernet, it jumps back to the
> > > > > Ethernet
> > > > > > element in the pcap.dfdl.xsd file, and then back into the
> > > > > > ethernetIP.dfdl.xsd for the next child, then back to the
> > > pcap.dfdl.xsd
> > > > > > file, etc.
> > > > > >
> > > > > > All this jumping around basically makes stepping useless. So I
> > > decide to
> > > > > > put a breakpoint on the Version element inside the IPv4Header
> element
> > > > > > inside ethernetIP.dfdl.xsd.
> > > > > >
> > > > > > Just adding the breakpoint causes the debug session to end with
> an
> > > NPE in
> > > > > > the dap setBreakpoint.
> > > > > >
> > > > > > ERROR o.a.d.d.d.DAPodil - unhandled error
> > > > > > java.lang.NullPointerException: null
> > > > > > at
> > > > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> > > > > > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> > > > > > at
> > > > > >
> java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> > > > > > at java.base/java.nio.file.Path.of(Path.java:147)
> > > > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > > at flatMap @
> > > > > >
> > > > >
> > >
> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > at handleErrorWith @
> > > > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > at flatMap @
> > > fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > 2023-05-24 11:09:10,717 [io-compute-1] DEBUG o.a.d.d.d.DAPodil -
> > > > > whenDone:
> > > > > > completed
> > > > > > java.lang.NullPointerException
> > > > > > at
> > > > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> > > > > > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> > > > > > at
> > > > > >
> java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> > > > > > at java.base/java.nio.file.Path.of(Path.java:147)
> > > > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > > at flatMap @
> > > > > >
> > > > >
> > >
> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > at handleErrorWith @
> > > > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > > at flatMap @
> > > fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, May 23, 2023 at 5:19 PM Mike Beckerle <
> mbeckerle@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > I really am trying to eval the RC2.
> > > > > > >
> > > > > > > Right now my VMWare Linux system has been broken by recent
> updates
> > > to
> > > > > the
> > > > > > > MS Windows host environment.
> > > > > > >
> > > > > > > I don't exactly understand this, but the VSCode front end can
> no
> > > longer
> > > > > > > connect to the server at localhost:4711. I get ECONNREFUSED.
> This
> > > > > started
> > > > > > > when a bunch of MS-Windows updates were done on the host OS. I
> run
> > > > > VSCode
> > > > > > > on the linux client machine, which lives behind a NAT firewall
> > > created
> > > > > by
> > > > > > > VMWare workstation - so I really don't get how these could be
> > > > > interacting
> > > > > > > this way, but... there it is.
> > > > > > >
> > > > > > > I will switch back to my standalone linux PC if this isn't
> resolved
> > > > > today.
> > > > > > >
> > > > > > >
> > > > > > > On Tue, May 16, 2023 at 1:24 PM Shane Dell <
> shanedell@apache.org>
> > > > > wrote:
> > > > > > >
> > > > > > >> Hello all,I'd like to call a vote to release Apache Daffodil
> VS
> > > Code
> > > > > > >> 1.3.0-rc2.
> > > > > > >>
> > > > > > >> All distribution packages, including signatures, digests, etc.
> > > can be
> > > > > > >> found at:
> > > > > > >>
> > > > >
> > >
> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
> > > > > > >>
> > > > > > >> This release has been signed with PGP key
> > > > > > >> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
> > > > > > >> to shanedell@apache.org, which is included in the KEYS file
> here:
> > > > > > >> https://downloads.apache.org/daffodil/KEYS
> > > > > > >>
> > > > > > >> The release candidate has been tagged in git with 1.3.0-rc2.
> The
> > > > > binaries
> > > > > > >>
> > > > > > >> For reference, here is a list of all closed GitHub issues
> tagged
> > > with
> > > > > > >> 1.3.0:
> > > > > > >>
> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
> > > > > > >>
> > > > > > >> Please review and vote. The vote will be open for at least 72
> > > hours
> > > > > > >> (Friday, 19 May 2023, 1:30pm EST).
> > > > > > >>
> > > > > > >> [ ] +1 approve
> > > > > > >> [ ] +0 no opinion
> > > > > > >> [ ] -1 disapprove (and reason why)
> > > > > > >>
> > > > > > >> Documentation for 1.3.0 can be found here
> > > > > > >> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
> > > > > > >> <
> > > > > > >>
> > > > >
> > >
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
> > > > > > >> >.
> > > > > > >>
> > > > > > >> Please note one of the large features that was focused on in
> > > 1.3.0 was
> > > > > > >> improving the data editor. To open the data editor you will
> need
> > > to
> > > > > open
> > > > > > >> the
> > > > > > >> command palette using:
> > > > > > >>
> > > > > > >> - Ctrl + Shift + P (windows/linux)
> > > > > > >> - Command + Shift + P (mac)
> > > > > > >>
> > > > > > >> Then typing "data.edit", make sure "Daffodil Debug: Data
> Editor"
> > > is
> > > > > > >> selected
> > > > > > >> and then hitting enter. By default the data editor will run on
> > > port
> > > > > 9000
> > > > > > >> of
> > > > > > >> your machine. However, if you want to use a different port
> follow
> > > > > steps
> > > > > > >> here
> > > > > > >> Data Editor Launch Settings
> > > > > > >> <
> > > > > > >>
> > > > >
> > >
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
> > > > > > >> >
> > > > > > >>
> > > > > > >> Thank you,
> > > > > > >>
> > > > > > >> - Shane Dell
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Shane Dell <sh...@apache.org>.
The extension does not depend on Java 11 it should work fine with Java 17. I remember back
porting to make sure we supported Java 8 also. Other than relying on the system Java, I am not
sure how on else we can really on Java. I know Davin mentioned packaging a JVM or JDK or
something like that but is that very common practice to do? The only time I ever heard of doing
that was for a language server so was wondering if its necessary or might be overkill. I agree
with the needing to fix the issue with the environment when it comes to shell and think what you
posted will help us do that. However, when it comes to Java I think if there is an issue with Java
17 we should figure that out first. Then maybe put it out for a vote or something about bundling
a Java version with the extension and then letting the user specify a different path to Java or
something like that?
On 2023/05/31 23:02:03 Mike Beckerle wrote:
> So, I have the 1.3.0 RC2 vscode extension debugger working now for PCAP w/
> ethernetIP and I can set a breakpoint in ethernetIP.dfdl.xsd and I no
> longer get an NPE.
>
> Configuring this is really far too difficult, and I have to have the schema
> working first with sbt in order to even hope to configure it.
>
> But I'll get back to that later.
>
> More basic issue is that I am unable to debug even dead-simple errors with
> it. I clobbered the magic number in the first bytes of an icmp1.cap file.
> This fails an assert at line 97 of pcap.dfdl.xsd, which is the first
> element of simple type.
>
> I step through with the debugger, and nothing displays *anything* about the
> assert failure. Execution ends. A little pop-up dialog says it wrote my
> infoset out, which it didn't. There's nothing showing that anything went
> wrong. All panels on the left are blank.
>
> If I modify the schema and make any mistake, then I get a blocking dialog
> showing the SDEs. This is basically intolerable, and is why I have to go
> back to sbt to debug this. You can't even go through the SDEs as you move
> the editor. It's a blocking dialog.
>
> So both for schema compilation, and runtime errors there simply has to be a
> non-modal display of errors, probably on the left with variables, watch,
> call-stack, and breakpoints. But a quasi-edit-buffer containing them would
> also be fine, or even just output them to the terminal window.
>
> The rest of this email is about the hoops I had to jump through to get this
> to work at all. Arguably, these issues are about working with a
> multi-component schema, and we could define our objectives to simpler
> schemas than this, but that puts it in the realm of toy-only schemas. The
> example is just PCAP + ethernetIP, which is not an extreme example.
>
> The existing shell environment of the terminal definitely causes problems.
> But just exiting that so that the debugger starts a new terminal doesn't
> fix it, as that inherits the environment from where vscode was started. It
> starts a new login shell unless you take steps to prevent that.
>
> A big trip-up in usability is that the wizard that creates the launch.json,
> doesn't actually let you populate the daffodilDebugClaspath. You have to
> edit that by hand, and nothing verifies it, so any typo and you get tedious
> failures that are hard to figure out. You just get an SDE - if you are
> lucky - via a blocking dialog.
>
> The wizard should help you put directories and corresponding jars on the
> daffodilDebugClasspath, and it should not let you put a non-existing
> directory or jar file on it. Ideally, at launch time the
> daffodilDebugClasspath should be checked, just in case something on it no
> longer exists.
>
> If the debugger happens to find a file on the daffodilDebugClasspath in a
> jar, vs. in a regular directory (maybe you have a typo in the regular
> directory path), so it passes that and hits the jar, then setting a
> breakpoint causes a NPE. Something requires a "hierarchical URL" which a
> jar-file URL is not. This is the bug that really has to be fixed so that
> you can set breakpoints regardless of whether the source DFDL schema file
> was in a jar or not.
>
> Note: at Owl we have DFDL schemas where the starting root element of the
> schema isn't even part of the schema, it's part of a component schema that
> is in a jar on the classpath. This happens when you have an
> envelope-payload idiom. The schema that combines the envelope and payload
> together contains only the glue part that connects the two together. The
> root element is the envelope's root element, and that schema is in a jar
> file.
>
> I did not see any feature in launch.json for specifying the name/namespace
> of the root element. There is just the "program" element, which must be to
> a file in the current schema. That needs to be able to be specified just as
> element name + namespace.
>
> The jars have to be on the daffodilDebugClasspath because of the plugins
> that daffodil must find in them. That's understandable, the problem is
> having to also have a regular file location for any schema you want to be
> able to put a breakpoint in and it must be before the corresponding jar.
>
> That all said, here are the scripts and settings I used to get it to work:
>
> First, I have this script I use to launch vscode in a minimal/empty
> environment. This technique is useful to
> determine if you are depending on the environment in any way:
>
> #! /bin/bash
> if [ "$1" = "emptyEnv" ] # is it the special first arg
> then
> shift # discard special first arg, and continue with the rest of the
> script
> else
> CMD=$0
> exec -c $CMD "emptyEnv" $* # exec with an empty environment, special
> first arg
> fi
> # the environment is empty when we get here
> # just add env vars we need to get vscode to work
> export LANG=en_US.UTF-8
> export HOME=/home/mbeckerle
> export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 # java 11 required
> export PATH=$JAVA_HOME/bin:$PATH
> export DISPLAY=:0
> export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus # vscode spits
> lots of errors without this
> exec /usr/bin/code
> # end
>
> Then, within ~/.config/Code/User/settings.json, I had to create these
> settings for terminal:
>
> "terminal.integrated.inheritEnv": false,
> "terminal.integrated.profiles.linux": {
> "Custom Init": {
> "path": "/bin/bash",
> "args": [
> "--norc"
> ]
> }
> },
> "terminal.integrated.defaultProfile.linux": "Custom Init"
>
> That instructs vscode not to create a login shell for the terminal. The
> --norc option means it does not run the .bashrc file so that the java
> version specified in the launch script sticks.
>
> I am not sure the inheritEnv setting actually does anything. That alone
> sounds like the right thing, but that alone didn't solve the problem.
> You still have everything in the login shell environment where vscode was
> launched. Hence the need for the environment-emptying launch script.
>
> With those two things, a terminal created by vscode does not inherit
> everything my typical shell has. For example there is no daffodil on the
> path, no DAFFODIL_CLASSPATH env var, etc.
>
> Given the above if I just create a terminal with Terminal > new Terminal:
> The environment is relatively simple:
>
> bash-5.0$ env
> COLORTERM=truecolor
> TERM_PROGRAM_VERSION=1.78.2
> JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
> NO_AT_BRIDGE=1
> PWD=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP
> VSCODE_GIT_ASKPASS_NODE=/usr/share/code/code
> HOME=/home/mbeckerle
> LANG=en_US.UTF-8
> GIT_ASKPASS=/usr/share/code/resources/app/extensions/git/dist/askpass.sh
> CHROME_DESKTOP=code-url-handler.desktop
> VSCODE_GIT_ASKPASS_EXTRA_ARGS=--ms-enable-electron-run-as-node
> TERM=xterm-256color
> VSCODE_GIT_IPC_HANDLE=/tmp/vscode-git-56a269c0f5.sock
> DISPLAY=:0
> SHLVL=1
> VSCODE_GIT_ASKPASS_MAIN=/usr/share/code/resources/app/extensions/git/dist/askpass-main.js
> GDK_BACKEND=x11
> PATH=/usr/lib/jvm/java-11-openjdk-amd64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
> ORIGINAL_XDG_CURRENT_DESKTOP=undefined
> DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
> TERM_PROGRAM=vscode
> _=/usr/bin/env
>
> If I run daffodil, then c-C in the terminal it creates, then the
> environment has this added environment variable only.
>
> DAFFODIL_DEBUG_CLASSPATH=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/../ethernetIP/src/main/resources:/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/lib_managed/jars/com.owlcyberdefense/dfdl-ethernetip/dfdl-ethernetip-1.2.0.jar
>
> This works. My launch.json is attached.
>
>
>
>
> On Fri, May 26, 2023 at 2:06 PM Shane Dell <sh...@apache.org> wrote:
>
> > Mike,
> >
> > The VSCode Extension should be using the daffodil command that is bundled
> > with the
> > extension. The issue might still be with that shell env variable you have
> > for
> > DAFFODIL_CLASSPATH, but this might only be an issue with a terminal that
> > is already opened
> > with the name "daffodil-debugger". If you close that terminal, the next
> > you time you run it should
> > create a new terminal with the env set properly. If you could try this and
> > see if it works how you
> > expect, that would help us single out this issue out to an already active
> > terminal.
> >
> > Thank you,
> >
> > Shane Dell
> >
> >
> > On 2023/05/25 23:59:53 Mike Beckerle wrote:
> > > There are so many environment interactions....
> > >
> > > So I have discovered that the VSCode extension just seems to run whatever
> > > daffodil command is on the path. It's not isolated at all from the shell
> > > environment.
> > >
> > > I found this out because I have a customized daffodil shell script that
> > > setups a bunch of DAFFODIL_CLASSPATH contents, and I couldn't figure out
> > > why those were part of the VSCode environment.
> > >
> > > Shouldn't vscode run exactly the daffodil bundled with it, unless you
> > > explicitly configure it not to?
> > >
> > > That's why it is finding the ethernetIP.dfdl.xsd file in the jar, not in
> > > the file system. Because my daffodil command sets up lots of jars on the
> > > DAFFODIL_CLASSPATH.
> > >
> > > It also depends on Java 11, breaks with Java 17. I also don't think this
> > > should be inherited from the environment.
> > >
> > > I really have a hard time using, or even testing the VSCode extension if
> > I
> > > have to scrub my shell environment of everything in it in order to make
> > the
> > > vscode extension work.
> > >
> > > But it is this kind of problem that caused people to invent docker
> > > containers.
> > >
> > > I am going to make a launcher script for VSCode that empties the
> > > environment, then supplies only what is absolutely necessary, with no
> > > outside environment assumptions.
> > >
> > > I will try to do this Friday if possible.
> > >
> > >
> > >
> > > On Wed, May 24, 2023 at 2:50 PM Shane Dell <sh...@apache.org> wrote:
> > >
> > > > Mike,
> > > >
> > > > I am not sure why you would be seeing this. I am able to set the
> > > > breakpoint where you
> > > > mentioned without any issues. Can you try removing the daffodil-dap
> > > > folder? This should be
> > > > somewhere like:
> > > > /home/$USER/.local/share/daffodil-dap
> > > > The issue I am thinking is possibly a bug with the rc1 debugger that
> > may
> > > > not be present on rc2.
> > > > But if the folder daffodil-dap/daffodil-debugger-3.4.0-1.3.0/ already
> > > > exists from rc1 then rc2 will
> > > > not remake it. So in order to remake it you have to manually remove
> > this
> > > > folder. Let me know if
> > > > this helps.
> > > >
> > > > Thank you,
> > > >
> > > > Shane Dell
> > > >
> > > > On 2023/05/24 15:15:33 Mike Beckerle wrote:
> > > > > I have been able to continue evaluating RC2.
> > > > >
> > > > > My vote is -1
> > > > >
> > > > > My testing steps listed below. I am trying to debug PCAP. I have a
> > broken
> > > > > icmp packet (bad magic number), and also just step through a correct
> > icmp
> > > > > packet.
> > > > >
> > > > > I am trying to debug icmp.badMagicNum.cap data file.
> > > > >
> > > > > I should get a failure of an assert because the magic number is not
> > one
> > > > of
> > > > > the expected values.
> > > > >
> > > > > I'm just trying to single-step through it until it hits the error.
> > > > >
> > > > > Once we get to the MagicNumber element, it doesn't position on the
> > assert
> > > > > while evaluating that expression or even on the element carrying the
> > > > assert.
> > > > > It jumps back to the top-level xs:schema position.
> > > > >
> > > > > I continue to click single step. Watching for the error to be
> > displayed
> > > > in
> > > > > the upper left under Variables > Parse > Diagnostics > Errors which
> > is
> > > > > where I think it ought to go.
> > > > >
> > > > > At some point the failure is somehow detected as the debug session
> > ends.
> > > > A
> > > > > dialog offers to open /tmp/infoset.xml for me, which I open and it is
> > > > empty
> > > > > of course.
> > > > >
> > > > > Nothing displayed the error message from the assert, or even
> > indicated
> > > > what
> > > > > went wrong at all. Not the state display in the upper left, not the
> > > > > terminal, nor debug console nor output.
> > > > >
> > > > > The upper left box with "Variables" is cleared at the end. It should
> > > > > certainly not be cleared at the end of a run when final state may be
> > of
> > > > > interest. At the start of a run, or manually, but not at the end of a
> > > > run.
> > > > >
> > > > > Second attempt to use it:
> > > > >
> > > > > I create a second launch config this time for what should be a
> > successful
> > > > > parse of icmp1.cap
> > > > >
> > > > > As I single step, the position of the cursor on the schema is jumping
> > > > > around all over the place back and forth between a containing
> > element and
> > > > > the child elements it contains.
> > > > > The infoset display does not get updated until long after the
> > display has
> > > > > moved on and is positioned on the next element.
> > > > >
> > > > > This jumping around gets completely out of hand once you step into
> > the
> > > > > complexType "Ethernet" which is in the ethernetIP jar file.
> > > > >
> > > > > At that point, after each child of Ethernet, it jumps back to the
> > > > Ethernet
> > > > > element in the pcap.dfdl.xsd file, and then back into the
> > > > > ethernetIP.dfdl.xsd for the next child, then back to the
> > pcap.dfdl.xsd
> > > > > file, etc.
> > > > >
> > > > > All this jumping around basically makes stepping useless. So I
> > decide to
> > > > > put a breakpoint on the Version element inside the IPv4Header element
> > > > > inside ethernetIP.dfdl.xsd.
> > > > >
> > > > > Just adding the breakpoint causes the debug session to end with an
> > NPE in
> > > > > the dap setBreakpoint.
> > > > >
> > > > > ERROR o.a.d.d.d.DAPodil - unhandled error
> > > > > java.lang.NullPointerException: null
> > > > > at
> > > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> > > > > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> > > > > at
> > > > > java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> > > > > at java.base/java.nio.file.Path.of(Path.java:147)
> > > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
> > > > > at
> > > > >
> > > >
> > org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > at flatMap @
> > > > >
> > > >
> > org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > at handleErrorWith @
> > > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > at flatMap @
> > fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > 2023-05-24 11:09:10,717 [io-compute-1] DEBUG o.a.d.d.d.DAPodil -
> > > > whenDone:
> > > > > completed
> > > > > java.lang.NullPointerException
> > > > > at
> > > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> > > > > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> > > > > at
> > > > > java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> > > > > at java.base/java.nio.file.Path.of(Path.java:147)
> > > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
> > > > > at
> > > > >
> > > >
> > org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > at flatMap @
> > > > >
> > > >
> > org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > at handleErrorWith @
> > > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > > at flatMap @
> > fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Tue, May 23, 2023 at 5:19 PM Mike Beckerle <mb...@apache.org>
> > > > wrote:
> > > > >
> > > > > > I really am trying to eval the RC2.
> > > > > >
> > > > > > Right now my VMWare Linux system has been broken by recent updates
> > to
> > > > the
> > > > > > MS Windows host environment.
> > > > > >
> > > > > > I don't exactly understand this, but the VSCode front end can no
> > longer
> > > > > > connect to the server at localhost:4711. I get ECONNREFUSED. This
> > > > started
> > > > > > when a bunch of MS-Windows updates were done on the host OS. I run
> > > > VSCode
> > > > > > on the linux client machine, which lives behind a NAT firewall
> > created
> > > > by
> > > > > > VMWare workstation - so I really don't get how these could be
> > > > interacting
> > > > > > this way, but... there it is.
> > > > > >
> > > > > > I will switch back to my standalone linux PC if this isn't resolved
> > > > today.
> > > > > >
> > > > > >
> > > > > > On Tue, May 16, 2023 at 1:24 PM Shane Dell <sh...@apache.org>
> > > > wrote:
> > > > > >
> > > > > >> Hello all,I'd like to call a vote to release Apache Daffodil VS
> > Code
> > > > > >> 1.3.0-rc2.
> > > > > >>
> > > > > >> All distribution packages, including signatures, digests, etc.
> > can be
> > > > > >> found at:
> > > > > >>
> > > >
> > https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
> > > > > >>
> > > > > >> This release has been signed with PGP key
> > > > > >> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
> > > > > >> to shanedell@apache.org, which is included in the KEYS file here:
> > > > > >> https://downloads.apache.org/daffodil/KEYS
> > > > > >>
> > > > > >> The release candidate has been tagged in git with 1.3.0-rc2. The
> > > > binaries
> > > > > >>
> > > > > >> For reference, here is a list of all closed GitHub issues tagged
> > with
> > > > > >> 1.3.0:
> > > > > >> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
> > > > > >>
> > > > > >> Please review and vote. The vote will be open for at least 72
> > hours
> > > > > >> (Friday, 19 May 2023, 1:30pm EST).
> > > > > >>
> > > > > >> [ ] +1 approve
> > > > > >> [ ] +0 no opinion
> > > > > >> [ ] -1 disapprove (and reason why)
> > > > > >>
> > > > > >> Documentation for 1.3.0 can be found here
> > > > > >> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
> > > > > >> <
> > > > > >>
> > > >
> > https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
> > > > > >> >.
> > > > > >>
> > > > > >> Please note one of the large features that was focused on in
> > 1.3.0 was
> > > > > >> improving the data editor. To open the data editor you will need
> > to
> > > > open
> > > > > >> the
> > > > > >> command palette using:
> > > > > >>
> > > > > >> - Ctrl + Shift + P (windows/linux)
> > > > > >> - Command + Shift + P (mac)
> > > > > >>
> > > > > >> Then typing "data.edit", make sure "Daffodil Debug: Data Editor"
> > is
> > > > > >> selected
> > > > > >> and then hitting enter. By default the data editor will run on
> > port
> > > > 9000
> > > > > >> of
> > > > > >> your machine. However, if you want to use a different port follow
> > > > steps
> > > > > >> here
> > > > > >> Data Editor Launch Settings
> > > > > >> <
> > > > > >>
> > > >
> > https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
> > > > > >> >
> > > > > >>
> > > > > >> Thank you,
> > > > > >>
> > > > > >> - Shane Dell
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: GPT-4 coding assist - was: Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Davin Shearer <sc...@gmail.com>.
Hi Steve, thanks for this question as it gets to the core of whether the
solution actually matches the problem.
One detail I want to get out of the way first: The debugger and Ωedit
Scala servers require only Java 8 for the VS Code extension, you can verify
this by reviewing the CI on the project's GitHub site (see for example
https://github.com/apache/daffodil-vscode/blob/main/.github/workflows/CI.yml#L101)
that builds tests and packages the extension. The previous release used
Java 11 only because there was a debugger server dependency that wasn't
available on Java 8, but we were able to work around that in version 1.3.0
and now we ship Java 8 bytecode. Java 8 compatibility is done to maximize
the JREs that can run the software -- I think this is a reasonable
definition of all, but there will be some differences among them with
different settings and tunables.
What are the benefits of bundling a JRE:
- We get to select the precise make and version of the JRE rather than be
at the mercy of whatever is installed on the user's system
- We no longer need to make a JRE a prerequisite, so it eases the install
and maintenance burden on the user (or administrator)
- The build and test environment will now match the user's runtime
environment
- If we control the make and version of the JRE, reproducing and debugging
problems ought to be easier
Is this _required_ to solve Mike's problems? Maybe (we don't test
in Mike's environment, which is why we need to control the JRE as much as
we can), but it's not sufficient. I think the most pressing problem is to
make sure we isolate the JRE environment so it's not being influenced by
the parent VS Code environment. If we do any bundling, I'm hoping we can
do so in a later release and that.we can get 1.3.0 released with some
lighter touches.
As for https://issues.apache.org/jira/browse/LEGAL-488, I'm also not clear
on the resolution, We would be distributing the binaries as provided by
adoptium.net without modification, so I think providing a copy of
the license and a link to the source ought to suffice, but I'm _not_ a
legal expert.
I hope this helps.
On Mon, Jun 5, 2023 at 2:08 PM Steve Lawrence <sl...@apache.org> wrote:
> Temurin falls under the same license:
>
> https://adoptium.net/docs/faq/#_is_temurin_free_to_use
>
> What are the JVM environment issues that bundling solves? Mike said that
> it requires Java 11, but it's not clear what Java 11 only features the
> debugger uses. I image there is an alternative to these features that
> works on all versions of Java (for a reasonable definition of all).
>
>
>
> On 2023-06-05 01:47 PM, Davin Shearer wrote:
> > We wouldn't bundle OpenJDK, we would bundle Temurin (see
> > https://adoptium.net/). The debugger and the Ωedit Scala servers are
> both
> > compiled using Java 8. At issue is control over the JVM environment and
> > what our options are to control some of the issues that Mike faced in the
> > RC2 evaluation.
> >
> > On Mon, Jun 5, 2023 at 12:39 PM Steve Lawrence <sl...@apache.org>
> wrote:
> >
> >> Can you detail what in the VS Code extension is specific to a particular
> >> Java version, and why that can't be fixed? It feels like that would be
> >> much less work than trying to bundle and distribute a JVM.
> >>
> >> There may even be issues with ASF and distributing OpenJDK. It looks
> >> like the license is GPLv2 which ASF considers category X. I found this
> >> ticket where netbeans wanted to distributed it here:
> >>
> >> https://issues.apache.org/jira/browse/LEGAL-488
> >>
> >> It's not clear what the resolution of that bug is to me
> >>
> >> - Steve
> >>
> >>
> >> On 2023-06-05 12:21 PM, Davin Shearer wrote:
> >>> Hi Mike,
> >>>
> >>> The answer is useful. It allows the VS Code user to select the version
> >> of
> >>> Java they wish to use and it will attempt to locate a compatible JVM
> >>> through various means. Sounds really useful for Java _development_ but
> >> not
> >>> so much for other extensions to leverage for their JVM dependencies
> >> unless
> >>> they are to be used _in support of Java development_. It's also
> >> important
> >>> to note that this support is only for Java 17+ (
> >>> https://marketplace.visualstudio.com/items?itemName=redhat.java). I
> >> don't
> >>> think this is a good option for our use case.
> >>>
> >>> GraalVM looks promising though I haven't used it before, so it's hard
> for
> >>> me to predict how successful that integration would be and how long it
> >>> would take to get up and running. The idea of having native machine
> code
> >>> applications has a lot of appeal. There is even support for C/C++ code
> >>> with the LLVM compiler. The Ωedit Scala server is linking to a C
> library
> >>> using it's foreign function interface (FFI), so I'm sure how that plays
> >> out
> >>> with GraalVM. GraalVM is GPL with a ClassPath exception. We won't be
> >>> distributing GraalVM, just the executables that come out (like
> gcc/g++),
> >> so
> >>> I don't foresee licensing problems. Does anyone have any experience
> with
> >>> GraalVM?
> >>>
> >>> I think the most straight-forward path is to bundle Temurin (
> >>> https://adoptium.net/), probably using version 17 LTS. We'll be able
> to
> >>> sync the version we build on with the version we test on, distribute
> and
> >>> run on. The downside is increased package size and complexity, but
> >>> debugging issues ought to be easier since more variables are controlled
> >>> this way.
> >>>
> >>> Either of these options will take a decent chunk of research,
> >> development,
> >>> and testing. The JVM bundling option is less opaque to me, though
> >>> the GraalVM option could _potentially_ provide a superior user
> >> experience.
> >>>
> >>>
> >>> On Mon, Jun 5, 2023 at 9:53 AM Mike Beckerle <mb...@apache.org>
> >> wrote:
> >>>
> >>>> So it seems it went round full circle by telling you yes, you could
> >> bundle
> >>>> the JVM, and this redhat java language thing does so, but then they
> tell
> >>>> you it doesn't bundle the JVM anyway, it just uses the system one.
> >>>>
> >>>> So was this answer useful, or just misleading?
> >>>>
> >>>
> >>
> >>
> >
>
>
Re: GPT-4 coding assist - was: Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Steve Lawrence <sl...@apache.org>.
Temurin falls under the same license:
https://adoptium.net/docs/faq/#_is_temurin_free_to_use
What are the JVM environment issues that bundling solves? Mike said that
it requires Java 11, but it's not clear what Java 11 only features the
debugger uses. I image there is an alternative to these features that
works on all versions of Java (for a reasonable definition of all).
On 2023-06-05 01:47 PM, Davin Shearer wrote:
> We wouldn't bundle OpenJDK, we would bundle Temurin (see
> https://adoptium.net/). The debugger and the Ωedit Scala servers are both
> compiled using Java 8. At issue is control over the JVM environment and
> what our options are to control some of the issues that Mike faced in the
> RC2 evaluation.
>
> On Mon, Jun 5, 2023 at 12:39 PM Steve Lawrence <sl...@apache.org> wrote:
>
>> Can you detail what in the VS Code extension is specific to a particular
>> Java version, and why that can't be fixed? It feels like that would be
>> much less work than trying to bundle and distribute a JVM.
>>
>> There may even be issues with ASF and distributing OpenJDK. It looks
>> like the license is GPLv2 which ASF considers category X. I found this
>> ticket where netbeans wanted to distributed it here:
>>
>> https://issues.apache.org/jira/browse/LEGAL-488
>>
>> It's not clear what the resolution of that bug is to me
>>
>> - Steve
>>
>>
>> On 2023-06-05 12:21 PM, Davin Shearer wrote:
>>> Hi Mike,
>>>
>>> The answer is useful. It allows the VS Code user to select the version
>> of
>>> Java they wish to use and it will attempt to locate a compatible JVM
>>> through various means. Sounds really useful for Java _development_ but
>> not
>>> so much for other extensions to leverage for their JVM dependencies
>> unless
>>> they are to be used _in support of Java development_. It's also
>> important
>>> to note that this support is only for Java 17+ (
>>> https://marketplace.visualstudio.com/items?itemName=redhat.java). I
>> don't
>>> think this is a good option for our use case.
>>>
>>> GraalVM looks promising though I haven't used it before, so it's hard for
>>> me to predict how successful that integration would be and how long it
>>> would take to get up and running. The idea of having native machine code
>>> applications has a lot of appeal. There is even support for C/C++ code
>>> with the LLVM compiler. The Ωedit Scala server is linking to a C library
>>> using it's foreign function interface (FFI), so I'm sure how that plays
>> out
>>> with GraalVM. GraalVM is GPL with a ClassPath exception. We won't be
>>> distributing GraalVM, just the executables that come out (like gcc/g++),
>> so
>>> I don't foresee licensing problems. Does anyone have any experience with
>>> GraalVM?
>>>
>>> I think the most straight-forward path is to bundle Temurin (
>>> https://adoptium.net/), probably using version 17 LTS. We'll be able to
>>> sync the version we build on with the version we test on, distribute and
>>> run on. The downside is increased package size and complexity, but
>>> debugging issues ought to be easier since more variables are controlled
>>> this way.
>>>
>>> Either of these options will take a decent chunk of research,
>> development,
>>> and testing. The JVM bundling option is less opaque to me, though
>>> the GraalVM option could _potentially_ provide a superior user
>> experience.
>>>
>>>
>>> On Mon, Jun 5, 2023 at 9:53 AM Mike Beckerle <mb...@apache.org>
>> wrote:
>>>
>>>> So it seems it went round full circle by telling you yes, you could
>> bundle
>>>> the JVM, and this redhat java language thing does so, but then they tell
>>>> you it doesn't bundle the JVM anyway, it just uses the system one.
>>>>
>>>> So was this answer useful, or just misleading?
>>>>
>>>
>>
>>
>
Re: GPT-4 coding assist - was: Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Davin Shearer <sc...@gmail.com>.
We wouldn't bundle OpenJDK, we would bundle Temurin (see
https://adoptium.net/). The debugger and the Ωedit Scala servers are both
compiled using Java 8. At issue is control over the JVM environment and
what our options are to control some of the issues that Mike faced in the
RC2 evaluation.
On Mon, Jun 5, 2023 at 12:39 PM Steve Lawrence <sl...@apache.org> wrote:
> Can you detail what in the VS Code extension is specific to a particular
> Java version, and why that can't be fixed? It feels like that would be
> much less work than trying to bundle and distribute a JVM.
>
> There may even be issues with ASF and distributing OpenJDK. It looks
> like the license is GPLv2 which ASF considers category X. I found this
> ticket where netbeans wanted to distributed it here:
>
> https://issues.apache.org/jira/browse/LEGAL-488
>
> It's not clear what the resolution of that bug is to me
>
> - Steve
>
>
> On 2023-06-05 12:21 PM, Davin Shearer wrote:
> > Hi Mike,
> >
> > The answer is useful. It allows the VS Code user to select the version
> of
> > Java they wish to use and it will attempt to locate a compatible JVM
> > through various means. Sounds really useful for Java _development_ but
> not
> > so much for other extensions to leverage for their JVM dependencies
> unless
> > they are to be used _in support of Java development_. It's also
> important
> > to note that this support is only for Java 17+ (
> > https://marketplace.visualstudio.com/items?itemName=redhat.java). I
> don't
> > think this is a good option for our use case.
> >
> > GraalVM looks promising though I haven't used it before, so it's hard for
> > me to predict how successful that integration would be and how long it
> > would take to get up and running. The idea of having native machine code
> > applications has a lot of appeal. There is even support for C/C++ code
> > with the LLVM compiler. The Ωedit Scala server is linking to a C library
> > using it's foreign function interface (FFI), so I'm sure how that plays
> out
> > with GraalVM. GraalVM is GPL with a ClassPath exception. We won't be
> > distributing GraalVM, just the executables that come out (like gcc/g++),
> so
> > I don't foresee licensing problems. Does anyone have any experience with
> > GraalVM?
> >
> > I think the most straight-forward path is to bundle Temurin (
> > https://adoptium.net/), probably using version 17 LTS. We'll be able to
> > sync the version we build on with the version we test on, distribute and
> > run on. The downside is increased package size and complexity, but
> > debugging issues ought to be easier since more variables are controlled
> > this way.
> >
> > Either of these options will take a decent chunk of research,
> development,
> > and testing. The JVM bundling option is less opaque to me, though
> > the GraalVM option could _potentially_ provide a superior user
> experience.
> >
> >
> > On Mon, Jun 5, 2023 at 9:53 AM Mike Beckerle <mb...@apache.org>
> wrote:
> >
> >> So it seems it went round full circle by telling you yes, you could
> bundle
> >> the JVM, and this redhat java language thing does so, but then they tell
> >> you it doesn't bundle the JVM anyway, it just uses the system one.
> >>
> >> So was this answer useful, or just misleading?
> >>
> >
>
>
Re: GPT-4 coding assist - was: Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Steve Lawrence <sl...@apache.org>.
Can you detail what in the VS Code extension is specific to a particular
Java version, and why that can't be fixed? It feels like that would be
much less work than trying to bundle and distribute a JVM.
There may even be issues with ASF and distributing OpenJDK. It looks
like the license is GPLv2 which ASF considers category X. I found this
ticket where netbeans wanted to distributed it here:
https://issues.apache.org/jira/browse/LEGAL-488
It's not clear what the resolution of that bug is to me
- Steve
On 2023-06-05 12:21 PM, Davin Shearer wrote:
> Hi Mike,
>
> The answer is useful. It allows the VS Code user to select the version of
> Java they wish to use and it will attempt to locate a compatible JVM
> through various means. Sounds really useful for Java _development_ but not
> so much for other extensions to leverage for their JVM dependencies unless
> they are to be used _in support of Java development_. It's also important
> to note that this support is only for Java 17+ (
> https://marketplace.visualstudio.com/items?itemName=redhat.java). I don't
> think this is a good option for our use case.
>
> GraalVM looks promising though I haven't used it before, so it's hard for
> me to predict how successful that integration would be and how long it
> would take to get up and running. The idea of having native machine code
> applications has a lot of appeal. There is even support for C/C++ code
> with the LLVM compiler. The Ωedit Scala server is linking to a C library
> using it's foreign function interface (FFI), so I'm sure how that plays out
> with GraalVM. GraalVM is GPL with a ClassPath exception. We won't be
> distributing GraalVM, just the executables that come out (like gcc/g++), so
> I don't foresee licensing problems. Does anyone have any experience with
> GraalVM?
>
> I think the most straight-forward path is to bundle Temurin (
> https://adoptium.net/), probably using version 17 LTS. We'll be able to
> sync the version we build on with the version we test on, distribute and
> run on. The downside is increased package size and complexity, but
> debugging issues ought to be easier since more variables are controlled
> this way.
>
> Either of these options will take a decent chunk of research, development,
> and testing. The JVM bundling option is less opaque to me, though
> the GraalVM option could _potentially_ provide a superior user experience.
>
>
> On Mon, Jun 5, 2023 at 9:53 AM Mike Beckerle <mb...@apache.org> wrote:
>
>> So it seems it went round full circle by telling you yes, you could bundle
>> the JVM, and this redhat java language thing does so, but then they tell
>> you it doesn't bundle the JVM anyway, it just uses the system one.
>>
>> So was this answer useful, or just misleading?
>>
>
Re: GPT-4 coding assist - was: Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Davin Shearer <sc...@gmail.com>.
Hi Mike,
The answer is useful. It allows the VS Code user to select the version of
Java they wish to use and it will attempt to locate a compatible JVM
through various means. Sounds really useful for Java _development_ but not
so much for other extensions to leverage for their JVM dependencies unless
they are to be used _in support of Java development_. It's also important
to note that this support is only for Java 17+ (
https://marketplace.visualstudio.com/items?itemName=redhat.java). I don't
think this is a good option for our use case.
GraalVM looks promising though I haven't used it before, so it's hard for
me to predict how successful that integration would be and how long it
would take to get up and running. The idea of having native machine code
applications has a lot of appeal. There is even support for C/C++ code
with the LLVM compiler. The Ωedit Scala server is linking to a C library
using it's foreign function interface (FFI), so I'm sure how that plays out
with GraalVM. GraalVM is GPL with a ClassPath exception. We won't be
distributing GraalVM, just the executables that come out (like gcc/g++), so
I don't foresee licensing problems. Does anyone have any experience with
GraalVM?
I think the most straight-forward path is to bundle Temurin (
https://adoptium.net/), probably using version 17 LTS. We'll be able to
sync the version we build on with the version we test on, distribute and
run on. The downside is increased package size and complexity, but
debugging issues ought to be easier since more variables are controlled
this way.
Either of these options will take a decent chunk of research, development,
and testing. The JVM bundling option is less opaque to me, though
the GraalVM option could _potentially_ provide a superior user experience.
On Mon, Jun 5, 2023 at 9:53 AM Mike Beckerle <mb...@apache.org> wrote:
> So it seems it went round full circle by telling you yes, you could bundle
> the JVM, and this redhat java language thing does so, but then they tell
> you it doesn't bundle the JVM anyway, it just uses the system one.
>
> So was this answer useful, or just misleading?
>
GPT-4 coding assist - was: Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Mike Beckerle <mb...@apache.org>.
So it seems it went round full circle by telling you yes, you could bundle
the JVM, and this redhat java language thing does so, but then they tell
you it doesn't bundle the JVM anyway, it just uses the system one.
So was this answer useful, or just misleading?
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Mike Beckerle <mb...@apache.org>.
I'll create the tickets for the individual issues in my review.
On Fri, Jun 2, 2023 at 3:24 PM Davin Shearer <sc...@gmail.com> wrote:
> Thanks for the detailed update and your patience Mike. This is
> definitely not the user experience we're hoping for. Are these regressions
> that have occurred since the v1.2.0 release?
>
> I'm a big fan of compartmentalization and technologies like docker and
> python's venv (virtual environment) among many others. Relying on system
> installs of virtual machines (e.g. Java's JVM) and interpreters (e.g.,
> Python, Perl, etc) introduces uncontrolled variables into our runtime
> environment, which I believe is one of the major encumbrances that you're
> experiencing and something I hope we can address as a community.
>
> I don't think docker is the right answer here because that introduces
> another system-level dependency... the docker service. I think packaging
> a JVM or pulling it in as a VS Code dependency (not a system level
> dependency like docker) will give us much tighter control over our runtime
> environment. We'll need to be careful too about how we execute our
> services on that JVM so its execution environment is properly
> compartmentalized. In addition to the Daffodil debugger service, the Data
> Editor part of the extension also uses a service running on the JVM which
> will benefit from this compartmentalization. This also will allow us to
> upgrade from Java 8 to something more recent (this will help with domain
> sockets and named fifos too). Anyway, that's the direction I'd be inclined
> to head in, but it's definitely worth a discussion and hopefully we can
> converge on solid design. I had an interesting conversation on this topic
> with GPT-4 that I think is worth sharing (see:
> https://chat.openai.com/share/da2ca068-714a-4eec-8a0e-8117069b3125).
>
> Each distinct failure that you've encountered in testing RC2 needs to have
> issues associated with them on the project's GitHub so we can track, triage
> and prioritize, schedule, and assign them. If you're willing to enter your
> issues into the GitHub tracker, that would be greatly appreciated and
> you'll be able to stay in the loop on how those issues progress and help us
> in the review phase to make sure that each issue is being properly and
> fully addressed.
>
> Shane is out, but I'm sure he'll have some additional follow up and things
> to try before making new pull requests.
>
>
> On Wed, May 31, 2023 at 7:02 PM Mike Beckerle <mb...@apache.org>
> wrote:
>
> >
> > So, I have the 1.3.0 RC2 vscode extension debugger working now for PCAP
> w/
> > ethernetIP and I can set a breakpoint in ethernetIP.dfdl.xsd and I no
> > longer get an NPE.
> >
> > Configuring this is really far too difficult, and I have to have the
> > schema working first with sbt in order to even hope to configure it.
> >
> > But I'll get back to that later.
> >
> > More basic issue is that I am unable to debug even dead-simple errors
> with
> > it. I clobbered the magic number in the first bytes of an icmp1.cap file.
> > This fails an assert at line 97 of pcap.dfdl.xsd, which is the first
> > element of simple type.
> >
> > I step through with the debugger, and nothing displays *anything* about
> > the assert failure. Execution ends. A little pop-up dialog says it wrote
> my
> > infoset out, which it didn't. There's nothing showing that anything went
> > wrong. All panels on the left are blank.
> >
> > If I modify the schema and make any mistake, then I get a blocking dialog
> > showing the SDEs. This is basically intolerable, and is why I have to go
> > back to sbt to debug this. You can't even go through the SDEs as you move
> > the editor. It's a blocking dialog.
> >
> > So both for schema compilation, and runtime errors there simply has to be
> > a non-modal display of errors, probably on the left with variables,
> watch,
> > call-stack, and breakpoints. But a quasi-edit-buffer containing them
> would
> > also be fine, or even just output them to the terminal window.
> >
> > The rest of this email is about the hoops I had to jump through to get
> > this to work at all. Arguably, these issues are about working with a
> > multi-component schema, and we could define our objectives to simpler
> > schemas than this, but that puts it in the realm of toy-only schemas. The
> > example is just PCAP + ethernetIP, which is not an extreme example.
> >
> > The existing shell environment of the terminal definitely causes
> problems.
> > But just exiting that so that the debugger starts a new terminal doesn't
> > fix it, as that inherits the environment from where vscode was started.
> It
> > starts a new login shell unless you take steps to prevent that.
> >
> > A big trip-up in usability is that the wizard that creates the
> > launch.json, doesn't actually let you populate the daffodilDebugClaspath.
> > You have to edit that by hand, and nothing verifies it, so any typo and
> you
> > get tedious failures that are hard to figure out. You just get an SDE -
> if
> > you are lucky - via a blocking dialog.
> >
> > The wizard should help you put directories and corresponding jars on the
> > daffodilDebugClasspath, and it should not let you put a non-existing
> > directory or jar file on it. Ideally, at launch time the
> > daffodilDebugClasspath should be checked, just in case something on it no
> > longer exists.
> >
> > If the debugger happens to find a file on the daffodilDebugClasspath in a
> > jar, vs. in a regular directory (maybe you have a typo in the regular
> > directory path), so it passes that and hits the jar, then setting a
> > breakpoint causes a NPE. Something requires a "hierarchical URL" which a
> > jar-file URL is not. This is the bug that really has to be fixed so that
> > you can set breakpoints regardless of whether the source DFDL schema file
> > was in a jar or not.
> >
> > Note: at Owl we have DFDL schemas where the starting root element of the
> > schema isn't even part of the schema, it's part of a component schema
> that
> > is in a jar on the classpath. This happens when you have an
> > envelope-payload idiom. The schema that combines the envelope and payload
> > together contains only the glue part that connects the two together. The
> > root element is the envelope's root element, and that schema is in a jar
> > file.
> >
> > I did not see any feature in launch.json for specifying the
> name/namespace
> > of the root element. There is just the "program" element, which must be
> to
> > a file in the current schema. That needs to be able to be specified just
> as
> > element name + namespace.
> >
> > The jars have to be on the daffodilDebugClasspath because of the plugins
> > that daffodil must find in them. That's understandable, the problem is
> > having to also have a regular file location for any schema you want to be
> > able to put a breakpoint in and it must be before the corresponding jar.
> >
> > That all said, here are the scripts and settings I used to get it to
> work:
> >
> > First, I have this script I use to launch vscode in a minimal/empty
> > environment. This technique is useful to
> > determine if you are depending on the environment in any way:
> >
> > #! /bin/bash
> > if [ "$1" = "emptyEnv" ] # is it the special first arg
> > then
> > shift # discard special first arg, and continue with the rest of the
> > script
> > else
> > CMD=$0
> > exec -c $CMD "emptyEnv" $* # exec with an empty environment, special
> > first arg
> > fi
> > # the environment is empty when we get here
> > # just add env vars we need to get vscode to work
> > export LANG=en_US.UTF-8
> > export HOME=/home/mbeckerle
> > export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 # java 11 required
> > export PATH=$JAVA_HOME/bin:$PATH
> > export DISPLAY=:0
> > export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus # vscode
> > spits lots of errors without this
> > exec /usr/bin/code
> > # end
> >
> > Then, within ~/.config/Code/User/settings.json, I had to create these
> > settings for terminal:
> >
> > "terminal.integrated.inheritEnv": false,
> > "terminal.integrated.profiles.linux": {
> > "Custom Init": {
> > "path": "/bin/bash",
> > "args": [
> > "--norc"
> > ]
> > }
> > },
> > "terminal.integrated.defaultProfile.linux": "Custom Init"
> >
> > That instructs vscode not to create a login shell for the terminal. The
> > --norc option means it does not run the .bashrc file so that the java
> > version specified in the launch script sticks.
> >
> > I am not sure the inheritEnv setting actually does anything. That alone
> > sounds like the right thing, but that alone didn't solve the problem.
> > You still have everything in the login shell environment where vscode was
> > launched. Hence the need for the environment-emptying launch script.
> >
> > With those two things, a terminal created by vscode does not inherit
> > everything my typical shell has. For example there is no daffodil on the
> > path, no DAFFODIL_CLASSPATH env var, etc.
> >
> > Given the above if I just create a terminal with Terminal > new Terminal:
> > The environment is relatively simple:
> >
> > bash-5.0$ env
> > COLORTERM=truecolor
> > TERM_PROGRAM_VERSION=1.78.2
> > JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
> > NO_AT_BRIDGE=1
> > PWD=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP
> > VSCODE_GIT_ASKPASS_NODE=/usr/share/code/code
> > HOME=/home/mbeckerle
> > LANG=en_US.UTF-8
> > GIT_ASKPASS=/usr/share/code/resources/app/extensions/git/dist/askpass.sh
> > CHROME_DESKTOP=code-url-handler.desktop
> > VSCODE_GIT_ASKPASS_EXTRA_ARGS=--ms-enable-electron-run-as-node
> > TERM=xterm-256color
> > VSCODE_GIT_IPC_HANDLE=/tmp/vscode-git-56a269c0f5.sock
> > DISPLAY=:0
> > SHLVL=1
> >
> >
> VSCODE_GIT_ASKPASS_MAIN=/usr/share/code/resources/app/extensions/git/dist/askpass-main.js
> > GDK_BACKEND=x11
> >
> >
> PATH=/usr/lib/jvm/java-11-openjdk-amd64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
> > ORIGINAL_XDG_CURRENT_DESKTOP=undefined
> > DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
> > TERM_PROGRAM=vscode
> > _=/usr/bin/env
> >
> > If I run daffodil, then c-C in the terminal it creates, then the
> > environment has this added environment variable only.
> >
> >
> >
> DAFFODIL_DEBUG_CLASSPATH=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/../ethernetIP/src/main/resources:/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/lib_managed/jars/com.owlcyberdefense/dfdl-ethernetip/dfdl-ethernetip-1.2.0.jar
> >
> > This works. My launch.json is attached.
> >
> >
> >
> >
> > On Fri, May 26, 2023 at 2:06 PM Shane Dell <sh...@apache.org> wrote:
> >
> >> Mike,
> >>
> >> The VSCode Extension should be using the daffodil command that is
> bundled
> >> with the
> >> extension. The issue might still be with that shell env variable you
> have
> >> for
> >> DAFFODIL_CLASSPATH, but this might only be an issue with a terminal that
> >> is already opened
> >> with the name "daffodil-debugger". If you close that terminal, the next
> >> you time you run it should
> >> create a new terminal with the env set properly. If you could try this
> >> and see if it works how you
> >> expect, that would help us single out this issue out to an already
> active
> >> terminal.
> >>
> >> Thank you,
> >>
> >> Shane Dell
> >>
> >>
> >> On 2023/05/25 23:59:53 Mike Beckerle wrote:
> >> > There are so many environment interactions....
> >> >
> >> > So I have discovered that the VSCode extension just seems to run
> >> whatever
> >> > daffodil command is on the path. It's not isolated at all from the
> shell
> >> > environment.
> >> >
> >> > I found this out because I have a customized daffodil shell script
> that
> >> > setups a bunch of DAFFODIL_CLASSPATH contents, and I couldn't figure
> out
> >> > why those were part of the VSCode environment.
> >> >
> >> > Shouldn't vscode run exactly the daffodil bundled with it, unless you
> >> > explicitly configure it not to?
> >> >
> >> > That's why it is finding the ethernetIP.dfdl.xsd file in the jar, not
> in
> >> > the file system. Because my daffodil command sets up lots of jars on
> the
> >> > DAFFODIL_CLASSPATH.
> >> >
> >> > It also depends on Java 11, breaks with Java 17. I also don't think
> this
> >> > should be inherited from the environment.
> >> >
> >> > I really have a hard time using, or even testing the VSCode extension
> >> if I
> >> > have to scrub my shell environment of everything in it in order to
> make
> >> the
> >> > vscode extension work.
> >> >
> >> > But it is this kind of problem that caused people to invent docker
> >> > containers.
> >> >
> >> > I am going to make a launcher script for VSCode that empties the
> >> > environment, then supplies only what is absolutely necessary, with no
> >> > outside environment assumptions.
> >> >
> >> > I will try to do this Friday if possible.
> >> >
> >> >
> >> >
> >> > On Wed, May 24, 2023 at 2:50 PM Shane Dell <sh...@apache.org>
> >> wrote:
> >> >
> >> > > Mike,
> >> > >
> >> > > I am not sure why you would be seeing this. I am able to set the
> >> > > breakpoint where you
> >> > > mentioned without any issues. Can you try removing the daffodil-dap
> >> > > folder? This should be
> >> > > somewhere like:
> >> > > /home/$USER/.local/share/daffodil-dap
> >> > > The issue I am thinking is possibly a bug with the rc1 debugger that
> >> may
> >> > > not be present on rc2.
> >> > > But if the folder daffodil-dap/daffodil-debugger-3.4.0-1.3.0/
> already
> >> > > exists from rc1 then rc2 will
> >> > > not remake it. So in order to remake it you have to manually remove
> >> this
> >> > > folder. Let me know if
> >> > > this helps.
> >> > >
> >> > > Thank you,
> >> > >
> >> > > Shane Dell
> >> > >
> >> > > On 2023/05/24 15:15:33 Mike Beckerle wrote:
> >> > > > I have been able to continue evaluating RC2.
> >> > > >
> >> > > > My vote is -1
> >> > > >
> >> > > > My testing steps listed below. I am trying to debug PCAP. I have a
> >> broken
> >> > > > icmp packet (bad magic number), and also just step through a
> >> correct icmp
> >> > > > packet.
> >> > > >
> >> > > > I am trying to debug icmp.badMagicNum.cap data file.
> >> > > >
> >> > > > I should get a failure of an assert because the magic number is
> not
> >> one
> >> > > of
> >> > > > the expected values.
> >> > > >
> >> > > > I'm just trying to single-step through it until it hits the error.
> >> > > >
> >> > > > Once we get to the MagicNumber element, it doesn't position on the
> >> assert
> >> > > > while evaluating that expression or even on the element carrying
> the
> >> > > assert.
> >> > > > It jumps back to the top-level xs:schema position.
> >> > > >
> >> > > > I continue to click single step. Watching for the error to be
> >> displayed
> >> > > in
> >> > > > the upper left under Variables > Parse > Diagnostics > Errors
> which
> >> is
> >> > > > where I think it ought to go.
> >> > > >
> >> > > > At some point the failure is somehow detected as the debug session
> >> ends.
> >> > > A
> >> > > > dialog offers to open /tmp/infoset.xml for me, which I open and it
> >> is
> >> > > empty
> >> > > > of course.
> >> > > >
> >> > > > Nothing displayed the error message from the assert, or even
> >> indicated
> >> > > what
> >> > > > went wrong at all. Not the state display in the upper left, not
> the
> >> > > > terminal, nor debug console nor output.
> >> > > >
> >> > > > The upper left box with "Variables" is cleared at the end. It
> should
> >> > > > certainly not be cleared at the end of a run when final state may
> >> be of
> >> > > > interest. At the start of a run, or manually, but not at the end
> of
> >> a
> >> > > run.
> >> > > >
> >> > > > Second attempt to use it:
> >> > > >
> >> > > > I create a second launch config this time for what should be a
> >> successful
> >> > > > parse of icmp1.cap
> >> > > >
> >> > > > As I single step, the position of the cursor on the schema is
> >> jumping
> >> > > > around all over the place back and forth between a containing
> >> element and
> >> > > > the child elements it contains.
> >> > > > The infoset display does not get updated until long after the
> >> display has
> >> > > > moved on and is positioned on the next element.
> >> > > >
> >> > > > This jumping around gets completely out of hand once you step into
> >> the
> >> > > > complexType "Ethernet" which is in the ethernetIP jar file.
> >> > > >
> >> > > > At that point, after each child of Ethernet, it jumps back to the
> >> > > Ethernet
> >> > > > element in the pcap.dfdl.xsd file, and then back into the
> >> > > > ethernetIP.dfdl.xsd for the next child, then back to the
> >> pcap.dfdl.xsd
> >> > > > file, etc.
> >> > > >
> >> > > > All this jumping around basically makes stepping useless. So I
> >> decide to
> >> > > > put a breakpoint on the Version element inside the IPv4Header
> >> element
> >> > > > inside ethernetIP.dfdl.xsd.
> >> > > >
> >> > > > Just adding the breakpoint causes the debug session to end with an
> >> NPE in
> >> > > > the dap setBreakpoint.
> >> > > >
> >> > > > ERROR o.a.d.d.d.DAPodil - unhandled error
> >> > > > java.lang.NullPointerException: null
> >> > > > at
> >> > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> >> > > > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> >> > > > at
> >> > > >
> java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> >> > > > at java.base/java.nio.file.Path.of(Path.java:147)
> >> > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
> >> > > > at
> >> > > >
> >> > >
> >>
> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> >> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > at flatMap @
> >> > > >
> >> > >
> >>
> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > at handleErrorWith @
> >> > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> >> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > at flatMap @
> >> fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> >> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > 2023-05-24 11:09:10,717 [io-compute-1] DEBUG o.a.d.d.d.DAPodil -
> >> > > whenDone:
> >> > > > completed
> >> > > > java.lang.NullPointerException
> >> > > > at
> >> > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> >> > > > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> >> > > > at
> >> > > >
> java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> >> > > > at java.base/java.nio.file.Path.of(Path.java:147)
> >> > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
> >> > > > at
> >> > > >
> >> > >
> >>
> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> >> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > at flatMap @
> >> > > >
> >> > >
> >>
> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > at handleErrorWith @
> >> > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> >> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > > at flatMap @
> >> fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> >> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> >> > > > at flatMap @
> fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Tue, May 23, 2023 at 5:19 PM Mike Beckerle <
> mbeckerle@apache.org
> >> >
> >> > > wrote:
> >> > > >
> >> > > > > I really am trying to eval the RC2.
> >> > > > >
> >> > > > > Right now my VMWare Linux system has been broken by recent
> >> updates to
> >> > > the
> >> > > > > MS Windows host environment.
> >> > > > >
> >> > > > > I don't exactly understand this, but the VSCode front end can no
> >> longer
> >> > > > > connect to the server at localhost:4711. I get ECONNREFUSED.
> This
> >> > > started
> >> > > > > when a bunch of MS-Windows updates were done on the host OS. I
> run
> >> > > VSCode
> >> > > > > on the linux client machine, which lives behind a NAT firewall
> >> created
> >> > > by
> >> > > > > VMWare workstation - so I really don't get how these could be
> >> > > interacting
> >> > > > > this way, but... there it is.
> >> > > > >
> >> > > > > I will switch back to my standalone linux PC if this isn't
> >> resolved
> >> > > today.
> >> > > > >
> >> > > > >
> >> > > > > On Tue, May 16, 2023 at 1:24 PM Shane Dell <
> shanedell@apache.org>
> >> > > wrote:
> >> > > > >
> >> > > > >> Hello all,I'd like to call a vote to release Apache Daffodil VS
> >> Code
> >> > > > >> 1.3.0-rc2.
> >> > > > >>
> >> > > > >> All distribution packages, including signatures, digests, etc.
> >> can be
> >> > > > >> found at:
> >> > > > >>
> >> > >
> >>
> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
> >> > > > >>
> >> > > > >> This release has been signed with PGP key
> >> > > > >> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
> >> > > > >> to shanedell@apache.org, which is included in the KEYS file
> >> here:
> >> > > > >> https://downloads.apache.org/daffodil/KEYS
> >> > > > >>
> >> > > > >> The release candidate has been tagged in git with 1.3.0-rc2.
> The
> >> > > binaries
> >> > > > >>
> >> > > > >> For reference, here is a list of all closed GitHub issues
> tagged
> >> with
> >> > > > >> 1.3.0:
> >> > > > >> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
> >> > > > >>
> >> > > > >> Please review and vote. The vote will be open for at least 72
> >> hours
> >> > > > >> (Friday, 19 May 2023, 1:30pm EST).
> >> > > > >>
> >> > > > >> [ ] +1 approve
> >> > > > >> [ ] +0 no opinion
> >> > > > >> [ ] -1 disapprove (and reason why)
> >> > > > >>
> >> > > > >> Documentation for 1.3.0 can be found here
> >> > > > >> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
> >> > > > >> <
> >> > > > >>
> >> > >
> >>
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
> >> > > > >> >.
> >> > > > >>
> >> > > > >> Please note one of the large features that was focused on in
> >> 1.3.0 was
> >> > > > >> improving the data editor. To open the data editor you will
> need
> >> to
> >> > > open
> >> > > > >> the
> >> > > > >> command palette using:
> >> > > > >>
> >> > > > >> - Ctrl + Shift + P (windows/linux)
> >> > > > >> - Command + Shift + P (mac)
> >> > > > >>
> >> > > > >> Then typing "data.edit", make sure "Daffodil Debug: Data
> Editor"
> >> is
> >> > > > >> selected
> >> > > > >> and then hitting enter. By default the data editor will run on
> >> port
> >> > > 9000
> >> > > > >> of
> >> > > > >> your machine. However, if you want to use a different port
> follow
> >> > > steps
> >> > > > >> here
> >> > > > >> Data Editor Launch Settings
> >> > > > >> <
> >> > > > >>
> >> > >
> >>
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
> >> > > > >> >
> >> > > > >>
> >> > > > >> Thank you,
> >> > > > >>
> >> > > > >> - Shane Dell
> >> > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Davin Shearer <sc...@gmail.com>.
Thanks for the detailed update and your patience Mike. This is
definitely not the user experience we're hoping for. Are these regressions
that have occurred since the v1.2.0 release?
I'm a big fan of compartmentalization and technologies like docker and
python's venv (virtual environment) among many others. Relying on system
installs of virtual machines (e.g. Java's JVM) and interpreters (e.g.,
Python, Perl, etc) introduces uncontrolled variables into our runtime
environment, which I believe is one of the major encumbrances that you're
experiencing and something I hope we can address as a community.
I don't think docker is the right answer here because that introduces
another system-level dependency... the docker service. I think packaging
a JVM or pulling it in as a VS Code dependency (not a system level
dependency like docker) will give us much tighter control over our runtime
environment. We'll need to be careful too about how we execute our
services on that JVM so its execution environment is properly
compartmentalized. In addition to the Daffodil debugger service, the Data
Editor part of the extension also uses a service running on the JVM which
will benefit from this compartmentalization. This also will allow us to
upgrade from Java 8 to something more recent (this will help with domain
sockets and named fifos too). Anyway, that's the direction I'd be inclined
to head in, but it's definitely worth a discussion and hopefully we can
converge on solid design. I had an interesting conversation on this topic
with GPT-4 that I think is worth sharing (see:
https://chat.openai.com/share/da2ca068-714a-4eec-8a0e-8117069b3125).
Each distinct failure that you've encountered in testing RC2 needs to have
issues associated with them on the project's GitHub so we can track, triage
and prioritize, schedule, and assign them. If you're willing to enter your
issues into the GitHub tracker, that would be greatly appreciated and
you'll be able to stay in the loop on how those issues progress and help us
in the review phase to make sure that each issue is being properly and
fully addressed.
Shane is out, but I'm sure he'll have some additional follow up and things
to try before making new pull requests.
On Wed, May 31, 2023 at 7:02 PM Mike Beckerle <mb...@apache.org> wrote:
>
> So, I have the 1.3.0 RC2 vscode extension debugger working now for PCAP w/
> ethernetIP and I can set a breakpoint in ethernetIP.dfdl.xsd and I no
> longer get an NPE.
>
> Configuring this is really far too difficult, and I have to have the
> schema working first with sbt in order to even hope to configure it.
>
> But I'll get back to that later.
>
> More basic issue is that I am unable to debug even dead-simple errors with
> it. I clobbered the magic number in the first bytes of an icmp1.cap file.
> This fails an assert at line 97 of pcap.dfdl.xsd, which is the first
> element of simple type.
>
> I step through with the debugger, and nothing displays *anything* about
> the assert failure. Execution ends. A little pop-up dialog says it wrote my
> infoset out, which it didn't. There's nothing showing that anything went
> wrong. All panels on the left are blank.
>
> If I modify the schema and make any mistake, then I get a blocking dialog
> showing the SDEs. This is basically intolerable, and is why I have to go
> back to sbt to debug this. You can't even go through the SDEs as you move
> the editor. It's a blocking dialog.
>
> So both for schema compilation, and runtime errors there simply has to be
> a non-modal display of errors, probably on the left with variables, watch,
> call-stack, and breakpoints. But a quasi-edit-buffer containing them would
> also be fine, or even just output them to the terminal window.
>
> The rest of this email is about the hoops I had to jump through to get
> this to work at all. Arguably, these issues are about working with a
> multi-component schema, and we could define our objectives to simpler
> schemas than this, but that puts it in the realm of toy-only schemas. The
> example is just PCAP + ethernetIP, which is not an extreme example.
>
> The existing shell environment of the terminal definitely causes problems.
> But just exiting that so that the debugger starts a new terminal doesn't
> fix it, as that inherits the environment from where vscode was started. It
> starts a new login shell unless you take steps to prevent that.
>
> A big trip-up in usability is that the wizard that creates the
> launch.json, doesn't actually let you populate the daffodilDebugClaspath.
> You have to edit that by hand, and nothing verifies it, so any typo and you
> get tedious failures that are hard to figure out. You just get an SDE - if
> you are lucky - via a blocking dialog.
>
> The wizard should help you put directories and corresponding jars on the
> daffodilDebugClasspath, and it should not let you put a non-existing
> directory or jar file on it. Ideally, at launch time the
> daffodilDebugClasspath should be checked, just in case something on it no
> longer exists.
>
> If the debugger happens to find a file on the daffodilDebugClasspath in a
> jar, vs. in a regular directory (maybe you have a typo in the regular
> directory path), so it passes that and hits the jar, then setting a
> breakpoint causes a NPE. Something requires a "hierarchical URL" which a
> jar-file URL is not. This is the bug that really has to be fixed so that
> you can set breakpoints regardless of whether the source DFDL schema file
> was in a jar or not.
>
> Note: at Owl we have DFDL schemas where the starting root element of the
> schema isn't even part of the schema, it's part of a component schema that
> is in a jar on the classpath. This happens when you have an
> envelope-payload idiom. The schema that combines the envelope and payload
> together contains only the glue part that connects the two together. The
> root element is the envelope's root element, and that schema is in a jar
> file.
>
> I did not see any feature in launch.json for specifying the name/namespace
> of the root element. There is just the "program" element, which must be to
> a file in the current schema. That needs to be able to be specified just as
> element name + namespace.
>
> The jars have to be on the daffodilDebugClasspath because of the plugins
> that daffodil must find in them. That's understandable, the problem is
> having to also have a regular file location for any schema you want to be
> able to put a breakpoint in and it must be before the corresponding jar.
>
> That all said, here are the scripts and settings I used to get it to work:
>
> First, I have this script I use to launch vscode in a minimal/empty
> environment. This technique is useful to
> determine if you are depending on the environment in any way:
>
> #! /bin/bash
> if [ "$1" = "emptyEnv" ] # is it the special first arg
> then
> shift # discard special first arg, and continue with the rest of the
> script
> else
> CMD=$0
> exec -c $CMD "emptyEnv" $* # exec with an empty environment, special
> first arg
> fi
> # the environment is empty when we get here
> # just add env vars we need to get vscode to work
> export LANG=en_US.UTF-8
> export HOME=/home/mbeckerle
> export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 # java 11 required
> export PATH=$JAVA_HOME/bin:$PATH
> export DISPLAY=:0
> export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus # vscode
> spits lots of errors without this
> exec /usr/bin/code
> # end
>
> Then, within ~/.config/Code/User/settings.json, I had to create these
> settings for terminal:
>
> "terminal.integrated.inheritEnv": false,
> "terminal.integrated.profiles.linux": {
> "Custom Init": {
> "path": "/bin/bash",
> "args": [
> "--norc"
> ]
> }
> },
> "terminal.integrated.defaultProfile.linux": "Custom Init"
>
> That instructs vscode not to create a login shell for the terminal. The
> --norc option means it does not run the .bashrc file so that the java
> version specified in the launch script sticks.
>
> I am not sure the inheritEnv setting actually does anything. That alone
> sounds like the right thing, but that alone didn't solve the problem.
> You still have everything in the login shell environment where vscode was
> launched. Hence the need for the environment-emptying launch script.
>
> With those two things, a terminal created by vscode does not inherit
> everything my typical shell has. For example there is no daffodil on the
> path, no DAFFODIL_CLASSPATH env var, etc.
>
> Given the above if I just create a terminal with Terminal > new Terminal:
> The environment is relatively simple:
>
> bash-5.0$ env
> COLORTERM=truecolor
> TERM_PROGRAM_VERSION=1.78.2
> JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
> NO_AT_BRIDGE=1
> PWD=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP
> VSCODE_GIT_ASKPASS_NODE=/usr/share/code/code
> HOME=/home/mbeckerle
> LANG=en_US.UTF-8
> GIT_ASKPASS=/usr/share/code/resources/app/extensions/git/dist/askpass.sh
> CHROME_DESKTOP=code-url-handler.desktop
> VSCODE_GIT_ASKPASS_EXTRA_ARGS=--ms-enable-electron-run-as-node
> TERM=xterm-256color
> VSCODE_GIT_IPC_HANDLE=/tmp/vscode-git-56a269c0f5.sock
> DISPLAY=:0
> SHLVL=1
>
> VSCODE_GIT_ASKPASS_MAIN=/usr/share/code/resources/app/extensions/git/dist/askpass-main.js
> GDK_BACKEND=x11
>
> PATH=/usr/lib/jvm/java-11-openjdk-amd64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
> ORIGINAL_XDG_CURRENT_DESKTOP=undefined
> DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
> TERM_PROGRAM=vscode
> _=/usr/bin/env
>
> If I run daffodil, then c-C in the terminal it creates, then the
> environment has this added environment variable only.
>
>
> DAFFODIL_DEBUG_CLASSPATH=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/../ethernetIP/src/main/resources:/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/lib_managed/jars/com.owlcyberdefense/dfdl-ethernetip/dfdl-ethernetip-1.2.0.jar
>
> This works. My launch.json is attached.
>
>
>
>
> On Fri, May 26, 2023 at 2:06 PM Shane Dell <sh...@apache.org> wrote:
>
>> Mike,
>>
>> The VSCode Extension should be using the daffodil command that is bundled
>> with the
>> extension. The issue might still be with that shell env variable you have
>> for
>> DAFFODIL_CLASSPATH, but this might only be an issue with a terminal that
>> is already opened
>> with the name "daffodil-debugger". If you close that terminal, the next
>> you time you run it should
>> create a new terminal with the env set properly. If you could try this
>> and see if it works how you
>> expect, that would help us single out this issue out to an already active
>> terminal.
>>
>> Thank you,
>>
>> Shane Dell
>>
>>
>> On 2023/05/25 23:59:53 Mike Beckerle wrote:
>> > There are so many environment interactions....
>> >
>> > So I have discovered that the VSCode extension just seems to run
>> whatever
>> > daffodil command is on the path. It's not isolated at all from the shell
>> > environment.
>> >
>> > I found this out because I have a customized daffodil shell script that
>> > setups a bunch of DAFFODIL_CLASSPATH contents, and I couldn't figure out
>> > why those were part of the VSCode environment.
>> >
>> > Shouldn't vscode run exactly the daffodil bundled with it, unless you
>> > explicitly configure it not to?
>> >
>> > That's why it is finding the ethernetIP.dfdl.xsd file in the jar, not in
>> > the file system. Because my daffodil command sets up lots of jars on the
>> > DAFFODIL_CLASSPATH.
>> >
>> > It also depends on Java 11, breaks with Java 17. I also don't think this
>> > should be inherited from the environment.
>> >
>> > I really have a hard time using, or even testing the VSCode extension
>> if I
>> > have to scrub my shell environment of everything in it in order to make
>> the
>> > vscode extension work.
>> >
>> > But it is this kind of problem that caused people to invent docker
>> > containers.
>> >
>> > I am going to make a launcher script for VSCode that empties the
>> > environment, then supplies only what is absolutely necessary, with no
>> > outside environment assumptions.
>> >
>> > I will try to do this Friday if possible.
>> >
>> >
>> >
>> > On Wed, May 24, 2023 at 2:50 PM Shane Dell <sh...@apache.org>
>> wrote:
>> >
>> > > Mike,
>> > >
>> > > I am not sure why you would be seeing this. I am able to set the
>> > > breakpoint where you
>> > > mentioned without any issues. Can you try removing the daffodil-dap
>> > > folder? This should be
>> > > somewhere like:
>> > > /home/$USER/.local/share/daffodil-dap
>> > > The issue I am thinking is possibly a bug with the rc1 debugger that
>> may
>> > > not be present on rc2.
>> > > But if the folder daffodil-dap/daffodil-debugger-3.4.0-1.3.0/ already
>> > > exists from rc1 then rc2 will
>> > > not remake it. So in order to remake it you have to manually remove
>> this
>> > > folder. Let me know if
>> > > this helps.
>> > >
>> > > Thank you,
>> > >
>> > > Shane Dell
>> > >
>> > > On 2023/05/24 15:15:33 Mike Beckerle wrote:
>> > > > I have been able to continue evaluating RC2.
>> > > >
>> > > > My vote is -1
>> > > >
>> > > > My testing steps listed below. I am trying to debug PCAP. I have a
>> broken
>> > > > icmp packet (bad magic number), and also just step through a
>> correct icmp
>> > > > packet.
>> > > >
>> > > > I am trying to debug icmp.badMagicNum.cap data file.
>> > > >
>> > > > I should get a failure of an assert because the magic number is not
>> one
>> > > of
>> > > > the expected values.
>> > > >
>> > > > I'm just trying to single-step through it until it hits the error.
>> > > >
>> > > > Once we get to the MagicNumber element, it doesn't position on the
>> assert
>> > > > while evaluating that expression or even on the element carrying the
>> > > assert.
>> > > > It jumps back to the top-level xs:schema position.
>> > > >
>> > > > I continue to click single step. Watching for the error to be
>> displayed
>> > > in
>> > > > the upper left under Variables > Parse > Diagnostics > Errors which
>> is
>> > > > where I think it ought to go.
>> > > >
>> > > > At some point the failure is somehow detected as the debug session
>> ends.
>> > > A
>> > > > dialog offers to open /tmp/infoset.xml for me, which I open and it
>> is
>> > > empty
>> > > > of course.
>> > > >
>> > > > Nothing displayed the error message from the assert, or even
>> indicated
>> > > what
>> > > > went wrong at all. Not the state display in the upper left, not the
>> > > > terminal, nor debug console nor output.
>> > > >
>> > > > The upper left box with "Variables" is cleared at the end. It should
>> > > > certainly not be cleared at the end of a run when final state may
>> be of
>> > > > interest. At the start of a run, or manually, but not at the end of
>> a
>> > > run.
>> > > >
>> > > > Second attempt to use it:
>> > > >
>> > > > I create a second launch config this time for what should be a
>> successful
>> > > > parse of icmp1.cap
>> > > >
>> > > > As I single step, the position of the cursor on the schema is
>> jumping
>> > > > around all over the place back and forth between a containing
>> element and
>> > > > the child elements it contains.
>> > > > The infoset display does not get updated until long after the
>> display has
>> > > > moved on and is positioned on the next element.
>> > > >
>> > > > This jumping around gets completely out of hand once you step into
>> the
>> > > > complexType "Ethernet" which is in the ethernetIP jar file.
>> > > >
>> > > > At that point, after each child of Ethernet, it jumps back to the
>> > > Ethernet
>> > > > element in the pcap.dfdl.xsd file, and then back into the
>> > > > ethernetIP.dfdl.xsd for the next child, then back to the
>> pcap.dfdl.xsd
>> > > > file, etc.
>> > > >
>> > > > All this jumping around basically makes stepping useless. So I
>> decide to
>> > > > put a breakpoint on the Version element inside the IPv4Header
>> element
>> > > > inside ethernetIP.dfdl.xsd.
>> > > >
>> > > > Just adding the breakpoint causes the debug session to end with an
>> NPE in
>> > > > the dap setBreakpoint.
>> > > >
>> > > > ERROR o.a.d.d.d.DAPodil - unhandled error
>> > > > java.lang.NullPointerException: null
>> > > > at
>> > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
>> > > > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
>> > > > at
>> > > > java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
>> > > > at java.base/java.nio.file.Path.of(Path.java:147)
>> > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
>> > > > at
>> > > >
>> > >
>> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
>> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > at flatMap @
>> > > >
>> > >
>> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > at handleErrorWith @
>> > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
>> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > at flatMap @
>> fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
>> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > 2023-05-24 11:09:10,717 [io-compute-1] DEBUG o.a.d.d.d.DAPodil -
>> > > whenDone:
>> > > > completed
>> > > > java.lang.NullPointerException
>> > > > at
>> > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
>> > > > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
>> > > > at
>> > > > java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
>> > > > at java.base/java.nio.file.Path.of(Path.java:147)
>> > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
>> > > > at
>> > > >
>> > >
>> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
>> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > at flatMap @
>> > > >
>> > >
>> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > at handleErrorWith @
>> > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
>> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > > at flatMap @
>> fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
>> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
>> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Tue, May 23, 2023 at 5:19 PM Mike Beckerle <mbeckerle@apache.org
>> >
>> > > wrote:
>> > > >
>> > > > > I really am trying to eval the RC2.
>> > > > >
>> > > > > Right now my VMWare Linux system has been broken by recent
>> updates to
>> > > the
>> > > > > MS Windows host environment.
>> > > > >
>> > > > > I don't exactly understand this, but the VSCode front end can no
>> longer
>> > > > > connect to the server at localhost:4711. I get ECONNREFUSED. This
>> > > started
>> > > > > when a bunch of MS-Windows updates were done on the host OS. I run
>> > > VSCode
>> > > > > on the linux client machine, which lives behind a NAT firewall
>> created
>> > > by
>> > > > > VMWare workstation - so I really don't get how these could be
>> > > interacting
>> > > > > this way, but... there it is.
>> > > > >
>> > > > > I will switch back to my standalone linux PC if this isn't
>> resolved
>> > > today.
>> > > > >
>> > > > >
>> > > > > On Tue, May 16, 2023 at 1:24 PM Shane Dell <sh...@apache.org>
>> > > wrote:
>> > > > >
>> > > > >> Hello all,I'd like to call a vote to release Apache Daffodil VS
>> Code
>> > > > >> 1.3.0-rc2.
>> > > > >>
>> > > > >> All distribution packages, including signatures, digests, etc.
>> can be
>> > > > >> found at:
>> > > > >>
>> > >
>> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
>> > > > >>
>> > > > >> This release has been signed with PGP key
>> > > > >> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
>> > > > >> to shanedell@apache.org, which is included in the KEYS file
>> here:
>> > > > >> https://downloads.apache.org/daffodil/KEYS
>> > > > >>
>> > > > >> The release candidate has been tagged in git with 1.3.0-rc2. The
>> > > binaries
>> > > > >>
>> > > > >> For reference, here is a list of all closed GitHub issues tagged
>> with
>> > > > >> 1.3.0:
>> > > > >> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
>> > > > >>
>> > > > >> Please review and vote. The vote will be open for at least 72
>> hours
>> > > > >> (Friday, 19 May 2023, 1:30pm EST).
>> > > > >>
>> > > > >> [ ] +1 approve
>> > > > >> [ ] +0 no opinion
>> > > > >> [ ] -1 disapprove (and reason why)
>> > > > >>
>> > > > >> Documentation for 1.3.0 can be found here
>> > > > >> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
>> > > > >> <
>> > > > >>
>> > >
>> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
>> > > > >> >.
>> > > > >>
>> > > > >> Please note one of the large features that was focused on in
>> 1.3.0 was
>> > > > >> improving the data editor. To open the data editor you will need
>> to
>> > > open
>> > > > >> the
>> > > > >> command palette using:
>> > > > >>
>> > > > >> - Ctrl + Shift + P (windows/linux)
>> > > > >> - Command + Shift + P (mac)
>> > > > >>
>> > > > >> Then typing "data.edit", make sure "Daffodil Debug: Data Editor"
>> is
>> > > > >> selected
>> > > > >> and then hitting enter. By default the data editor will run on
>> port
>> > > 9000
>> > > > >> of
>> > > > >> your machine. However, if you want to use a different port follow
>> > > steps
>> > > > >> here
>> > > > >> Data Editor Launch Settings
>> > > > >> <
>> > > > >>
>> > >
>> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
>> > > > >> >
>> > > > >>
>> > > > >> Thank you,
>> > > > >>
>> > > > >> - Shane Dell
>> > > > >>
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Mike Beckerle <mb...@apache.org>.
So, I have the 1.3.0 RC2 vscode extension debugger working now for PCAP w/
ethernetIP and I can set a breakpoint in ethernetIP.dfdl.xsd and I no
longer get an NPE.
Configuring this is really far too difficult, and I have to have the schema
working first with sbt in order to even hope to configure it.
But I'll get back to that later.
More basic issue is that I am unable to debug even dead-simple errors with
it. I clobbered the magic number in the first bytes of an icmp1.cap file.
This fails an assert at line 97 of pcap.dfdl.xsd, which is the first
element of simple type.
I step through with the debugger, and nothing displays *anything* about the
assert failure. Execution ends. A little pop-up dialog says it wrote my
infoset out, which it didn't. There's nothing showing that anything went
wrong. All panels on the left are blank.
If I modify the schema and make any mistake, then I get a blocking dialog
showing the SDEs. This is basically intolerable, and is why I have to go
back to sbt to debug this. You can't even go through the SDEs as you move
the editor. It's a blocking dialog.
So both for schema compilation, and runtime errors there simply has to be a
non-modal display of errors, probably on the left with variables, watch,
call-stack, and breakpoints. But a quasi-edit-buffer containing them would
also be fine, or even just output them to the terminal window.
The rest of this email is about the hoops I had to jump through to get this
to work at all. Arguably, these issues are about working with a
multi-component schema, and we could define our objectives to simpler
schemas than this, but that puts it in the realm of toy-only schemas. The
example is just PCAP + ethernetIP, which is not an extreme example.
The existing shell environment of the terminal definitely causes problems.
But just exiting that so that the debugger starts a new terminal doesn't
fix it, as that inherits the environment from where vscode was started. It
starts a new login shell unless you take steps to prevent that.
A big trip-up in usability is that the wizard that creates the launch.json,
doesn't actually let you populate the daffodilDebugClaspath. You have to
edit that by hand, and nothing verifies it, so any typo and you get tedious
failures that are hard to figure out. You just get an SDE - if you are
lucky - via a blocking dialog.
The wizard should help you put directories and corresponding jars on the
daffodilDebugClasspath, and it should not let you put a non-existing
directory or jar file on it. Ideally, at launch time the
daffodilDebugClasspath should be checked, just in case something on it no
longer exists.
If the debugger happens to find a file on the daffodilDebugClasspath in a
jar, vs. in a regular directory (maybe you have a typo in the regular
directory path), so it passes that and hits the jar, then setting a
breakpoint causes a NPE. Something requires a "hierarchical URL" which a
jar-file URL is not. This is the bug that really has to be fixed so that
you can set breakpoints regardless of whether the source DFDL schema file
was in a jar or not.
Note: at Owl we have DFDL schemas where the starting root element of the
schema isn't even part of the schema, it's part of a component schema that
is in a jar on the classpath. This happens when you have an
envelope-payload idiom. The schema that combines the envelope and payload
together contains only the glue part that connects the two together. The
root element is the envelope's root element, and that schema is in a jar
file.
I did not see any feature in launch.json for specifying the name/namespace
of the root element. There is just the "program" element, which must be to
a file in the current schema. That needs to be able to be specified just as
element name + namespace.
The jars have to be on the daffodilDebugClasspath because of the plugins
that daffodil must find in them. That's understandable, the problem is
having to also have a regular file location for any schema you want to be
able to put a breakpoint in and it must be before the corresponding jar.
That all said, here are the scripts and settings I used to get it to work:
First, I have this script I use to launch vscode in a minimal/empty
environment. This technique is useful to
determine if you are depending on the environment in any way:
#! /bin/bash
if [ "$1" = "emptyEnv" ] # is it the special first arg
then
shift # discard special first arg, and continue with the rest of the
script
else
CMD=$0
exec -c $CMD "emptyEnv" $* # exec with an empty environment, special
first arg
fi
# the environment is empty when we get here
# just add env vars we need to get vscode to work
export LANG=en_US.UTF-8
export HOME=/home/mbeckerle
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 # java 11 required
export PATH=$JAVA_HOME/bin:$PATH
export DISPLAY=:0
export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus # vscode spits
lots of errors without this
exec /usr/bin/code
# end
Then, within ~/.config/Code/User/settings.json, I had to create these
settings for terminal:
"terminal.integrated.inheritEnv": false,
"terminal.integrated.profiles.linux": {
"Custom Init": {
"path": "/bin/bash",
"args": [
"--norc"
]
}
},
"terminal.integrated.defaultProfile.linux": "Custom Init"
That instructs vscode not to create a login shell for the terminal. The
--norc option means it does not run the .bashrc file so that the java
version specified in the launch script sticks.
I am not sure the inheritEnv setting actually does anything. That alone
sounds like the right thing, but that alone didn't solve the problem.
You still have everything in the login shell environment where vscode was
launched. Hence the need for the environment-emptying launch script.
With those two things, a terminal created by vscode does not inherit
everything my typical shell has. For example there is no daffodil on the
path, no DAFFODIL_CLASSPATH env var, etc.
Given the above if I just create a terminal with Terminal > new Terminal:
The environment is relatively simple:
bash-5.0$ env
COLORTERM=truecolor
TERM_PROGRAM_VERSION=1.78.2
JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
NO_AT_BRIDGE=1
PWD=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP
VSCODE_GIT_ASKPASS_NODE=/usr/share/code/code
HOME=/home/mbeckerle
LANG=en_US.UTF-8
GIT_ASKPASS=/usr/share/code/resources/app/extensions/git/dist/askpass.sh
CHROME_DESKTOP=code-url-handler.desktop
VSCODE_GIT_ASKPASS_EXTRA_ARGS=--ms-enable-electron-run-as-node
TERM=xterm-256color
VSCODE_GIT_IPC_HANDLE=/tmp/vscode-git-56a269c0f5.sock
DISPLAY=:0
SHLVL=1
VSCODE_GIT_ASKPASS_MAIN=/usr/share/code/resources/app/extensions/git/dist/askpass-main.js
GDK_BACKEND=x11
PATH=/usr/lib/jvm/java-11-openjdk-amd64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ORIGINAL_XDG_CURRENT_DESKTOP=undefined
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
TERM_PROGRAM=vscode
_=/usr/bin/env
If I run daffodil, then c-C in the terminal it creates, then the
environment has this added environment variable only.
DAFFODIL_DEBUG_CLASSPATH=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/../ethernetIP/src/main/resources:/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/lib_managed/jars/com.owlcyberdefense/dfdl-ethernetip/dfdl-ethernetip-1.2.0.jar
This works. My launch.json is attached.
On Fri, May 26, 2023 at 2:06 PM Shane Dell <sh...@apache.org> wrote:
> Mike,
>
> The VSCode Extension should be using the daffodil command that is bundled
> with the
> extension. The issue might still be with that shell env variable you have
> for
> DAFFODIL_CLASSPATH, but this might only be an issue with a terminal that
> is already opened
> with the name "daffodil-debugger". If you close that terminal, the next
> you time you run it should
> create a new terminal with the env set properly. If you could try this and
> see if it works how you
> expect, that would help us single out this issue out to an already active
> terminal.
>
> Thank you,
>
> Shane Dell
>
>
> On 2023/05/25 23:59:53 Mike Beckerle wrote:
> > There are so many environment interactions....
> >
> > So I have discovered that the VSCode extension just seems to run whatever
> > daffodil command is on the path. It's not isolated at all from the shell
> > environment.
> >
> > I found this out because I have a customized daffodil shell script that
> > setups a bunch of DAFFODIL_CLASSPATH contents, and I couldn't figure out
> > why those were part of the VSCode environment.
> >
> > Shouldn't vscode run exactly the daffodil bundled with it, unless you
> > explicitly configure it not to?
> >
> > That's why it is finding the ethernetIP.dfdl.xsd file in the jar, not in
> > the file system. Because my daffodil command sets up lots of jars on the
> > DAFFODIL_CLASSPATH.
> >
> > It also depends on Java 11, breaks with Java 17. I also don't think this
> > should be inherited from the environment.
> >
> > I really have a hard time using, or even testing the VSCode extension if
> I
> > have to scrub my shell environment of everything in it in order to make
> the
> > vscode extension work.
> >
> > But it is this kind of problem that caused people to invent docker
> > containers.
> >
> > I am going to make a launcher script for VSCode that empties the
> > environment, then supplies only what is absolutely necessary, with no
> > outside environment assumptions.
> >
> > I will try to do this Friday if possible.
> >
> >
> >
> > On Wed, May 24, 2023 at 2:50 PM Shane Dell <sh...@apache.org> wrote:
> >
> > > Mike,
> > >
> > > I am not sure why you would be seeing this. I am able to set the
> > > breakpoint where you
> > > mentioned without any issues. Can you try removing the daffodil-dap
> > > folder? This should be
> > > somewhere like:
> > > /home/$USER/.local/share/daffodil-dap
> > > The issue I am thinking is possibly a bug with the rc1 debugger that
> may
> > > not be present on rc2.
> > > But if the folder daffodil-dap/daffodil-debugger-3.4.0-1.3.0/ already
> > > exists from rc1 then rc2 will
> > > not remake it. So in order to remake it you have to manually remove
> this
> > > folder. Let me know if
> > > this helps.
> > >
> > > Thank you,
> > >
> > > Shane Dell
> > >
> > > On 2023/05/24 15:15:33 Mike Beckerle wrote:
> > > > I have been able to continue evaluating RC2.
> > > >
> > > > My vote is -1
> > > >
> > > > My testing steps listed below. I am trying to debug PCAP. I have a
> broken
> > > > icmp packet (bad magic number), and also just step through a correct
> icmp
> > > > packet.
> > > >
> > > > I am trying to debug icmp.badMagicNum.cap data file.
> > > >
> > > > I should get a failure of an assert because the magic number is not
> one
> > > of
> > > > the expected values.
> > > >
> > > > I'm just trying to single-step through it until it hits the error.
> > > >
> > > > Once we get to the MagicNumber element, it doesn't position on the
> assert
> > > > while evaluating that expression or even on the element carrying the
> > > assert.
> > > > It jumps back to the top-level xs:schema position.
> > > >
> > > > I continue to click single step. Watching for the error to be
> displayed
> > > in
> > > > the upper left under Variables > Parse > Diagnostics > Errors which
> is
> > > > where I think it ought to go.
> > > >
> > > > At some point the failure is somehow detected as the debug session
> ends.
> > > A
> > > > dialog offers to open /tmp/infoset.xml for me, which I open and it is
> > > empty
> > > > of course.
> > > >
> > > > Nothing displayed the error message from the assert, or even
> indicated
> > > what
> > > > went wrong at all. Not the state display in the upper left, not the
> > > > terminal, nor debug console nor output.
> > > >
> > > > The upper left box with "Variables" is cleared at the end. It should
> > > > certainly not be cleared at the end of a run when final state may be
> of
> > > > interest. At the start of a run, or manually, but not at the end of a
> > > run.
> > > >
> > > > Second attempt to use it:
> > > >
> > > > I create a second launch config this time for what should be a
> successful
> > > > parse of icmp1.cap
> > > >
> > > > As I single step, the position of the cursor on the schema is jumping
> > > > around all over the place back and forth between a containing
> element and
> > > > the child elements it contains.
> > > > The infoset display does not get updated until long after the
> display has
> > > > moved on and is positioned on the next element.
> > > >
> > > > This jumping around gets completely out of hand once you step into
> the
> > > > complexType "Ethernet" which is in the ethernetIP jar file.
> > > >
> > > > At that point, after each child of Ethernet, it jumps back to the
> > > Ethernet
> > > > element in the pcap.dfdl.xsd file, and then back into the
> > > > ethernetIP.dfdl.xsd for the next child, then back to the
> pcap.dfdl.xsd
> > > > file, etc.
> > > >
> > > > All this jumping around basically makes stepping useless. So I
> decide to
> > > > put a breakpoint on the Version element inside the IPv4Header element
> > > > inside ethernetIP.dfdl.xsd.
> > > >
> > > > Just adding the breakpoint causes the debug session to end with an
> NPE in
> > > > the dap setBreakpoint.
> > > >
> > > > ERROR o.a.d.d.d.DAPodil - unhandled error
> > > > java.lang.NullPointerException: null
> > > > at
> > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> > > > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> > > > at
> > > > java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> > > > at java.base/java.nio.file.Path.of(Path.java:147)
> > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
> > > > at
> > > >
> > >
> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > at flatMap @
> > > >
> > >
> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > at handleErrorWith @
> > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > at flatMap @
> fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > 2023-05-24 11:09:10,717 [io-compute-1] DEBUG o.a.d.d.d.DAPodil -
> > > whenDone:
> > > > completed
> > > > java.lang.NullPointerException
> > > > at
> > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> > > > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> > > > at
> > > > java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> > > > at java.base/java.nio.file.Path.of(Path.java:147)
> > > > at java.base/java.nio.file.Paths.get(Paths.java:69)
> > > > at
> > > >
> > >
> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > at flatMap @
> > > >
> > >
> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > at handleErrorWith @
> > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > > at flatMap @
> fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Tue, May 23, 2023 at 5:19 PM Mike Beckerle <mb...@apache.org>
> > > wrote:
> > > >
> > > > > I really am trying to eval the RC2.
> > > > >
> > > > > Right now my VMWare Linux system has been broken by recent updates
> to
> > > the
> > > > > MS Windows host environment.
> > > > >
> > > > > I don't exactly understand this, but the VSCode front end can no
> longer
> > > > > connect to the server at localhost:4711. I get ECONNREFUSED. This
> > > started
> > > > > when a bunch of MS-Windows updates were done on the host OS. I run
> > > VSCode
> > > > > on the linux client machine, which lives behind a NAT firewall
> created
> > > by
> > > > > VMWare workstation - so I really don't get how these could be
> > > interacting
> > > > > this way, but... there it is.
> > > > >
> > > > > I will switch back to my standalone linux PC if this isn't resolved
> > > today.
> > > > >
> > > > >
> > > > > On Tue, May 16, 2023 at 1:24 PM Shane Dell <sh...@apache.org>
> > > wrote:
> > > > >
> > > > >> Hello all,I'd like to call a vote to release Apache Daffodil VS
> Code
> > > > >> 1.3.0-rc2.
> > > > >>
> > > > >> All distribution packages, including signatures, digests, etc.
> can be
> > > > >> found at:
> > > > >>
> > >
> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
> > > > >>
> > > > >> This release has been signed with PGP key
> > > > >> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
> > > > >> to shanedell@apache.org, which is included in the KEYS file here:
> > > > >> https://downloads.apache.org/daffodil/KEYS
> > > > >>
> > > > >> The release candidate has been tagged in git with 1.3.0-rc2. The
> > > binaries
> > > > >>
> > > > >> For reference, here is a list of all closed GitHub issues tagged
> with
> > > > >> 1.3.0:
> > > > >> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
> > > > >>
> > > > >> Please review and vote. The vote will be open for at least 72
> hours
> > > > >> (Friday, 19 May 2023, 1:30pm EST).
> > > > >>
> > > > >> [ ] +1 approve
> > > > >> [ ] +0 no opinion
> > > > >> [ ] -1 disapprove (and reason why)
> > > > >>
> > > > >> Documentation for 1.3.0 can be found here
> > > > >> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
> > > > >> <
> > > > >>
> > >
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
> > > > >> >.
> > > > >>
> > > > >> Please note one of the large features that was focused on in
> 1.3.0 was
> > > > >> improving the data editor. To open the data editor you will need
> to
> > > open
> > > > >> the
> > > > >> command palette using:
> > > > >>
> > > > >> - Ctrl + Shift + P (windows/linux)
> > > > >> - Command + Shift + P (mac)
> > > > >>
> > > > >> Then typing "data.edit", make sure "Daffodil Debug: Data Editor"
> is
> > > > >> selected
> > > > >> and then hitting enter. By default the data editor will run on
> port
> > > 9000
> > > > >> of
> > > > >> your machine. However, if you want to use a different port follow
> > > steps
> > > > >> here
> > > > >> Data Editor Launch Settings
> > > > >> <
> > > > >>
> > >
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
> > > > >> >
> > > > >>
> > > > >> Thank you,
> > > > >>
> > > > >> - Shane Dell
> > > > >>
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Shane Dell <sh...@apache.org>.
Mike,
The VSCode Extension should be using the daffodil command that is bundled with the
extension. The issue might still be with that shell env variable you have for
DAFFODIL_CLASSPATH, but this might only be an issue with a terminal that is already opened
with the name "daffodil-debugger". If you close that terminal, the next you time you run it should
create a new terminal with the env set properly. If you could try this and see if it works how you
expect, that would help us single out this issue out to an already active terminal.
Thank you,
Shane Dell
On 2023/05/25 23:59:53 Mike Beckerle wrote:
> There are so many environment interactions....
>
> So I have discovered that the VSCode extension just seems to run whatever
> daffodil command is on the path. It's not isolated at all from the shell
> environment.
>
> I found this out because I have a customized daffodil shell script that
> setups a bunch of DAFFODIL_CLASSPATH contents, and I couldn't figure out
> why those were part of the VSCode environment.
>
> Shouldn't vscode run exactly the daffodil bundled with it, unless you
> explicitly configure it not to?
>
> That's why it is finding the ethernetIP.dfdl.xsd file in the jar, not in
> the file system. Because my daffodil command sets up lots of jars on the
> DAFFODIL_CLASSPATH.
>
> It also depends on Java 11, breaks with Java 17. I also don't think this
> should be inherited from the environment.
>
> I really have a hard time using, or even testing the VSCode extension if I
> have to scrub my shell environment of everything in it in order to make the
> vscode extension work.
>
> But it is this kind of problem that caused people to invent docker
> containers.
>
> I am going to make a launcher script for VSCode that empties the
> environment, then supplies only what is absolutely necessary, with no
> outside environment assumptions.
>
> I will try to do this Friday if possible.
>
>
>
> On Wed, May 24, 2023 at 2:50 PM Shane Dell <sh...@apache.org> wrote:
>
> > Mike,
> >
> > I am not sure why you would be seeing this. I am able to set the
> > breakpoint where you
> > mentioned without any issues. Can you try removing the daffodil-dap
> > folder? This should be
> > somewhere like:
> > /home/$USER/.local/share/daffodil-dap
> > The issue I am thinking is possibly a bug with the rc1 debugger that may
> > not be present on rc2.
> > But if the folder daffodil-dap/daffodil-debugger-3.4.0-1.3.0/ already
> > exists from rc1 then rc2 will
> > not remake it. So in order to remake it you have to manually remove this
> > folder. Let me know if
> > this helps.
> >
> > Thank you,
> >
> > Shane Dell
> >
> > On 2023/05/24 15:15:33 Mike Beckerle wrote:
> > > I have been able to continue evaluating RC2.
> > >
> > > My vote is -1
> > >
> > > My testing steps listed below. I am trying to debug PCAP. I have a broken
> > > icmp packet (bad magic number), and also just step through a correct icmp
> > > packet.
> > >
> > > I am trying to debug icmp.badMagicNum.cap data file.
> > >
> > > I should get a failure of an assert because the magic number is not one
> > of
> > > the expected values.
> > >
> > > I'm just trying to single-step through it until it hits the error.
> > >
> > > Once we get to the MagicNumber element, it doesn't position on the assert
> > > while evaluating that expression or even on the element carrying the
> > assert.
> > > It jumps back to the top-level xs:schema position.
> > >
> > > I continue to click single step. Watching for the error to be displayed
> > in
> > > the upper left under Variables > Parse > Diagnostics > Errors which is
> > > where I think it ought to go.
> > >
> > > At some point the failure is somehow detected as the debug session ends.
> > A
> > > dialog offers to open /tmp/infoset.xml for me, which I open and it is
> > empty
> > > of course.
> > >
> > > Nothing displayed the error message from the assert, or even indicated
> > what
> > > went wrong at all. Not the state display in the upper left, not the
> > > terminal, nor debug console nor output.
> > >
> > > The upper left box with "Variables" is cleared at the end. It should
> > > certainly not be cleared at the end of a run when final state may be of
> > > interest. At the start of a run, or manually, but not at the end of a
> > run.
> > >
> > > Second attempt to use it:
> > >
> > > I create a second launch config this time for what should be a successful
> > > parse of icmp1.cap
> > >
> > > As I single step, the position of the cursor on the schema is jumping
> > > around all over the place back and forth between a containing element and
> > > the child elements it contains.
> > > The infoset display does not get updated until long after the display has
> > > moved on and is positioned on the next element.
> > >
> > > This jumping around gets completely out of hand once you step into the
> > > complexType "Ethernet" which is in the ethernetIP jar file.
> > >
> > > At that point, after each child of Ethernet, it jumps back to the
> > Ethernet
> > > element in the pcap.dfdl.xsd file, and then back into the
> > > ethernetIP.dfdl.xsd for the next child, then back to the pcap.dfdl.xsd
> > > file, etc.
> > >
> > > All this jumping around basically makes stepping useless. So I decide to
> > > put a breakpoint on the Version element inside the IPv4Header element
> > > inside ethernetIP.dfdl.xsd.
> > >
> > > Just adding the breakpoint causes the debug session to end with an NPE in
> > > the dap setBreakpoint.
> > >
> > > ERROR o.a.d.d.d.DAPodil - unhandled error
> > > java.lang.NullPointerException: null
> > > at
> > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> > > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> > > at
> > > java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> > > at java.base/java.nio.file.Path.of(Path.java:147)
> > > at java.base/java.nio.file.Paths.get(Paths.java:69)
> > > at
> > >
> > org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > at flatMap @
> > >
> > org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > at handleErrorWith @
> > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > at flatMap @ fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > 2023-05-24 11:09:10,717 [io-compute-1] DEBUG o.a.d.d.d.DAPodil -
> > whenDone:
> > > completed
> > > java.lang.NullPointerException
> > > at
> > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> > > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> > > at
> > > java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> > > at java.base/java.nio.file.Path.of(Path.java:147)
> > > at java.base/java.nio.file.Paths.get(Paths.java:69)
> > > at
> > >
> > org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > at flatMap @
> > >
> > org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > at handleErrorWith @
> > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > > at flatMap @ fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> > > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Tue, May 23, 2023 at 5:19 PM Mike Beckerle <mb...@apache.org>
> > wrote:
> > >
> > > > I really am trying to eval the RC2.
> > > >
> > > > Right now my VMWare Linux system has been broken by recent updates to
> > the
> > > > MS Windows host environment.
> > > >
> > > > I don't exactly understand this, but the VSCode front end can no longer
> > > > connect to the server at localhost:4711. I get ECONNREFUSED. This
> > started
> > > > when a bunch of MS-Windows updates were done on the host OS. I run
> > VSCode
> > > > on the linux client machine, which lives behind a NAT firewall created
> > by
> > > > VMWare workstation - so I really don't get how these could be
> > interacting
> > > > this way, but... there it is.
> > > >
> > > > I will switch back to my standalone linux PC if this isn't resolved
> > today.
> > > >
> > > >
> > > > On Tue, May 16, 2023 at 1:24 PM Shane Dell <sh...@apache.org>
> > wrote:
> > > >
> > > >> Hello all,I'd like to call a vote to release Apache Daffodil VS Code
> > > >> 1.3.0-rc2.
> > > >>
> > > >> All distribution packages, including signatures, digests, etc. can be
> > > >> found at:
> > > >>
> > https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
> > > >>
> > > >> This release has been signed with PGP key
> > > >> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
> > > >> to shanedell@apache.org, which is included in the KEYS file here:
> > > >> https://downloads.apache.org/daffodil/KEYS
> > > >>
> > > >> The release candidate has been tagged in git with 1.3.0-rc2. The
> > binaries
> > > >>
> > > >> For reference, here is a list of all closed GitHub issues tagged with
> > > >> 1.3.0:
> > > >> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
> > > >>
> > > >> Please review and vote. The vote will be open for at least 72 hours
> > > >> (Friday, 19 May 2023, 1:30pm EST).
> > > >>
> > > >> [ ] +1 approve
> > > >> [ ] +0 no opinion
> > > >> [ ] -1 disapprove (and reason why)
> > > >>
> > > >> Documentation for 1.3.0 can be found here
> > > >> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
> > > >> <
> > > >>
> > https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
> > > >> >.
> > > >>
> > > >> Please note one of the large features that was focused on in 1.3.0 was
> > > >> improving the data editor. To open the data editor you will need to
> > open
> > > >> the
> > > >> command palette using:
> > > >>
> > > >> - Ctrl + Shift + P (windows/linux)
> > > >> - Command + Shift + P (mac)
> > > >>
> > > >> Then typing "data.edit", make sure "Daffodil Debug: Data Editor" is
> > > >> selected
> > > >> and then hitting enter. By default the data editor will run on port
> > 9000
> > > >> of
> > > >> your machine. However, if you want to use a different port follow
> > steps
> > > >> here
> > > >> Data Editor Launch Settings
> > > >> <
> > > >>
> > https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
> > > >> >
> > > >>
> > > >> Thank you,
> > > >>
> > > >> - Shane Dell
> > > >>
> > > >
> > >
> >
>
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Mike Beckerle <mb...@apache.org>.
There are so many environment interactions....
So I have discovered that the VSCode extension just seems to run whatever
daffodil command is on the path. It's not isolated at all from the shell
environment.
I found this out because I have a customized daffodil shell script that
setups a bunch of DAFFODIL_CLASSPATH contents, and I couldn't figure out
why those were part of the VSCode environment.
Shouldn't vscode run exactly the daffodil bundled with it, unless you
explicitly configure it not to?
That's why it is finding the ethernetIP.dfdl.xsd file in the jar, not in
the file system. Because my daffodil command sets up lots of jars on the
DAFFODIL_CLASSPATH.
It also depends on Java 11, breaks with Java 17. I also don't think this
should be inherited from the environment.
I really have a hard time using, or even testing the VSCode extension if I
have to scrub my shell environment of everything in it in order to make the
vscode extension work.
But it is this kind of problem that caused people to invent docker
containers.
I am going to make a launcher script for VSCode that empties the
environment, then supplies only what is absolutely necessary, with no
outside environment assumptions.
I will try to do this Friday if possible.
On Wed, May 24, 2023 at 2:50 PM Shane Dell <sh...@apache.org> wrote:
> Mike,
>
> I am not sure why you would be seeing this. I am able to set the
> breakpoint where you
> mentioned without any issues. Can you try removing the daffodil-dap
> folder? This should be
> somewhere like:
> /home/$USER/.local/share/daffodil-dap
> The issue I am thinking is possibly a bug with the rc1 debugger that may
> not be present on rc2.
> But if the folder daffodil-dap/daffodil-debugger-3.4.0-1.3.0/ already
> exists from rc1 then rc2 will
> not remake it. So in order to remake it you have to manually remove this
> folder. Let me know if
> this helps.
>
> Thank you,
>
> Shane Dell
>
> On 2023/05/24 15:15:33 Mike Beckerle wrote:
> > I have been able to continue evaluating RC2.
> >
> > My vote is -1
> >
> > My testing steps listed below. I am trying to debug PCAP. I have a broken
> > icmp packet (bad magic number), and also just step through a correct icmp
> > packet.
> >
> > I am trying to debug icmp.badMagicNum.cap data file.
> >
> > I should get a failure of an assert because the magic number is not one
> of
> > the expected values.
> >
> > I'm just trying to single-step through it until it hits the error.
> >
> > Once we get to the MagicNumber element, it doesn't position on the assert
> > while evaluating that expression or even on the element carrying the
> assert.
> > It jumps back to the top-level xs:schema position.
> >
> > I continue to click single step. Watching for the error to be displayed
> in
> > the upper left under Variables > Parse > Diagnostics > Errors which is
> > where I think it ought to go.
> >
> > At some point the failure is somehow detected as the debug session ends.
> A
> > dialog offers to open /tmp/infoset.xml for me, which I open and it is
> empty
> > of course.
> >
> > Nothing displayed the error message from the assert, or even indicated
> what
> > went wrong at all. Not the state display in the upper left, not the
> > terminal, nor debug console nor output.
> >
> > The upper left box with "Variables" is cleared at the end. It should
> > certainly not be cleared at the end of a run when final state may be of
> > interest. At the start of a run, or manually, but not at the end of a
> run.
> >
> > Second attempt to use it:
> >
> > I create a second launch config this time for what should be a successful
> > parse of icmp1.cap
> >
> > As I single step, the position of the cursor on the schema is jumping
> > around all over the place back and forth between a containing element and
> > the child elements it contains.
> > The infoset display does not get updated until long after the display has
> > moved on and is positioned on the next element.
> >
> > This jumping around gets completely out of hand once you step into the
> > complexType "Ethernet" which is in the ethernetIP jar file.
> >
> > At that point, after each child of Ethernet, it jumps back to the
> Ethernet
> > element in the pcap.dfdl.xsd file, and then back into the
> > ethernetIP.dfdl.xsd for the next child, then back to the pcap.dfdl.xsd
> > file, etc.
> >
> > All this jumping around basically makes stepping useless. So I decide to
> > put a breakpoint on the Version element inside the IPv4Header element
> > inside ethernetIP.dfdl.xsd.
> >
> > Just adding the breakpoint causes the debug session to end with an NPE in
> > the dap setBreakpoint.
> >
> > ERROR o.a.d.d.d.DAPodil - unhandled error
> > java.lang.NullPointerException: null
> > at
> java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> > at
> > java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> > at java.base/java.nio.file.Path.of(Path.java:147)
> > at java.base/java.nio.file.Paths.get(Paths.java:69)
> > at
> >
> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > at flatMap @
> >
> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > at handleErrorWith @
> > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > at flatMap @ fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > 2023-05-24 11:09:10,717 [io-compute-1] DEBUG o.a.d.d.d.DAPodil -
> whenDone:
> > completed
> > java.lang.NullPointerException
> > at
> java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> > at
> > java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> > at java.base/java.nio.file.Path.of(Path.java:147)
> > at java.base/java.nio.file.Paths.get(Paths.java:69)
> > at
> >
> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > at flatMap @
> >
> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > at handleErrorWith @
> > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> > at flatMap @ fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> > at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> > at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Tue, May 23, 2023 at 5:19 PM Mike Beckerle <mb...@apache.org>
> wrote:
> >
> > > I really am trying to eval the RC2.
> > >
> > > Right now my VMWare Linux system has been broken by recent updates to
> the
> > > MS Windows host environment.
> > >
> > > I don't exactly understand this, but the VSCode front end can no longer
> > > connect to the server at localhost:4711. I get ECONNREFUSED. This
> started
> > > when a bunch of MS-Windows updates were done on the host OS. I run
> VSCode
> > > on the linux client machine, which lives behind a NAT firewall created
> by
> > > VMWare workstation - so I really don't get how these could be
> interacting
> > > this way, but... there it is.
> > >
> > > I will switch back to my standalone linux PC if this isn't resolved
> today.
> > >
> > >
> > > On Tue, May 16, 2023 at 1:24 PM Shane Dell <sh...@apache.org>
> wrote:
> > >
> > >> Hello all,I'd like to call a vote to release Apache Daffodil VS Code
> > >> 1.3.0-rc2.
> > >>
> > >> All distribution packages, including signatures, digests, etc. can be
> > >> found at:
> > >>
> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
> > >>
> > >> This release has been signed with PGP key
> > >> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
> > >> to shanedell@apache.org, which is included in the KEYS file here:
> > >> https://downloads.apache.org/daffodil/KEYS
> > >>
> > >> The release candidate has been tagged in git with 1.3.0-rc2. The
> binaries
> > >>
> > >> For reference, here is a list of all closed GitHub issues tagged with
> > >> 1.3.0:
> > >> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
> > >>
> > >> Please review and vote. The vote will be open for at least 72 hours
> > >> (Friday, 19 May 2023, 1:30pm EST).
> > >>
> > >> [ ] +1 approve
> > >> [ ] +0 no opinion
> > >> [ ] -1 disapprove (and reason why)
> > >>
> > >> Documentation for 1.3.0 can be found here
> > >> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
> > >> <
> > >>
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
> > >> >.
> > >>
> > >> Please note one of the large features that was focused on in 1.3.0 was
> > >> improving the data editor. To open the data editor you will need to
> open
> > >> the
> > >> command palette using:
> > >>
> > >> - Ctrl + Shift + P (windows/linux)
> > >> - Command + Shift + P (mac)
> > >>
> > >> Then typing "data.edit", make sure "Daffodil Debug: Data Editor" is
> > >> selected
> > >> and then hitting enter. By default the data editor will run on port
> 9000
> > >> of
> > >> your machine. However, if you want to use a different port follow
> steps
> > >> here
> > >> Data Editor Launch Settings
> > >> <
> > >>
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
> > >> >
> > >>
> > >> Thank you,
> > >>
> > >> - Shane Dell
> > >>
> > >
> >
>
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Shane Dell <sh...@apache.org>.
Mike,
I am not sure why you would be seeing this. I am able to set the breakpoint where you
mentioned without any issues. Can you try removing the daffodil-dap folder? This should be
somewhere like:
/home/$USER/.local/share/daffodil-dap
The issue I am thinking is possibly a bug with the rc1 debugger that may not be present on rc2.
But if the folder daffodil-dap/daffodil-debugger-3.4.0-1.3.0/ already exists from rc1 then rc2 will
not remake it. So in order to remake it you have to manually remove this folder. Let me know if
this helps.
Thank you,
Shane Dell
On 2023/05/24 15:15:33 Mike Beckerle wrote:
> I have been able to continue evaluating RC2.
>
> My vote is -1
>
> My testing steps listed below. I am trying to debug PCAP. I have a broken
> icmp packet (bad magic number), and also just step through a correct icmp
> packet.
>
> I am trying to debug icmp.badMagicNum.cap data file.
>
> I should get a failure of an assert because the magic number is not one of
> the expected values.
>
> I'm just trying to single-step through it until it hits the error.
>
> Once we get to the MagicNumber element, it doesn't position on the assert
> while evaluating that expression or even on the element carrying the assert.
> It jumps back to the top-level xs:schema position.
>
> I continue to click single step. Watching for the error to be displayed in
> the upper left under Variables > Parse > Diagnostics > Errors which is
> where I think it ought to go.
>
> At some point the failure is somehow detected as the debug session ends. A
> dialog offers to open /tmp/infoset.xml for me, which I open and it is empty
> of course.
>
> Nothing displayed the error message from the assert, or even indicated what
> went wrong at all. Not the state display in the upper left, not the
> terminal, nor debug console nor output.
>
> The upper left box with "Variables" is cleared at the end. It should
> certainly not be cleared at the end of a run when final state may be of
> interest. At the start of a run, or manually, but not at the end of a run.
>
> Second attempt to use it:
>
> I create a second launch config this time for what should be a successful
> parse of icmp1.cap
>
> As I single step, the position of the cursor on the schema is jumping
> around all over the place back and forth between a containing element and
> the child elements it contains.
> The infoset display does not get updated until long after the display has
> moved on and is positioned on the next element.
>
> This jumping around gets completely out of hand once you step into the
> complexType "Ethernet" which is in the ethernetIP jar file.
>
> At that point, after each child of Ethernet, it jumps back to the Ethernet
> element in the pcap.dfdl.xsd file, and then back into the
> ethernetIP.dfdl.xsd for the next child, then back to the pcap.dfdl.xsd
> file, etc.
>
> All this jumping around basically makes stepping useless. So I decide to
> put a breakpoint on the Version element inside the IPv4Header element
> inside ethernetIP.dfdl.xsd.
>
> Just adding the breakpoint causes the debug session to end with an NPE in
> the dap setBreakpoint.
>
> ERROR o.a.d.d.d.DAPodil - unhandled error
> java.lang.NullPointerException: null
> at java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> at
> java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> at java.base/java.nio.file.Path.of(Path.java:147)
> at java.base/java.nio.file.Paths.get(Paths.java:69)
> at
> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> at flatMap @
> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> at handleErrorWith @
> fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> at flatMap @ fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> 2023-05-24 11:09:10,717 [io-compute-1] DEBUG o.a.d.d.d.DAPodil - whenDone:
> completed
> java.lang.NullPointerException
> at java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
> at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
> at
> java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
> at java.base/java.nio.file.Path.of(Path.java:147)
> at java.base/java.nio.file.Paths.get(Paths.java:69)
> at
> org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
> at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> at flatMap @
> org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> at handleErrorWith @
> fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
> at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
> at flatMap @ fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
> at get @ fs2.internal.Scope.openScope(Scope.scala:281)
> at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Tue, May 23, 2023 at 5:19 PM Mike Beckerle <mb...@apache.org> wrote:
>
> > I really am trying to eval the RC2.
> >
> > Right now my VMWare Linux system has been broken by recent updates to the
> > MS Windows host environment.
> >
> > I don't exactly understand this, but the VSCode front end can no longer
> > connect to the server at localhost:4711. I get ECONNREFUSED. This started
> > when a bunch of MS-Windows updates were done on the host OS. I run VSCode
> > on the linux client machine, which lives behind a NAT firewall created by
> > VMWare workstation - so I really don't get how these could be interacting
> > this way, but... there it is.
> >
> > I will switch back to my standalone linux PC if this isn't resolved today.
> >
> >
> > On Tue, May 16, 2023 at 1:24 PM Shane Dell <sh...@apache.org> wrote:
> >
> >> Hello all,I'd like to call a vote to release Apache Daffodil VS Code
> >> 1.3.0-rc2.
> >>
> >> All distribution packages, including signatures, digests, etc. can be
> >> found at:
> >> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
> >>
> >> This release has been signed with PGP key
> >> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
> >> to shanedell@apache.org, which is included in the KEYS file here:
> >> https://downloads.apache.org/daffodil/KEYS
> >>
> >> The release candidate has been tagged in git with 1.3.0-rc2. The binaries
> >>
> >> For reference, here is a list of all closed GitHub issues tagged with
> >> 1.3.0:
> >> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
> >>
> >> Please review and vote. The vote will be open for at least 72 hours
> >> (Friday, 19 May 2023, 1:30pm EST).
> >>
> >> [ ] +1 approve
> >> [ ] +0 no opinion
> >> [ ] -1 disapprove (and reason why)
> >>
> >> Documentation for 1.3.0 can be found here
> >> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
> >> <
> >> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
> >> >.
> >>
> >> Please note one of the large features that was focused on in 1.3.0 was
> >> improving the data editor. To open the data editor you will need to open
> >> the
> >> command palette using:
> >>
> >> - Ctrl + Shift + P (windows/linux)
> >> - Command + Shift + P (mac)
> >>
> >> Then typing "data.edit", make sure "Daffodil Debug: Data Editor" is
> >> selected
> >> and then hitting enter. By default the data editor will run on port 9000
> >> of
> >> your machine. However, if you want to use a different port follow steps
> >> here
> >> Data Editor Launch Settings
> >> <
> >> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
> >> >
> >>
> >> Thank you,
> >>
> >> - Shane Dell
> >>
> >
>
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Mike Beckerle <mb...@apache.org>.
I have been able to continue evaluating RC2.
My vote is -1
My testing steps listed below. I am trying to debug PCAP. I have a broken
icmp packet (bad magic number), and also just step through a correct icmp
packet.
I am trying to debug icmp.badMagicNum.cap data file.
I should get a failure of an assert because the magic number is not one of
the expected values.
I'm just trying to single-step through it until it hits the error.
Once we get to the MagicNumber element, it doesn't position on the assert
while evaluating that expression or even on the element carrying the assert.
It jumps back to the top-level xs:schema position.
I continue to click single step. Watching for the error to be displayed in
the upper left under Variables > Parse > Diagnostics > Errors which is
where I think it ought to go.
At some point the failure is somehow detected as the debug session ends. A
dialog offers to open /tmp/infoset.xml for me, which I open and it is empty
of course.
Nothing displayed the error message from the assert, or even indicated what
went wrong at all. Not the state display in the upper left, not the
terminal, nor debug console nor output.
The upper left box with "Variables" is cleared at the end. It should
certainly not be cleared at the end of a run when final state may be of
interest. At the start of a run, or manually, but not at the end of a run.
Second attempt to use it:
I create a second launch config this time for what should be a successful
parse of icmp1.cap
As I single step, the position of the cursor on the schema is jumping
around all over the place back and forth between a containing element and
the child elements it contains.
The infoset display does not get updated until long after the display has
moved on and is positioned on the next element.
This jumping around gets completely out of hand once you step into the
complexType "Ethernet" which is in the ethernetIP jar file.
At that point, after each child of Ethernet, it jumps back to the Ethernet
element in the pcap.dfdl.xsd file, and then back into the
ethernetIP.dfdl.xsd for the next child, then back to the pcap.dfdl.xsd
file, etc.
All this jumping around basically makes stepping useless. So I decide to
put a breakpoint on the Version element inside the IPv4Header element
inside ethernetIP.dfdl.xsd.
Just adding the breakpoint causes the debug session to end with an NPE in
the dap setBreakpoint.
ERROR o.a.d.d.d.DAPodil - unhandled error
java.lang.NullPointerException: null
at java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
at
java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
at java.base/java.nio.file.Path.of(Path.java:147)
at java.base/java.nio.file.Paths.get(Paths.java:69)
at
org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
at get @ fs2.internal.Scope.openScope(Scope.scala:281)
at flatMap @
org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at handleErrorWith @
fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at get @ fs2.internal.Scope.openScope(Scope.scala:281)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
at get @ fs2.internal.Scope.openScope(Scope.scala:281)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
at get @ fs2.internal.Scope.openScope(Scope.scala:281)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
2023-05-24 11:09:10,717 [io-compute-1] DEBUG o.a.d.d.d.DAPodil - whenDone:
completed
java.lang.NullPointerException
at java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75)
at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69)
at
java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
at java.base/java.nio.file.Path.of(Path.java:147)
at java.base/java.nio.file.Paths.get(Paths.java:69)
at
org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267)
at get @ fs2.internal.Scope.openScope(Scope.scala:281)
at flatMap @
org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at handleErrorWith @
fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at get @ fs2.internal.Scope.openScope(Scope.scala:281)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057)
at get @ fs2.internal.Scope.openScope(Scope.scala:281)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929)
at get @ fs2.internal.Scope.openScope(Scope.scala:281)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
On Tue, May 23, 2023 at 5:19 PM Mike Beckerle <mb...@apache.org> wrote:
> I really am trying to eval the RC2.
>
> Right now my VMWare Linux system has been broken by recent updates to the
> MS Windows host environment.
>
> I don't exactly understand this, but the VSCode front end can no longer
> connect to the server at localhost:4711. I get ECONNREFUSED. This started
> when a bunch of MS-Windows updates were done on the host OS. I run VSCode
> on the linux client machine, which lives behind a NAT firewall created by
> VMWare workstation - so I really don't get how these could be interacting
> this way, but... there it is.
>
> I will switch back to my standalone linux PC if this isn't resolved today.
>
>
> On Tue, May 16, 2023 at 1:24 PM Shane Dell <sh...@apache.org> wrote:
>
>> Hello all,I'd like to call a vote to release Apache Daffodil VS Code
>> 1.3.0-rc2.
>>
>> All distribution packages, including signatures, digests, etc. can be
>> found at:
>> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
>>
>> This release has been signed with PGP key
>> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
>> to shanedell@apache.org, which is included in the KEYS file here:
>> https://downloads.apache.org/daffodil/KEYS
>>
>> The release candidate has been tagged in git with 1.3.0-rc2. The binaries
>>
>> For reference, here is a list of all closed GitHub issues tagged with
>> 1.3.0:
>> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
>>
>> Please review and vote. The vote will be open for at least 72 hours
>> (Friday, 19 May 2023, 1:30pm EST).
>>
>> [ ] +1 approve
>> [ ] +0 no opinion
>> [ ] -1 disapprove (and reason why)
>>
>> Documentation for 1.3.0 can be found here
>> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
>> <
>> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
>> >.
>>
>> Please note one of the large features that was focused on in 1.3.0 was
>> improving the data editor. To open the data editor you will need to open
>> the
>> command palette using:
>>
>> - Ctrl + Shift + P (windows/linux)
>> - Command + Shift + P (mac)
>>
>> Then typing "data.edit", make sure "Daffodil Debug: Data Editor" is
>> selected
>> and then hitting enter. By default the data editor will run on port 9000
>> of
>> your machine. However, if you want to use a different port follow steps
>> here
>> Data Editor Launch Settings
>> <
>> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
>> >
>>
>> Thank you,
>>
>> - Shane Dell
>>
>
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Mike Beckerle <mb...@apache.org>.
I really am trying to eval the RC2.
Right now my VMWare Linux system has been broken by recent updates to the
MS Windows host environment.
I don't exactly understand this, but the VSCode front end can no longer
connect to the server at localhost:4711. I get ECONNREFUSED. This started
when a bunch of MS-Windows updates were done on the host OS. I run VSCode
on the linux client machine, which lives behind a NAT firewall created by
VMWare workstation - so I really don't get how these could be interacting
this way, but... there it is.
I will switch back to my standalone linux PC if this isn't resolved today.
On Tue, May 16, 2023 at 1:24 PM Shane Dell <sh...@apache.org> wrote:
> Hello all,I'd like to call a vote to release Apache Daffodil VS Code
> 1.3.0-rc2.
>
> All distribution packages, including signatures, digests, etc. can be
> found at:
> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
>
> This release has been signed with PGP key
> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
> to shanedell@apache.org, which is included in the KEYS file here:
> https://downloads.apache.org/daffodil/KEYS
>
> The release candidate has been tagged in git with 1.3.0-rc2. The binaries
>
> For reference, here is a list of all closed GitHub issues tagged with
> 1.3.0:
> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
>
> Please review and vote. The vote will be open for at least 72 hours
> (Friday, 19 May 2023, 1:30pm EST).
>
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>
> Documentation for 1.3.0 can be found here
> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
> <
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
> >.
>
> Please note one of the large features that was focused on in 1.3.0 was
> improving the data editor. To open the data editor you will need to open
> the
> command palette using:
>
> - Ctrl + Shift + P (windows/linux)
> - Command + Shift + P (mac)
>
> Then typing "data.edit", make sure "Daffodil Debug: Data Editor" is
> selected
> and then hitting enter. By default the data editor will run on port 9000 of
> your machine. However, if you want to use a different port follow steps
> here
> Data Editor Launch Settings
> <
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
> >
>
> Thank you,
>
> - Shane Dell
>
Re: [VOTE] Apache Daffodil VS Code Extension 1.3.0-rc2
Posted by Mike Beckerle <mb...@apache.org>.
I need more time. I won't be able to get to this before Monday.
On Tue, May 16, 2023 at 1:24 PM Shane Dell <sh...@apache.org> wrote:
> Hello all,I'd like to call a vote to release Apache Daffodil VS Code
> 1.3.0-rc2.
>
> All distribution packages, including signatures, digests, etc. can be
> found at:
> https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2
>
> This release has been signed with PGP key
> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding
> to shanedell@apache.org, which is included in the KEYS file here:
> https://downloads.apache.org/daffodil/KEYS
>
> The release candidate has been tagged in git with 1.3.0-rc2. The binaries
>
> For reference, here is a list of all closed GitHub issues tagged with
> 1.3.0:
> https://github.com/apache/daffodil-vscode/milestone/4?closed=1
>
> Please review and vote. The vote will be open for at least 72 hours
> (Friday, 19 May 2023, 1:30pm EST).
>
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and reason why)
>
> Documentation for 1.3.0 can be found here
> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki
> <
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0
> >.
>
> Please note one of the large features that was focused on in 1.3.0 was
> improving the data editor. To open the data editor you will need to open
> the
> command palette using:
>
> - Ctrl + Shift + P (windows/linux)
> - Command + Shift + P (mac)
>
> Then typing "data.edit", make sure "Daffodil Debug: Data Editor" is
> selected
> and then hitting enter. By default the data editor will run on port 9000 of
> your machine. However, if you want to use a different port follow steps
> here
> Data Editor Launch Settings
> <
> https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings
> >
>
> Thank you,
>
> - Shane Dell
>