You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by Shane Dell <sh...@apache.org> on 2022/07/19 17:44:59 UTC
Daffodil Debugger Scala Source Code
All,
This thread is to discuss splitting out the code for the Apache Daffodil
Scala Debugger from the apache/daffodil-vscode repository. The two options
would be:
- 1.) Move the debugger source code into the apache/daffodil repo. However,
will this work because the debugger code depends on certain daffodil Scala
libraries to be published? This is mostly an option since both are
mostly/fully Scala based.
- 2.) Create a new repo (apache/daffodil-debugger?,
apache/daffodil-debugger-scala?) where the Scala code will live.
This would be helpful as the apache/daffodil-vscode repo is heavily based
on creating the VS Code extension for Daffodil. With this the debugger
source code is rarely updated, when it is they are pretty minor fixes or
dependency updates. Currently it is a bit of a cluster having a mix of
node/JavaScript/TypeScript and Scala which causes an issue with dependency
bots checking as these are checking for both npm and sbt/Scala
dependencies, causing many PRs to be made. The extension code also runs a
sbt universal:packageBin command in multiple occurences, being able to
remove this and downloading the debugger package would simplify a couple
different processes for the extension.
My personal vote is for creating a new repo called something like either
apache/daffodil-debugger or apache/daffodil-debugger-scala.
Sincerely,
Shane Dell
Re: Daffodil Debugger Scala Source Code
Posted by Mike Beckerle <mb...@apache.org>.
We will also need some tests for this mode to insure we haven't broken it.
A test rig would be needed of some sort.
On Tue, Jul 19, 2022 at 2:15 PM Steve Lawrence <sl...@apache.org> wrote:
> +1 to this
>
> I was thinking the same thing as I read this. My only thought is using
> "daffodil dap" since the debug server is just a thing that implements
> the DAP protocol, and theoretically it could be used any any IDE that
> supports DAP (I think?). But the name is the easy part.
>
> My only other (minor) concern is that this pulls in a number of
> dependencies. It's not an issue from a licensing perspective since we
> know they are fine, but it does make the CLI release a bit bigger and
> more to manage, e.g. lots of Cats jars. I suspect it's not enough to
> really matter (and most of the size is probably daffodil jars), but if
> we do go this route, during or prior to migration it might be worth
> looking to see if anything can be replaced. For example, one thing that
> jumps out is VSCode debugger currently depends on logback for logging,
> but daffodil use log4j.
>
> The other concern is that currently the VS Code stuff can sort of
> develop at its own pace. If we integrate the DAP portion into the CLI,
> it means VSCode can't get DAP updates without a Daffodil release. But it
> seems DAP is fairly stable, and as long as we keep a quick release cycle
> and plan accordingly, that shouldn't be too much of an issue.
>
> I would perfer this approach over creating a new repo. It's a bit less
> to keep track of, and makes it easier to ensure we don't have any
> breaking changes with changes to Daffodil.
>
> On 7/19/22 1:52 PM, Mike Beckerle wrote:
> > I think of this as a daffodil server mode, for the front end VSCode stuff
> > to use.
> >
> > So, is it plausable to add the code to daffodil proper. Make it a CLI
> > command mode to start up this server, so that
> >
> > daffodil vscodeServer
> >
> > starts it, optionally with connection options like what ports to use,
> etc.
> >
> >
> > On Tue, Jul 19, 2022 at 1:45 PM Shane Dell <sh...@apache.org> wrote:
> >
> >> All,
> >>
> >> This thread is to discuss splitting out the code for the Apache Daffodil
> >> Scala Debugger from the apache/daffodil-vscode repository. The two
> options
> >> would be:
> >>
> >> - 1.) Move the debugger source code into the apache/daffodil repo.
> However,
> >> will this work because the debugger code depends on certain daffodil
> Scala
> >> libraries to be published? This is mostly an option since both are
> >> mostly/fully Scala based.
> >> - 2.) Create a new repo (apache/daffodil-debugger?,
> >> apache/daffodil-debugger-scala?) where the Scala code will live.
> >>
> >> This would be helpful as the apache/daffodil-vscode repo is heavily
> based
> >> on creating the VS Code extension for Daffodil. With this the debugger
> >> source code is rarely updated, when it is they are pretty minor fixes or
> >> dependency updates. Currently it is a bit of a cluster having a mix of
> >> node/JavaScript/TypeScript and Scala which causes an issue with
> dependency
> >> bots checking as these are checking for both npm and sbt/Scala
> >> dependencies, causing many PRs to be made. The extension code also runs
> a
> >> sbt universal:packageBin command in multiple occurences, being able to
> >> remove this and downloading the debugger package would simplify a couple
> >> different processes for the extension.
> >>
> >> My personal vote is for creating a new repo called something like either
> >> apache/daffodil-debugger or apache/daffodil-debugger-scala.
> >>
> >> Sincerely,
> >>
> >> Shane Dell
> >>
> >
>
>
Re: Daffodil Debugger Scala Source Code
Posted by "Shearer, Davin" <sh...@ctc.com>.
Just piling on with another +1 to the idea of splitting the daffodil debugger server from the VSCode extension.
The server is more mature / stable versus the heavy development that is occurring on the extension. The extension leverages the server as a loosely coupled dependency service (using DAP). Consuming it as a dependency from the extension repo will be helpful and is a pattern we're already using for Ωedit integration, another loosely coupled dependency service (using gRPC).
I agree with Steve on most of what he's said, though I'm more in the separate repo camp because it allows us to independently maintain, develop, test, version, and release the debugger service outside of the established daffodil development cycle. There are benefits to developing in layers, and using loose coupling between layers. There are also appreciable downsides to having "yet another" daffodil repo that needs to be stood up and maintained. A positive to having it live in the main daffodil repo is also the additional exposure it will get. I recognize that this is additional burden on Mike, Steve, and ASF, so I defer out of fairness.
I'm in favor of ""daffodil-debugger ", instead of "daffodil-debugger-scala" as I don't think anyone should care about the programming language used to develop the server, since DAP is the interface being used, not language bindings.
-Davin
On 7/19/22, 2:16 PM, "Steve Lawrence" <sl...@apache.org> wrote:
+1 to this
I was thinking the same thing as I read this. My only thought is using
"daffodil dap" since the debug server is just a thing that implements
the DAP protocol, and theoretically it could be used any any IDE that
supports DAP (I think?). But the name is the easy part.
My only other (minor) concern is that this pulls in a number of
dependencies. It's not an issue from a licensing perspective since we
know they are fine, but it does make the CLI release a bit bigger and
more to manage, e.g. lots of Cats jars. I suspect it's not enough to
really matter (and most of the size is probably daffodil jars), but if
we do go this route, during or prior to migration it might be worth
looking to see if anything can be replaced. For example, one thing that
jumps out is VSCode debugger currently depends on logback for logging,
but daffodil use log4j.
The other concern is that currently the VS Code stuff can sort of
develop at its own pace. If we integrate the DAP portion into the CLI,
it means VSCode can't get DAP updates without a Daffodil release. But it
seems DAP is fairly stable, and as long as we keep a quick release cycle
and plan accordingly, that shouldn't be too much of an issue.
I would perfer this approach over creating a new repo. It's a bit less
to keep track of, and makes it easier to ensure we don't have any
breaking changes with changes to Daffodil.
On 7/19/22 1:52 PM, Mike Beckerle wrote:
> I think of this as a daffodil server mode, for the front end VSCode stuff
> to use.
>
> So, is it plausable to add the code to daffodil proper. Make it a CLI
> command mode to start up this server, so that
>
> daffodil vscodeServer
>
> starts it, optionally with connection options like what ports to use, etc.
>
>
> On Tue, Jul 19, 2022 at 1:45 PM Shane Dell <sh...@apache.org> wrote:
>
>> All,
>>
>> This thread is to discuss splitting out the code for the Apache Daffodil
>> Scala Debugger from the apache/daffodil-vscode repository. The two options
>> would be:
>>
>> - 1.) Move the debugger source code into the apache/daffodil repo. However,
>> will this work because the debugger code depends on certain daffodil Scala
>> libraries to be published? This is mostly an option since both are
>> mostly/fully Scala based.
>> - 2.) Create a new repo (apache/daffodil-debugger?,
>> apache/daffodil-debugger-scala?) where the Scala code will live.
>>
>> This would be helpful as the apache/daffodil-vscode repo is heavily based
>> on creating the VS Code extension for Daffodil. With this the debugger
>> source code is rarely updated, when it is they are pretty minor fixes or
>> dependency updates. Currently it is a bit of a cluster having a mix of
>> node/JavaScript/TypeScript and Scala which causes an issue with dependency
>> bots checking as these are checking for both npm and sbt/Scala
>> dependencies, causing many PRs to be made. The extension code also runs a
>> sbt universal:packageBin command in multiple occurences, being able to
>> remove this and downloading the debugger package would simplify a couple
>> different processes for the extension.
>>
>> My personal vote is for creating a new repo called something like either
>> apache/daffodil-debugger or apache/daffodil-debugger-scala.
>>
>> Sincerely,
>>
>> Shane Dell
>>
>
-----------------------------------------------------------------
This message and any files transmitted within are intended
solely for the addressee or its representative and may contain
company proprietary information. If you are not the intended
recipient, notify the sender immediately and delete this
message. Publication, reproduction, forwarding, or content
disclosure is prohibited without the consent of the original
sender and may be unlawful.
Concurrent Technologies Corporation and its Affiliates.
www.ctc.com 1-800-282-4392
-----------------------------------------------------------------
Re: Daffodil Debugger Scala Source Code
Posted by "Shearer, Davin" <sh...@ctc.com>.
Hi Steve,
I don’t think anyone stated or implied that hosting the debugger service in separate repo would be trivial (it's not), but it _is_ a solved problem and a frequently used development pattern.
We can begin detangling the debugger from the daffoldil-vscode repo and relocate it to its new home (wherever it will be) after we release 1.2.0 of the VSCode extension, which we're pushing hard to have done before the end of the month.
On 7/20/22, 9:57 AM, "Steve Lawrence" <sl...@apache.org> wrote:
I don't think we necessarily need to support "daffodil dap". I think
this is just a nice side effect of bundling everything in the main repo.
But if it poses technical challenges I don't think it's worth it.
I would like us to at least have an idea about how release/publish/VS
Code integration will work if we do separate repos. It doesn't sound
trivial to me.
On 7/20/22 9:39 AM, Interrante, John A (GE Research, US) wrote:
> The motivation for putting dap into the main daffodil repo is that bundling dap into the daffodil cli (allowing "daffodil dap" to start dap) will simplify configuring the VS Code debugger extension to work with different daffodil and dap versions.
>
> I suggest that there is almost certainly a way to put dap into its own repo while still allowing "daffodil dap" to start a compatible dap version. I will let others figure out the most optimal method, but I don't see any technical obstacle that would prevent "daffodil dap" from starting a compatible dap version and therefore simplifying configuring the VS Code debugger extension.
>
> -----Original Message-----
> From: Mike Beckerle <mb...@apache.org>
> Sent: Wednesday, July 20, 2022 9:05 AM
> To: dev@daffodil.apache.org
> Subject: EXT: Re: Daffodil Debugger Scala Source Code
>
> WARNING: This email originated from outside of GE. Please validate the sender's email address before clicking on links or attachments as they may not be safe.
>
> I think putting dap into the main daffodil repo is a false economy.
> Consider:
>
> 1) more sbt hacks to exclude the tests of the dap from 'sbt test' and to add an alternative 'sbt dap:test' which runs those tests.
> 2) that won't work with IDEs automatically, so they will be unable to run the dap tests without renaming directories or such.
> (CLI has this same problem today. I do not yet have a convenient workaround for this, and it is a pain in the neck routinely)
> 3) another module inside daffodil to compile when 'sbt compile' is issued.
> So more delay in the routine edit-compile-debug cycle.
> 4) more IDE customizations to save that extra compilation time when doing IDE builds.
>
> The above is a huge hassle to maintain compared with one more repo that is completely ordinary with respect to build configuration, IDE support, etc.
>
> Creating a separate repo has none of these drawbacks. 'sbt test' can work normally to test it. Tests live in src/test as normal. No need for special exclusions or sbt hackery.
> IDEs will do the right thing by default.
>
>
>
>
>
>
>
>
> On Tue, Jul 19, 2022 at 4:51 PM Steve Lawrence <sl...@apache.org> wrote:
>
>> DAP stands for Debug Adapter Protocol:
>>
>> https://microsoft.github.io/debug-adapter-protocol/
>>
>> If the goal is to simplify development, then including the dap stuff
>> in the main repo with the CLI would probably make things the simplest.
>> Just
>> run:
>>
>> sbt daffodil-cli/stage
>>
>> in the daffodil repo, and then point VS Code to execute "daffodil dap
>> .." using the staged files (i.e.
>> daffodil-cli/target/universal/stage/bin/daffodil). We may need changes
>> in the VS Code to support alternative locations of the daffodil
>> binary, but I think that's something we want to support anyways to
>> allow testing different versions of Daffodil without having to install
>> a different extension.
>>
>> On 7/19/22 4:10 PM, Mike Beckerle wrote:
>>> I am leaning toward a separate repo (called daffodil-dap-server)
>>> which
>> uses
>>> (depends on) daffodil.
>>> (btw: what does DAP stand for again? Daffodil Access Protocol ?)
>>>
>>> This is for separation of concerns. I'd like daffodil proper
>>> libraries to be about DFDL mostly, though it does contain the basic
>>> CLI. Things having to do with client/server and testing of the
>>> server side in isolation,
>> etc.
>>> all that belongs only in the daffodil-dap-server which people fixing
>>> ordinary DFDL/Daffodil bugs don't test when they run 'sbt test'.
>>>
>>> I do think the server is going to be quite coupled to Daffodil internals.
>>> For example, to provide better visual animation of delimiter
>>> scanning we will need new debug hooks providing visibility or
>>> callbacks from those parse operations, and DAP will then want to
>>> make this available to the vscode front end for proper
>>> visualization. Similar story is needed around animating what goes on
>>> with unparsing and suspend/resume of expressions, etc. I expect we
>>> will create internal-only versions of these features before we try to firm up any published APIs for them.
>>>
>>> It is pretty important for me to be able to easily and quickly point
>> VSCode
>>> at a new DAP server that is depending on a locally updated daffodil.
>>> But I think something like this is sufficiently easy:
>>>
>>> ... fix a bug in daffodil then
>>> cd daffodil
>>> sbt test
>>> sbt publishLocal // creates a snapshot such as 4.2.1-SNAPSHOT
>>> cd dap-server
>>> sbt test // runs test rig to check out DAP server in isolation
>>> from front end // has dependency on daffodil 4.2.1-SNAPSHOT
>>> sbt package // or publishLocal or... // has dependency on
>>> daffodil 4.2.1-SNAPSHOT
>>> ... then start VSCode using updated configuration file giving
>> dap-server
>>> package location
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Jul 19, 2022 at 3:13 PM Steve Lawrence
>>> <sl...@apache.org>
>> wrote:
>>>
>>>> Some other positives if DAP were combined it into the existing CLI:
>>>>
>>>> There was once talk of making it so the the VS Code debugger could
>>>> switch between different Daffodil versions. If it were part of the
>>>> CLI, this just means downloading a new version of the Daffodil CLI
>>>> and pointing VS Code to it--that process could either be manual or
>>>> automated in the VS Code settings.
>>>>
>>>> I think this would also make it easier to test Daffodil snapshots
>>>> with the VSCode extension? I believe the current process requires
>>>> you to publishLocal a Daffodil snapshot, then update the
>>>> daffodil-vscode repo (or if we split it, the daffodil-debugger
>>>> repo) to have the snapshot version, and the build that. If it was
>>>> all part of the CLI, you could just run "sbt stage" in the Daffodil
>>>> repo and point VSCode to those staged files. It only avoids a
>>>> couple extra steps, but if you do it frequently it would be nice.
>>>>
>>>> On 7/19/22 2:15 PM, Steve Lawrence wrote:
>>>>> +1 to this
>>>>>
>>>>> I was thinking the same thing as I read this. My only thought is
>>>>> using "daffodil dap" since the debug server is just a thing that
>>>>> implements the DAP protocol, and theoretically it could be used
>>>>> any any IDE that supports DAP (I think?). But the name is the easy part.
>>>>>
>>>>> My only other (minor) concern is that this pulls in a number of
>>>>> dependencies. It's not an issue from a licensing perspective since
>>>>> we know they are fine, but it does make the CLI release a bit
>>>>> bigger and more to manage, e.g. lots of Cats jars. I suspect it's
>>>>> not enough to really matter (and most of the size is probably
>>>>> daffodil jars), but if we do go this route, during or prior to
>>>>> migration it might be worth looking to see if anything can be
>>>>> replaced. For example, one thing that jumps out is VSCode debugger
>>>>> currently depends on logback for logging, but daffodil use log4j.
>>>>>
>>>>> The other concern is that currently the VS Code stuff can sort of
>>>>> develop at its own pace. If we integrate the DAP portion into the
>>>>> CLI, it means VSCode can't get DAP updates without a Daffodil
>>>>> release. But
>> it
>>>>> seems DAP is fairly stable, and as long as we keep a quick release
>> cycle
>>>>> and plan accordingly, that shouldn't be too much of an issue.
>>>>>
>>>>> I would perfer this approach over creating a new repo. It's a bit
>>>>> less to keep track of, and makes it easier to ensure we don't have
>>>>> any breaking changes with changes to Daffodil.
>>>>>
>>>>> On 7/19/22 1:52 PM, Mike Beckerle wrote:
>>>>>> I think of this as a daffodil server mode, for the front end
>>>>>> VSCode
>>>> stuff
>>>>>> to use.
>>>>>>
>>>>>> So, is it plausable to add the code to daffodil proper. Make it a
>>>>>> CLI command mode to start up this server, so that
>>>>>>
>>>>>> daffodil vscodeServer
>>>>>>
>>>>>> starts it, optionally with connection options like what ports to
>>>>>> use, etc.
>>>>>>
>>>>>>
>>>>>> On Tue, Jul 19, 2022 at 1:45 PM Shane Dell <sh...@apache.org>
>>>> wrote:
>>>>>>
>>>>>>> All,
>>>>>>>
>>>>>>> This thread is to discuss splitting out the code for the Apache
>>>> Daffodil
>>>>>>> Scala Debugger from the apache/daffodil-vscode repository. The
>>>>>>> two options would be:
>>>>>>>
>>>>>>> - 1.) Move the debugger source code into the apache/daffodil repo.
>>>>>>> However,
>>>>>>> will this work because the debugger code depends on certain
>>>>>>> daffodil Scala libraries to be published? This is mostly an
>>>>>>> option since both are mostly/fully Scala based.
>>>>>>> - 2.) Create a new repo (apache/daffodil-debugger?,
>>>>>>> apache/daffodil-debugger-scala?) where the Scala code will live.
>>>>>>>
>>>>>>> This would be helpful as the apache/daffodil-vscode repo is
>>>>>>> heavily based on creating the VS Code extension for Daffodil.
>>>>>>> With this the
>> debugger
>>>>>>> source code is rarely updated, when it is they are pretty minor
>>>>>>> fixes
>>>> or
>>>>>>> dependency updates. Currently it is a bit of a cluster having a
>>>>>>> mix
>> of
>>>>>>> node/JavaScript/TypeScript and Scala which causes an issue with
>>>>>>> dependency bots checking as these are checking for both npm and
>>>>>>> sbt/Scala dependencies, causing many PRs to be made. The
>>>>>>> extension code also runs a sbt universal:packageBin command in
>>>>>>> multiple occurences, being able
>> to
>>>>>>> remove this and downloading the debugger package would simplify
>>>>>>> a
>>>> couple
>>>>>>> different processes for the extension.
>>>>>>>
>>>>>>> My personal vote is for creating a new repo called something
>>>>>>> like
>>>> either
>>>>>>> apache/daffodil-debugger or apache/daffodil-debugger-scala.
>>>>>>>
>>>>>>> Sincerely,
>>>>>>>
>>>>>>> Shane Dell
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
-----------------------------------------------------------------
This message and any files transmitted within are intended
solely for the addressee or its representative and may contain
company proprietary information. If you are not the intended
recipient, notify the sender immediately and delete this
message. Publication, reproduction, forwarding, or content
disclosure is prohibited without the consent of the original
sender and may be unlawful.
Concurrent Technologies Corporation and its Affiliates.
www.ctc.com 1-800-282-4392
-----------------------------------------------------------------
Re: Daffodil Debugger Scala Source Code
Posted by Steve Lawrence <sl...@apache.org>.
I don't think we necessarily need to support "daffodil dap". I think
this is just a nice side effect of bundling everything in the main repo.
But if it poses technical challenges I don't think it's worth it.
I would like us to at least have an idea about how release/publish/VS
Code integration will work if we do separate repos. It doesn't sound
trivial to me.
On 7/20/22 9:39 AM, Interrante, John A (GE Research, US) wrote:
> The motivation for putting dap into the main daffodil repo is that bundling dap into the daffodil cli (allowing "daffodil dap" to start dap) will simplify configuring the VS Code debugger extension to work with different daffodil and dap versions.
>
> I suggest that there is almost certainly a way to put dap into its own repo while still allowing "daffodil dap" to start a compatible dap version. I will let others figure out the most optimal method, but I don't see any technical obstacle that would prevent "daffodil dap" from starting a compatible dap version and therefore simplifying configuring the VS Code debugger extension.
>
> -----Original Message-----
> From: Mike Beckerle <mb...@apache.org>
> Sent: Wednesday, July 20, 2022 9:05 AM
> To: dev@daffodil.apache.org
> Subject: EXT: Re: Daffodil Debugger Scala Source Code
>
> WARNING: This email originated from outside of GE. Please validate the sender's email address before clicking on links or attachments as they may not be safe.
>
> I think putting dap into the main daffodil repo is a false economy.
> Consider:
>
> 1) more sbt hacks to exclude the tests of the dap from 'sbt test' and to add an alternative 'sbt dap:test' which runs those tests.
> 2) that won't work with IDEs automatically, so they will be unable to run the dap tests without renaming directories or such.
> (CLI has this same problem today. I do not yet have a convenient workaround for this, and it is a pain in the neck routinely)
> 3) another module inside daffodil to compile when 'sbt compile' is issued.
> So more delay in the routine edit-compile-debug cycle.
> 4) more IDE customizations to save that extra compilation time when doing IDE builds.
>
> The above is a huge hassle to maintain compared with one more repo that is completely ordinary with respect to build configuration, IDE support, etc.
>
> Creating a separate repo has none of these drawbacks. 'sbt test' can work normally to test it. Tests live in src/test as normal. No need for special exclusions or sbt hackery.
> IDEs will do the right thing by default.
>
>
>
>
>
>
>
>
> On Tue, Jul 19, 2022 at 4:51 PM Steve Lawrence <sl...@apache.org> wrote:
>
>> DAP stands for Debug Adapter Protocol:
>>
>> https://microsoft.github.io/debug-adapter-protocol/
>>
>> If the goal is to simplify development, then including the dap stuff
>> in the main repo with the CLI would probably make things the simplest.
>> Just
>> run:
>>
>> sbt daffodil-cli/stage
>>
>> in the daffodil repo, and then point VS Code to execute "daffodil dap
>> .." using the staged files (i.e.
>> daffodil-cli/target/universal/stage/bin/daffodil). We may need changes
>> in the VS Code to support alternative locations of the daffodil
>> binary, but I think that's something we want to support anyways to
>> allow testing different versions of Daffodil without having to install
>> a different extension.
>>
>> On 7/19/22 4:10 PM, Mike Beckerle wrote:
>>> I am leaning toward a separate repo (called daffodil-dap-server)
>>> which
>> uses
>>> (depends on) daffodil.
>>> (btw: what does DAP stand for again? Daffodil Access Protocol ?)
>>>
>>> This is for separation of concerns. I'd like daffodil proper
>>> libraries to be about DFDL mostly, though it does contain the basic
>>> CLI. Things having to do with client/server and testing of the
>>> server side in isolation,
>> etc.
>>> all that belongs only in the daffodil-dap-server which people fixing
>>> ordinary DFDL/Daffodil bugs don't test when they run 'sbt test'.
>>>
>>> I do think the server is going to be quite coupled to Daffodil internals.
>>> For example, to provide better visual animation of delimiter
>>> scanning we will need new debug hooks providing visibility or
>>> callbacks from those parse operations, and DAP will then want to
>>> make this available to the vscode front end for proper
>>> visualization. Similar story is needed around animating what goes on
>>> with unparsing and suspend/resume of expressions, etc. I expect we
>>> will create internal-only versions of these features before we try to firm up any published APIs for them.
>>>
>>> It is pretty important for me to be able to easily and quickly point
>> VSCode
>>> at a new DAP server that is depending on a locally updated daffodil.
>>> But I think something like this is sufficiently easy:
>>>
>>> ... fix a bug in daffodil then
>>> cd daffodil
>>> sbt test
>>> sbt publishLocal // creates a snapshot such as 4.2.1-SNAPSHOT
>>> cd dap-server
>>> sbt test // runs test rig to check out DAP server in isolation
>>> from front end // has dependency on daffodil 4.2.1-SNAPSHOT
>>> sbt package // or publishLocal or... // has dependency on
>>> daffodil 4.2.1-SNAPSHOT
>>> ... then start VSCode using updated configuration file giving
>> dap-server
>>> package location
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Jul 19, 2022 at 3:13 PM Steve Lawrence
>>> <sl...@apache.org>
>> wrote:
>>>
>>>> Some other positives if DAP were combined it into the existing CLI:
>>>>
>>>> There was once talk of making it so the the VS Code debugger could
>>>> switch between different Daffodil versions. If it were part of the
>>>> CLI, this just means downloading a new version of the Daffodil CLI
>>>> and pointing VS Code to it--that process could either be manual or
>>>> automated in the VS Code settings.
>>>>
>>>> I think this would also make it easier to test Daffodil snapshots
>>>> with the VSCode extension? I believe the current process requires
>>>> you to publishLocal a Daffodil snapshot, then update the
>>>> daffodil-vscode repo (or if we split it, the daffodil-debugger
>>>> repo) to have the snapshot version, and the build that. If it was
>>>> all part of the CLI, you could just run "sbt stage" in the Daffodil
>>>> repo and point VSCode to those staged files. It only avoids a
>>>> couple extra steps, but if you do it frequently it would be nice.
>>>>
>>>> On 7/19/22 2:15 PM, Steve Lawrence wrote:
>>>>> +1 to this
>>>>>
>>>>> I was thinking the same thing as I read this. My only thought is
>>>>> using "daffodil dap" since the debug server is just a thing that
>>>>> implements the DAP protocol, and theoretically it could be used
>>>>> any any IDE that supports DAP (I think?). But the name is the easy part.
>>>>>
>>>>> My only other (minor) concern is that this pulls in a number of
>>>>> dependencies. It's not an issue from a licensing perspective since
>>>>> we know they are fine, but it does make the CLI release a bit
>>>>> bigger and more to manage, e.g. lots of Cats jars. I suspect it's
>>>>> not enough to really matter (and most of the size is probably
>>>>> daffodil jars), but if we do go this route, during or prior to
>>>>> migration it might be worth looking to see if anything can be
>>>>> replaced. For example, one thing that jumps out is VSCode debugger
>>>>> currently depends on logback for logging, but daffodil use log4j.
>>>>>
>>>>> The other concern is that currently the VS Code stuff can sort of
>>>>> develop at its own pace. If we integrate the DAP portion into the
>>>>> CLI, it means VSCode can't get DAP updates without a Daffodil
>>>>> release. But
>> it
>>>>> seems DAP is fairly stable, and as long as we keep a quick release
>> cycle
>>>>> and plan accordingly, that shouldn't be too much of an issue.
>>>>>
>>>>> I would perfer this approach over creating a new repo. It's a bit
>>>>> less to keep track of, and makes it easier to ensure we don't have
>>>>> any breaking changes with changes to Daffodil.
>>>>>
>>>>> On 7/19/22 1:52 PM, Mike Beckerle wrote:
>>>>>> I think of this as a daffodil server mode, for the front end
>>>>>> VSCode
>>>> stuff
>>>>>> to use.
>>>>>>
>>>>>> So, is it plausable to add the code to daffodil proper. Make it a
>>>>>> CLI command mode to start up this server, so that
>>>>>>
>>>>>> daffodil vscodeServer
>>>>>>
>>>>>> starts it, optionally with connection options like what ports to
>>>>>> use, etc.
>>>>>>
>>>>>>
>>>>>> On Tue, Jul 19, 2022 at 1:45 PM Shane Dell <sh...@apache.org>
>>>> wrote:
>>>>>>
>>>>>>> All,
>>>>>>>
>>>>>>> This thread is to discuss splitting out the code for the Apache
>>>> Daffodil
>>>>>>> Scala Debugger from the apache/daffodil-vscode repository. The
>>>>>>> two options would be:
>>>>>>>
>>>>>>> - 1.) Move the debugger source code into the apache/daffodil repo.
>>>>>>> However,
>>>>>>> will this work because the debugger code depends on certain
>>>>>>> daffodil Scala libraries to be published? This is mostly an
>>>>>>> option since both are mostly/fully Scala based.
>>>>>>> - 2.) Create a new repo (apache/daffodil-debugger?,
>>>>>>> apache/daffodil-debugger-scala?) where the Scala code will live.
>>>>>>>
>>>>>>> This would be helpful as the apache/daffodil-vscode repo is
>>>>>>> heavily based on creating the VS Code extension for Daffodil.
>>>>>>> With this the
>> debugger
>>>>>>> source code is rarely updated, when it is they are pretty minor
>>>>>>> fixes
>>>> or
>>>>>>> dependency updates. Currently it is a bit of a cluster having a
>>>>>>> mix
>> of
>>>>>>> node/JavaScript/TypeScript and Scala which causes an issue with
>>>>>>> dependency bots checking as these are checking for both npm and
>>>>>>> sbt/Scala dependencies, causing many PRs to be made. The
>>>>>>> extension code also runs a sbt universal:packageBin command in
>>>>>>> multiple occurences, being able
>> to
>>>>>>> remove this and downloading the debugger package would simplify
>>>>>>> a
>>>> couple
>>>>>>> different processes for the extension.
>>>>>>>
>>>>>>> My personal vote is for creating a new repo called something
>>>>>>> like
>>>> either
>>>>>>> apache/daffodil-debugger or apache/daffodil-debugger-scala.
>>>>>>>
>>>>>>> Sincerely,
>>>>>>>
>>>>>>> Shane Dell
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
RE: Re: Daffodil Debugger Scala Source Code
Posted by "Interrante, John A (GE Research, US)" <Jo...@ge.com>.
The motivation for putting dap into the main daffodil repo is that bundling dap into the daffodil cli (allowing "daffodil dap" to start dap) will simplify configuring the VS Code debugger extension to work with different daffodil and dap versions.
I suggest that there is almost certainly a way to put dap into its own repo while still allowing "daffodil dap" to start a compatible dap version. I will let others figure out the most optimal method, but I don't see any technical obstacle that would prevent "daffodil dap" from starting a compatible dap version and therefore simplifying configuring the VS Code debugger extension.
-----Original Message-----
From: Mike Beckerle <mb...@apache.org>
Sent: Wednesday, July 20, 2022 9:05 AM
To: dev@daffodil.apache.org
Subject: EXT: Re: Daffodil Debugger Scala Source Code
WARNING: This email originated from outside of GE. Please validate the sender's email address before clicking on links or attachments as they may not be safe.
I think putting dap into the main daffodil repo is a false economy.
Consider:
1) more sbt hacks to exclude the tests of the dap from 'sbt test' and to add an alternative 'sbt dap:test' which runs those tests.
2) that won't work with IDEs automatically, so they will be unable to run the dap tests without renaming directories or such.
(CLI has this same problem today. I do not yet have a convenient workaround for this, and it is a pain in the neck routinely)
3) another module inside daffodil to compile when 'sbt compile' is issued.
So more delay in the routine edit-compile-debug cycle.
4) more IDE customizations to save that extra compilation time when doing IDE builds.
The above is a huge hassle to maintain compared with one more repo that is completely ordinary with respect to build configuration, IDE support, etc.
Creating a separate repo has none of these drawbacks. 'sbt test' can work normally to test it. Tests live in src/test as normal. No need for special exclusions or sbt hackery.
IDEs will do the right thing by default.
On Tue, Jul 19, 2022 at 4:51 PM Steve Lawrence <sl...@apache.org> wrote:
> DAP stands for Debug Adapter Protocol:
>
> https://microsoft.github.io/debug-adapter-protocol/
>
> If the goal is to simplify development, then including the dap stuff
> in the main repo with the CLI would probably make things the simplest.
> Just
> run:
>
> sbt daffodil-cli/stage
>
> in the daffodil repo, and then point VS Code to execute "daffodil dap
> .." using the staged files (i.e.
> daffodil-cli/target/universal/stage/bin/daffodil). We may need changes
> in the VS Code to support alternative locations of the daffodil
> binary, but I think that's something we want to support anyways to
> allow testing different versions of Daffodil without having to install
> a different extension.
>
> On 7/19/22 4:10 PM, Mike Beckerle wrote:
> > I am leaning toward a separate repo (called daffodil-dap-server)
> > which
> uses
> > (depends on) daffodil.
> > (btw: what does DAP stand for again? Daffodil Access Protocol ?)
> >
> > This is for separation of concerns. I'd like daffodil proper
> > libraries to be about DFDL mostly, though it does contain the basic
> > CLI. Things having to do with client/server and testing of the
> > server side in isolation,
> etc.
> > all that belongs only in the daffodil-dap-server which people fixing
> > ordinary DFDL/Daffodil bugs don't test when they run 'sbt test'.
> >
> > I do think the server is going to be quite coupled to Daffodil internals.
> > For example, to provide better visual animation of delimiter
> > scanning we will need new debug hooks providing visibility or
> > callbacks from those parse operations, and DAP will then want to
> > make this available to the vscode front end for proper
> > visualization. Similar story is needed around animating what goes on
> > with unparsing and suspend/resume of expressions, etc. I expect we
> > will create internal-only versions of these features before we try to firm up any published APIs for them.
> >
> > It is pretty important for me to be able to easily and quickly point
> VSCode
> > at a new DAP server that is depending on a locally updated daffodil.
> > But I think something like this is sufficiently easy:
> >
> > ... fix a bug in daffodil then
> > cd daffodil
> > sbt test
> > sbt publishLocal // creates a snapshot such as 4.2.1-SNAPSHOT
> > cd dap-server
> > sbt test // runs test rig to check out DAP server in isolation
> > from front end // has dependency on daffodil 4.2.1-SNAPSHOT
> > sbt package // or publishLocal or... // has dependency on
> > daffodil 4.2.1-SNAPSHOT
> > ... then start VSCode using updated configuration file giving
> dap-server
> > package location
> >
> >
> >
> >
> >
> > On Tue, Jul 19, 2022 at 3:13 PM Steve Lawrence
> > <sl...@apache.org>
> wrote:
> >
> >> Some other positives if DAP were combined it into the existing CLI:
> >>
> >> There was once talk of making it so the the VS Code debugger could
> >> switch between different Daffodil versions. If it were part of the
> >> CLI, this just means downloading a new version of the Daffodil CLI
> >> and pointing VS Code to it--that process could either be manual or
> >> automated in the VS Code settings.
> >>
> >> I think this would also make it easier to test Daffodil snapshots
> >> with the VSCode extension? I believe the current process requires
> >> you to publishLocal a Daffodil snapshot, then update the
> >> daffodil-vscode repo (or if we split it, the daffodil-debugger
> >> repo) to have the snapshot version, and the build that. If it was
> >> all part of the CLI, you could just run "sbt stage" in the Daffodil
> >> repo and point VSCode to those staged files. It only avoids a
> >> couple extra steps, but if you do it frequently it would be nice.
> >>
> >> On 7/19/22 2:15 PM, Steve Lawrence wrote:
> >>> +1 to this
> >>>
> >>> I was thinking the same thing as I read this. My only thought is
> >>> using "daffodil dap" since the debug server is just a thing that
> >>> implements the DAP protocol, and theoretically it could be used
> >>> any any IDE that supports DAP (I think?). But the name is the easy part.
> >>>
> >>> My only other (minor) concern is that this pulls in a number of
> >>> dependencies. It's not an issue from a licensing perspective since
> >>> we know they are fine, but it does make the CLI release a bit
> >>> bigger and more to manage, e.g. lots of Cats jars. I suspect it's
> >>> not enough to really matter (and most of the size is probably
> >>> daffodil jars), but if we do go this route, during or prior to
> >>> migration it might be worth looking to see if anything can be
> >>> replaced. For example, one thing that jumps out is VSCode debugger
> >>> currently depends on logback for logging, but daffodil use log4j.
> >>>
> >>> The other concern is that currently the VS Code stuff can sort of
> >>> develop at its own pace. If we integrate the DAP portion into the
> >>> CLI, it means VSCode can't get DAP updates without a Daffodil
> >>> release. But
> it
> >>> seems DAP is fairly stable, and as long as we keep a quick release
> cycle
> >>> and plan accordingly, that shouldn't be too much of an issue.
> >>>
> >>> I would perfer this approach over creating a new repo. It's a bit
> >>> less to keep track of, and makes it easier to ensure we don't have
> >>> any breaking changes with changes to Daffodil.
> >>>
> >>> On 7/19/22 1:52 PM, Mike Beckerle wrote:
> >>>> I think of this as a daffodil server mode, for the front end
> >>>> VSCode
> >> stuff
> >>>> to use.
> >>>>
> >>>> So, is it plausable to add the code to daffodil proper. Make it a
> >>>> CLI command mode to start up this server, so that
> >>>>
> >>>> daffodil vscodeServer
> >>>>
> >>>> starts it, optionally with connection options like what ports to
> >>>> use, etc.
> >>>>
> >>>>
> >>>> On Tue, Jul 19, 2022 at 1:45 PM Shane Dell <sh...@apache.org>
> >> wrote:
> >>>>
> >>>>> All,
> >>>>>
> >>>>> This thread is to discuss splitting out the code for the Apache
> >> Daffodil
> >>>>> Scala Debugger from the apache/daffodil-vscode repository. The
> >>>>> two options would be:
> >>>>>
> >>>>> - 1.) Move the debugger source code into the apache/daffodil repo.
> >>>>> However,
> >>>>> will this work because the debugger code depends on certain
> >>>>> daffodil Scala libraries to be published? This is mostly an
> >>>>> option since both are mostly/fully Scala based.
> >>>>> - 2.) Create a new repo (apache/daffodil-debugger?,
> >>>>> apache/daffodil-debugger-scala?) where the Scala code will live.
> >>>>>
> >>>>> This would be helpful as the apache/daffodil-vscode repo is
> >>>>> heavily based on creating the VS Code extension for Daffodil.
> >>>>> With this the
> debugger
> >>>>> source code is rarely updated, when it is they are pretty minor
> >>>>> fixes
> >> or
> >>>>> dependency updates. Currently it is a bit of a cluster having a
> >>>>> mix
> of
> >>>>> node/JavaScript/TypeScript and Scala which causes an issue with
> >>>>> dependency bots checking as these are checking for both npm and
> >>>>> sbt/Scala dependencies, causing many PRs to be made. The
> >>>>> extension code also runs a sbt universal:packageBin command in
> >>>>> multiple occurences, being able
> to
> >>>>> remove this and downloading the debugger package would simplify
> >>>>> a
> >> couple
> >>>>> different processes for the extension.
> >>>>>
> >>>>> My personal vote is for creating a new repo called something
> >>>>> like
> >> either
> >>>>> apache/daffodil-debugger or apache/daffodil-debugger-scala.
> >>>>>
> >>>>> Sincerely,
> >>>>>
> >>>>> Shane Dell
> >>>>>
> >>>>
> >>>
> >>
> >>
> >
>
>
Re: Daffodil Debugger Scala Source Code
Posted by "Shearer, Davin" <sh...@ctc.com>.
Hi Mike,
Each layer in the stack ought to have its own test suite, comprehensive (as best we can) set of test cases, and kicked off via CI (and on-demand from the CLI via `sbt test`). In this case. The test suite will be communicating with the debugger over the DAP protocol and asserting expectations. It doesn't require a front-end or human interaction.
On 7/20/22, 10:18 AM, "Mike Beckerle" <mb...@apache.org> wrote:
I do have one further argument opposed to making it part of the CLI.
To me, if it cannot be tested in isolation from the VSCode front end, then
we should not make it part of Daffodil proper.
So are there tests and test-rig for exercising the dap server in isolation?
If not I think any PR to add it to the daffodil CLI should get -1. We have
a general principle that tests are required for things added to the
daffodil code base.
Furthermore, test coverage is supposed not to be falling, but increasing
over time.
Once there are tests for it that will confirm it operating properly in
isolation from the front end, then I would not have an objection to adding
it to daffodil's CLI.
I suspect we're going to need to enhance the dap server to have some sort
of capture capability so that manual testing of the front end can be
captured in terms of the commands sent to the dap server, and the responses
received from it, and those can be then turned into test cases to drive the
dap server in isolation and compare actual with expected responses.
...mikeb
On Wed, Jul 20, 2022 at 9:40 AM Steve Lawrence <sl...@apache.org> wrote:
>
> I'm not sure I'm convinced by these arguments (note that I don't feel
> strongly either way, though I lean towards including in the repo).
> Comments inline.
>
> On 7/20/22 9:04 AM, Mike Beckerle wrote:
> > I think putting dap into the main daffodil repo is a false economy.
> > Consider:
> >
> > 1) more sbt hacks to exclude the tests of the dap from 'sbt test' and to
> > add an alternative 'sbt dap:test' which runs those tests.
>
> We could just put them as part of the IntegratoonTest if we don't want
> to run them as normal tests. IntegrationTests already take forever, and
> "sbt testOnly" can run a subset of tests if you only want to test DAP
> stuff. Note that we also don't even have any DAP tests right now so I'm
> not even sure what mechanisms they would use or how slow/fast they would
> be, so it might not actually add a significant amount of time to testing.
>
> > 2) that won't work with IDEs automatically, so they will be unable to run
> > the dap tests without renaming directories or such.
> > (CLI has this same problem today. I do not yet have a convenient
> workaround
> > for this, and it is a pain in the neck routinely)
>
> I didn't know integration tests couldn't be run with IDE's. I think
> that's worth solving, and the solution should work the same for DAP tests.
>
> > 3) another module inside daffodil to compile when 'sbt compile' is
> issued.
> > So more delay in the routine edit-compile-debug cycle.
>
> From a clean/compile, it takes me on average about 4 seconds to compile
> the server. Compared to Daffodil, which takes me about 40 seconds. It's
> not nothing, but I'm not sure I would notice a 4 second difference. And
> it wouldn't surprise me if that's mostly Scala/Sbt overhead anyways.
> There's only 2000 lines of DAP code, that should compile pretty fast.
>
> > 4) more IDE customizations to save that extra compilation time when doing
> > IDE builds.
>
> I'm not sure I understand what this means, but there shouldn't be any
> extra IDE configuration, it should work the same way any other
> subproject works.
>
> >
> > The above is a huge hassle to maintain compared with one more repo that
> is
> > completely ordinary with respect to build configuration, IDE support,
> etc.
>
> I would argue we could make it completely ordinary. It's just another
> Daffodil subproject.
>
> > Creating a separate repo has none of these drawbacks. 'sbt test' can work
> > normally to test it. Tests live in src/test as normal. No need for
> special
> > exclusions or sbt hackery.
> > IDEs will do the right thing by default.
>
> Another tought, from a testing perspective I think we're much less
> likely to accidentally break DAP if it's integrated in Daffodil. I know
> I'm very rarely going to want to run "sbt publishLocal" Daffodil, modify
> daffdoil-debugger to use a SNAPSHOT version, and then run "sbt test"
> from there. We have this same issue with DFDL schemas where we rearely
> test schemas because they're in a separate repo and only find out when
> we broke something because of some nightly test or someone starts using
> the schema.
>
> Maybe some GitHub action trigger could do this for Daffodil changes, or
> someone include a publishLocal, download debugger and test in the GitHub
> Actions, but that's just more complication.
>
>
> This also potentially adds extra complication to the release process.
> I'm not even sure how it would work. Would we first do a Daffodil
> release (discuss, vote, publish), and then update the daffodil-debugger
> repo to point to the new daffodil version and then do another release
> for that (discuss, vote, publish)? So essentially two separate release
> processes? Or do we always do them at the same time and the
> release-candidate script builds both Daffodil and the debugger, and the
> vote/release is for both? Whatever the case, this probably requires
> changes/maintenance to the release-candidate tool, and is a change to
> our current process. I'm sure we can manage, but if we integrate it into
> the main repo, nothing with the release process changes.
>
-----------------------------------------------------------------
This message and any files transmitted within are intended
solely for the addressee or its representative and may contain
company proprietary information. If you are not the intended
recipient, notify the sender immediately and delete this
message. Publication, reproduction, forwarding, or content
disclosure is prohibited without the consent of the original
sender and may be unlawful.
Concurrent Technologies Corporation and its Affiliates.
www.ctc.com 1-800-282-4392
-----------------------------------------------------------------
Re: Daffodil Debugger Scala Source Code
Posted by Mike Beckerle <mb...@apache.org>.
I do have one further argument opposed to making it part of the CLI.
To me, if it cannot be tested in isolation from the VSCode front end, then
we should not make it part of Daffodil proper.
So are there tests and test-rig for exercising the dap server in isolation?
If not I think any PR to add it to the daffodil CLI should get -1. We have
a general principle that tests are required for things added to the
daffodil code base.
Furthermore, test coverage is supposed not to be falling, but increasing
over time.
Once there are tests for it that will confirm it operating properly in
isolation from the front end, then I would not have an objection to adding
it to daffodil's CLI.
I suspect we're going to need to enhance the dap server to have some sort
of capture capability so that manual testing of the front end can be
captured in terms of the commands sent to the dap server, and the responses
received from it, and those can be then turned into test cases to drive the
dap server in isolation and compare actual with expected responses.
...mikeb
On Wed, Jul 20, 2022 at 9:40 AM Steve Lawrence <sl...@apache.org> wrote:
>
> I'm not sure I'm convinced by these arguments (note that I don't feel
> strongly either way, though I lean towards including in the repo).
> Comments inline.
>
> On 7/20/22 9:04 AM, Mike Beckerle wrote:
> > I think putting dap into the main daffodil repo is a false economy.
> > Consider:
> >
> > 1) more sbt hacks to exclude the tests of the dap from 'sbt test' and to
> > add an alternative 'sbt dap:test' which runs those tests.
>
> We could just put them as part of the IntegratoonTest if we don't want
> to run them as normal tests. IntegrationTests already take forever, and
> "sbt testOnly" can run a subset of tests if you only want to test DAP
> stuff. Note that we also don't even have any DAP tests right now so I'm
> not even sure what mechanisms they would use or how slow/fast they would
> be, so it might not actually add a significant amount of time to testing.
>
> > 2) that won't work with IDEs automatically, so they will be unable to run
> > the dap tests without renaming directories or such.
> > (CLI has this same problem today. I do not yet have a convenient
> workaround
> > for this, and it is a pain in the neck routinely)
>
> I didn't know integration tests couldn't be run with IDE's. I think
> that's worth solving, and the solution should work the same for DAP tests.
>
> > 3) another module inside daffodil to compile when 'sbt compile' is
> issued.
> > So more delay in the routine edit-compile-debug cycle.
>
> From a clean/compile, it takes me on average about 4 seconds to compile
> the server. Compared to Daffodil, which takes me about 40 seconds. It's
> not nothing, but I'm not sure I would notice a 4 second difference. And
> it wouldn't surprise me if that's mostly Scala/Sbt overhead anyways.
> There's only 2000 lines of DAP code, that should compile pretty fast.
>
> > 4) more IDE customizations to save that extra compilation time when doing
> > IDE builds.
>
> I'm not sure I understand what this means, but there shouldn't be any
> extra IDE configuration, it should work the same way any other
> subproject works.
>
> >
> > The above is a huge hassle to maintain compared with one more repo that
> is
> > completely ordinary with respect to build configuration, IDE support,
> etc.
>
> I would argue we could make it completely ordinary. It's just another
> Daffodil subproject.
>
> > Creating a separate repo has none of these drawbacks. 'sbt test' can work
> > normally to test it. Tests live in src/test as normal. No need for
> special
> > exclusions or sbt hackery.
> > IDEs will do the right thing by default.
>
> Another tought, from a testing perspective I think we're much less
> likely to accidentally break DAP if it's integrated in Daffodil. I know
> I'm very rarely going to want to run "sbt publishLocal" Daffodil, modify
> daffdoil-debugger to use a SNAPSHOT version, and then run "sbt test"
> from there. We have this same issue with DFDL schemas where we rearely
> test schemas because they're in a separate repo and only find out when
> we broke something because of some nightly test or someone starts using
> the schema.
>
> Maybe some GitHub action trigger could do this for Daffodil changes, or
> someone include a publishLocal, download debugger and test in the GitHub
> Actions, but that's just more complication.
>
>
> This also potentially adds extra complication to the release process.
> I'm not even sure how it would work. Would we first do a Daffodil
> release (discuss, vote, publish), and then update the daffodil-debugger
> repo to point to the new daffodil version and then do another release
> for that (discuss, vote, publish)? So essentially two separate release
> processes? Or do we always do them at the same time and the
> release-candidate script builds both Daffodil and the debugger, and the
> vote/release is for both? Whatever the case, this probably requires
> changes/maintenance to the release-candidate tool, and is a change to
> our current process. I'm sure we can manage, but if we integrate it into
> the main repo, nothing with the release process changes.
>
Re: Daffodil Debugger Scala Source Code
Posted by Steve Lawrence <sl...@apache.org>.
I'm not sure I'm convinced by these arguments (note that I don't feel
strongly either way, though I lean towards including in the repo).
Comments inline.
On 7/20/22 9:04 AM, Mike Beckerle wrote:
> I think putting dap into the main daffodil repo is a false economy.
> Consider:
>
> 1) more sbt hacks to exclude the tests of the dap from 'sbt test' and to
> add an alternative 'sbt dap:test' which runs those tests.
We could just put them as part of the IntegratoonTest if we don't want
to run them as normal tests. IntegrationTests already take forever, and
"sbt testOnly" can run a subset of tests if you only want to test DAP
stuff. Note that we also don't even have any DAP tests right now so I'm
not even sure what mechanisms they would use or how slow/fast they would
be, so it might not actually add a significant amount of time to testing.
> 2) that won't work with IDEs automatically, so they will be unable to run
> the dap tests without renaming directories or such.
> (CLI has this same problem today. I do not yet have a convenient workaround
> for this, and it is a pain in the neck routinely)
I didn't know integration tests couldn't be run with IDE's. I think
that's worth solving, and the solution should work the same for DAP tests.
> 3) another module inside daffodil to compile when 'sbt compile' is issued.
> So more delay in the routine edit-compile-debug cycle.
From a clean/compile, it takes me on average about 4 seconds to compile
the server. Compared to Daffodil, which takes me about 40 seconds. It's
not nothing, but I'm not sure I would notice a 4 second difference. And
it wouldn't surprise me if that's mostly Scala/Sbt overhead anyways.
There's only 2000 lines of DAP code, that should compile pretty fast.
> 4) more IDE customizations to save that extra compilation time when doing
> IDE builds.
I'm not sure I understand what this means, but there shouldn't be any
extra IDE configuration, it should work the same way any other
subproject works.
>
> The above is a huge hassle to maintain compared with one more repo that is
> completely ordinary with respect to build configuration, IDE support, etc.
I would argue we could make it completely ordinary. It's just another
Daffodil subproject.
> Creating a separate repo has none of these drawbacks. 'sbt test' can work
> normally to test it. Tests live in src/test as normal. No need for special
> exclusions or sbt hackery.
> IDEs will do the right thing by default.
Another tought, from a testing perspective I think we're much less
likely to accidentally break DAP if it's integrated in Daffodil. I know
I'm very rarely going to want to run "sbt publishLocal" Daffodil, modify
daffdoil-debugger to use a SNAPSHOT version, and then run "sbt test"
from there. We have this same issue with DFDL schemas where we rearely
test schemas because they're in a separate repo and only find out when
we broke something because of some nightly test or someone starts using
the schema.
Maybe some GitHub action trigger could do this for Daffodil changes, or
someone include a publishLocal, download debugger and test in the GitHub
Actions, but that's just more complication.
This also potentially adds extra complication to the release process.
I'm not even sure how it would work. Would we first do a Daffodil
release (discuss, vote, publish), and then update the daffodil-debugger
repo to point to the new daffodil version and then do another release
for that (discuss, vote, publish)? So essentially two separate release
processes? Or do we always do them at the same time and the
release-candidate script builds both Daffodil and the debugger, and the
vote/release is for both? Whatever the case, this probably requires
changes/maintenance to the release-candidate tool, and is a change to
our current process. I'm sure we can manage, but if we integrate it into
the main repo, nothing with the release process changes.
Re: Daffodil Debugger Scala Source Code
Posted by Mike Beckerle <mb...@apache.org>.
I think putting dap into the main daffodil repo is a false economy.
Consider:
1) more sbt hacks to exclude the tests of the dap from 'sbt test' and to
add an alternative 'sbt dap:test' which runs those tests.
2) that won't work with IDEs automatically, so they will be unable to run
the dap tests without renaming directories or such.
(CLI has this same problem today. I do not yet have a convenient workaround
for this, and it is a pain in the neck routinely)
3) another module inside daffodil to compile when 'sbt compile' is issued.
So more delay in the routine edit-compile-debug cycle.
4) more IDE customizations to save that extra compilation time when doing
IDE builds.
The above is a huge hassle to maintain compared with one more repo that is
completely ordinary with respect to build configuration, IDE support, etc.
Creating a separate repo has none of these drawbacks. 'sbt test' can work
normally to test it. Tests live in src/test as normal. No need for special
exclusions or sbt hackery.
IDEs will do the right thing by default.
On Tue, Jul 19, 2022 at 4:51 PM Steve Lawrence <sl...@apache.org> wrote:
> DAP stands for Debug Adapter Protocol:
>
> https://microsoft.github.io/debug-adapter-protocol/
>
> If the goal is to simplify development, then including the dap stuff in
> the main repo with the CLI would probably make things the simplest. Just
> run:
>
> sbt daffodil-cli/stage
>
> in the daffodil repo, and then point VS Code to execute "daffodil dap
> .." using the staged files (i.e.
> daffodil-cli/target/universal/stage/bin/daffodil). We may need changes
> in the VS Code to support alternative locations of the daffodil binary,
> but I think that's something we want to support anyways to allow testing
> different versions of Daffodil without having to install a different
> extension.
>
> On 7/19/22 4:10 PM, Mike Beckerle wrote:
> > I am leaning toward a separate repo (called daffodil-dap-server) which
> uses
> > (depends on) daffodil.
> > (btw: what does DAP stand for again? Daffodil Access Protocol ?)
> >
> > This is for separation of concerns. I'd like daffodil proper libraries to
> > be about DFDL mostly, though it does contain the basic CLI. Things having
> > to do with client/server and testing of the server side in isolation,
> etc.
> > all that belongs only in the daffodil-dap-server which people fixing
> > ordinary DFDL/Daffodil bugs don't test when they run 'sbt test'.
> >
> > I do think the server is going to be quite coupled to Daffodil internals.
> > For example, to provide better visual animation of delimiter scanning we
> > will need new debug hooks providing visibility or callbacks from those
> > parse operations, and DAP will then want to make this available to the
> > vscode front end for proper visualization. Similar story is needed around
> > animating what goes on with unparsing and suspend/resume of expressions,
> > etc. I expect we will create internal-only versions of these features
> > before we try to firm up any published APIs for them.
> >
> > It is pretty important for me to be able to easily and quickly point
> VSCode
> > at a new DAP server that is depending on a locally updated daffodil.
> > But I think something like this is sufficiently easy:
> >
> > ... fix a bug in daffodil then
> > cd daffodil
> > sbt test
> > sbt publishLocal // creates a snapshot such as 4.2.1-SNAPSHOT
> > cd dap-server
> > sbt test // runs test rig to check out DAP server in isolation from
> > front end // has dependency on daffodil 4.2.1-SNAPSHOT
> > sbt package // or publishLocal or... // has dependency on daffodil
> > 4.2.1-SNAPSHOT
> > ... then start VSCode using updated configuration file giving
> dap-server
> > package location
> >
> >
> >
> >
> >
> > On Tue, Jul 19, 2022 at 3:13 PM Steve Lawrence <sl...@apache.org>
> wrote:
> >
> >> Some other positives if DAP were combined it into the existing CLI:
> >>
> >> There was once talk of making it so the the VS Code debugger could
> >> switch between different Daffodil versions. If it were part of the CLI,
> >> this just means downloading a new version of the Daffodil CLI and
> >> pointing VS Code to it--that process could either be manual or automated
> >> in the VS Code settings.
> >>
> >> I think this would also make it easier to test Daffodil snapshots with
> >> the VSCode extension? I believe the current process requires you to
> >> publishLocal a Daffodil snapshot, then update the daffodil-vscode repo
> >> (or if we split it, the daffodil-debugger repo) to have the snapshot
> >> version, and the build that. If it was all part of the CLI, you could
> >> just run "sbt stage" in the Daffodil repo and point VSCode to those
> >> staged files. It only avoids a couple extra steps, but if you do it
> >> frequently it would be nice.
> >>
> >> On 7/19/22 2:15 PM, Steve Lawrence wrote:
> >>> +1 to this
> >>>
> >>> I was thinking the same thing as I read this. My only thought is using
> >>> "daffodil dap" since the debug server is just a thing that implements
> >>> the DAP protocol, and theoretically it could be used any any IDE that
> >>> supports DAP (I think?). But the name is the easy part.
> >>>
> >>> My only other (minor) concern is that this pulls in a number of
> >>> dependencies. It's not an issue from a licensing perspective since we
> >>> know they are fine, but it does make the CLI release a bit bigger and
> >>> more to manage, e.g. lots of Cats jars. I suspect it's not enough to
> >>> really matter (and most of the size is probably daffodil jars), but if
> >>> we do go this route, during or prior to migration it might be worth
> >>> looking to see if anything can be replaced. For example, one thing that
> >>> jumps out is VSCode debugger currently depends on logback for logging,
> >>> but daffodil use log4j.
> >>>
> >>> The other concern is that currently the VS Code stuff can sort of
> >>> develop at its own pace. If we integrate the DAP portion into the CLI,
> >>> it means VSCode can't get DAP updates without a Daffodil release. But
> it
> >>> seems DAP is fairly stable, and as long as we keep a quick release
> cycle
> >>> and plan accordingly, that shouldn't be too much of an issue.
> >>>
> >>> I would perfer this approach over creating a new repo. It's a bit less
> >>> to keep track of, and makes it easier to ensure we don't have any
> >>> breaking changes with changes to Daffodil.
> >>>
> >>> On 7/19/22 1:52 PM, Mike Beckerle wrote:
> >>>> I think of this as a daffodil server mode, for the front end VSCode
> >> stuff
> >>>> to use.
> >>>>
> >>>> So, is it plausable to add the code to daffodil proper. Make it a CLI
> >>>> command mode to start up this server, so that
> >>>>
> >>>> daffodil vscodeServer
> >>>>
> >>>> starts it, optionally with connection options like what ports to use,
> >>>> etc.
> >>>>
> >>>>
> >>>> On Tue, Jul 19, 2022 at 1:45 PM Shane Dell <sh...@apache.org>
> >> wrote:
> >>>>
> >>>>> All,
> >>>>>
> >>>>> This thread is to discuss splitting out the code for the Apache
> >> Daffodil
> >>>>> Scala Debugger from the apache/daffodil-vscode repository. The two
> >>>>> options
> >>>>> would be:
> >>>>>
> >>>>> - 1.) Move the debugger source code into the apache/daffodil repo.
> >>>>> However,
> >>>>> will this work because the debugger code depends on certain daffodil
> >>>>> Scala
> >>>>> libraries to be published? This is mostly an option since both are
> >>>>> mostly/fully Scala based.
> >>>>> - 2.) Create a new repo (apache/daffodil-debugger?,
> >>>>> apache/daffodil-debugger-scala?) where the Scala code will live.
> >>>>>
> >>>>> This would be helpful as the apache/daffodil-vscode repo is heavily
> >>>>> based
> >>>>> on creating the VS Code extension for Daffodil. With this the
> debugger
> >>>>> source code is rarely updated, when it is they are pretty minor fixes
> >> or
> >>>>> dependency updates. Currently it is a bit of a cluster having a mix
> of
> >>>>> node/JavaScript/TypeScript and Scala which causes an issue with
> >>>>> dependency
> >>>>> bots checking as these are checking for both npm and sbt/Scala
> >>>>> dependencies, causing many PRs to be made. The extension code also
> >>>>> runs a
> >>>>> sbt universal:packageBin command in multiple occurences, being able
> to
> >>>>> remove this and downloading the debugger package would simplify a
> >> couple
> >>>>> different processes for the extension.
> >>>>>
> >>>>> My personal vote is for creating a new repo called something like
> >> either
> >>>>> apache/daffodil-debugger or apache/daffodil-debugger-scala.
> >>>>>
> >>>>> Sincerely,
> >>>>>
> >>>>> Shane Dell
> >>>>>
> >>>>
> >>>
> >>
> >>
> >
>
>
Re: Daffodil Debugger Scala Source Code
Posted by Steve Lawrence <sl...@apache.org>.
DAP stands for Debug Adapter Protocol:
https://microsoft.github.io/debug-adapter-protocol/
If the goal is to simplify development, then including the dap stuff in
the main repo with the CLI would probably make things the simplest. Just
run:
sbt daffodil-cli/stage
in the daffodil repo, and then point VS Code to execute "daffodil dap
.." using the staged files (i.e.
daffodil-cli/target/universal/stage/bin/daffodil). We may need changes
in the VS Code to support alternative locations of the daffodil binary,
but I think that's something we want to support anyways to allow testing
different versions of Daffodil without having to install a different
extension.
On 7/19/22 4:10 PM, Mike Beckerle wrote:
> I am leaning toward a separate repo (called daffodil-dap-server) which uses
> (depends on) daffodil.
> (btw: what does DAP stand for again? Daffodil Access Protocol ?)
>
> This is for separation of concerns. I'd like daffodil proper libraries to
> be about DFDL mostly, though it does contain the basic CLI. Things having
> to do with client/server and testing of the server side in isolation, etc.
> all that belongs only in the daffodil-dap-server which people fixing
> ordinary DFDL/Daffodil bugs don't test when they run 'sbt test'.
>
> I do think the server is going to be quite coupled to Daffodil internals.
> For example, to provide better visual animation of delimiter scanning we
> will need new debug hooks providing visibility or callbacks from those
> parse operations, and DAP will then want to make this available to the
> vscode front end for proper visualization. Similar story is needed around
> animating what goes on with unparsing and suspend/resume of expressions,
> etc. I expect we will create internal-only versions of these features
> before we try to firm up any published APIs for them.
>
> It is pretty important for me to be able to easily and quickly point VSCode
> at a new DAP server that is depending on a locally updated daffodil.
> But I think something like this is sufficiently easy:
>
> ... fix a bug in daffodil then
> cd daffodil
> sbt test
> sbt publishLocal // creates a snapshot such as 4.2.1-SNAPSHOT
> cd dap-server
> sbt test // runs test rig to check out DAP server in isolation from
> front end // has dependency on daffodil 4.2.1-SNAPSHOT
> sbt package // or publishLocal or... // has dependency on daffodil
> 4.2.1-SNAPSHOT
> ... then start VSCode using updated configuration file giving dap-server
> package location
>
>
>
>
>
> On Tue, Jul 19, 2022 at 3:13 PM Steve Lawrence <sl...@apache.org> wrote:
>
>> Some other positives if DAP were combined it into the existing CLI:
>>
>> There was once talk of making it so the the VS Code debugger could
>> switch between different Daffodil versions. If it were part of the CLI,
>> this just means downloading a new version of the Daffodil CLI and
>> pointing VS Code to it--that process could either be manual or automated
>> in the VS Code settings.
>>
>> I think this would also make it easier to test Daffodil snapshots with
>> the VSCode extension? I believe the current process requires you to
>> publishLocal a Daffodil snapshot, then update the daffodil-vscode repo
>> (or if we split it, the daffodil-debugger repo) to have the snapshot
>> version, and the build that. If it was all part of the CLI, you could
>> just run "sbt stage" in the Daffodil repo and point VSCode to those
>> staged files. It only avoids a couple extra steps, but if you do it
>> frequently it would be nice.
>>
>> On 7/19/22 2:15 PM, Steve Lawrence wrote:
>>> +1 to this
>>>
>>> I was thinking the same thing as I read this. My only thought is using
>>> "daffodil dap" since the debug server is just a thing that implements
>>> the DAP protocol, and theoretically it could be used any any IDE that
>>> supports DAP (I think?). But the name is the easy part.
>>>
>>> My only other (minor) concern is that this pulls in a number of
>>> dependencies. It's not an issue from a licensing perspective since we
>>> know they are fine, but it does make the CLI release a bit bigger and
>>> more to manage, e.g. lots of Cats jars. I suspect it's not enough to
>>> really matter (and most of the size is probably daffodil jars), but if
>>> we do go this route, during or prior to migration it might be worth
>>> looking to see if anything can be replaced. For example, one thing that
>>> jumps out is VSCode debugger currently depends on logback for logging,
>>> but daffodil use log4j.
>>>
>>> The other concern is that currently the VS Code stuff can sort of
>>> develop at its own pace. If we integrate the DAP portion into the CLI,
>>> it means VSCode can't get DAP updates without a Daffodil release. But it
>>> seems DAP is fairly stable, and as long as we keep a quick release cycle
>>> and plan accordingly, that shouldn't be too much of an issue.
>>>
>>> I would perfer this approach over creating a new repo. It's a bit less
>>> to keep track of, and makes it easier to ensure we don't have any
>>> breaking changes with changes to Daffodil.
>>>
>>> On 7/19/22 1:52 PM, Mike Beckerle wrote:
>>>> I think of this as a daffodil server mode, for the front end VSCode
>> stuff
>>>> to use.
>>>>
>>>> So, is it plausable to add the code to daffodil proper. Make it a CLI
>>>> command mode to start up this server, so that
>>>>
>>>> daffodil vscodeServer
>>>>
>>>> starts it, optionally with connection options like what ports to use,
>>>> etc.
>>>>
>>>>
>>>> On Tue, Jul 19, 2022 at 1:45 PM Shane Dell <sh...@apache.org>
>> wrote:
>>>>
>>>>> All,
>>>>>
>>>>> This thread is to discuss splitting out the code for the Apache
>> Daffodil
>>>>> Scala Debugger from the apache/daffodil-vscode repository. The two
>>>>> options
>>>>> would be:
>>>>>
>>>>> - 1.) Move the debugger source code into the apache/daffodil repo.
>>>>> However,
>>>>> will this work because the debugger code depends on certain daffodil
>>>>> Scala
>>>>> libraries to be published? This is mostly an option since both are
>>>>> mostly/fully Scala based.
>>>>> - 2.) Create a new repo (apache/daffodil-debugger?,
>>>>> apache/daffodil-debugger-scala?) where the Scala code will live.
>>>>>
>>>>> This would be helpful as the apache/daffodil-vscode repo is heavily
>>>>> based
>>>>> on creating the VS Code extension for Daffodil. With this the debugger
>>>>> source code is rarely updated, when it is they are pretty minor fixes
>> or
>>>>> dependency updates. Currently it is a bit of a cluster having a mix of
>>>>> node/JavaScript/TypeScript and Scala which causes an issue with
>>>>> dependency
>>>>> bots checking as these are checking for both npm and sbt/Scala
>>>>> dependencies, causing many PRs to be made. The extension code also
>>>>> runs a
>>>>> sbt universal:packageBin command in multiple occurences, being able to
>>>>> remove this and downloading the debugger package would simplify a
>> couple
>>>>> different processes for the extension.
>>>>>
>>>>> My personal vote is for creating a new repo called something like
>> either
>>>>> apache/daffodil-debugger or apache/daffodil-debugger-scala.
>>>>>
>>>>> Sincerely,
>>>>>
>>>>> Shane Dell
>>>>>
>>>>
>>>
>>
>>
>
Re: Daffodil Debugger Scala Source Code
Posted by Mike Beckerle <mb...@apache.org>.
I am leaning toward a separate repo (called daffodil-dap-server) which uses
(depends on) daffodil.
(btw: what does DAP stand for again? Daffodil Access Protocol ?)
This is for separation of concerns. I'd like daffodil proper libraries to
be about DFDL mostly, though it does contain the basic CLI. Things having
to do with client/server and testing of the server side in isolation, etc.
all that belongs only in the daffodil-dap-server which people fixing
ordinary DFDL/Daffodil bugs don't test when they run 'sbt test'.
I do think the server is going to be quite coupled to Daffodil internals.
For example, to provide better visual animation of delimiter scanning we
will need new debug hooks providing visibility or callbacks from those
parse operations, and DAP will then want to make this available to the
vscode front end for proper visualization. Similar story is needed around
animating what goes on with unparsing and suspend/resume of expressions,
etc. I expect we will create internal-only versions of these features
before we try to firm up any published APIs for them.
It is pretty important for me to be able to easily and quickly point VSCode
at a new DAP server that is depending on a locally updated daffodil.
But I think something like this is sufficiently easy:
... fix a bug in daffodil then
cd daffodil
sbt test
sbt publishLocal // creates a snapshot such as 4.2.1-SNAPSHOT
cd dap-server
sbt test // runs test rig to check out DAP server in isolation from
front end // has dependency on daffodil 4.2.1-SNAPSHOT
sbt package // or publishLocal or... // has dependency on daffodil
4.2.1-SNAPSHOT
... then start VSCode using updated configuration file giving dap-server
package location
On Tue, Jul 19, 2022 at 3:13 PM Steve Lawrence <sl...@apache.org> wrote:
> Some other positives if DAP were combined it into the existing CLI:
>
> There was once talk of making it so the the VS Code debugger could
> switch between different Daffodil versions. If it were part of the CLI,
> this just means downloading a new version of the Daffodil CLI and
> pointing VS Code to it--that process could either be manual or automated
> in the VS Code settings.
>
> I think this would also make it easier to test Daffodil snapshots with
> the VSCode extension? I believe the current process requires you to
> publishLocal a Daffodil snapshot, then update the daffodil-vscode repo
> (or if we split it, the daffodil-debugger repo) to have the snapshot
> version, and the build that. If it was all part of the CLI, you could
> just run "sbt stage" in the Daffodil repo and point VSCode to those
> staged files. It only avoids a couple extra steps, but if you do it
> frequently it would be nice.
>
> On 7/19/22 2:15 PM, Steve Lawrence wrote:
> > +1 to this
> >
> > I was thinking the same thing as I read this. My only thought is using
> > "daffodil dap" since the debug server is just a thing that implements
> > the DAP protocol, and theoretically it could be used any any IDE that
> > supports DAP (I think?). But the name is the easy part.
> >
> > My only other (minor) concern is that this pulls in a number of
> > dependencies. It's not an issue from a licensing perspective since we
> > know they are fine, but it does make the CLI release a bit bigger and
> > more to manage, e.g. lots of Cats jars. I suspect it's not enough to
> > really matter (and most of the size is probably daffodil jars), but if
> > we do go this route, during or prior to migration it might be worth
> > looking to see if anything can be replaced. For example, one thing that
> > jumps out is VSCode debugger currently depends on logback for logging,
> > but daffodil use log4j.
> >
> > The other concern is that currently the VS Code stuff can sort of
> > develop at its own pace. If we integrate the DAP portion into the CLI,
> > it means VSCode can't get DAP updates without a Daffodil release. But it
> > seems DAP is fairly stable, and as long as we keep a quick release cycle
> > and plan accordingly, that shouldn't be too much of an issue.
> >
> > I would perfer this approach over creating a new repo. It's a bit less
> > to keep track of, and makes it easier to ensure we don't have any
> > breaking changes with changes to Daffodil.
> >
> > On 7/19/22 1:52 PM, Mike Beckerle wrote:
> >> I think of this as a daffodil server mode, for the front end VSCode
> stuff
> >> to use.
> >>
> >> So, is it plausable to add the code to daffodil proper. Make it a CLI
> >> command mode to start up this server, so that
> >>
> >> daffodil vscodeServer
> >>
> >> starts it, optionally with connection options like what ports to use,
> >> etc.
> >>
> >>
> >> On Tue, Jul 19, 2022 at 1:45 PM Shane Dell <sh...@apache.org>
> wrote:
> >>
> >>> All,
> >>>
> >>> This thread is to discuss splitting out the code for the Apache
> Daffodil
> >>> Scala Debugger from the apache/daffodil-vscode repository. The two
> >>> options
> >>> would be:
> >>>
> >>> - 1.) Move the debugger source code into the apache/daffodil repo.
> >>> However,
> >>> will this work because the debugger code depends on certain daffodil
> >>> Scala
> >>> libraries to be published? This is mostly an option since both are
> >>> mostly/fully Scala based.
> >>> - 2.) Create a new repo (apache/daffodil-debugger?,
> >>> apache/daffodil-debugger-scala?) where the Scala code will live.
> >>>
> >>> This would be helpful as the apache/daffodil-vscode repo is heavily
> >>> based
> >>> on creating the VS Code extension for Daffodil. With this the debugger
> >>> source code is rarely updated, when it is they are pretty minor fixes
> or
> >>> dependency updates. Currently it is a bit of a cluster having a mix of
> >>> node/JavaScript/TypeScript and Scala which causes an issue with
> >>> dependency
> >>> bots checking as these are checking for both npm and sbt/Scala
> >>> dependencies, causing many PRs to be made. The extension code also
> >>> runs a
> >>> sbt universal:packageBin command in multiple occurences, being able to
> >>> remove this and downloading the debugger package would simplify a
> couple
> >>> different processes for the extension.
> >>>
> >>> My personal vote is for creating a new repo called something like
> either
> >>> apache/daffodil-debugger or apache/daffodil-debugger-scala.
> >>>
> >>> Sincerely,
> >>>
> >>> Shane Dell
> >>>
> >>
> >
>
>
Re: Daffodil Debugger Scala Source Code
Posted by Steve Lawrence <sl...@apache.org>.
Some other positives if DAP were combined it into the existing CLI:
There was once talk of making it so the the VS Code debugger could
switch between different Daffodil versions. If it were part of the CLI,
this just means downloading a new version of the Daffodil CLI and
pointing VS Code to it--that process could either be manual or automated
in the VS Code settings.
I think this would also make it easier to test Daffodil snapshots with
the VSCode extension? I believe the current process requires you to
publishLocal a Daffodil snapshot, then update the daffodil-vscode repo
(or if we split it, the daffodil-debugger repo) to have the snapshot
version, and the build that. If it was all part of the CLI, you could
just run "sbt stage" in the Daffodil repo and point VSCode to those
staged files. It only avoids a couple extra steps, but if you do it
frequently it would be nice.
On 7/19/22 2:15 PM, Steve Lawrence wrote:
> +1 to this
>
> I was thinking the same thing as I read this. My only thought is using
> "daffodil dap" since the debug server is just a thing that implements
> the DAP protocol, and theoretically it could be used any any IDE that
> supports DAP (I think?). But the name is the easy part.
>
> My only other (minor) concern is that this pulls in a number of
> dependencies. It's not an issue from a licensing perspective since we
> know they are fine, but it does make the CLI release a bit bigger and
> more to manage, e.g. lots of Cats jars. I suspect it's not enough to
> really matter (and most of the size is probably daffodil jars), but if
> we do go this route, during or prior to migration it might be worth
> looking to see if anything can be replaced. For example, one thing that
> jumps out is VSCode debugger currently depends on logback for logging,
> but daffodil use log4j.
>
> The other concern is that currently the VS Code stuff can sort of
> develop at its own pace. If we integrate the DAP portion into the CLI,
> it means VSCode can't get DAP updates without a Daffodil release. But it
> seems DAP is fairly stable, and as long as we keep a quick release cycle
> and plan accordingly, that shouldn't be too much of an issue.
>
> I would perfer this approach over creating a new repo. It's a bit less
> to keep track of, and makes it easier to ensure we don't have any
> breaking changes with changes to Daffodil.
>
> On 7/19/22 1:52 PM, Mike Beckerle wrote:
>> I think of this as a daffodil server mode, for the front end VSCode stuff
>> to use.
>>
>> So, is it plausable to add the code to daffodil proper. Make it a CLI
>> command mode to start up this server, so that
>>
>> daffodil vscodeServer
>>
>> starts it, optionally with connection options like what ports to use,
>> etc.
>>
>>
>> On Tue, Jul 19, 2022 at 1:45 PM Shane Dell <sh...@apache.org> wrote:
>>
>>> All,
>>>
>>> This thread is to discuss splitting out the code for the Apache Daffodil
>>> Scala Debugger from the apache/daffodil-vscode repository. The two
>>> options
>>> would be:
>>>
>>> - 1.) Move the debugger source code into the apache/daffodil repo.
>>> However,
>>> will this work because the debugger code depends on certain daffodil
>>> Scala
>>> libraries to be published? This is mostly an option since both are
>>> mostly/fully Scala based.
>>> - 2.) Create a new repo (apache/daffodil-debugger?,
>>> apache/daffodil-debugger-scala?) where the Scala code will live.
>>>
>>> This would be helpful as the apache/daffodil-vscode repo is heavily
>>> based
>>> on creating the VS Code extension for Daffodil. With this the debugger
>>> source code is rarely updated, when it is they are pretty minor fixes or
>>> dependency updates. Currently it is a bit of a cluster having a mix of
>>> node/JavaScript/TypeScript and Scala which causes an issue with
>>> dependency
>>> bots checking as these are checking for both npm and sbt/Scala
>>> dependencies, causing many PRs to be made. The extension code also
>>> runs a
>>> sbt universal:packageBin command in multiple occurences, being able to
>>> remove this and downloading the debugger package would simplify a couple
>>> different processes for the extension.
>>>
>>> My personal vote is for creating a new repo called something like either
>>> apache/daffodil-debugger or apache/daffodil-debugger-scala.
>>>
>>> Sincerely,
>>>
>>> Shane Dell
>>>
>>
>
Re: Daffodil Debugger Scala Source Code
Posted by Steve Lawrence <sl...@apache.org>.
+1 to this
I was thinking the same thing as I read this. My only thought is using
"daffodil dap" since the debug server is just a thing that implements
the DAP protocol, and theoretically it could be used any any IDE that
supports DAP (I think?). But the name is the easy part.
My only other (minor) concern is that this pulls in a number of
dependencies. It's not an issue from a licensing perspective since we
know they are fine, but it does make the CLI release a bit bigger and
more to manage, e.g. lots of Cats jars. I suspect it's not enough to
really matter (and most of the size is probably daffodil jars), but if
we do go this route, during or prior to migration it might be worth
looking to see if anything can be replaced. For example, one thing that
jumps out is VSCode debugger currently depends on logback for logging,
but daffodil use log4j.
The other concern is that currently the VS Code stuff can sort of
develop at its own pace. If we integrate the DAP portion into the CLI,
it means VSCode can't get DAP updates without a Daffodil release. But it
seems DAP is fairly stable, and as long as we keep a quick release cycle
and plan accordingly, that shouldn't be too much of an issue.
I would perfer this approach over creating a new repo. It's a bit less
to keep track of, and makes it easier to ensure we don't have any
breaking changes with changes to Daffodil.
On 7/19/22 1:52 PM, Mike Beckerle wrote:
> I think of this as a daffodil server mode, for the front end VSCode stuff
> to use.
>
> So, is it plausable to add the code to daffodil proper. Make it a CLI
> command mode to start up this server, so that
>
> daffodil vscodeServer
>
> starts it, optionally with connection options like what ports to use, etc.
>
>
> On Tue, Jul 19, 2022 at 1:45 PM Shane Dell <sh...@apache.org> wrote:
>
>> All,
>>
>> This thread is to discuss splitting out the code for the Apache Daffodil
>> Scala Debugger from the apache/daffodil-vscode repository. The two options
>> would be:
>>
>> - 1.) Move the debugger source code into the apache/daffodil repo. However,
>> will this work because the debugger code depends on certain daffodil Scala
>> libraries to be published? This is mostly an option since both are
>> mostly/fully Scala based.
>> - 2.) Create a new repo (apache/daffodil-debugger?,
>> apache/daffodil-debugger-scala?) where the Scala code will live.
>>
>> This would be helpful as the apache/daffodil-vscode repo is heavily based
>> on creating the VS Code extension for Daffodil. With this the debugger
>> source code is rarely updated, when it is they are pretty minor fixes or
>> dependency updates. Currently it is a bit of a cluster having a mix of
>> node/JavaScript/TypeScript and Scala which causes an issue with dependency
>> bots checking as these are checking for both npm and sbt/Scala
>> dependencies, causing many PRs to be made. The extension code also runs a
>> sbt universal:packageBin command in multiple occurences, being able to
>> remove this and downloading the debugger package would simplify a couple
>> different processes for the extension.
>>
>> My personal vote is for creating a new repo called something like either
>> apache/daffodil-debugger or apache/daffodil-debugger-scala.
>>
>> Sincerely,
>>
>> Shane Dell
>>
>
Re: Daffodil Debugger Scala Source Code
Posted by Mike Beckerle <mb...@apache.org>.
I think of this as a daffodil server mode, for the front end VSCode stuff
to use.
So, is it plausable to add the code to daffodil proper. Make it a CLI
command mode to start up this server, so that
daffodil vscodeServer
starts it, optionally with connection options like what ports to use, etc.
On Tue, Jul 19, 2022 at 1:45 PM Shane Dell <sh...@apache.org> wrote:
> All,
>
> This thread is to discuss splitting out the code for the Apache Daffodil
> Scala Debugger from the apache/daffodil-vscode repository. The two options
> would be:
>
> - 1.) Move the debugger source code into the apache/daffodil repo. However,
> will this work because the debugger code depends on certain daffodil Scala
> libraries to be published? This is mostly an option since both are
> mostly/fully Scala based.
> - 2.) Create a new repo (apache/daffodil-debugger?,
> apache/daffodil-debugger-scala?) where the Scala code will live.
>
> This would be helpful as the apache/daffodil-vscode repo is heavily based
> on creating the VS Code extension for Daffodil. With this the debugger
> source code is rarely updated, when it is they are pretty minor fixes or
> dependency updates. Currently it is a bit of a cluster having a mix of
> node/JavaScript/TypeScript and Scala which causes an issue with dependency
> bots checking as these are checking for both npm and sbt/Scala
> dependencies, causing many PRs to be made. The extension code also runs a
> sbt universal:packageBin command in multiple occurences, being able to
> remove this and downloading the debugger package would simplify a couple
> different processes for the extension.
>
> My personal vote is for creating a new repo called something like either
> apache/daffodil-debugger or apache/daffodil-debugger-scala.
>
> Sincerely,
>
> Shane Dell
>