DFDL Intellisense work - which repo?

[Mike Beckerle <mb...@apache.org>]

Where should we put the additional VSCode DFDL Intellisense editing
capability?

It's not integrated into the VSCode debugger at this time, though they may
get combined some day in the future. Arguably, some people may want the
editing without the debugging, but the opposite is a lot less likely.

Two possibilities (maybe there are others?) are:

(a) create an orphan git branch in the VSCode Debugger repo.

or

(b) create a new repo for this work.

W.r.t. (b) Down the road if these two VSCode-related bases do converge into
one code base, then we'd need also to merge the issues/tickets, wiki pages,
and any other artifacts. Not sure how hard that is, but since they're both
github.... I would hope we're not the first people to need to do this.

RE: DFDL Intellisense work - which repo?

[Regis Thomas <re...@nteligen.com>]

I agree that dfdl intellisense should be combined with the daffodil debugger/editor extension.  There are some issues that would need to be resolved.  The daffodil debugger package.json files has a number of configuration entries where the language is defined as "xml".  The language configuration provides the editor portion of the daffodil debugger with auto-completion, formatting, and colorization from XML language extensions.  The daffodil debugger is configured to a use a snippet file to provide auto-completion of dfdl specific items.  The auto-completion from the snippet file makes suggestions based on matching letters typed in the editor to the name of a snippet.  These suggestions are combined with the suggestions from the XML language extension and any matching words that have been previously typed.

The dfdl intellisense extension defines a new "dfdl" language and the extension is configured to use the "dfdl" language.  By using "dfdl" as a language, the intellisense suggestions come only from the dfdl-intellisense extension.  The current version of the dfdl intellisense extension provides suggestions based on the Data Format Description Language (https://daffodil.apache.org/docs/dfdl/).  The extension still uses snippets, but they are wrapped in TypeScript code which allows for logic to control which suggestions are given based on the context where they are triggered.  This limits the suggestions much more than the original snippet file plus XML language plus matching words based suggestions.  However, the current version of the dfdl intellisense extension lacks a dfdl grammar file which supplies additional formatting and colorization.

Both extensions use the common extension file names like package.json, extension.json, extension.ts, launch.json, settings.json, tasks.json, etc.  If we merge the source code for the two projects at this point we would lose colorization for the dfdl schema code in the editor of the daffodil debug extension.  We might also lose some additional functionality provided by the XML grammar, for instance curly brace completion with quotes is provided by a work around in the dfdl intellisense extension.

Java Script provides a utility that can be used in VS Code to "bundle" two (or more) extensions together (https://code.visualstudio.com/api/working-with-extensions/bundling-extension).  Bundling allows for the packaging of extensions into a single install package, but doesn't provide merging of source code.  Bundling of the two extensions has not been tested.  I'm not sure if the "xml" language configuration and the "dfdl' language configuration would be compatible with each other.

The information above may provide more context in deciding how best to get the dfdl intellisense code into the repository.

-----Original Message-----
From: Steve Lawrence <sl...@apache.org> 
Sent: Friday, February 18, 2022 10:33 AM
To: dev@daffodil.apache.org
Subject: Re: DFDL Intellisense work - which repo?

> Arguably, some people may want the editing without the debugging, but the opposite is a lot less likely.

I'm not sure how likely it would be to only want editing. I would think if someone wants to edit schemas using VSCode it's very likely they'll also to want to actually parse/unparse some files using that schema. 
They may not necessarily need all the functionality like step/breakpoint/hex editor/etc that the debugger provides, but at the very least they'll  want the ability to parse/unparse and see the output, which the debugger can do. And when the parse/unparse doesn't go as expected (which will happen), already having the debugger might be useful.


I'm not familiar with Intellisense and VSCode extensions, how different is it from the existing debugger infrastructure? Is it completely different or is it just adding a new directory/file/properties/something, and yarn build can put it in the same vsix file?

If they really are the same build infrastructure, maybe another option:

(c) add a new directory to the daffodil-vscode repo and integrate it all into one extension


I'm not really a fan of (a) creating an orphan git branch. If one branch 
is called "main" and the other "intellisense" or something, it makes it 
feel like intellisense is a second class thing, and will potentially 
have less visibility. Where ever it lives, I think it should be on a 
main branch.

So I'd rather we pick between (b) create a new repo and (c) add a new 
directory to the existing repo (or maybe another option?).

If the intellisense and vscode infrastructure is really different or we 
want completely different release schedules/bug tracking/etc, then maybe 
(b) makes more sense. I dont' have a strong preference one way or the 
other, but I'd like to hear from people with more experience with VS 
Code and intellisense.


Regarding the concern about the need to transfer issues if we start with 
(b) but eventually move towards (c), github does have a Transfer Issue 
option. It doesn't look like it actually works though--I think Infra 
might need to enable something.


Lastly, where ever it lives, the intellisense code should be integrated 
sooner rather than later, even if it's not fully functional. I would 
like to avoid the whole IP clearance stuff we had to through with the VS 
Code debugger.



On 2/16/22 3:15 PM, Mike Beckerle wrote:
> Where should we put the additional VSCode DFDL Intellisense editing
> capability?
> 
> It's not integrated into the VSCode debugger at this time, though they may
> get combined some day in the future. Arguably, some people may want the
> editing without the debugging, but the opposite is a lot less likely.
> 
> Two possibilities (maybe there are others?) are:
> 
> (a) create an orphan git branch in the VSCode Debugger repo.
> 
> or
> 
> (b) create a new repo for this work.
> 
> W.r.t. (b) Down the road if these two VSCode-related bases do converge into
> one code base, then we'd need also to merge the issues/tickets, wiki pages,
> and any other artifacts. Not sure how hard that is, but since they're both
> github.... I would hope we're not the first people to need to do this.
> 

Re: DFDL Intellisense work - which repo?

[Steve Lawrence <sl...@apache.org>]

> Arguably, some people may want the editing without the debugging, but the opposite is a lot less likely.

I'm not sure how likely it would be to only want editing. I would think 
if someone wants to edit schemas using VSCode it's very likely they'll 
also to want to actually parse/unparse some files using that schema. 
They may not necessarily need all the functionality like 
step/breakpoint/hex editor/etc that the debugger provides, but at the 
very least they'll  want the ability to parse/unparse and see the 
output, which the debugger can do. And when the parse/unparse doesn't go 
as expected (which will happen), already having the debugger might be 
useful.


I'm not familiar with Intellisense and VSCode extensions, how different 
is it from the existing debugger infrastructure? Is it completely 
different or is it just adding a new 
directory/file/properties/something, and yarn build can put it in the 
same vsix file?

If they really are the same build infrastructure, maybe another option:

(c) add a new directory to the daffodil-vscode repo and integrate it all 
into one extension


I'm not really a fan of (a) creating an orphan git branch. If one branch 
is called "main" and the other "intellisense" or something, it makes it 
feel like intellisense is a second class thing, and will potentially 
have less visibility. Where ever it lives, I think it should be on a 
main branch.

So I'd rather we pick between (b) create a new repo and (c) add a new 
directory to the existing repo (or maybe another option?).

If the intellisense and vscode infrastructure is really different or we 
want completely different release schedules/bug tracking/etc, then maybe 
(b) makes more sense. I dont' have a strong preference one way or the 
other, but I'd like to hear from people with more experience with VS 
Code and intellisense.


Regarding the concern about the need to transfer issues if we start with 
(b) but eventually move towards (c), github does have a Transfer Issue 
option. It doesn't look like it actually works though--I think Infra 
might need to enable something.


Lastly, where ever it lives, the intellisense code should be integrated 
sooner rather than later, even if it's not fully functional. I would 
like to avoid the whole IP clearance stuff we had to through with the VS 
Code debugger.



On 2/16/22 3:15 PM, Mike Beckerle wrote:
> Where should we put the additional VSCode DFDL Intellisense editing
> capability?
> 
> It's not integrated into the VSCode debugger at this time, though they may
> get combined some day in the future. Arguably, some people may want the
> editing without the debugging, but the opposite is a lot less likely.
> 
> Two possibilities (maybe there are others?) are:
> 
> (a) create an orphan git branch in the VSCode Debugger repo.
> 
> or
> 
> (b) create a new repo for this work.
> 
> W.r.t. (b) Down the road if these two VSCode-related bases do converge into
> one code base, then we'd need also to merge the issues/tickets, wiki pages,
> and any other artifacts. Not sure how hard that is, but since they're both
> github.... I would hope we're not the first people to need to do this.
>