You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemml.apache.org by Nakul Jindal <na...@gmail.com> on 2015/12/21 23:44:53 UTC
POC Eclipse IDE for DML
Hi,
I've been trying to build a Proof of Concept IDE for DML using Xtext.
https://github.com/nakul02/sysml.dml
Grammar File :
https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext
Here are some screenshots:
http://imgur.com/a/ZJyg7
The files shown are from the algorithms folder.
https://github.com/apache/incubator-systemml/tree/master/scripts/algorithms
(There was no particular reason to have chosen these files over others)
For this POC, my only goal was to get syntax highlighting to work in
Eclipse.
SystemML DML grammar is written in ANTLRv4 whereas Xtext uses ANTLRv3. Some
amount of work was needed on the grammar before it could be used it for
Xtext.
>From the screenshots, you can see that eclipse thinks there are syntactic
errors in the files. This has to do with statements ending in an optional
semicolon.
I only supplied xtext a grammar file from which to generate the IDE. I have
not yet worked on the generated code to add any other features.
I could not come up with a LL(*) grammar that would let me express optional
semicolons at the end of statements. For the time being, the grammar
requires semicolons at the end of statements (as you can see in the grammar
file :
https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext
)
I discussed this problem with Fred(@frreiss) and he suggested 3 ways of
dealing with this (assuming no way of modifying the grammar to fix this):
1. Support a subset of DML in the XText based tools
2. Modify the xtext generated tools to fix this
3. Write an eclipse plugin from scratch with complete control over lexer
and parser (among other components)
Option 1 requires the least amount of work.
I am not sure between options 2 & 3, which would be more work.
Is this useful to anyone?
If so, are there any thoughts or suggestions on how to approach the
optional semicolon problem?
Nakul Jindal
https://github.com/nakul02/
Re: POC Eclipse IDE for DML
Posted by Nakul Jindal <na...@gmail.com>.
@Derron -
To play around with the Xtext plugin, set up the env (I've tried it
eclipse):
1. Get the xtext eclipse from https://eclipse.org/Xtext/download.html
or install components in you existing eclipse using the update site.
2. Clone the project from https://github.com/nakul02/sysml.dml
3. In eclipse, import the project.
"File -> Import", "General > Existing Project into Workspace"
Navigate to your cloned copy and tick the "Search for nested projects"
4. Once everything shows up in Eclipse, right click on the
"GenerateDml.mwe2" file and "Run As -> MWE2 Workflow"
5. If needed, clean up ("Project -> Clean")
6. Make an eclipse application Run Configuration (with the defaults
selected) and run it.
7. In the new Eclipse instance, make a Java project and add ".dml" files.
The IDE should do syntax highlighting on it.
You can modify the grammar, run the "MWE2 Workflow" and launch the new
eclipse instance to see the changes.
Alternatively, you can get the Xtext eclipse, make a new project (project
name - sysml.dml, language name - sysml.Dml) and copy-paste the grammar
into the generated stub.
Other resources:
http://blogs.itemis.com/xtext/get-started-with-xtext-and-eclipse-in-5-minutes
https://eclipse.org/Xtext/documentation/102_domainmodelwalkthrough.html
https://eclipse.org/Xtext/documentation/301_grammarlanguage.html
http://www.euclideanspace.com/software/development/eclipse/xtext/grammar/lexer/index.htm
About hard coding the builtin functions - for now they are in the grammar,
because that was the easier thing to do (and they make for nice
screenshots). Since the language allows users to define a function with the
same name as standard functions, putting it into content assist (with a
bit more smartness) might be right thing to do.
Also, I've missed a couple of builtin functions like read, write.
On Sat, Jan 2, 2016 at 4:33 PM, Matthias Boehm <mb...@us.ibm.com> wrote:
> well, in general all of our builtin functions are handled in
> BuiltinFunctionExpression - so that's the place to look at if the language
> reference is not enough. The only reason why as.scalar and similar
> functions appear in the grammar is a workaround for functions which violate
> our own function naming convention (i.e., user-defined functions must not
> contain a '.'). However, this was a compromise in favor of users familiar
> with R (which provides functions like as.integer, as.logical, as.matrix etc
> as well).
>
> Regards,
> Matthias
>
>
> [image: Inactive hide details for Deron Eriksson ---01/02/2016 04:24:16
> PM---Hi Matthias, Thank you for the clarification with regards]Deron
> Eriksson ---01/02/2016 04:24:16 PM---Hi Matthias, Thank you for the
> clarification with regards to the parser grammar and
>
> From: Deron Eriksson <de...@gmail.com>
> To: dev@systemml.incubator.apache.org
> Date: 01/02/2016 04:24 PM
> Subject: Re: POC Eclipse IDE for DML
> ------------------------------
>
>
>
> Hi Matthias,
>
> Thank you for the clarification with regards to the parser grammar and
> built-in functions. To me, as a relative newbie, in one way, the built-in
> functions seem similar to java.lang.* classes in Java, but in another way,
> they seem almost even more intrinsic than that.
>
> I guess what I was mainly trying to get at is that Dml.g4 is fantastic in
> terms of flexibility, especially when considered in conjunction with the
> supporting referenced Java classes, but Nakul's grammar for IDE support is
> able to convey the essence of the language capabilities (such as built-in
> functions) in one place.
>
> Also, consider the following. The icdf() function is not in Dml.g4 since it
> is a built-in function. It is handled in the
> parser.ParameterizedBuiltinFunctionExpression class. On the other hand,
> as.scalar() is in Dml.g4 and in parser.BuiltinFunctionExpression. However,
> castAsScalar() is not in Dml.g4 but is handled similarly in
> parser.BuiltinFunctionExpression.
>
> As a result of this, I am now confused. If I am trying to understand what I
> can do with the language, do I look in Dml.g4 (such as for as.scalar()), do
> I look in BuiltinFunctionExpression (such as for as.scalar() and
> castAsScalar()), or do I look in ParameterizedBuiltinFunctionExpression
> (such as for icdf())?
>
> Perhaps as.scalar() needs to be removed from the grammar and as.scalar()
> and castAsScalar() need to be condensed to one function and I will be less
> confused. :-)
>
> Maybe I am overthinking things. The DML Language Reference does contain a
> tremendous amount of useful information with regards to the language
> capabilities.
>
> Deron
>
>
>
>
> On Sat, Jan 2, 2016 at 12:25 PM, Matthias Boehm <mb...@us.ibm.com> wrote:
>
> > just to clarify, our parser grammar does not include builtin functions
> > because they are -- in contrast to keywords and language constructs --
> not
> > part of the DML/PyDML syntax. This is important for both maintainability
> > and flexibility. For example, it allows you to define a variable or
> > user-defined function with the name 't' although there exists a builtin
> > function t() for transpose operations.
> >
> > Regards,
> > Matthias
> >
> >
> > [image: Inactive hide details for Deron Eriksson ---01/01/2016 11:38:27
> > AM---Hi Nakul, This is very cool! I see great value in IDE tool]Deron
> > Eriksson ---01/01/2016 11:38:27 AM---Hi Nakul, This is very cool! I see
> > great value in IDE tools such as an Eclipse editor
> >
> > From: Deron Eriksson <de...@gmail.com>
> > To: dev@systemml.incubator.apache.org
> > Date: 01/01/2016 11:38 AM
> > Subject: Re: POC Eclipse IDE for DML
> > ------------------------------
> >
> >
> >
> > Hi Nakul,
> >
> > This is very cool! I see great value in IDE tools such as an Eclipse
> editor
> > for DML/PyDML. Highlighting syntax errors helps people catch syntax
> > mistakes before running the script, and IDE code completion is a great
> > productivity boost.
> >
> > Also, I like how clean and readable the grammar is. I don't know how
> others
> > feel, but it is so clean that, if flushed out, it might be a useful
> > addition to the DML Language Reference. It does things such as enumerate
> > the built-in functions, which is handy when trying to wrap your mind
> around
> > the language. The Dml.g4 file is very powerful and useful but I have a
> > harder time understanding the language syntax when looking at it since
> > things like built-in functions aren't included in it. I wonder if there
> is
> > a way to integrate some of your work into Dml.g4?
> >
> > I haven't done Xtext before. Do you have any advice to get started if I'd
> > like to try this out in my Eclipse (for instance, to look at the
> semicolon
> > issue)?
> >
> > Deron
> >
> >
> >
> > On Mon, Dec 21, 2015 at 2:44 PM, Nakul Jindal <na...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I've been trying to build a Proof of Concept IDE for DML using Xtext.
> > >
> > > https://github.com/nakul02/sysml.dml
> > >
> > > Grammar File :
> > >
> > >
> >
> https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext
> > >
> > > Here are some screenshots:
> > > http://imgur.com/a/ZJyg7
> > >
> > > The files shown are from the algorithms folder.
> > >
> >
> https://github.com/apache/incubator-systemml/tree/master/scripts/algorithms
> > >
> > > (There was no particular reason to have chosen these files over others)
> > >
> > > For this POC, my only goal was to get syntax highlighting to work in
> > > Eclipse.
> > > SystemML DML grammar is written in ANTLRv4 whereas Xtext uses ANTLRv3.
> > Some
> > > amount of work was needed on the grammar before it could be used it for
> > > Xtext.
> > >
> > > From the screenshots, you can see that eclipse thinks there are
> syntactic
> > > errors in the files. This has to do with statements ending in an
> optional
> > > semicolon.
> > >
> > > I only supplied xtext a grammar file from which to generate the IDE. I
> > have
> > > not yet worked on the generated code to add any other features.
> > > I could not come up with a LL(*) grammar that would let me express
> > optional
> > > semicolons at the end of statements. For the time being, the grammar
> > > requires semicolons at the end of statements (as you can see in the
> > grammar
> > > file :
> > >
> > >
> >
> https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext
> > > )
> > >
> > > I discussed this problem with Fred(@frreiss) and he suggested 3 ways of
> > > dealing with this (assuming no way of modifying the grammar to fix
> this):
> > >
> > > 1. Support a subset of DML in the XText based tools
> > > 2. Modify the xtext generated tools to fix this
> > > 3. Write an eclipse plugin from scratch with complete control over
> lexer
> > > and parser (among other components)
> > >
> > > Option 1 requires the least amount of work.
> > > I am not sure between options 2 & 3, which would be more work.
> > >
> > >
> > > Is this useful to anyone?
> > > If so, are there any thoughts or suggestions on how to approach the
> > > optional semicolon problem?
> > >
> > >
> > > Nakul Jindal
> > > https://github.com/nakul02/
> > >
> >
> >
> >
>
>
>
Re: POC Eclipse IDE for DML
Posted by Matthias Boehm <mb...@us.ibm.com>.
well, in general all of our builtin functions are handled in
BuiltinFunctionExpression - so that's the place to look at if the language
reference is not enough. The only reason why as.scalar and similar
functions appear in the grammar is a workaround for functions which violate
our own function naming convention (i.e., user-defined functions must not
contain a '.'). However, this was a compromise in favor of users familiar
with R (which provides functions like as.integer, as.logical, as.matrix etc
as well).
Regards,
Matthias
From: Deron Eriksson <de...@gmail.com>
To: dev@systemml.incubator.apache.org
Date: 01/02/2016 04:24 PM
Subject: Re: POC Eclipse IDE for DML
Hi Matthias,
Thank you for the clarification with regards to the parser grammar and
built-in functions. To me, as a relative newbie, in one way, the built-in
functions seem similar to java.lang.* classes in Java, but in another way,
they seem almost even more intrinsic than that.
I guess what I was mainly trying to get at is that Dml.g4 is fantastic in
terms of flexibility, especially when considered in conjunction with the
supporting referenced Java classes, but Nakul's grammar for IDE support is
able to convey the essence of the language capabilities (such as built-in
functions) in one place.
Also, consider the following. The icdf() function is not in Dml.g4 since it
is a built-in function. It is handled in the
parser.ParameterizedBuiltinFunctionExpression class. On the other hand,
as.scalar() is in Dml.g4 and in parser.BuiltinFunctionExpression. However,
castAsScalar() is not in Dml.g4 but is handled similarly in
parser.BuiltinFunctionExpression.
As a result of this, I am now confused. If I am trying to understand what I
can do with the language, do I look in Dml.g4 (such as for as.scalar()), do
I look in BuiltinFunctionExpression (such as for as.scalar() and
castAsScalar()), or do I look in ParameterizedBuiltinFunctionExpression
(such as for icdf())?
Perhaps as.scalar() needs to be removed from the grammar and as.scalar()
and castAsScalar() need to be condensed to one function and I will be less
confused. :-)
Maybe I am overthinking things. The DML Language Reference does contain a
tremendous amount of useful information with regards to the language
capabilities.
Deron
On Sat, Jan 2, 2016 at 12:25 PM, Matthias Boehm <mb...@us.ibm.com> wrote:
> just to clarify, our parser grammar does not include builtin functions
> because they are -- in contrast to keywords and language constructs --
not
> part of the DML/PyDML syntax. This is important for both maintainability
> and flexibility. For example, it allows you to define a variable or
> user-defined function with the name 't' although there exists a builtin
> function t() for transpose operations.
>
> Regards,
> Matthias
>
>
> [image: Inactive hide details for Deron Eriksson ---01/01/2016 11:38:27
> AM---Hi Nakul, This is very cool! I see great value in IDE tool]Deron
> Eriksson ---01/01/2016 11:38:27 AM---Hi Nakul, This is very cool! I see
> great value in IDE tools such as an Eclipse editor
>
> From: Deron Eriksson <de...@gmail.com>
> To: dev@systemml.incubator.apache.org
> Date: 01/01/2016 11:38 AM
> Subject: Re: POC Eclipse IDE for DML
> ------------------------------
>
>
>
> Hi Nakul,
>
> This is very cool! I see great value in IDE tools such as an Eclipse
editor
> for DML/PyDML. Highlighting syntax errors helps people catch syntax
> mistakes before running the script, and IDE code completion is a great
> productivity boost.
>
> Also, I like how clean and readable the grammar is. I don't know how
others
> feel, but it is so clean that, if flushed out, it might be a useful
> addition to the DML Language Reference. It does things such as enumerate
> the built-in functions, which is handy when trying to wrap your mind
around
> the language. The Dml.g4 file is very powerful and useful but I have a
> harder time understanding the language syntax when looking at it since
> things like built-in functions aren't included in it. I wonder if there
is
> a way to integrate some of your work into Dml.g4?
>
> I haven't done Xtext before. Do you have any advice to get started if I'd
> like to try this out in my Eclipse (for instance, to look at the
semicolon
> issue)?
>
> Deron
>
>
>
> On Mon, Dec 21, 2015 at 2:44 PM, Nakul Jindal <na...@gmail.com> wrote:
>
> > Hi,
> >
> > I've been trying to build a Proof of Concept IDE for DML using Xtext.
> >
> > https://github.com/nakul02/sysml.dml
> >
> > Grammar File :
> >
> >
>
https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext
> >
> > Here are some screenshots:
> > http://imgur.com/a/ZJyg7
> >
> > The files shown are from the algorithms folder.
> >
>
https://github.com/apache/incubator-systemml/tree/master/scripts/algorithms
> >
> > (There was no particular reason to have chosen these files over others)
> >
> > For this POC, my only goal was to get syntax highlighting to work in
> > Eclipse.
> > SystemML DML grammar is written in ANTLRv4 whereas Xtext uses ANTLRv3.
> Some
> > amount of work was needed on the grammar before it could be used it for
> > Xtext.
> >
> > From the screenshots, you can see that eclipse thinks there are
syntactic
> > errors in the files. This has to do with statements ending in an
optional
> > semicolon.
> >
> > I only supplied xtext a grammar file from which to generate the IDE. I
> have
> > not yet worked on the generated code to add any other features.
> > I could not come up with a LL(*) grammar that would let me express
> optional
> > semicolons at the end of statements. For the time being, the grammar
> > requires semicolons at the end of statements (as you can see in the
> grammar
> > file :
> >
> >
>
https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext
> > )
> >
> > I discussed this problem with Fred(@frreiss) and he suggested 3 ways of
> > dealing with this (assuming no way of modifying the grammar to fix
this):
> >
> > 1. Support a subset of DML in the XText based tools
> > 2. Modify the xtext generated tools to fix this
> > 3. Write an eclipse plugin from scratch with complete control over
lexer
> > and parser (among other components)
> >
> > Option 1 requires the least amount of work.
> > I am not sure between options 2 & 3, which would be more work.
> >
> >
> > Is this useful to anyone?
> > If so, are there any thoughts or suggestions on how to approach the
> > optional semicolon problem?
> >
> >
> > Nakul Jindal
> > https://github.com/nakul02/
> >
>
>
>
Re: POC Eclipse IDE for DML
Posted by Deron Eriksson <de...@gmail.com>.
Hi Matthias,
Thank you for the clarification with regards to the parser grammar and
built-in functions. To me, as a relative newbie, in one way, the built-in
functions seem similar to java.lang.* classes in Java, but in another way,
they seem almost even more intrinsic than that.
I guess what I was mainly trying to get at is that Dml.g4 is fantastic in
terms of flexibility, especially when considered in conjunction with the
supporting referenced Java classes, but Nakul's grammar for IDE support is
able to convey the essence of the language capabilities (such as built-in
functions) in one place.
Also, consider the following. The icdf() function is not in Dml.g4 since it
is a built-in function. It is handled in the
parser.ParameterizedBuiltinFunctionExpression class. On the other hand,
as.scalar() is in Dml.g4 and in parser.BuiltinFunctionExpression. However,
castAsScalar() is not in Dml.g4 but is handled similarly in
parser.BuiltinFunctionExpression.
As a result of this, I am now confused. If I am trying to understand what I
can do with the language, do I look in Dml.g4 (such as for as.scalar()), do
I look in BuiltinFunctionExpression (such as for as.scalar() and
castAsScalar()), or do I look in ParameterizedBuiltinFunctionExpression
(such as for icdf())?
Perhaps as.scalar() needs to be removed from the grammar and as.scalar()
and castAsScalar() need to be condensed to one function and I will be less
confused. :-)
Maybe I am overthinking things. The DML Language Reference does contain a
tremendous amount of useful information with regards to the language
capabilities.
Deron
On Sat, Jan 2, 2016 at 12:25 PM, Matthias Boehm <mb...@us.ibm.com> wrote:
> just to clarify, our parser grammar does not include builtin functions
> because they are -- in contrast to keywords and language constructs -- not
> part of the DML/PyDML syntax. This is important for both maintainability
> and flexibility. For example, it allows you to define a variable or
> user-defined function with the name 't' although there exists a builtin
> function t() for transpose operations.
>
> Regards,
> Matthias
>
>
> [image: Inactive hide details for Deron Eriksson ---01/01/2016 11:38:27
> AM---Hi Nakul, This is very cool! I see great value in IDE tool]Deron
> Eriksson ---01/01/2016 11:38:27 AM---Hi Nakul, This is very cool! I see
> great value in IDE tools such as an Eclipse editor
>
> From: Deron Eriksson <de...@gmail.com>
> To: dev@systemml.incubator.apache.org
> Date: 01/01/2016 11:38 AM
> Subject: Re: POC Eclipse IDE for DML
> ------------------------------
>
>
>
> Hi Nakul,
>
> This is very cool! I see great value in IDE tools such as an Eclipse editor
> for DML/PyDML. Highlighting syntax errors helps people catch syntax
> mistakes before running the script, and IDE code completion is a great
> productivity boost.
>
> Also, I like how clean and readable the grammar is. I don't know how others
> feel, but it is so clean that, if flushed out, it might be a useful
> addition to the DML Language Reference. It does things such as enumerate
> the built-in functions, which is handy when trying to wrap your mind around
> the language. The Dml.g4 file is very powerful and useful but I have a
> harder time understanding the language syntax when looking at it since
> things like built-in functions aren't included in it. I wonder if there is
> a way to integrate some of your work into Dml.g4?
>
> I haven't done Xtext before. Do you have any advice to get started if I'd
> like to try this out in my Eclipse (for instance, to look at the semicolon
> issue)?
>
> Deron
>
>
>
> On Mon, Dec 21, 2015 at 2:44 PM, Nakul Jindal <na...@gmail.com> wrote:
>
> > Hi,
> >
> > I've been trying to build a Proof of Concept IDE for DML using Xtext.
> >
> > https://github.com/nakul02/sysml.dml
> >
> > Grammar File :
> >
> >
> https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext
> >
> > Here are some screenshots:
> > http://imgur.com/a/ZJyg7
> >
> > The files shown are from the algorithms folder.
> >
> https://github.com/apache/incubator-systemml/tree/master/scripts/algorithms
> >
> > (There was no particular reason to have chosen these files over others)
> >
> > For this POC, my only goal was to get syntax highlighting to work in
> > Eclipse.
> > SystemML DML grammar is written in ANTLRv4 whereas Xtext uses ANTLRv3.
> Some
> > amount of work was needed on the grammar before it could be used it for
> > Xtext.
> >
> > From the screenshots, you can see that eclipse thinks there are syntactic
> > errors in the files. This has to do with statements ending in an optional
> > semicolon.
> >
> > I only supplied xtext a grammar file from which to generate the IDE. I
> have
> > not yet worked on the generated code to add any other features.
> > I could not come up with a LL(*) grammar that would let me express
> optional
> > semicolons at the end of statements. For the time being, the grammar
> > requires semicolons at the end of statements (as you can see in the
> grammar
> > file :
> >
> >
> https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext
> > )
> >
> > I discussed this problem with Fred(@frreiss) and he suggested 3 ways of
> > dealing with this (assuming no way of modifying the grammar to fix this):
> >
> > 1. Support a subset of DML in the XText based tools
> > 2. Modify the xtext generated tools to fix this
> > 3. Write an eclipse plugin from scratch with complete control over lexer
> > and parser (among other components)
> >
> > Option 1 requires the least amount of work.
> > I am not sure between options 2 & 3, which would be more work.
> >
> >
> > Is this useful to anyone?
> > If so, are there any thoughts or suggestions on how to approach the
> > optional semicolon problem?
> >
> >
> > Nakul Jindal
> > https://github.com/nakul02/
> >
>
>
>
Re: POC Eclipse IDE for DML
Posted by Matthias Boehm <mb...@us.ibm.com>.
just to clarify, our parser grammar does not include builtin functions
because they are -- in contrast to keywords and language constructs -- not
part of the DML/PyDML syntax. This is important for both maintainability
and flexibility. For example, it allows you to define a variable or
user-defined function with the name 't' although there exists a builtin
function t() for transpose operations.
Regards,
Matthias
From: Deron Eriksson <de...@gmail.com>
To: dev@systemml.incubator.apache.org
Date: 01/01/2016 11:38 AM
Subject: Re: POC Eclipse IDE for DML
Hi Nakul,
This is very cool! I see great value in IDE tools such as an Eclipse editor
for DML/PyDML. Highlighting syntax errors helps people catch syntax
mistakes before running the script, and IDE code completion is a great
productivity boost.
Also, I like how clean and readable the grammar is. I don't know how others
feel, but it is so clean that, if flushed out, it might be a useful
addition to the DML Language Reference. It does things such as enumerate
the built-in functions, which is handy when trying to wrap your mind around
the language. The Dml.g4 file is very powerful and useful but I have a
harder time understanding the language syntax when looking at it since
things like built-in functions aren't included in it. I wonder if there is
a way to integrate some of your work into Dml.g4?
I haven't done Xtext before. Do you have any advice to get started if I'd
like to try this out in my Eclipse (for instance, to look at the semicolon
issue)?
Deron
On Mon, Dec 21, 2015 at 2:44 PM, Nakul Jindal <na...@gmail.com> wrote:
> Hi,
>
> I've been trying to build a Proof of Concept IDE for DML using Xtext.
>
> https://github.com/nakul02/sysml.dml
>
> Grammar File :
>
>
https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext
>
> Here are some screenshots:
> http://imgur.com/a/ZJyg7
>
> The files shown are from the algorithms folder.
>
https://github.com/apache/incubator-systemml/tree/master/scripts/algorithms
>
> (There was no particular reason to have chosen these files over others)
>
> For this POC, my only goal was to get syntax highlighting to work in
> Eclipse.
> SystemML DML grammar is written in ANTLRv4 whereas Xtext uses ANTLRv3.
Some
> amount of work was needed on the grammar before it could be used it for
> Xtext.
>
> From the screenshots, you can see that eclipse thinks there are syntactic
> errors in the files. This has to do with statements ending in an optional
> semicolon.
>
> I only supplied xtext a grammar file from which to generate the IDE. I
have
> not yet worked on the generated code to add any other features.
> I could not come up with a LL(*) grammar that would let me express
optional
> semicolons at the end of statements. For the time being, the grammar
> requires semicolons at the end of statements (as you can see in the
grammar
> file :
>
>
https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext
> )
>
> I discussed this problem with Fred(@frreiss) and he suggested 3 ways of
> dealing with this (assuming no way of modifying the grammar to fix this):
>
> 1. Support a subset of DML in the XText based tools
> 2. Modify the xtext generated tools to fix this
> 3. Write an eclipse plugin from scratch with complete control over lexer
> and parser (among other components)
>
> Option 1 requires the least amount of work.
> I am not sure between options 2 & 3, which would be more work.
>
>
> Is this useful to anyone?
> If so, are there any thoughts or suggestions on how to approach the
> optional semicolon problem?
>
>
> Nakul Jindal
> https://github.com/nakul02/
>
Re: POC Eclipse IDE for DML
Posted by Deron Eriksson <de...@gmail.com>.
Hi Nakul,
This is very cool! I see great value in IDE tools such as an Eclipse editor
for DML/PyDML. Highlighting syntax errors helps people catch syntax
mistakes before running the script, and IDE code completion is a great
productivity boost.
Also, I like how clean and readable the grammar is. I don't know how others
feel, but it is so clean that, if flushed out, it might be a useful
addition to the DML Language Reference. It does things such as enumerate
the built-in functions, which is handy when trying to wrap your mind around
the language. The Dml.g4 file is very powerful and useful but I have a
harder time understanding the language syntax when looking at it since
things like built-in functions aren't included in it. I wonder if there is
a way to integrate some of your work into Dml.g4?
I haven't done Xtext before. Do you have any advice to get started if I'd
like to try this out in my Eclipse (for instance, to look at the semicolon
issue)?
Deron
On Mon, Dec 21, 2015 at 2:44 PM, Nakul Jindal <na...@gmail.com> wrote:
> Hi,
>
> I've been trying to build a Proof of Concept IDE for DML using Xtext.
>
> https://github.com/nakul02/sysml.dml
>
> Grammar File :
>
> https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext
>
> Here are some screenshots:
> http://imgur.com/a/ZJyg7
>
> The files shown are from the algorithms folder.
> https://github.com/apache/incubator-systemml/tree/master/scripts/algorithms
>
> (There was no particular reason to have chosen these files over others)
>
> For this POC, my only goal was to get syntax highlighting to work in
> Eclipse.
> SystemML DML grammar is written in ANTLRv4 whereas Xtext uses ANTLRv3. Some
> amount of work was needed on the grammar before it could be used it for
> Xtext.
>
> From the screenshots, you can see that eclipse thinks there are syntactic
> errors in the files. This has to do with statements ending in an optional
> semicolon.
>
> I only supplied xtext a grammar file from which to generate the IDE. I have
> not yet worked on the generated code to add any other features.
> I could not come up with a LL(*) grammar that would let me express optional
> semicolons at the end of statements. For the time being, the grammar
> requires semicolons at the end of statements (as you can see in the grammar
> file :
>
> https://github.com/nakul02/sysml.dml/blob/master/sysml.dml/src/sysml/Dml.xtext
> )
>
> I discussed this problem with Fred(@frreiss) and he suggested 3 ways of
> dealing with this (assuming no way of modifying the grammar to fix this):
>
> 1. Support a subset of DML in the XText based tools
> 2. Modify the xtext generated tools to fix this
> 3. Write an eclipse plugin from scratch with complete control over lexer
> and parser (among other components)
>
> Option 1 requires the least amount of work.
> I am not sure between options 2 & 3, which would be more work.
>
>
> Is this useful to anyone?
> If so, are there any thoughts or suggestions on how to approach the
> optional semicolon problem?
>
>
> Nakul Jindal
> https://github.com/nakul02/
>