You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@netbeans.apache.org by GitBox <gi...@apache.org> on 2022/10/16 16:23:51 UTC
[GitHub] [netbeans] lkishalmi opened a new pull request, #4800: ANTLRv4 indent and code snippet support.
lkishalmi opened a new pull request, #4800:
URL: https://github.com/apache/netbeans/pull/4800
This is my last round of features for ANTLR support, mainly for v4 grammars. Add indent support in rule definitions, and a few code shortcuts.
Beside of that, prevented a few of NPE-s that could happen on parsing an incomplete grammar. Also removed the ConsoleErrorListener.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] lkishalmi merged pull request #4800: ANTLRv4 indent and code snippet support.
Posted by GitBox <gi...@apache.org>.
lkishalmi merged PR #4800:
URL: https://github.com/apache/netbeans/pull/4800
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] lkishalmi commented on a diff in pull request #4800: ANTLRv4 indent and code snippet support.
Posted by GitBox <gi...@apache.org>.
lkishalmi commented on code in PR #4800:
URL: https://github.com/apache/netbeans/pull/4800#discussion_r997188284
##########
java/languages.antlr/src/org/netbeans/modules/languages/antlr/v4/Antlr4CompletionProvider.java:
##########
@@ -143,12 +129,12 @@ protected void query(CompletionResultSet resultSet, Document doc, int caretOffse
tokens.previous();
lookAround(fo, tokens, caretOffset, prefix, resultSet);
} else {
- //Empty grammar so far offer lexer and grammar
- addTokens("", caretOffset, resultSet, "lexer", "grammar");
+ //Empty grammar so far offer lexer, parser and grammar
+ addTokens("", caretOffset, resultSet, "lexer", "parser", "grammar");
}
}
} catch (Throwable th) {
- System.out.println(th);
+ //System.out.println(th);
Review Comment:
Well, removed that one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] mbien commented on a diff in pull request #4800: ANTLRv4 indent and code snippet support.
Posted by GitBox <gi...@apache.org>.
mbien commented on code in PR #4800:
URL: https://github.com/apache/netbeans/pull/4800#discussion_r996996970
##########
java/languages.antlr/src/org/netbeans/modules/languages/antlr/v4/Antlr4Formatter.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.netbeans.modules.languages.antlr.v4;
+
+import java.util.prefs.Preferences;
+import javax.swing.text.BadLocationException;
+import javax.swing.text.Document;
+import org.antlr.parser.antlr4.ANTLRv4Lexer;
+import org.antlr.v4.runtime.CharStreams;
+import org.netbeans.modules.csl.api.Formatter;
+import org.netbeans.modules.csl.spi.ParserResult;
+import org.netbeans.modules.editor.indent.spi.Context;
+
+import static org.antlr.parser.antlr4.ANTLRv4Lexer.*;
+import org.antlr.v4.runtime.Token;
+import org.netbeans.api.editor.document.LineDocument;
+import org.netbeans.api.editor.document.LineDocumentUtils;
+import org.netbeans.api.editor.settings.SimpleValueNames;
+import org.netbeans.modules.editor.indent.spi.CodeStylePreferences;
+
+/**
+ *
+ * @author lkishalmi
+ */
+public class Antlr4Formatter implements Formatter {
+
+ public Antlr4Formatter() {
+ }
+
+ @Override
+ public void reformat(Context context, ParserResult compilationInfo) {
+ LineDocument doc = LineDocumentUtils.as(context.document(), LineDocument.class);
+ String text;
+ int indentSize = getIndentSize(context.document());
+ try {
+ text = doc.getText(0, doc.getLength());
+ } catch (BadLocationException ex) {
+ return;
+ }
+
+ ANTLRv4Lexer lexer = new ANTLRv4Lexer(CharStreams.fromString(text));
+ Token token = lexer.nextToken();
+ if (token.getType() == EOF) {
+ return;
+ }
+
+ final int cstart = context.startOffset();
+ final int cend = context.endOffset();
+
+ int tstart = token.getStartIndex();
+ int tstop = token.getStopIndex();
+
+ try {
+ boolean inRule = false;
+ int ruleLine = 0;
+ int parenDepth = 0;
+ int textDelta = 0;
+ int prevTokenType = -1;
+ while ((token.getType() != EOF) && (tstart < cend)) {
+ switch (token.getType()) {
+ case RULE_REF:
+ case TOKEN_REF:
+ // @header , @member, etc, are not real rules;
+ if (!inRule && (prevTokenType != AT)) {
+ inRule = true;
+ }
+ break;
+ case LPAREN:
+ parenDepth++;
+ break;
+ case RPAREN:
+ parenDepth--;
+ break;
+ case SEMI:
+ inRule = false;
+ break;
+ }
+ if (tstop >= cstart) {
+ if (!context.isIndent()) {
+ //TODO: Do non-indent formatting
+ }
+ if (token.getChannel() == OFF_CHANNEL) {
+ String ttext = token.getText();
+ int nl = ttext.indexOf('\n');
+ while ((nl != -1) && (tstart + nl <= cend)) {
+ int lineStart = context.lineStartOffset(tstart + textDelta + nl);
+ if (inRule || ruleLine > 0) {
+ if (tstart + nl >= cstart) {
+ // Indent the first rule line to 0 the rest to indentSize
+ int originalIndent = context.lineIndent(lineStart);
+ int newIndent = ruleLine > 0 ? indentSize : 0;
+ context.modifyIndent(lineStart, newIndent);
+ textDelta += newIndent - originalIndent;
+ }
+ ruleLine = inRule ? ruleLine + 1 : 0;
+ }
+ nl = ttext.indexOf('\n', nl + 1);
+ }
+ }
+ }
+ prevTokenType = token.getType();
+ token = lexer.nextToken();
+ tstart = token.getStartIndex();
+ tstop = token.getStopIndex();
+ }
+ } catch (BadLocationException ex) {}
+ }
+
+ @Override
+ public void reindent(Context context) {
+ reformat(context, null);
+ }
+
+ @Override
+ public boolean needsParserResult() {
+ return false;
+ }
+
+ @Override
+ public int indentSize() {
+ return 3;
+ }
Review Comment:
3 space indent is evil since it looks almost right in both directions :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] mbien commented on a diff in pull request #4800: ANTLRv4 indent and code snippet support.
Posted by GitBox <gi...@apache.org>.
mbien commented on code in PR #4800:
URL: https://github.com/apache/netbeans/pull/4800#discussion_r996996970
##########
java/languages.antlr/src/org/netbeans/modules/languages/antlr/v4/Antlr4Formatter.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.netbeans.modules.languages.antlr.v4;
+
+import java.util.prefs.Preferences;
+import javax.swing.text.BadLocationException;
+import javax.swing.text.Document;
+import org.antlr.parser.antlr4.ANTLRv4Lexer;
+import org.antlr.v4.runtime.CharStreams;
+import org.netbeans.modules.csl.api.Formatter;
+import org.netbeans.modules.csl.spi.ParserResult;
+import org.netbeans.modules.editor.indent.spi.Context;
+
+import static org.antlr.parser.antlr4.ANTLRv4Lexer.*;
+import org.antlr.v4.runtime.Token;
+import org.netbeans.api.editor.document.LineDocument;
+import org.netbeans.api.editor.document.LineDocumentUtils;
+import org.netbeans.api.editor.settings.SimpleValueNames;
+import org.netbeans.modules.editor.indent.spi.CodeStylePreferences;
+
+/**
+ *
+ * @author lkishalmi
+ */
+public class Antlr4Formatter implements Formatter {
+
+ public Antlr4Formatter() {
+ }
+
+ @Override
+ public void reformat(Context context, ParserResult compilationInfo) {
+ LineDocument doc = LineDocumentUtils.as(context.document(), LineDocument.class);
+ String text;
+ int indentSize = getIndentSize(context.document());
+ try {
+ text = doc.getText(0, doc.getLength());
+ } catch (BadLocationException ex) {
+ return;
+ }
+
+ ANTLRv4Lexer lexer = new ANTLRv4Lexer(CharStreams.fromString(text));
+ Token token = lexer.nextToken();
+ if (token.getType() == EOF) {
+ return;
+ }
+
+ final int cstart = context.startOffset();
+ final int cend = context.endOffset();
+
+ int tstart = token.getStartIndex();
+ int tstop = token.getStopIndex();
+
+ try {
+ boolean inRule = false;
+ int ruleLine = 0;
+ int parenDepth = 0;
+ int textDelta = 0;
+ int prevTokenType = -1;
+ while ((token.getType() != EOF) && (tstart < cend)) {
+ switch (token.getType()) {
+ case RULE_REF:
+ case TOKEN_REF:
+ // @header , @member, etc, are not real rules;
+ if (!inRule && (prevTokenType != AT)) {
+ inRule = true;
+ }
+ break;
+ case LPAREN:
+ parenDepth++;
+ break;
+ case RPAREN:
+ parenDepth--;
+ break;
+ case SEMI:
+ inRule = false;
+ break;
+ }
+ if (tstop >= cstart) {
+ if (!context.isIndent()) {
+ //TODO: Do non-indent formatting
+ }
+ if (token.getChannel() == OFF_CHANNEL) {
+ String ttext = token.getText();
+ int nl = ttext.indexOf('\n');
+ while ((nl != -1) && (tstart + nl <= cend)) {
+ int lineStart = context.lineStartOffset(tstart + textDelta + nl);
+ if (inRule || ruleLine > 0) {
+ if (tstart + nl >= cstart) {
+ // Indent the first rule line to 0 the rest to indentSize
+ int originalIndent = context.lineIndent(lineStart);
+ int newIndent = ruleLine > 0 ? indentSize : 0;
+ context.modifyIndent(lineStart, newIndent);
+ textDelta += newIndent - originalIndent;
+ }
+ ruleLine = inRule ? ruleLine + 1 : 0;
+ }
+ nl = ttext.indexOf('\n', nl + 1);
+ }
+ }
+ }
+ prevTokenType = token.getType();
+ token = lexer.nextToken();
+ tstart = token.getStartIndex();
+ tstop = token.getStopIndex();
+ }
+ } catch (BadLocationException ex) {}
+ }
+
+ @Override
+ public void reindent(Context context) {
+ reformat(context, null);
+ }
+
+ @Override
+ public boolean needsParserResult() {
+ return false;
+ }
+
+ @Override
+ public int indentSize() {
+ return 3;
+ }
Review Comment:
3 space indent is evil since it looks almost right :)
##########
java/languages.antlr/src/org/netbeans/modules/languages/antlr/v4/Antlr4CompletionProvider.java:
##########
@@ -143,12 +129,12 @@ protected void query(CompletionResultSet resultSet, Document doc, int caretOffse
tokens.previous();
lookAround(fo, tokens, caretOffset, prefix, resultSet);
} else {
- //Empty grammar so far offer lexer and grammar
- addTokens("", caretOffset, resultSet, "lexer", "grammar");
+ //Empty grammar so far offer lexer, parser and grammar
+ addTokens("", caretOffset, resultSet, "lexer", "parser", "grammar");
}
}
} catch (Throwable th) {
- System.out.println(th);
+ //System.out.println(th);
Review Comment:
was this intended? Would be better if it wouldn't be `Throwable`, It is always a bit dangerous to catch that without handling the edge cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] lkishalmi commented on pull request #4800: ANTLRv4 indent and code snippet support.
Posted by GitBox <gi...@apache.org>.
lkishalmi commented on PR #4800:
URL: https://github.com/apache/netbeans/pull/4800#issuecomment-1281040919
Well, a formatter code is never nice, as far as I checked. The basic idea here is using a heuristic based on the Lexer to determine whether we are in a rule or not. While in rule, the whitespace is examined for each line break, and indent the line according to the inrule state.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] mbien commented on a diff in pull request #4800: ANTLRv4 indent and code snippet support.
Posted by GitBox <gi...@apache.org>.
mbien commented on code in PR #4800:
URL: https://github.com/apache/netbeans/pull/4800#discussion_r996995861
##########
java/languages.antlr/src/org/netbeans/modules/languages/antlr/v4/Antlr4CompletionProvider.java:
##########
@@ -143,12 +129,12 @@ protected void query(CompletionResultSet resultSet, Document doc, int caretOffse
tokens.previous();
lookAround(fo, tokens, caretOffset, prefix, resultSet);
} else {
- //Empty grammar so far offer lexer and grammar
- addTokens("", caretOffset, resultSet, "lexer", "grammar");
+ //Empty grammar so far offer lexer, parser and grammar
+ addTokens("", caretOffset, resultSet, "lexer", "parser", "grammar");
}
}
} catch (Throwable th) {
- System.out.println(th);
+ //System.out.println(th);
Review Comment:
was this intended? Would also be better if it wouldn't be `Throwable`, It is always a bit dangerous to catch that without handling the edge cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
[GitHub] [netbeans] lkishalmi commented on a diff in pull request #4800: ANTLRv4 indent and code snippet support.
Posted by GitBox <gi...@apache.org>.
lkishalmi commented on code in PR #4800:
URL: https://github.com/apache/netbeans/pull/4800#discussion_r997186368
##########
java/languages.antlr/src/org/netbeans/modules/languages/antlr/v4/Antlr4Formatter.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.netbeans.modules.languages.antlr.v4;
+
+import java.util.prefs.Preferences;
+import javax.swing.text.BadLocationException;
+import javax.swing.text.Document;
+import org.antlr.parser.antlr4.ANTLRv4Lexer;
+import org.antlr.v4.runtime.CharStreams;
+import org.netbeans.modules.csl.api.Formatter;
+import org.netbeans.modules.csl.spi.ParserResult;
+import org.netbeans.modules.editor.indent.spi.Context;
+
+import static org.antlr.parser.antlr4.ANTLRv4Lexer.*;
+import org.antlr.v4.runtime.Token;
+import org.netbeans.api.editor.document.LineDocument;
+import org.netbeans.api.editor.document.LineDocumentUtils;
+import org.netbeans.api.editor.settings.SimpleValueNames;
+import org.netbeans.modules.editor.indent.spi.CodeStylePreferences;
+
+/**
+ *
+ * @author lkishalmi
+ */
+public class Antlr4Formatter implements Formatter {
+
+ public Antlr4Formatter() {
+ }
+
+ @Override
+ public void reformat(Context context, ParserResult compilationInfo) {
+ LineDocument doc = LineDocumentUtils.as(context.document(), LineDocument.class);
+ String text;
+ int indentSize = getIndentSize(context.document());
+ try {
+ text = doc.getText(0, doc.getLength());
+ } catch (BadLocationException ex) {
+ return;
+ }
+
+ ANTLRv4Lexer lexer = new ANTLRv4Lexer(CharStreams.fromString(text));
+ Token token = lexer.nextToken();
+ if (token.getType() == EOF) {
+ return;
+ }
+
+ final int cstart = context.startOffset();
+ final int cend = context.endOffset();
+
+ int tstart = token.getStartIndex();
+ int tstop = token.getStopIndex();
+
+ try {
+ boolean inRule = false;
+ int ruleLine = 0;
+ int parenDepth = 0;
+ int textDelta = 0;
+ int prevTokenType = -1;
+ while ((token.getType() != EOF) && (tstart < cend)) {
+ switch (token.getType()) {
+ case RULE_REF:
+ case TOKEN_REF:
+ // @header , @member, etc, are not real rules;
+ if (!inRule && (prevTokenType != AT)) {
+ inRule = true;
+ }
+ break;
+ case LPAREN:
+ parenDepth++;
+ break;
+ case RPAREN:
+ parenDepth--;
+ break;
+ case SEMI:
+ inRule = false;
+ break;
+ }
+ if (tstop >= cstart) {
+ if (!context.isIndent()) {
+ //TODO: Do non-indent formatting
+ }
+ if (token.getChannel() == OFF_CHANNEL) {
+ String ttext = token.getText();
+ int nl = ttext.indexOf('\n');
+ while ((nl != -1) && (tstart + nl <= cend)) {
+ int lineStart = context.lineStartOffset(tstart + textDelta + nl);
+ if (inRule || ruleLine > 0) {
+ if (tstart + nl >= cstart) {
+ // Indent the first rule line to 0 the rest to indentSize
+ int originalIndent = context.lineIndent(lineStart);
+ int newIndent = ruleLine > 0 ? indentSize : 0;
+ context.modifyIndent(lineStart, newIndent);
+ textDelta += newIndent - originalIndent;
+ }
+ ruleLine = inRule ? ruleLine + 1 : 0;
+ }
+ nl = ttext.indexOf('\n', nl + 1);
+ }
+ }
+ }
+ prevTokenType = token.getType();
+ token = lexer.nextToken();
+ tstart = token.getStartIndex();
+ tstop = token.getStopIndex();
+ }
+ } catch (BadLocationException ex) {}
+ }
+
+ @Override
+ public void reindent(Context context) {
+ reformat(context, null);
+ }
+
+ @Override
+ public boolean needsParserResult() {
+ return false;
+ }
+
+ @Override
+ public int indentSize() {
+ return 3;
+ }
Review Comment:
Just checked a few grammars today randomly. Most of them were using 4 spaces, but a few of them were using 3. It seems our ANTLR v3 and v4 grammars using 3 spaces are kind of a minority.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists