You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@netbeans.apache.org by "mbien (via GitHub)" <gi...@apache.org> on 2025/03/07 20:22:15 UTC
[PR] Code scanner and search should tolerate CoderMalfunctionErrors [netbeans]
mbien opened a new pull request, #8304:
URL: https://github.com/apache/netbeans/pull/8304
- scanner will skip the file and log a warning
- search will add it to the regular search result error list
cleanup:
- removed unused UI component of search results window
- simplified exception handling in Source#createSnapshot
indexer log:
```
WARNING [org.netbeans.modules.parsing.api.Source]: Can't create snapshot of /tmp/mavenproject1/src/main/java/com/mycompany/mavenproject1/error_file2.xml@cbec52aa:59023f9d, mimeType=text/xml. CoderMalfunctionError: java.lang.IllegalArgumentException: newPosition > limit: (1 > 0)
```
fixes #7839
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: [PR] Code scanner and search should tolerate CoderMalfunctionErrors [netbeans]
Posted by "matthiasblaesing (via GitHub)" <gi...@apache.org>.
matthiasblaesing commented on code in PR #8304:
URL: https://github.com/apache/netbeans/pull/8304#discussion_r1985687746
##########
ide/parsing.api/src/org/netbeans/modules/parsing/api/Source.java:
##########
@@ -334,81 +332,72 @@ public Snapshot createSnapshot () {
// The code below is a copy of CharacterConversions.lineSeparatorToLineFeed() method
// and it handles line-end conversion. In general EditorKits can do more conversions,
// but they usually don't and this should be good enough for Snapshots.
- try {
- if (fileObject.isValid ()) {
- if (fileObject.getSize() <= Utilities.getMaxFileSize()) {
- final InputStream is = fileObject.getInputStream ();
- assert is != null : "FileObject.getInputStream() returned null for FileObject: " + FileUtil.getFileDisplayName(fileObject); //NOI18N
- try {
- InputStreamReader reader = new InputStreamReader (
- is,
- FileEncodingQuery.getEncoding (fileObject)
- );
- try {
- StringBuilder output = new StringBuilder(Math.max(16, (int) fileObject.getSize()));
- List<Integer> lso = new LinkedList<Integer>();
- boolean lastCharCR = false;
- char [] buffer = new char [65536];
- int size = -1;
-
- final char LF = '\n'; //NOI18N, Unicode line feed (0x000A)
- final char CR = '\r'; //NOI18N, Unicode carriage return (0x000D)
- final char LS = 0x2028; // Unicode line separator (0x2028)
- final char PS = 0x2029; // Unicode paragraph separator (0x2029)
-
- lso.add(0);
- while(-1 != (size = reader.read(buffer, 0, buffer.length))) {
- for(int i = 0; i < size; i++) {
- char ch = buffer[i];
- if (lastCharCR && ch == LF) {
- // CR-LF pair changed to single LF
- continue;
- }
- if (ch == CR) {
- // convert to LF; subsequent LF will be skipped
- output.append(LF);
- lso.add(output.length());
- lastCharCR = true;
- } else if (ch == LS || ch == PS) { // Unicode LS, PS
- output.append(LF);
- lso.add(output.length());
- lastCharCR = false;
- } else { // current char not CR
- lastCharCR = false;
- output.append(ch);
- }
- }
+ if (fileObject.isValid()) {
+ if (fileObject.getSize() <= Utilities.getMaxFileSize()) {
+ try (InputStreamReader reader = new InputStreamReader(fileObject.getInputStream(), FileEncodingQuery.getEncoding(fileObject))) {
+ StringBuilder output = new StringBuilder(Math.max(16, (int) fileObject.getSize()));
+ List<Integer> lso = new ArrayList<>();
+ boolean lastCharCR = false;
+ char[] buffer = new char [65536];
+ int size = -1;
+
+ final char LF = '\n'; //NOI18N, Unicode line feed (0x000A)
+ final char CR = '\r'; //NOI18N, Unicode carriage return (0x000D)
+ final char LS = 0x2028; // Unicode line separator (0x2028)
+ final char PS = 0x2029; // Unicode paragraph separator (0x2029)
+
+ lso.add(0);
+ while(-1 != (size = reader.read(buffer, 0, buffer.length))) {
+ for(int i = 0; i < size; i++) {
+ char ch = buffer[i];
+ if (lastCharCR && ch == LF) {
+ // CR-LF pair changed to single LF
+ continue;
}
-
- int [] lsoArr = new int [lso.size()];
- int idx = 0;
- for(Integer offset : lso) {
- lsoArr[idx++] = offset;
+ if (ch == CR) {
+ // convert to LF; subsequent LF will be skipped
+ output.append(LF);
+ lso.add(output.length());
+ lastCharCR = true;
+ } else if (ch == LS || ch == PS) { // Unicode LS, PS
+ output.append(LF);
+ lso.add(output.length());
+ lastCharCR = false;
+ } else { // current char not CR
+ lastCharCR = false;
+ output.append(ch);
}
-
- text[0] = output;
- lineStartOffsets[0] = lsoArr;
- } finally {
- reader.close ();
}
- } finally {
- is.close ();
}
- } else {
- LOG.log(
- Level.WARNING,
- "Source {0} of size: {1} has been ignored due to large size. Files large then {2} bytes are ignored, you can increase the size by parse.max.file.size property.", //NOI18N
- new Object[]{
- FileUtil.getFileDisplayName(fileObject),
- fileObject.getSize(),
- Utilities.getMaxFileSize()
- });
+
+ int[] lsoArr = new int[lso.size()];
+ int idx = 0;
+ for(Integer offset : lso) {
+ lsoArr[idx++] = offset;
+ }
+
+ text[0] = output;
+ lineStartOffsets[0] = lsoArr;
+ } catch (FileNotFoundException fnfe) {
+ // working with a stale FileObject, just ignore this (eg see #158119)
+ } catch (CoderMalfunctionError cme) {
+ text[0] = ""; //NOI18N
+ lineStartOffsets[0] = new int[] { 0 };
Review Comment:
Question/nitpick: is there a reason to treat `CoderMalfunctionError` different compared to `IOException` with regard to the variables? I see a reason for the different logging, but why handle the variable differently?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: [PR] Code scanner and search should tolerate CoderMalfunctionErrors [netbeans]
Posted by "mbien (via GitHub)" <gi...@apache.org>.
mbien commented on code in PR #8304:
URL: https://github.com/apache/netbeans/pull/8304#discussion_r1985708913
##########
ide/parsing.api/src/org/netbeans/modules/parsing/api/Source.java:
##########
@@ -334,81 +332,72 @@ public Snapshot createSnapshot () {
// The code below is a copy of CharacterConversions.lineSeparatorToLineFeed() method
// and it handles line-end conversion. In general EditorKits can do more conversions,
// but they usually don't and this should be good enough for Snapshots.
- try {
- if (fileObject.isValid ()) {
- if (fileObject.getSize() <= Utilities.getMaxFileSize()) {
- final InputStream is = fileObject.getInputStream ();
- assert is != null : "FileObject.getInputStream() returned null for FileObject: " + FileUtil.getFileDisplayName(fileObject); //NOI18N
- try {
- InputStreamReader reader = new InputStreamReader (
- is,
- FileEncodingQuery.getEncoding (fileObject)
- );
- try {
- StringBuilder output = new StringBuilder(Math.max(16, (int) fileObject.getSize()));
- List<Integer> lso = new LinkedList<Integer>();
- boolean lastCharCR = false;
- char [] buffer = new char [65536];
- int size = -1;
-
- final char LF = '\n'; //NOI18N, Unicode line feed (0x000A)
- final char CR = '\r'; //NOI18N, Unicode carriage return (0x000D)
- final char LS = 0x2028; // Unicode line separator (0x2028)
- final char PS = 0x2029; // Unicode paragraph separator (0x2029)
-
- lso.add(0);
- while(-1 != (size = reader.read(buffer, 0, buffer.length))) {
- for(int i = 0; i < size; i++) {
- char ch = buffer[i];
- if (lastCharCR && ch == LF) {
- // CR-LF pair changed to single LF
- continue;
- }
- if (ch == CR) {
- // convert to LF; subsequent LF will be skipped
- output.append(LF);
- lso.add(output.length());
- lastCharCR = true;
- } else if (ch == LS || ch == PS) { // Unicode LS, PS
- output.append(LF);
- lso.add(output.length());
- lastCharCR = false;
- } else { // current char not CR
- lastCharCR = false;
- output.append(ch);
- }
- }
+ if (fileObject.isValid()) {
+ if (fileObject.getSize() <= Utilities.getMaxFileSize()) {
+ try (InputStreamReader reader = new InputStreamReader(fileObject.getInputStream(), FileEncodingQuery.getEncoding(fileObject))) {
+ StringBuilder output = new StringBuilder(Math.max(16, (int) fileObject.getSize()));
+ List<Integer> lso = new ArrayList<>();
+ boolean lastCharCR = false;
+ char[] buffer = new char [65536];
+ int size = -1;
+
+ final char LF = '\n'; //NOI18N, Unicode line feed (0x000A)
+ final char CR = '\r'; //NOI18N, Unicode carriage return (0x000D)
+ final char LS = 0x2028; // Unicode line separator (0x2028)
+ final char PS = 0x2029; // Unicode paragraph separator (0x2029)
+
+ lso.add(0);
+ while(-1 != (size = reader.read(buffer, 0, buffer.length))) {
+ for(int i = 0; i < size; i++) {
+ char ch = buffer[i];
+ if (lastCharCR && ch == LF) {
+ // CR-LF pair changed to single LF
+ continue;
}
-
- int [] lsoArr = new int [lso.size()];
- int idx = 0;
- for(Integer offset : lso) {
- lsoArr[idx++] = offset;
+ if (ch == CR) {
+ // convert to LF; subsequent LF will be skipped
+ output.append(LF);
+ lso.add(output.length());
+ lastCharCR = true;
+ } else if (ch == LS || ch == PS) { // Unicode LS, PS
+ output.append(LF);
+ lso.add(output.length());
+ lastCharCR = false;
+ } else { // current char not CR
+ lastCharCR = false;
+ output.append(ch);
}
-
- text[0] = output;
- lineStartOffsets[0] = lsoArr;
- } finally {
- reader.close ();
}
- } finally {
- is.close ();
}
- } else {
- LOG.log(
- Level.WARNING,
- "Source {0} of size: {1} has been ignored due to large size. Files large then {2} bytes are ignored, you can increase the size by parse.max.file.size property.", //NOI18N
- new Object[]{
- FileUtil.getFileDisplayName(fileObject),
- fileObject.getSize(),
- Utilities.getMaxFileSize()
- });
+
+ int[] lsoArr = new int[lso.size()];
+ int idx = 0;
+ for(Integer offset : lso) {
+ lsoArr[idx++] = offset;
+ }
+
+ text[0] = output;
+ lineStartOffsets[0] = lsoArr;
+ } catch (FileNotFoundException fnfe) {
+ // working with a stale FileObject, just ignore this (eg see #158119)
+ } catch (CoderMalfunctionError cme) {
+ text[0] = ""; //NOI18N
+ lineStartOffsets[0] = new int[] { 0 };
Review Comment:
yeah I suppose we can handle both the same way. I took the two lines from the OutOfMemory catch block - so I guess all paths which are not able to complete normally (like out of memory) should be the same handler with a different message.
will take another look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: [PR] Code scanner and search should tolerate CoderMalfunctionErrors [netbeans]
Posted by "mbien (via GitHub)" <gi...@apache.org>.
mbien merged PR #8304:
URL: https://github.com/apache/netbeans/pull/8304
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: [PR] Code scanner and search should tolerate CoderMalfunctionErrors [netbeans]
Posted by "mbien (via GitHub)" <gi...@apache.org>.
mbien commented on PR #8304:
URL: https://github.com/apache/netbeans/pull/8304#issuecomment-2707399410
note that search in multi-line mode did already correctly catch the error and skipped the file since it uses different code paths (`BufferedCharSequence`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: [PR] Code scanner and search should tolerate CoderMalfunctionErrors [netbeans]
Posted by "mbien (via GitHub)" <gi...@apache.org>.
mbien commented on code in PR #8304:
URL: https://github.com/apache/netbeans/pull/8304#discussion_r1985743240
##########
ide/parsing.api/src/org/netbeans/modules/parsing/api/Source.java:
##########
@@ -334,81 +332,72 @@ public Snapshot createSnapshot () {
// The code below is a copy of CharacterConversions.lineSeparatorToLineFeed() method
// and it handles line-end conversion. In general EditorKits can do more conversions,
// but they usually don't and this should be good enough for Snapshots.
- try {
- if (fileObject.isValid ()) {
- if (fileObject.getSize() <= Utilities.getMaxFileSize()) {
- final InputStream is = fileObject.getInputStream ();
- assert is != null : "FileObject.getInputStream() returned null for FileObject: " + FileUtil.getFileDisplayName(fileObject); //NOI18N
- try {
- InputStreamReader reader = new InputStreamReader (
- is,
- FileEncodingQuery.getEncoding (fileObject)
- );
- try {
- StringBuilder output = new StringBuilder(Math.max(16, (int) fileObject.getSize()));
- List<Integer> lso = new LinkedList<Integer>();
- boolean lastCharCR = false;
- char [] buffer = new char [65536];
- int size = -1;
-
- final char LF = '\n'; //NOI18N, Unicode line feed (0x000A)
- final char CR = '\r'; //NOI18N, Unicode carriage return (0x000D)
- final char LS = 0x2028; // Unicode line separator (0x2028)
- final char PS = 0x2029; // Unicode paragraph separator (0x2029)
-
- lso.add(0);
- while(-1 != (size = reader.read(buffer, 0, buffer.length))) {
- for(int i = 0; i < size; i++) {
- char ch = buffer[i];
- if (lastCharCR && ch == LF) {
- // CR-LF pair changed to single LF
- continue;
- }
- if (ch == CR) {
- // convert to LF; subsequent LF will be skipped
- output.append(LF);
- lso.add(output.length());
- lastCharCR = true;
- } else if (ch == LS || ch == PS) { // Unicode LS, PS
- output.append(LF);
- lso.add(output.length());
- lastCharCR = false;
- } else { // current char not CR
- lastCharCR = false;
- output.append(ch);
- }
- }
+ if (fileObject.isValid()) {
+ if (fileObject.getSize() <= Utilities.getMaxFileSize()) {
+ try (InputStreamReader reader = new InputStreamReader(fileObject.getInputStream(), FileEncodingQuery.getEncoding(fileObject))) {
+ StringBuilder output = new StringBuilder(Math.max(16, (int) fileObject.getSize()));
+ List<Integer> lso = new ArrayList<>();
+ boolean lastCharCR = false;
+ char[] buffer = new char [65536];
+ int size = -1;
+
+ final char LF = '\n'; //NOI18N, Unicode line feed (0x000A)
+ final char CR = '\r'; //NOI18N, Unicode carriage return (0x000D)
+ final char LS = 0x2028; // Unicode line separator (0x2028)
+ final char PS = 0x2029; // Unicode paragraph separator (0x2029)
+
+ lso.add(0);
+ while(-1 != (size = reader.read(buffer, 0, buffer.length))) {
+ for(int i = 0; i < size; i++) {
+ char ch = buffer[i];
+ if (lastCharCR && ch == LF) {
+ // CR-LF pair changed to single LF
+ continue;
}
-
- int [] lsoArr = new int [lso.size()];
- int idx = 0;
- for(Integer offset : lso) {
- lsoArr[idx++] = offset;
+ if (ch == CR) {
+ // convert to LF; subsequent LF will be skipped
+ output.append(LF);
+ lso.add(output.length());
+ lastCharCR = true;
+ } else if (ch == LS || ch == PS) { // Unicode LS, PS
+ output.append(LF);
+ lso.add(output.length());
+ lastCharCR = false;
+ } else { // current char not CR
+ lastCharCR = false;
+ output.append(ch);
}
-
- text[0] = output;
- lineStartOffsets[0] = lsoArr;
- } finally {
- reader.close ();
}
- } finally {
- is.close ();
}
- } else {
- LOG.log(
- Level.WARNING,
- "Source {0} of size: {1} has been ignored due to large size. Files large then {2} bytes are ignored, you can increase the size by parse.max.file.size property.", //NOI18N
- new Object[]{
- FileUtil.getFileDisplayName(fileObject),
- fileObject.getSize(),
- Utilities.getMaxFileSize()
- });
+
+ int[] lsoArr = new int[lso.size()];
+ int idx = 0;
+ for(Integer offset : lso) {
+ lsoArr[idx++] = offset;
+ }
+
+ text[0] = output;
+ lineStartOffsets[0] = lsoArr;
+ } catch (FileNotFoundException fnfe) {
+ // working with a stale FileObject, just ignore this (eg see #158119)
+ } catch (CoderMalfunctionError cme) {
+ text[0] = ""; //NOI18N
+ lineStartOffsets[0] = new int[] { 0 };
Review Comment:
done, used multicatch - FileNotFound is also implicitly handled via IOException.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@netbeans.apache.org
For additional commands, e-mail: notifications-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists