[GitHub] [jena] kinow commented on a change in pull request #884: JENA-2012: Test for UTF-16 high-low surrogates pairs

[GitBox <gi...@apache.org>]

kinow commented on a change in pull request #884:
URL: https://github.com/apache/jena/pull/884#discussion_r538742111



##########
File path: jena-arq/src/main/java/org/apache/jena/sparql/lang/ParserBase.java
##########
@@ -190,6 +190,24 @@ protected Node stripSign(Node node) {
         return NodeFactory.createLiteral(lex, lang, dt) ;
     }
 
+    protected void checkString(String string, int line, int column) {
+        for ( int i = 0 ; i < string.length() ; i++ ) {
+            // Not "codePointAt" which does surrogate processing.
+            char ch = string.charAt(i);
+            // Check surrogate pairs are pairs.
+            if ( Character.isHighSurrogate(ch) ) {
+                i++;
+                if ( i == string.length() )
+                    throw new QueryParseException("Bad surrogate pair (end of string)", line, column);
+                char ch1 = string.charAt(i);
+                if ( ! Character.isLowSurrogate(ch1) ) {
+                    throw new QueryParseException("Bad surrogate pair (high surrogate not followed by low surrogate)", line, column);
+                }
+            } else if ( Character.isLowSurrogate(ch) )
+                throw new QueryParseException("Bad surrogate pair (low surrogate without high surrogate)", line, column);
+        }
+    }

Review comment:
       :+1: 
   
   I checked for a method that does something like this on Apache Commons Text, and Apache Commons Lang, but there's none. I thought some user would have requested that, interesting.

##########
File path: jena-base/src/main/java/org/apache/jena/atlas/io/InStreamUTF8.java
##########
@@ -144,7 +144,18 @@ public final int advance()
     public static final int advance(InputStreamBuffered input) {
         int x = input.advance() ;
         if ( x == -1 ) return -1 ;
-        return advance(input, x) ;
+
+        int codepoint = advance(input, x) ;
+        //System.out.printf("U+%04X\n", codepoint);
+
+        if ( false ) {
+//          if ( codepoint > Character.MAX_VALUE ) {
+//              throw new AtlasException("Out of range character (must use a surrogate pair)");
+//          }

Review comment:
       Delete? :point_up: 

##########
File path: jena-cmds/src/main/java/riotcmd/utf8.java
##########
@@ -60,7 +60,8 @@ public static void main(String... args) {
                     } else
                         colNum++ ;
                     if ( !Character.isDefined(ch) )
-                        throw new AtlasException(String.format("No such codepoint: 0x%04X", ch)) ;
+                        //throw new AtlasException(String.format("No such codepoint: 0x%04X", ch)) ;

Review comment:
       Delete? :point_up: 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@jena.apache.org
For additional commands, e-mail: pr-help@jena.apache.org