You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@jena.apache.org by "mcb5637 (via GitHub)" <gi...@apache.org> on 2024/10/11 11:37:54 UTC
[I] turtle command wrongly warns about invalid characters in an IRI containing 4-byte unicode [jena]
mcb5637 opened a new issue, #2766:
URL: https://github.com/apache/jena/issues/2766
### Version
Apache Jena RIOT version 5.1.0
### What happened?
I am trying to validate a turtle file with the turtle command described here: https://jena.apache.org/documentation/io/
The result are a bunch of warnings, which i reduced down to cases simmilar to the following:
Trying to parse the triple `<http://en.wikipedia.org/wiki/𝁰> a <http://xmlns.com/foaf/0.1/Document> .` (the last character of the subject is unicode codepoint 1D070, in case it does not get rendered properly). The turtle command issues the following warnings:
```
11:04:40 WARN riot :: [line: 1, col: 32] Illegal character in IRI (Not a ucschar: 0xD834): <http://en.wikipedia.org/wiki/[U+D834]...>
11:04:40 WARN riot :: [line: 1, col: 33] Illegal character in IRI (Not a ucschar: 0xDC70): <http://en.wikipedia.org/wiki/?[U+DC70]...>
```
(Note that 0xD834 and 0xDC70 are the 2 halves of 1D070, if you represent it in utf-16)
(I also validated the input and it does contain the correct representation of 1D070 in utf-8)
In case it is relevant:
OS: `Ubuntu 24.04.1 LTS`
Java version:
```
openjdk 21.0.4 2024-07-16
OpenJDK Runtime Environment (build 21.0.4+7-Ubuntu-1ubuntu224.04)
OpenJDK 64-Bit Server VM (build 21.0.4+7-Ubuntu-1ubuntu224.04, mixed mode, sharing)
```
Command run: `cat test.ttl | apache-jena-5.1.0/bin/turtle --output=TURTLE > /dev/null`, where test.tll only contains the relevant triple. (Although different types of input (like loading the file directly with turtle) result in the same warnings)
### Relevant output and stacktrace
_No response_
### Are you interested in making a pull request?
None
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] turtle command wrongly warns about invalid characters in an IRI containing 4-byte unicode [jena]
Posted by "afs (via GitHub)" <gi...@apache.org>.
afs commented on issue #2766:
URL: https://github.com/apache/jena/issues/2766#issuecomment-2409042272
Hi @mcb5637 - thank you for the report.
Due to Java bytes to string conversion using the JDK standard charset, Jena can't see multibyte characters translated to surrogates (which is legal) and surrogates actually in the in UTF-8 (which is illegal - UTF-8 does not allow surrogates).
As this would be silently legal in a string, #2769 removes the warning (it is only a warning - the code does insert the character into the IRI).
A deeper solution is to use a UTF-8 processor that generates ints, not chars (and hence needs surrogates). But that is delicate. When written the tokenizer was faster using teh JDK built-in conversion; they may not be the case nowadays (java improvements, CPU architectures). But not a change to be done lightly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org
Re: [I] turtle command wrongly warns about invalid characters in an IRI containing 4-byte unicode [jena]
Posted by "afs (via GitHub)" <gi...@apache.org>.
afs closed issue #2766: turtle command wrongly warns about invalid characters in an IRI containing 4-byte unicode
URL: https://github.com/apache/jena/issues/2766
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@jena.apache.org
For additional commands, e-mail: issues-help@jena.apache.org