You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Andy Seaborne <an...@apache.org> on 2013/11/12 19:55:59 UTC
Re: svn commit: r1541118 - in /jena/trunk/jena-arq/src/main/java/org/apache/jena/riot:
lang/BlankNodeAllocatorFixedSeedHash.java lang/BlankNodeAllocatorHash.java
lang/LabelToNode.java tokens/TokenizerFactory.java
> jena/trunk/jena-arq/src/main/java/org/apache/jena/riot/tokens/TokenizerFactory.java
>
> Modified: jena/trunk/jena-arq/src/main/java/org/apache/jena/riot/tokens/TokenizerFactory.java
> URL: http://svn.apache.org/viewvc/jena/trunk/jena-arq/src/main/java/org/apache/jena/riot/tokens/TokenizerFactory.java?rev=1541118&r1=1541117&r2=1541118&view=diff
> ==============================================================================
> --- jena/trunk/jena-arq/src/main/java/org/apache/jena/riot/tokens/TokenizerFactory.java (original)
> +++ jena/trunk/jena-arq/src/main/java/org/apache/jena/riot/tokens/TokenizerFactory.java Tue Nov 12 15:53:36 2013
> @@ -42,6 +42,13 @@ public class TokenizerFactory
> Tokenizer tokenizer = new TokenizerText(peekReader) ;
> return tokenizer ;
> }
> +
> + public static Tokenizer makeTokenizerUTF8(String string)
> + {
> + PeekReader peekReader = PeekReader.readString(string);
> + Tokenizer tokenizer = new TokenizerText(peekReader);
> + return tokenizer;
> + }
>
> public static Tokenizer makeTokenizerASCII(InputStream in)
> {
>
>
Rob -
There is TokenizerFactory.makeTokenizerString which is identical to
makeTokenizerUTF8. "String" was a better name because a string isn't
UTF8 in Java.
Andy
Re: svn commit: r1541118 - in
/jena/trunk/jena-arq/src/main/java/org/apache/jena/riot:
lang/BlankNodeAllocatorFixedSeedHash.java lang/BlankNodeAllocatorHash.java
lang/LabelToNode.java tokens/TokenizerFactory.java
Posted by Rob Vesse <rv...@dotnetrdf.org>.
Gah - character encoding in Java is still a horrific minefield :(
Removed the unnecessary overload
Rob
On 12/11/2013 18:55, "Andy Seaborne" <an...@apache.org> wrote:
>>
>>jena/trunk/jena-arq/src/main/java/org/apache/jena/riot/tokens/TokenizerFa
>>ctory.java
>>
>
>
>> Modified:
>>jena/trunk/jena-arq/src/main/java/org/apache/jena/riot/tokens/TokenizerFa
>>ctory.java
>> URL:
>>http://svn.apache.org/viewvc/jena/trunk/jena-arq/src/main/java/org/apache
>>/jena/riot/tokens/TokenizerFactory.java?rev=1541118&r1=1541117&r2=1541118
>>&view=diff
>>
>>=========================================================================
>>=====
>> ---
>>jena/trunk/jena-arq/src/main/java/org/apache/jena/riot/tokens/TokenizerFa
>>ctory.java (original)
>> +++
>>jena/trunk/jena-arq/src/main/java/org/apache/jena/riot/tokens/TokenizerFa
>>ctory.java Tue Nov 12 15:53:36 2013
>> @@ -42,6 +42,13 @@ public class TokenizerFactory
>> Tokenizer tokenizer = new TokenizerText(peekReader) ;
>> return tokenizer ;
>> }
>> +
>> + public static Tokenizer makeTokenizerUTF8(String string)
>> + {
>> + PeekReader peekReader = PeekReader.readString(string);
>> + Tokenizer tokenizer = new TokenizerText(peekReader);
>> + return tokenizer;
>> + }
>>
>> public static Tokenizer makeTokenizerASCII(InputStream in)
>> {
>>
>>
>
>Rob -
>
>There is TokenizerFactory.makeTokenizerString which is identical to
>makeTokenizerUTF8. "String" was a better name because a string isn't
>UTF8 in Java.
>
> Andy
>