You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by rm...@apache.org on 2011/08/01 03:39:12 UTC
svn commit: r1152673 -
/lucene/dev/branches/branch_3x/lucene/src/test/org/apache/lucene/index/TestPayloads.java
Author: rmuir
Date: Mon Aug 1 01:39:11 2011
New Revision: 1152673
URL: http://svn.apache.org/viewvc?rev=1152673&view=rev
Log:
fix test to not create invalid unicode
Modified:
lucene/dev/branches/branch_3x/lucene/src/test/org/apache/lucene/index/TestPayloads.java
Modified: lucene/dev/branches/branch_3x/lucene/src/test/org/apache/lucene/index/TestPayloads.java
URL: http://svn.apache.org/viewvc/lucene/dev/branches/branch_3x/lucene/src/test/org/apache/lucene/index/TestPayloads.java?rev=1152673&r1=1152672&r2=1152673&view=diff
==============================================================================
--- lucene/dev/branches/branch_3x/lucene/src/test/org/apache/lucene/index/TestPayloads.java (original)
+++ lucene/dev/branches/branch_3x/lucene/src/test/org/apache/lucene/index/TestPayloads.java Mon Aug 1 01:39:11 2011
@@ -342,16 +342,10 @@ public class TestPayloads extends Lucene
}
private void generateRandomData(byte[] data) {
- // this test needs the random data to be valid unicode
- String s = _TestUtil.randomFixedByteLengthUnicodeString(random, data.length);
- byte b[];
- try {
- b = s.getBytes("UTF-8");
- } catch (UnsupportedEncodingException e) {
- throw new RuntimeException(e);
+ // this test needs the random data to be valid unicode: historically it hasn't done a great job
+ for (int i = 0; i < data.length; i++) {
+ data[i] = (byte) _TestUtil.nextInt(random, 'a', 'z');
}
- assert b.length == data.length;
- System.arraycopy(b, 0, data, 0, b.length);
}
private byte[] generateRandomData(int n) {
Re: svn commit: r1152673 - /lucene/dev/branches/branch_3x/lucene/src/test/org/apache/lucene/index/Tes
tPayloads.java
Posted by Michael McCandless <lu...@mikemccandless.com>.
OK I found the problem: 3.x can't handle the U+FFFF character (we
replace it on indexing), while trunk can. So I think we just have to
fix randomFixedByteLengthUnicodeString to never use that char. I'll
commit...
Mike McCandless
http://blog.mikemccandless.com
On Sun, Jul 31, 2011 at 10:33 PM, Robert Muir <rc...@gmail.com> wrote:
> On Sun, Jul 31, 2011 at 10:24 PM, Chris Hostetter
> <ho...@fucit.org> wrote:
>>
>> : fix test to not create invalid unicode
>>
>> I'm confused ... when/why does randomFixedByteLengthUnicodeString not
>> return valid unicode?
>>
>
> I'm confused too, but all of the issues are with replacement chars for
> invalid unicode:
>
> Since the intent of this test is to test thread safety, not to test
> unicode enc/dec back and forth, I switched it to ascii until the test
> grows up (e.g. trunk, which now uses full unicode range correctly,
> maybe I backported this wrong before)
>
> --
> lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: svn commit: r1152673 - /lucene/dev/branches/branch_3x/lucene/src/test/org/apache/lucene/index/Tes
tPayloads.java
Posted by Robert Muir <rc...@gmail.com>.
On Sun, Jul 31, 2011 at 10:24 PM, Chris Hostetter
<ho...@fucit.org> wrote:
>
> : fix test to not create invalid unicode
>
> I'm confused ... when/why does randomFixedByteLengthUnicodeString not
> return valid unicode?
>
I'm confused too, but all of the issues are with replacement chars for
invalid unicode:
Since the intent of this test is to test thread safety, not to test
unicode enc/dec back and forth, I switched it to ascii until the test
grows up (e.g. trunk, which now uses full unicode range correctly,
maybe I backported this wrong before)
--
lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: svn commit: r1152673 - /lucene/dev/branches/branch_3x/lucene/src/test/org/apache/lucene/index/Tes
tPayloads.java
Posted by Chris Hostetter <ho...@fucit.org>.
: fix test to not create invalid unicode
I'm confused ... when/why does randomFixedByteLengthUnicodeString not
return valid unicode?
: - // this test needs the random data to be valid unicode
: - String s = _TestUtil.randomFixedByteLengthUnicodeString(random, data.length);
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org