Codec/UtilityTest issue?

[Adam Constabaris <ad...@clownsinmycoffee.net>]

I'm still kind of puzzled by this one.  UtilityTest in the server module 
(tests for encoding utitlities) fails on a (Intel) Mac, but not on any 
Linux system I've tried.   The results coming back from the EncodingUtil 
aren't agreeing with the string literals in the test case.

Attached is a diff against 
server/src/test/java/org/apache/abdera/test/server/UtilityTest.java that 
replaces all the literals used as the test values with calls to the 
relevant methods from the commons codec library.  I'm not sure whether 
this is a good idea, although it lets the test pass =)

Re: Codec/UtilityTest issue?

[James M Snell <ja...@gmail.com>]

I'll take a look at this in detail later but it's actually of less
importance now than it used to be.  While we do need to support RFC2047
generally, the APP spec has switched to using percent-encoding for the
Slug header instead of rfc2047 encoding.

- James

Adam Constabaris wrote:
> I'm still kind of puzzled by this one.  UtilityTest in the server module
> (tests for encoding utitlities) fails on a (Intel) Mac, but not on any
> Linux system I've tried.   The results coming back from the EncodingUtil
> aren't agreeing with the string literals in the test case.
> 
> Attached is a diff against
> server/src/test/java/org/apache/abdera/test/server/UtilityTest.java that
> replaces all the literals used as the test values with calls to the
> relevant methods from the commons codec library.  I'm not sure whether
> this is a good idea, although it lets the test pass =)
> 

Re: Codec/UtilityTest issue?

[James M Snell <ja...@gmail.com>]

That's what I would prefer.  At this point the code should be up to date
with the pending -15 draft that will go to the IESG.  While I'm not
completely happy with the client API, we've got the functionality
needed.  We can work on refining the client API in the 0.4.0 stream.

- James

Adam Constabaris wrote:
> [snip]
> On another topic: is the current plan to have a 0.30 release soon after
> the APP draft is submitted to the IESG?
> 
> AC
> 
> 

Re: Codec/UtilityTest issue?

[James M Snell <ja...@gmail.com>]

Checked in!  thanks!

Adam Constabaris wrote:
> Exchanging a literal é for \u0059 allows the tests to pass.  Patch
> attached.
> 
> cheers,
> 
> AC
> 
> 
> .Adam Constabaris wrote:
>> James M Snell wrote:
>>> Yeah, the challenge I have with this is that it makes the test
>>> circular... testing the output of a function against the output of the
>>> same function.  We need to understand why the test is failing on the Mac
>>> and figure out if it's something we need to fix.
>>>
>>> - James
>>>   
>>
>> Both OSes think their respective copies are encoded as UTF-8;  I tried
>> iconv to iso-8859-1, and for fun, to utf-8, with the same results. 
>> One thing I didn't try yet was replacing the é  in the test string
>> with \u00e9, which should answer that question.  I'll try that when I
>> can get some time later.  If none of that pans out, I'll start
>> exploring the JVM angle.
>>

Re: Codec/UtilityTest issue?

[Adam Constabaris <ad...@clownsinmycoffee.net>]

Exchanging a literal é for \u0059 allows the tests to pass.  Patch attached.

cheers,

AC


.Adam Constabaris wrote:
> James M Snell wrote:
>> Yeah, the challenge I have with this is that it makes the test
>> circular... testing the output of a function against the output of the
>> same function.  We need to understand why the test is failing on the Mac
>> and figure out if it's something we need to fix.
>>
>> - James
>>   
>
> Both OSes think their respective copies are encoded as UTF-8;  I tried 
> iconv to iso-8859-1, and for fun, to utf-8, with the same results.  
> One thing I didn't try yet was replacing the é  in the test string 
> with \u00e9, which should answer that question.  I'll try that when I 
> can get some time later.  If none of that pans out, I'll start 
> exploring the JVM angle.
>

Re: Codec/UtilityTest issue?

[Adam Constabaris <ad...@clownsinmycoffee.net>]

James M Snell wrote:
> Yeah, the challenge I have with this is that it makes the test
> circular... testing the output of a function against the output of the
> same function.  We need to understand why the test is failing on the Mac
> and figure out if it's something we need to fix.
>
> - James
>   
Yeah, the only thing that's still being actually tested in my diff are 
that the default encoding is what it's supposed to be.  I can see 
arguing that beyond that, the tests are superfluous in that it's commons 
codec's responsibility to generate correct output; that said, what you 
want is proper encoding handling, so the way the tests are actually 
structured is preferable.  At any rate, if the encoding is crucial and 
it's not working on a popular(?) platform, that's probably worth knowing.

IMO, the three most likely causes are a file encoding problem, an Apple 
JVM bug, a commons codec bug, or some kind of interaction between the 
latter.  Since I'm unfamiliar with the other stuff, though, I'm not sure 
where I'd start, so I've been looking at the file encoding angle.

Both OSes think their respective copies are encoded as UTF-8;  I tried 
iconv to iso-8859-1, and for fun, to utf-8, with the same results.  One 
thing I didn't try yet was replacing the é  in the test string with 
\u00e9, which should answer that question.  I'll try that when I can get 
some time later.  If none of that pans out, I'll start exploring the JVM 
angle.

On another topic: is the current plan to have a 0.30 release soon after 
the APP draft is submitted to the IESG?

AC

Re: Codec/UtilityTest issue?

[James M Snell <ja...@gmail.com>]

Yeah, the challenge I have with this is that it makes the test
circular... testing the output of a function against the output of the
same function.  We need to understand why the test is failing on the Mac
and figure out if it's something we need to fix.

- James

Adam Constabaris wrote:
> I'm still kind of puzzled by this one.  UtilityTest in the server module
> (tests for encoding utitlities) fails on a (Intel) Mac, but not on any
> Linux system I've tried.   The results coming back from the EncodingUtil
> aren't agreeing with the string literals in the test case.
> 
> Attached is a diff against
> server/src/test/java/org/apache/abdera/test/server/UtilityTest.java that
> replaces all the literals used as the test values with calls to the
> relevant methods from the commons codec library.  I'm not sure whether
> this is a good idea, although it lets the test pass =)
>