specification of type params in org.apache.hadoop.io.*Writable impls

[Andy Schlaikjer <an...@gmail.com>]

Can anyone comment on why the base *Writable implementations (e.g.
IntWritable, LongWritable, etc) don't specify WritableComparable<T>'s
type param T?

I noticed this when I tried to use DoubleWritable [1] within a custom
WritableComparable parameterized type which constrains its own type
params:

public class Pair<
  First extends WritableComparable<First>,
  Second extends WritableComparable<Second>>
  implements WritableComparable<Pair<First, Second>> {
...
}

With this declaration, something like the following will not compile:

Pair<DoubleWritable, DoubleWritable> pair = null;

ERROR: Bound mismatch: The type DoubleWritable is not a valid
substitute for the bounded parameter <First extends
WritableComparable<First>> of the type Pair<First,Second>

However, if DoubleWritable were declared as follows, the above line
would work just fine:

public class DoubleWritable implements WritableComparable<DoubleWritable> {
...
}

[1] http://svn.apache.org/viewvc/hadoop/common/trunk/src/java/org/apache/hadoop/io/DoubleWritable.java?revision=786726&view=markup

Best,

Re: specification of type params in org.apache.hadoop.io.*Writable impls

[Andy Schlaikjer <an...@gmail.com>]

I've received a number of responses from helpful Hadoop list members. Thanks!

But I should clarify here; I'm not looking for workarounds for my Pair
declaration or explanations of Java's generics facilities. I'm looking
for justifications for Hadoop's approach.

Looking at Java's core library, there are 86 classes which implement
java.lang.Comparable<T> [1]. 80 of these specify T = the type being
declared, or some parent of the type being declared.

[1] http://java.sun.com/javase/6/docs/api/java/lang/Comparable.html

So, I'm curious, are there specific use cases the Hadoop community has
run into which support the current design of the primitive *Writable
types which excludes specification of Comparable<T>'s T param? If not
I'll try and find time to work on a patch to push more conventional
use of generics into Hadoop Common.

Best,

Re: specification of type params in org.apache.hadoop.io.*Writable impls

[Julia Smith <in...@mac.com>]

Java Generics are finicky. Basically the parameterized generic is not considered the same thing (type) as a class extended from it. I went through this when writing a deeply generic-ed package just to test its limits with a heavily templatized c++ package.

On Mar 29, 2010, at 4:06 PM, Andy Schlaikjer wrote:

> Can anyone comment on why the base *Writable implementations (e.g.
> IntWritable, LongWritable, etc) don't specify WritableComparable<T>'s
> type param T?
> 
> I noticed this when I tried to use DoubleWritable [1] within a custom
> WritableComparable parameterized type which constrains its own type
> params:
> 
> public class Pair<
>  First extends WritableComparable<First>,
>  Second extends WritableComparable<Second>>
>  implements WritableComparable<Pair<First, Second>> {
> ...
> }
> 
> With this declaration, something like the following will not compile:
> 
> Pair<DoubleWritable, DoubleWritable> pair = null;
> 
> ERROR: Bound mismatch: The type DoubleWritable is not a valid
> substitute for the bounded parameter <First extends
> WritableComparable<First>> of the type Pair<First,Second>
> 
> However, if DoubleWritable were declared as follows, the above line
> would work just fine:
> 
> public class DoubleWritable implements WritableComparable<DoubleWritable> {
> ...
> }
> 
> [1] http://svn.apache.org/viewvc/hadoop/common/trunk/src/java/org/apache/hadoop/io/DoubleWritable.java?revision=786726&view=markup
> 
> Best,

Re: specification of type params in org.apache.hadoop.io.*Writable impls

[Julia Smith <in...@mac.com>]

It's easy enough to change the prototype to Comparable<?>, should fix it.

On Mar 30, 2010, at 9:51 AM, Doug Cutting wrote:

> Andy Schlaikjer wrote:
>> Can anyone comment on why the base *Writable implementations (e.g.
>> IntWritable, LongWritable, etc) don't specify WritableComparable<T>'s
>> type param T?
> 
> I suspect it's just historic.  When they were first written we were not using Java generics, and no one has ever retrofitted them with generics.
> 
> Doug

Re: specification of type params in org.apache.hadoop.io.*Writable impls

[Doug Cutting <cu...@apache.org>]

Andy Schlaikjer wrote:
> Can anyone comment on why the base *Writable implementations (e.g.
> IntWritable, LongWritable, etc) don't specify WritableComparable<T>'s
> type param T?

I suspect it's just historic.  When they were first written we were not 
using Java generics, and no one has ever retrofitted them with generics.

Doug