Compiler for Hadoop WritableComparable objects (similar to protobufs)

[Slava Chernyak <sl...@gmail.com>]

Hello All,

I have developed a simple compiler that compiles object specifications into
Hadoop Writable and WritableComparable objects. The purpose of this is to
avoid writing boilerplate code every time you need to have a custom data
type passed around in Hadoop. As input it takes a specification containing a
list of objects and their fields and generates Java classes that implement
Hadoop's Writable or WritableComparable and can serialize and de-serialize
these fields. This compiler is still in its very early stages but I have
found it useful throughout my work with different Hadoop based projects. I
would appreciate any feedback on this!

The project (download, wiki, and source code) is here:
http://code.google.com/p/hadoop-data-adapters/
Thank you,
- Slava Chernyak

Re: Compiler for Hadoop WritableComparable objects (similar to protobufs)

[Enis Soztutar <en...@gmail.com>]

Hi,

Have you tried record-io in Hadoop? What is the difference between this 
and recordio?

Slava Chernyak wrote:
> Hello All,
>
> I have developed a simple compiler that compiles object specifications into
> Hadoop Writable and WritableComparable objects. The purpose of this is to
> avoid writing boilerplate code every time you need to have a custom data
> type passed around in Hadoop. As input it takes a specification containing a
> list of objects and their fields and generates Java classes that implement
> Hadoop's Writable or WritableComparable and can serialize and de-serialize
> these fields. This compiler is still in its very early stages but I have
> found it useful throughout my work with different Hadoop based projects. I
> would appreciate any feedback on this!
>
> The project (download, wiki, and source code) is here:
> http://code.google.com/p/hadoop-data-adapters/
> Thank you,
> - Slava Chernyak
>
>