Hadoop RPC

[Tom Davis <ma...@gmail.com>]

Hi,

Hope this hasn't been asked before.

I'm interested in finding out more about Hadoop RPC and how it works.  I'd
like to write a service that cleanly fits into the ecosystem and making use
of the Hadoop RPC library looks like a good start, particularly as my
service will be called (at least) once for each mapper.

I've found this page:
https://wiki.apache.org/hadoop/HadoopRpc

I've followed the code-snippets and gotten a simple client / server app
working.

There's not a huge amount of documentation available, but from:
http://hortonworks.com/blog/rpc-improvements-and-wire-compatibility-in-apache-hadoop/
it sounds like Hadoop has migrated from the original RPC over-the-wire
serialization to use Google Protobufs.

Does that mean that the approach documented in the first link is deprecated
(or soon to be), or that there's a better approach?  Or is it just the
on-the-wire serialization that's changed?

Also interested in how authentication works on a secure cluster.  Is this
provided 'for free' as part of the RPC library?  That first link refers to
a follow up post regarding authentication, but I can't find it anyway.

Many thanks,

Tom