You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by Apache Wiki <wi...@apache.org> on 2014/09/04 23:16:31 UTC

[Hadoop Wiki] Update of "ProtocolBuffers" by ArpitAgarwal

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "ProtocolBuffers" page has been changed by ArpitAgarwal:
https://wiki.apache.org/hadoop/ProtocolBuffers?action=diff&rev1=4&rev2=5

  
  One criticism of both !ProtocolBuffers and Thrift is that the content is not self-describing; it is expected that the reader has compile-time expectations for the specific datatypes and interfaces, though possibly different versions. [[http://avro.apache.org/|Apache Avro]] does include in-content type declarations and runtime parsing, which is why some organizations using Hadoop consider it a significantly better format for persistent data: it becomes possible to parse files without advance knowledge of their structure.
  
+ == Installation Guide ==
+ Hadoop 0.23+ must have Google's ProtocolBuffers for compilation to work. These are native binaries which need to be downloaded, compiled and then installed locally.  See [[https://git-wip-us.apache.org/repos/asf?p=hadoop.git;a=blob_plain;f=BUILDING.txt;hb=HEAD|BUILDING.txt]]. 
+ 
+ This is a good opportunity to get the GNU C/C++ toolchain installed, which is useful for working on the native code used in the HDFS project.
+ 
+ To install and use ProtocolBuffers
+ 
+ ==== Unix ====
+ 
+ Install the protobuf packages ''provided they are current enough'' -see the README file for the current version. If they are too old, uninstall any version you have and follow the instructions.
+ 
+ ==== Local build and installation ====
+ 
+  * you need a copy of GCC 4.1+ including the {{{g++}}} C++ compiler, {{{make}}} and the rest of the GNU C++ development chain.
+  * Linux: you need a copy of autoconf installed, which your local package manager will do -along with automake.
+  * Download the version of protocol buffers that the BUILDING.txt recommends from [[http://code.google.com/p/protobuf/ |the protocol buffers project]].
+  * unzip it/untar it
+  * {{{cd}}} into the directory that has been created.
+  * run {{{./configure}}}
+  * If configure fails with "C++ preprocessor "/lib/cpp" fails sanity check" that means you don't have g++ installed. Install it.
+  * run {{{make}}} to build the libraries.
+  * on a Unix system, after building the libraries, you must install it ''as root''. {{{su}}} to root, then run {{{make install}}}
+ 
+ ==== Testing your Protocol Buffers installation ====
+ 
+ The test for this is verifying that {{{protoc}}} is on the command line. You should expect something like
+ 
+ {{{
+ $ protoc
+ Missing input file.
+ }}}
+ 
+ You may see the error message
+ {{{
+ $ protoc
+ protoc: error while loading shared libraries: libprotobuf.so.7: cannot open shared object file: No such file or directory
+ }}}
+ 
+ This is a [[http://code.google.com/p/protobuf/issues/detail?id=213 |known issue]] for Linux, and is caused by a stale cache of libraries. Run {{{ldconfig}}} and try again.
+