You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by Apache Wiki <wi...@apache.org> on 2014/09/04 23:16:31 UTC
[Hadoop Wiki] Update of "ProtocolBuffers" by ArpitAgarwal
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.
The "ProtocolBuffers" page has been changed by ArpitAgarwal:
https://wiki.apache.org/hadoop/ProtocolBuffers?action=diff&rev1=4&rev2=5
One criticism of both !ProtocolBuffers and Thrift is that the content is not self-describing; it is expected that the reader has compile-time expectations for the specific datatypes and interfaces, though possibly different versions. [[http://avro.apache.org/|Apache Avro]] does include in-content type declarations and runtime parsing, which is why some organizations using Hadoop consider it a significantly better format for persistent data: it becomes possible to parse files without advance knowledge of their structure.
+ == Installation Guide ==
+ Hadoop 0.23+ must have Google's ProtocolBuffers for compilation to work. These are native binaries which need to be downloaded, compiled and then installed locally. See [[https://git-wip-us.apache.org/repos/asf?p=hadoop.git;a=blob_plain;f=BUILDING.txt;hb=HEAD|BUILDING.txt]].
+
+ This is a good opportunity to get the GNU C/C++ toolchain installed, which is useful for working on the native code used in the HDFS project.
+
+ To install and use ProtocolBuffers
+
+ ==== Unix ====
+
+ Install the protobuf packages ''provided they are current enough'' -see the README file for the current version. If they are too old, uninstall any version you have and follow the instructions.
+
+ ==== Local build and installation ====
+
+ * you need a copy of GCC 4.1+ including the {{{g++}}} C++ compiler, {{{make}}} and the rest of the GNU C++ development chain.
+ * Linux: you need a copy of autoconf installed, which your local package manager will do -along with automake.
+ * Download the version of protocol buffers that the BUILDING.txt recommends from [[http://code.google.com/p/protobuf/ |the protocol buffers project]].
+ * unzip it/untar it
+ * {{{cd}}} into the directory that has been created.
+ * run {{{./configure}}}
+ * If configure fails with "C++ preprocessor "/lib/cpp" fails sanity check" that means you don't have g++ installed. Install it.
+ * run {{{make}}} to build the libraries.
+ * on a Unix system, after building the libraries, you must install it ''as root''. {{{su}}} to root, then run {{{make install}}}
+
+ ==== Testing your Protocol Buffers installation ====
+
+ The test for this is verifying that {{{protoc}}} is on the command line. You should expect something like
+
+ {{{
+ $ protoc
+ Missing input file.
+ }}}
+
+ You may see the error message
+ {{{
+ $ protoc
+ protoc: error while loading shared libraries: libprotobuf.so.7: cannot open shared object file: No such file or directory
+ }}}
+
+ This is a [[http://code.google.com/p/protobuf/issues/detail?id=213 |known issue]] for Linux, and is caused by a stale cache of libraries. Run {{{ldconfig}}} and try again.
+