You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@whirr.apache.org by Marco Didonna <m....@gmail.com> on 2013/05/29 12:16:49 UTC

Cdh4 and pig

Hello everyone,
I'm using latest stable whirr and I was trying to fire up a toy hadoop
cluster to use with pig. Therefore I took a look at the appropriate recipes
and I tried to accomplish my goals...here's the problems I found:

- If I don't specify install_cdh_hadoop hadoop is not installed and neither
is pig. If I do specify that option cdh4 is properly installed and I can
successfully run a wordcount job.

- Looks like cdh4 ships with MRv2 as I get
"org.apache.pig.backend.hadoop20.PigJobControl -   falling back to default
JobControl (not using hadoop 0.20 ?)
java.lang.NoSuchFieldException: runnerState" but I did not choose the -yarn
recipe

- I'd like to customize the hadoop-env.HADOOP_HEAPSIZE variable in
different ways according to the different node type (DN+TT vs NN+JT), is
that possible? Using

whirr.templates.hadoop-namenode+hadoop-jobtracker+pig-client.hadoop-env.HADOOP_HEAPSIZE=2048
whirr.templates.hadoop-datanode+hadoop-tasktracker.hadoop-env.HADOOP_HEAPSIZE=512

has no effect at all :)

I hope somebody can clear my doubts.

Thank you,

Marco