basic, dumb getting started question (single-node)

[Nicholas Murphy <ni...@oracle.com>]

Sorry for the (as per subject) basic, dumb getting started question but: I'm trying to just set up Hive on a single node.  Hadoop is set up and running happily, but following the directions on the Getting Started guide don't actually work (following the directions literally results in it complaining about a lack of a hive-site.conf file; putting a dummy one in results in, among other things, the HCatalog server not starting...looking at logs, it looks like it's encountering configuration parsing problems, which suggests I just don't have the configuration files set up correctly).  HADOOP_HOME is set, the correct directories are set up in HDFS, etc.

Hadoop version 2.5.1, Hive version 0.13.1, Oracle JDK (1.6, I believe), Debian 7.7.

I notice the default conf/ directory has a bunch of template files, but only that.  Can someone point me to a resource, or to an example of what configuration I need, to actually run Hive correctly out of the box on a single machine?

Thanks,
Nick

Re: basic, dumb getting started question (single-node)

[Nicholas Murphy <ni...@oracle.com>]

Default installation only comes with a hive-env.sh.template (lots of stuff, all of it commented out):

-----
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Set Hive and Hadoop environment variables here. These variables can be used
# to control the execution of Hive. It should be used by admins to configure
# the Hive installation (so that users do not have to set environment variables
# or set command line parameters to get correct behavior).
#
# The hive service being invoked (CLI/HWI etc.) is available via the environment
# variable SERVICE


# Hive Client memory usage can be an issue if a large number of clients
# are running at the same time. The flags below have been useful in 
# reducing memory usage:
#
# if [ "$SERVICE" = "cli" ]; then
#   if [ -z "$DEBUG" ]; then
#     export HADOOP_OPTS="$HADOOP_OPTS -XX:NewRatio=12 -Xms10m -XX:MaxHeapFreeRa
tio=40 -XX:MinHeapFreeRatio=15 -XX:+UseParNewGC -XX:-UseGCOverheadLimit"
#   else
#     export HADOOP_OPTS="$HADOOP_OPTS -XX:NewRatio=12 -Xms10m -XX:MaxHeapFreeRa
tio=40 -XX:MinHeapFreeRatio=15 -XX:-UseGCOverheadLimit"
#   fi
# fi

# The heap size of the jvm stared by hive shell script can be controlled via:
#
# export HADOOP_HEAPSIZE=1024
#
# Larger heap size may be required when running queries over large number of fil
es or partitions. 
# By default hive shell scripts use a heap size of 256 (MB).  Larger heap size w
ould also be 
# appropriate for hive server (hwi etc).


# Set HADOOP_HOME to point to a specific hadoop install directory
# HADOOP_HOME=${bin}/../../hadoop

# Hive Configuration Directory can be controlled by:
# export HIVE_CONF_DIR=

# Folder containing extra ibraries required for hive compilation/execution can b
e controlled by:
# export HIVE_AUX_JARS_PATH=
-----

Thanks,
Nick

> On Nov 12, 2014, at 2:25 PM, Ritesh Kumar Singh <ri...@gmail.com> wrote:
> 
> If you have correctly set your HIVE_HOME variable in your .bash_rc file,
> What's the output of:
> 
> >cat $HIVE_HOME/conf/hive-env.sh
> 
> If it doesn't work, run
> 
> >cat $HIVE_HOME/conf/hive-env.sh.template
> 
> On Thu, Nov 13, 2014 at 2:57 AM, Nicholas Murphy <ni...@oracle.com> wrote:
> Sorry for the (as per subject) basic, dumb getting started question but: I'm trying to just set up Hive on a single node.  Hadoop is set up and running happily, but following the directions on the Getting Started guide don't actually work (following the directions literally results in it complaining about a lack of a hive-site.conf file; putting a dummy one in results in, among other things, the HCatalog server not starting...looking at logs, it looks like it's encountering configuration parsing problems, which suggests I just don't have the configuration files set up correctly).  HADOOP_HOME is set, the correct directories are set up in HDFS, etc.
> 
> Hadoop version 2.5.1, Hive version 0.13.1, Oracle JDK (1.6, I believe), Debian 7.7.
> 
> I notice the default conf/ directory has a bunch of template files, but only that.  Can someone point me to a resource, or to an example of what configuration I need, to actually run Hive correctly out of the box on a single machine?
> 
> Thanks,
> Nick
> 

Re: basic, dumb getting started question (single-node)

[Ritesh Kumar Singh <ri...@gmail.com>]

If you have correctly set your HIVE_HOME variable in your .bash_rc file,
What's the output of:

>cat $HIVE_HOME/conf/hive-env.sh

If it doesn't work, run

>cat $HIVE_HOME/conf/hive-env.sh.template

On Thu, Nov 13, 2014 at 2:57 AM, Nicholas Murphy <ni...@oracle.com>
wrote:

> Sorry for the (as per subject) basic, dumb getting started question but:
> I'm trying to just set up Hive on a single node.  Hadoop is set up and
> running happily, but following the directions on the Getting Started guide
> don't actually work (following the directions literally results in it
> complaining about a lack of a hive-site.conf file; putting a dummy one in
> results in, among other things, the HCatalog server not starting...looking
> at logs, it looks like it's encountering configuration parsing problems,
> which suggests I just don't have the configuration files set up
> correctly).  HADOOP_HOME is set, the correct directories are set up in
> HDFS, etc.
>
> Hadoop version 2.5.1, Hive version 0.13.1, Oracle JDK (1.6, I believe),
> Debian 7.7.
>
> I notice the default conf/ directory has a bunch of template files, but
> only that.  Can someone point me to a resource, or to an example of what
> configuration I need, to actually run Hive correctly out of the box on a
> single machine?
>
> Thanks,
> Nick