You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Mohit Anchlia <mo...@gmail.com> on 2012/05/24 01:48:28 UTC
Local hadoop
When I launch simple SQL I see "local hadoop". And when I do hadoop job fs
-list in my hadoop cluster I don't see any jobs. Am I doing something wrong
here?
# hive
Hive history file=/tmp/root/hive_job_log_root_201205231
Execution log at:
/tmp/root/root_20120523163636_18c8cce4-7568-401f-b502-223011ee7877.log
Job running in-process (local Hadoop)
2012-05-23 16:36:51,648 null map = 0%, reduce = 0%
Re: Local hadoop
Posted by Mohit Anchlia <mo...@gmail.com>.
Thanks! Do most of the people run hive on the datanodes? I was running hive
on a non-hadoop node.
On Wed, May 23, 2012 at 5:36 PM, Edward Capriolo <ed...@gmail.com>wrote:
> If your HADOOP_HOME specified the correct path your hadoop should
> already be picking up this setting as well as many others. Be careful
> here. If you have the default settings your dfs.replication would be 1
> or you might get some other nasty surprises.
>
> On Wed, May 23, 2012 at 7:59 PM, Mohit Anchlia <mo...@gmail.com>
> wrote:
> > Thanks. I used --hiveconf to set the jobtracker and it worked.
> >
> > On Wed, May 23, 2012 at 4:57 PM, Edward Capriolo <ed...@gmail.com>
> > wrote:
> >>
> >> Hive will chose local mode when the inputs files are "small" as an
> >> optimization. This also happens if mapred.job.tracker is set to local.
> >>
> >> On Wed, May 23, 2012 at 7:48 PM, Mohit Anchlia <mo...@gmail.com>
> >> wrote:
> >> > When I launch simple SQL I see "local hadoop". And when I do hadoop
> job
> >> > fs
> >> > -list in my hadoop cluster I don't see any jobs. Am I doing something
> >> > wrong
> >> > here?
> >> >
> >> >
> >> > # hive
> >> >
> >> > Hive history file=/tmp/root/hive_job_log_root_201205231
> >> >
> >> > Execution log at:
> >> > /tmp/root/root_20120523163636_18c8cce4-7568-401f-b502-223011ee7877.log
> >> >
> >> > Job running in-process (local Hadoop)
> >> >
> >> > 2012-05-23 16:36:51,648 null map = 0%, reduce = 0%
> >
> >
>
Re: Local hadoop
Posted by Edward Capriolo <ed...@gmail.com>.
If your HADOOP_HOME specified the correct path your hadoop should
already be picking up this setting as well as many others. Be careful
here. If you have the default settings your dfs.replication would be 1
or you might get some other nasty surprises.
On Wed, May 23, 2012 at 7:59 PM, Mohit Anchlia <mo...@gmail.com> wrote:
> Thanks. I used --hiveconf to set the jobtracker and it worked.
>
> On Wed, May 23, 2012 at 4:57 PM, Edward Capriolo <ed...@gmail.com>
> wrote:
>>
>> Hive will chose local mode when the inputs files are "small" as an
>> optimization. This also happens if mapred.job.tracker is set to local.
>>
>> On Wed, May 23, 2012 at 7:48 PM, Mohit Anchlia <mo...@gmail.com>
>> wrote:
>> > When I launch simple SQL I see "local hadoop". And when I do hadoop job
>> > fs
>> > -list in my hadoop cluster I don't see any jobs. Am I doing something
>> > wrong
>> > here?
>> >
>> >
>> > # hive
>> >
>> > Hive history file=/tmp/root/hive_job_log_root_201205231
>> >
>> > Execution log at:
>> > /tmp/root/root_20120523163636_18c8cce4-7568-401f-b502-223011ee7877.log
>> >
>> > Job running in-process (local Hadoop)
>> >
>> > 2012-05-23 16:36:51,648 null map = 0%, reduce = 0%
>
>
Re: Local hadoop
Posted by Mohit Anchlia <mo...@gmail.com>.
Thanks. I used --hiveconf to set the jobtracker and it worked.
On Wed, May 23, 2012 at 4:57 PM, Edward Capriolo <ed...@gmail.com>wrote:
> Hive will chose local mode when the inputs files are "small" as an
> optimization. This also happens if mapred.job.tracker is set to local.
>
> On Wed, May 23, 2012 at 7:48 PM, Mohit Anchlia <mo...@gmail.com>
> wrote:
> > When I launch simple SQL I see "local hadoop". And when I do hadoop job
> fs
> > -list in my hadoop cluster I don't see any jobs. Am I doing something
> wrong
> > here?
> >
> >
> > # hive
> >
> > Hive history file=/tmp/root/hive_job_log_root_201205231
> >
> > Execution log at:
> > /tmp/root/root_20120523163636_18c8cce4-7568-401f-b502-223011ee7877.log
> >
> > Job running in-process (local Hadoop)
> >
> > 2012-05-23 16:36:51,648 null map = 0%, reduce = 0%
>
Re: Local hadoop
Posted by Edward Capriolo <ed...@gmail.com>.
Hive will chose local mode when the inputs files are "small" as an
optimization. This also happens if mapred.job.tracker is set to local.
On Wed, May 23, 2012 at 7:48 PM, Mohit Anchlia <mo...@gmail.com> wrote:
> When I launch simple SQL I see "local hadoop". And when I do hadoop job fs
> -list in my hadoop cluster I don't see any jobs. Am I doing something wrong
> here?
>
>
> # hive
>
> Hive history file=/tmp/root/hive_job_log_root_201205231
>
> Execution log at:
> /tmp/root/root_20120523163636_18c8cce4-7568-401f-b502-223011ee7877.log
>
> Job running in-process (local Hadoop)
>
> 2012-05-23 16:36:51,648 null map = 0%, reduce = 0%