You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Fengdong Yu (JIRA)" <ji...@apache.org> on 2014/05/11 00:16:05 UTC

[jira] [Updated] (HIVE-7039) Hive run on MR2 must set mapreduce.jobtracker.address

     [ https://issues.apache.org/jira/browse/HIVE-7039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Fengdong Yu updated HIVE-7039:
------------------------------

    Description: 
I am using hive-0.13.0 and hadoop-2.4.0,




why I must set 'mapreduce.jobtracker.address' in yarn-site.xml? otherwise, there are exceptions and job failed.




And, 'mapreduce.jobtracker.address' can be set to any value.




The following messages are gened without set 'mapreduce.jobtracker.address'.




Job output on the console:
{code}

Execution log at: /tmp/test/test_20140507180505_bcd4d89f-017c-4cf4-81a3-5fa619de0ad0.log
Job running in-process (local Hadoop)
Hadoop job information for null: number of mappers: 1; number of reducers: 1
 2014-05-07 18:06:25,782 null map = 0%,  reduce = 0%
2014-05-07 18:06:33,699 null map = 100%,  reduce = 0%
2014-05-07 18:06:34,774 null map = 0%,  reduce = 0%
2014-05-07 18:06:49,222 null map = 100%,  reduce = 100%
 Ended Job = job_1399453944131_0006 with errors
Error during job, obtaining debugging information...
{code}



Container error:

{code}
2014-05-07 18:06:33,634 INFO [main] org.apache.hadoop.hive.ql.exec.Utilities: No plan file found: file:/tmp/test/hive_2014-05-07_18-06-08_349_1526907284076641211-1/-mr-10001/0a1c9ebe-cdb0-4adc-9e93-8f176019f19a/map.xml
 2014-05-07 18:06:33,635 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.NullPointerException
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437)
 at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430)
	at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587)
	at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:168)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:409)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
	at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
{code}


  was:
I am using hive-0.13.0 and hadoop-2.4.0,




why I must set 'mapreduce.jobtracker.address' in yarn-site.xml? otherwise, there are exceptions and job failed.




And, 'mapreduce.jobtracker.address' can be set to any value.




The following messages are gened without set 'mapreduce.jobtracker.address'.




Job output on the console:

Execution log at: /tmp/test/test_20140507180505_bcd4d89f-017c-4cf4-81a3-5fa619de0ad0.log
Job running in-process (local Hadoop)
Hadoop job information for null: number of mappers: 1; number of reducers: 1
 2014-05-07 18:06:25,782 null map = 0%,  reduce = 0%
2014-05-07 18:06:33,699 null map = 100%,  reduce = 0%
2014-05-07 18:06:34,774 null map = 0%,  reduce = 0%
2014-05-07 18:06:49,222 null map = 100%,  reduce = 100%
 Ended Job = job_1399453944131_0006 with errors
Error during job, obtaining debugging information...




Container error:

2014-05-07 18:06:33,634 INFO [main] org.apache.hadoop.hive.ql.exec.Utilities: No plan file found: file:/tmp/test/hive_2014-05-07_18-06-08_349_1526907284076641211-1/-mr-10001/0a1c9ebe-cdb0-4adc-9e93-8f176019f19a/map.xml
 2014-05-07 18:06:33,635 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.NullPointerException
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
	at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437)
 at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430)
	at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587)
	at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:168)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:409)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
	at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)


> Hive run on MR2 must set mapreduce.jobtracker.address
> -----------------------------------------------------
>
>                 Key: HIVE-7039
>                 URL: https://issues.apache.org/jira/browse/HIVE-7039
>             Project: Hive
>          Issue Type: Bug
>          Components: Clients
>    Affects Versions: 0.13.0
>            Reporter: Fengdong Yu
>            Priority: Minor
>
> I am using hive-0.13.0 and hadoop-2.4.0,
> why I must set 'mapreduce.jobtracker.address' in yarn-site.xml? otherwise, there are exceptions and job failed.
> And, 'mapreduce.jobtracker.address' can be set to any value.
> The following messages are gened without set 'mapreduce.jobtracker.address'.
> Job output on the console:
> {code}
> Execution log at: /tmp/test/test_20140507180505_bcd4d89f-017c-4cf4-81a3-5fa619de0ad0.log
> Job running in-process (local Hadoop)
> Hadoop job information for null: number of mappers: 1; number of reducers: 1
>  2014-05-07 18:06:25,782 null map = 0%,  reduce = 0%
> 2014-05-07 18:06:33,699 null map = 100%,  reduce = 0%
> 2014-05-07 18:06:34,774 null map = 0%,  reduce = 0%
> 2014-05-07 18:06:49,222 null map = 100%,  reduce = 100%
>  Ended Job = job_1399453944131_0006 with errors
> Error during job, obtaining debugging information...
> {code}
> Container error:
> {code}
> 2014-05-07 18:06:33,634 INFO [main] org.apache.hadoop.hive.ql.exec.Utilities: No plan file found: file:/tmp/test/hive_2014-05-07_18-06-08_349_1526907284076641211-1/-mr-10001/0a1c9ebe-cdb0-4adc-9e93-8f176019f19a/map.xml
>  2014-05-07 18:06:33,635 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.NullPointerException
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437)
>  at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430)
> 	at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587)
> 	at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:168)
>  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:409)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
> 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
> 	at java.security.AccessController.doPrivileged(Native Method)
>  at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
> 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)