You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by prasenjit mukherjee <pr...@gmail.com> on 2010/02/17 11:48:58 UTC
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
Here is my std-error :
hive> insert overwrite local directory '/tmp/mystuff' select transform(*)
using 'my.py' FROM myhivetable;
Total MapReduce jobs = 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201002160457_0033, Tracking URL =
http://ec2-204-236-205-98.compute-1.amazonaws.com:50030/jobdetails.jsp?jobid=job_201002160457_0033
Kill Command = /usr/lib/hadoop/bin/hadoop job -Dmapred.job.tracker=
ec2-204-236-205-98.compute-1.amazonaws.com:8021 -kill job_201002160457_0033
2010-02-17 05:40:28,380 map = 0%, reduce =0%
2010-02-17 05:41:12,469 map = 100%, reduce =100%
Ended Job = job_201002160457_0033 with errors
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.ExecDriver
I am trying to use the following command :
hive ql :
add file /root/my.py
insert overwrite local directory '/tmp/mystuff' select transform(*) using
'my.py' FROM myhivetable;
and following is my my.py:
#!/usr/bin/python
import sys
for line in sys.stdin:
line = line.strip()
flds = line.split('\t')
(cl_id,cook_id)=flds[:2]
sub_id=cl_id
if cl_id.startswith('foo'): sub_id=cook_id;
print ','.join([sub_id,flds[2],flds[3]])
This works fine, as I tested it in commandline using : echo -e
'aa\tbb\tcc\tdd' | /root/my.py
Any pointers ?
Re: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
Posted by Edward Capriolo <ed...@gmail.com>.
On Thu, Feb 18, 2010 at 12:54 AM, prasenjit mukherjee
<pm...@quattrowireless.com> wrote:
> Thanks a lot, that helped me to fix the problem. I ran with "hive -hiveconf
> hive.root.logger=DEBUG,console" and it threw some derby_lock_error, I re-ran
> after deleting the file '/var/lib/hive/metastore/${user.name}_db' and
> everything worked ok.
>
> Thanks again,
> -Prasen
>
> On Thu, Feb 18, 2010 at 11:06 AM, Sonal Goyal <so...@gmail.com> wrote:
>>
>> Can you edit your logging configurations to DEBUG, try again and check the
>> logs? Also check hadoop mapper logs.
>>
>> Thanks and Regards,
>> Sonal
>>
>>
>>
>
>
The meta-store lock: Is this because you are not running derby in server mode?
http://wiki.apache.org/hadoop/HiveDerbyServerMode
You need to run in server mode for multiple-concurrent access.
Re: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
Posted by prasenjit mukherjee <pm...@quattrowireless.com>.
Thanks a lot, that helped me to fix the problem. I ran with "hive -hiveconf
hive.root.logger=DEBUG,console" and it threw some derby_lock_error, I re-ran
after deleting the file '/var/lib/hive/metastore/${user.name}_db' and
everything worked ok.
Thanks again,
-Prasen
On Thu, Feb 18, 2010 at 11:06 AM, Sonal Goyal <so...@gmail.com> wrote:
> Can you edit your logging configurations to DEBUG, try again and check the
> logs? Also check hadoop mapper logs.
>
> Thanks and Regards,
> Sonal
>
>
>
>
>
Re: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
Posted by Sonal Goyal <so...@gmail.com>.
Can you edit your logging configurations to DEBUG, try again and check the
logs? Also check hadoop mapper logs.
Thanks and Regards,
Sonal
On Thu, Feb 18, 2010 at 11:04 AM, prasenjit mukherjee
<pr...@gmail.com>wrote:
> Sorry for the delay. Here is from my /tmp/root/hive.log file. Any other
> files I should be looking into.
>
> 2010-02-18 00:29:56,082 WARN mapred.JobClient
> (JobClient.java:configureCommandLineOptions(580)) - Use GenericOptionsParser
> for parsing the arguments. Applications should implement Tool for the same.
> 2010-02-18 00:30:39,506 ERROR exec.ExecDriver
> (SessionState.java:printError(279)) - Ended Job = job_201002171050_0011 with
> errors
> 2010-02-18 00:30:39,514 ERROR ql.Driver (SessionState.java:printError(279))
> - FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.ExecDriver
>
>
>
> On Wed, Feb 17, 2010 at 6:36 PM, Sonal Goyal <so...@gmail.com>wrote:
>
>> Hi,
>>
>> What do your Hive logs say? You can also check the Hadoop mapper and
>> reduce job logs.
>>
>> Thanks and Regards,
>> Sonal
>>
>>
>>
>> On Wed, Feb 17, 2010 at 4:18 PM, prasenjit mukherjee <
>> prasen.bea@gmail.com> wrote:
>>
>>>
>>> Here is my std-error :
>>> hive> insert overwrite local directory '/tmp/mystuff' select transform(*)
>>> using 'my.py' FROM myhivetable;
>>> Total MapReduce jobs = 1
>>> Number of reduce tasks is set to 0 since there's no reduce operator
>>> Starting Job = job_201002160457_0033, Tracking URL =
>>> http://ec2-204-236-205-98.compute-1.amazonaws.com:50030/jobdetails.jsp?jobid=job_201002160457_0033
>>> Kill Command = /usr/lib/hadoop/bin/hadoop job -Dmapred.job.tracker=
>>> ec2-204-236-205-98.compute-1.amazonaws.com:8021 -kill
>>> job_201002160457_0033
>>> 2010-02-17 05:40:28,380 map = 0%, reduce =0%
>>> 2010-02-17 05:41:12,469 map = 100%, reduce =100%
>>> Ended Job = job_201002160457_0033 with errors
>>> FAILED: Execution Error, return code 2 from
>>> org.apache.hadoop.hive.ql.exec.ExecDriver
>>>
>>>
>>> I am trying to use the following command :
>>>
>>> hive ql :
>>>
>>> add file /root/my.py
>>> insert overwrite local directory '/tmp/mystuff' select transform(*)
>>> using 'my.py' FROM myhivetable;
>>>
>>> and following is my my.py:
>>> #!/usr/bin/python
>>> import sys
>>> for line in sys.stdin:
>>> line = line.strip()
>>> flds = line.split('\t')
>>> (cl_id,cook_id)=flds[:2]
>>> sub_id=cl_id
>>> if cl_id.startswith('foo'): sub_id=cook_id;
>>> print ','.join([sub_id,flds[2],flds[3]])
>>>
>>> This works fine, as I tested it in commandline using : echo -e
>>> 'aa\tbb\tcc\tdd' | /root/my.py
>>>
>>> Any pointers ?
>>>
>>
>>
>
Re: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
Posted by prasenjit mukherjee <pr...@gmail.com>.
Sorry for the delay. Here is from my /tmp/root/hive.log file. Any other
files I should be looking into.
2010-02-18 00:29:56,082 WARN mapred.JobClient
(JobClient.java:configureCommandLineOptions(580)) - Use GenericOptionsParser
for parsing the arguments. Applications should implement Tool for the same.
2010-02-18 00:30:39,506 ERROR exec.ExecDriver
(SessionState.java:printError(279)) - Ended Job = job_201002171050_0011 with
errors
2010-02-18 00:30:39,514 ERROR ql.Driver (SessionState.java:printError(279))
- FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.ExecDriver
On Wed, Feb 17, 2010 at 6:36 PM, Sonal Goyal <so...@gmail.com> wrote:
> Hi,
>
> What do your Hive logs say? You can also check the Hadoop mapper and reduce
> job logs.
>
> Thanks and Regards,
> Sonal
>
>
>
> On Wed, Feb 17, 2010 at 4:18 PM, prasenjit mukherjee <prasen.bea@gmail.com
> > wrote:
>
>>
>> Here is my std-error :
>> hive> insert overwrite local directory '/tmp/mystuff' select transform(*)
>> using 'my.py' FROM myhivetable;
>> Total MapReduce jobs = 1
>> Number of reduce tasks is set to 0 since there's no reduce operator
>> Starting Job = job_201002160457_0033, Tracking URL =
>> http://ec2-204-236-205-98.compute-1.amazonaws.com:50030/jobdetails.jsp?jobid=job_201002160457_0033
>> Kill Command = /usr/lib/hadoop/bin/hadoop job -Dmapred.job.tracker=
>> ec2-204-236-205-98.compute-1.amazonaws.com:8021 -kill
>> job_201002160457_0033
>> 2010-02-17 05:40:28,380 map = 0%, reduce =0%
>> 2010-02-17 05:41:12,469 map = 100%, reduce =100%
>> Ended Job = job_201002160457_0033 with errors
>> FAILED: Execution Error, return code 2 from
>> org.apache.hadoop.hive.ql.exec.ExecDriver
>>
>>
>> I am trying to use the following command :
>>
>> hive ql :
>>
>> add file /root/my.py
>> insert overwrite local directory '/tmp/mystuff' select transform(*) using
>> 'my.py' FROM myhivetable;
>>
>> and following is my my.py:
>> #!/usr/bin/python
>> import sys
>> for line in sys.stdin:
>> line = line.strip()
>> flds = line.split('\t')
>> (cl_id,cook_id)=flds[:2]
>> sub_id=cl_id
>> if cl_id.startswith('foo'): sub_id=cook_id;
>> print ','.join([sub_id,flds[2],flds[3]])
>>
>> This works fine, as I tested it in commandline using : echo -e
>> 'aa\tbb\tcc\tdd' | /root/my.py
>>
>> Any pointers ?
>>
>
>
Re: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
Posted by Sonal Goyal <so...@gmail.com>.
Hi,
What do your Hive logs say? You can also check the Hadoop mapper and reduce
job logs.
Thanks and Regards,
Sonal
On Wed, Feb 17, 2010 at 4:18 PM, prasenjit mukherjee
<pr...@gmail.com>wrote:
>
> Here is my std-error :
> hive> insert overwrite local directory '/tmp/mystuff' select transform(*)
> using 'my.py' FROM myhivetable;
> Total MapReduce jobs = 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> Starting Job = job_201002160457_0033, Tracking URL =
> http://ec2-204-236-205-98.compute-1.amazonaws.com:50030/jobdetails.jsp?jobid=job_201002160457_0033
> Kill Command = /usr/lib/hadoop/bin/hadoop job -Dmapred.job.tracker=
> ec2-204-236-205-98.compute-1.amazonaws.com:8021 -kill
> job_201002160457_0033
> 2010-02-17 05:40:28,380 map = 0%, reduce =0%
> 2010-02-17 05:41:12,469 map = 100%, reduce =100%
> Ended Job = job_201002160457_0033 with errors
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.ExecDriver
>
>
> I am trying to use the following command :
>
> hive ql :
>
> add file /root/my.py
> insert overwrite local directory '/tmp/mystuff' select transform(*) using
> 'my.py' FROM myhivetable;
>
> and following is my my.py:
> #!/usr/bin/python
> import sys
> for line in sys.stdin:
> line = line.strip()
> flds = line.split('\t')
> (cl_id,cook_id)=flds[:2]
> sub_id=cl_id
> if cl_id.startswith('foo'): sub_id=cook_id;
> print ','.join([sub_id,flds[2],flds[3]])
>
> This works fine, as I tested it in commandline using : echo -e
> 'aa\tbb\tcc\tdd' | /root/my.py
>
> Any pointers ?
>