You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by wi...@thomsonreuters.com on 2013/03/25 20:32:13 UTC
missing error log
Dear pig users,
What does it mean when pig [Cloudera Pig version 0.10.0-cdh4.1.2] reports
2013-03-25 14:46:31,186 [main] INFO org.apache.pig.Main - Logging error messages to: /proj/ac/acComponents/blocker/pig_1364237191181.log
but that file is not created? I think there are errors in my pig script or something it depends on because it has STORE statements (and I tried DUMP too) yet there was no output. Also the runtime is too quick for any real processing to have happened. But without an error log it’s hard to track down. How can I make pig reporting more verbose, and force it to the console?
Thanks,
Will
William F Dowling
Senior Technologist
Thomson Reuters
RE: missing error log
Posted by wi...@thomsonreuters.com.
Thanks Johnny for your reply. Working backwards: I am using MRv1. I did try the logging suggestion you made, but did not get any other info.
So I did it the old-fashioned way with code bisection, commenting out swaths of my script to localize the error. It turned out it was this statement:
%default analysis_func = positive_features
When my code had that statement, it did not run productively, nor did it show any useful error message. When I changed that to
%default analysis_func positive_features
then the script ran as expected, and when I intentionally introduced an error, the line
lkl lkl
then that error was logged with a good message
Pig Stack Trace
---------------
ERROR 1000: Error during parsing. Encountered " <IDENTIFIER> "lkl "" at line 100, column 1.
Was expecting one of:
<EOF>
"cat" ...
"fs" ...
"sh" ...
"cd" ...
So the problem was the '=' character in the %default statement, but I was not able to find that out through an error message.
Thanks again,
Will
William F Dowling
Senior Technologist
Thomson Reuters
-----Original Message-----
From: Johnny Zhang [mailto:xiaoyuz@cloudera.com]
Sent: Monday, March 25, 2013 4:10 PM
To: user@pig.apache.org
Subject: Re: missing error log
Hi, Will:
Can you try to check below make sure the logger is property configured:
1. Edit conf/log4j.properties as follows:
log4j.logger.org.apache.hadoop.conf.Configuration=ERROR, A
2. Edit conf/pig.properties as follows:
# log4jconf log4j configuration file
log4jconf=<path_to_log4j.properties>/log4j.properties
If there is still no error message, bump the level to INFO to make sure you
can see the logger output in the log file.
BTW, are you using MRv1 or MRv2 ?
Johnny
On Mon, Mar 25, 2013 at 12:32 PM, <wi...@thomsonreuters.com>wrote:
> Dear pig users,
>
> What does it mean when pig [Cloudera Pig version 0.10.0-cdh4.1.2] reports
>
> 2013-03-25 14:46:31,186 [main] INFO org.apache.pig.Main - Logging error
> messages to: /proj/ac/acComponents/blocker/pig_1364237191181.log
>
> but that file is not created? I think there are errors in my pig script or
> something it depends on because it has STORE statements (and I tried DUMP
> too) yet there was no output. Also the runtime is too quick for any real
> processing to have happened. But without an error log it’s hard to track
> down. How can I make pig reporting more verbose, and force it to the
> console?
> Thanks,
>
> Will
>
> William F Dowling
> Senior Technologist
>
> Thomson Reuters
>
>
>
>
Re: missing error log
Posted by Johnny Zhang <xi...@cloudera.com>.
Hi, Will:
Can you try to check below make sure the logger is property configured:
1. Edit conf/log4j.properties as follows:
log4j.logger.org.apache.hadoop.conf.Configuration=ERROR, A
2. Edit conf/pig.properties as follows:
# log4jconf log4j configuration file
log4jconf=<path_to_log4j.properties>/log4j.properties
If there is still no error message, bump the level to INFO to make sure you
can see the logger output in the log file.
BTW, are you using MRv1 or MRv2 ?
Johnny
On Mon, Mar 25, 2013 at 12:32 PM, <wi...@thomsonreuters.com>wrote:
> Dear pig users,
>
> What does it mean when pig [Cloudera Pig version 0.10.0-cdh4.1.2] reports
>
> 2013-03-25 14:46:31,186 [main] INFO org.apache.pig.Main - Logging error
> messages to: /proj/ac/acComponents/blocker/pig_1364237191181.log
>
> but that file is not created? I think there are errors in my pig script or
> something it depends on because it has STORE statements (and I tried DUMP
> too) yet there was no output. Also the runtime is too quick for any real
> processing to have happened. But without an error log it’s hard to track
> down. How can I make pig reporting more verbose, and force it to the
> console?
> Thanks,
>
> Will
>
> William F Dowling
> Senior Technologist
>
> Thomson Reuters
>
>
>
>