You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Vinay Kumar Thota (JIRA)" <ji...@apache.org> on 2010/05/17 12:42:43 UTC
[jira] Created: (MAPREDUCE-1793) Exception exculsion functionality
is not working correctly.
Exception exculsion functionality is not working correctly.
-----------------------------------------------------------
Key: MAPREDUCE-1793
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1793
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Reporter: Vinay Kumar Thota
Assignee: Balaji Rajagopalan
Exception exclusion functionality is not working correctly because of that tests are failing by not matching the error count.
I debugged the issue and found that the problem with shell command which is generating in the getNumberOfMatchesInLogFile function.
Currently building the shell command in the following way.
if(list != null){
for(int i =0; i < list.length; ++i)
{
filePattern.append(" | grep -v " + list[i] );
}
}
String[] cmd =
new String[] {
"bash",
"-c",
"grep -c "
+ pattern + " " + filePattern
+ " | awk -F: '{s+=$2} END {print s}'" };
However, The above commnad won't work correctly because you are counting the exceptions in the file before excluding the known exceptions.
In this case it gives the mismatch error counts everytime.The shell command should be in the following way to work correctly.
if (list != null) {
int index = 0;
for (String excludeExp : list) {
filePattern.append((++index < list.length)? "| grep -v " :
"| grep -vc " + list[i] );
}
}
String[] cmd =
new String[] {
"bash",
"-c",
"grep "
+ pattern + " " + filePattern
+ " | awk -F: '{s+=$2} END {print s}'" };
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.