You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Some Body <so...@squareplanet.de> on 2010/07/16 14:12:06 UTC

Killed : GC overhead limit exceeded

I'm seeing this error in my tasktracker's log.

FATAL org.apache.hadoop.mapred.TaskTracker: 
   Task: attempt_201007160344_0001_m_000005_1 
     - Killed : GC overhead limit exceed

more detail from my task's log states:

FATAL org.apache.hadoop.mapred.TaskTracker: 
    Error running child : java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.util.regex.Pattern$BitClass.<init>(Pattern.java:2910)
        at java.util.regex.Pattern.clazz(Pattern.java:2190)
        at java.util.regex.Pattern.sequence(Pattern.java:1818)
        at java.util.regex.Pattern.expr(Pattern.java:1752)
        at java.util.regex.Pattern.compile(Pattern.java:1460)
        at java.util.regex.Pattern.<init>(Pattern.java:1133)
        at java.util.regex.Pattern.compile(Pattern.java:823)
        at java.lang.String.replaceAll(String.java:2189)
        at com.synopsys.hadoop.RecordParser.parse(MyRecordParser.java:71)
        at com.synopsys.hadoop.ComputeSeqMapper.map(MyMapper.java:106)
        at com.synopsys.hadoop.ComputeSeqMapper.map(MyMapper.java:35)

Any ideas where to look further? I don't see anything wrong codewise?

line 106 MyMapper.java is from my map() and calls:

  parser.parse(line.toString()); 

which calls line 71 from MyParser.java which is basically this:

  public class RecordParser {
      .....    
      int curFieldCount = 0;
      String[] values = {};
      .....
      public void parse(String record) {
          values = record.split ("\t");
          // strip preceeding/trailing spaces 
          for (int i=0; i<values.length; i++) {
              values[i] = values[i].replaceAll("^[\\s]*", "");
              values[i] = values[i].replaceAll("[\\s]*$", "");
          }
          curFieldCount = values.length;
      }
  }

Alan