You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by "Jihoon Son (JIRA)" <ji...@apache.org> on 2015/12/08 04:01:11 UTC
[jira] [Created] (TAJO-2012) OOM while writing history
Jihoon Son created TAJO-2012:
--------------------------------
Summary: OOM while writing history
Key: TAJO-2012
URL: https://issues.apache.org/jira/browse/TAJO-2012
Project: Tajo
Issue Type: Bug
Reporter: Jihoon Son
Fix For: 0.12.0
I met OOM when querying on 10TB data set.
Here is the log.
{noformat}
2015-12-08 03:39:13,591 INFO org.apache.tajo.util.history.HistoryWriter: Saving query unit: hdfs://appliance007:9010/tmp/tajo-hadoop/staging/history/20151208/query-detail/q_1449490267507_0003/eb_1449490267507_0003_000001.hist
2015-12-08 03:39:22,520 ERROR org.apache.tajo.util.history.HistoryWriter: Error while saving query history: q_1449490267507_0003:Requested array size exceeds VM limit
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at java.util.Arrays.copyOf(Arrays.java:3332)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:137)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:121)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:421)
at java.lang.StringBuffer.append(StringBuffer.java:272)
at java.io.StringWriter.write(StringWriter.java:112)
at com.google.gson.stream.JsonWriter.string(JsonWriter.java:538)
at com.google.gson.stream.JsonWriter.value(JsonWriter.java:404)
at com.google.gson.internal.bind.TypeAdapters$13.write(TypeAdapters.java:353)
at com.google.gson.internal.bind.TypeAdapters$13.write(TypeAdapters.java:337)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
at com.google.gson.internal.bind.ArrayTypeAdapter.write(ArrayTypeAdapter.java:93)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
at com.google.gson.internal.bind.ArrayTypeAdapter.write(ArrayTypeAdapter.java:93)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:89)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:195)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:96)
at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:60)
at com.google.gson.Gson.toJson(Gson.java:586)
at com.google.gson.Gson.toJson(Gson.java:565)
at com.google.gson.Gson.toJson(Gson.java:520)
at org.apache.tajo.util.history.StageHistory.toTasksJson(StageHistory.java:233)
at org.apache.tajo.util.history.HistoryWriter$WriterThread.writeQueryHistory(HistoryWriter.java:378)
at org.apache.tajo.util.history.HistoryWriter$WriterThread.writeHistory(HistoryWriter.java:318)
at org.apache.tajo.util.history.HistoryWriter$WriterThread.run(HistoryWriter.java:244)
2015-12-08 09:11:08,538 INFO org.apache.tajo.util.history.HistoryCleaner: Cleaning task history dir: file:/tmp/tajo-hadoop/history/20151201
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)