You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "Akash R Nilugal (JIRA)" <ji...@apache.org> on 2019/01/07 13:52:01 UTC
[jira] [Updated] (CARBONDATA-3233) JVM is getting crashed during
dataload while compressing in snappy
[ https://issues.apache.org/jira/browse/CARBONDATA-3233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Akash R Nilugal updated CARBONDATA-3233:
----------------------------------------
Description:
when huge dataload is done, some times dataload is failed and jvm is crashed during snappy compression
Below is the logs:
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j org.xerial.snappy.SnappyNative.rawCompress(JJJ)J+0
j org.apache.carbondata.core.datastore.compression.SnappyCompressor.rawCompress(JIJ)J+9
j org.apache.carbondata.core.datastore.page.UnsafeFixLengthColumnPage.compress(Lorg/apache/carbondata/core/datastore/compression/Compressor[B+50
j org.apache.carbondata.core.datastore.page.encoding.adaptive.AdaptiveCodec.encodeAndCompressPage(Lorg/apache/carbondata/core/datastore/page/ColumnPage;Lorg/apache/carbondata/core/datastore/page/ColumnPageValueConverter;Lorg/apache/carbondata/core/datastore/compression/Compressor[B+85
j org.apache.carbondata.core.datastore.page.encoding.adaptive.AdaptiveDeltaIntegralCodec$1.encodeData(Lorg/apache/carbondata/core/datastore/page/ColumnPage[B+45
j org.apache.carbondata.core.datastore.page.encoding.ColumnPageEncoder.encode(Lorg/apache/carbondata/core/datastore/page/ColumnPage;)Lorg/apache/carbondata/core/datastore/page/encoding/EncodedColumnPage;+2
j org.apache.carbondata.processing.store.TablePage.encodeAndCompressMeasures()[Lorg/apache/carbondata/core/datastore/page/encoding/EncodedColumnPage;+54
j org.apache.carbondata.processing.store.TablePage.encode()V+6
j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.processDataRows(Ljava/util/List;)Lorg/apache/carbondata/processing/store/TablePage;+86
j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.access$500(Lorg/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar;Ljava/util/List;)Lorg/apache/carbondata/processing/store/TablePage;+2
j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call()Ljava/lang/Void;+8
j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call()Ljava/lang/Object;+1
j java.util.concurrent.FutureTask.run()V+42
j java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub
was:
when huge dataload is done, some times dataload is failed and jvm is crashed during snappy compression
Below is the logs:
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j org.xerial.snappy.SnappyNative.rawCompress(JJJ)J+0
j org.apache.carbondata.core.datastore.compression.SnappyCompressor.rawCompress(JIJ)J+9
j org.apache.carbondata.core.datastore.page.UnsafeFixLengthColumnPage.compress(Lorg/apache/carbondata/core/datastore/compression/Compressor;)[B+50
j org.apache.carbondata.core.datastore.page.encoding.adaptive.AdaptiveCodec.encodeAndCompressPage(Lorg/apache/carbondata/core/datastore/page/ColumnPage;Lorg/apache/carbondata/core/datastore/page/ColumnPageValueConverter;Lorg/apache/carbondata/core/datastore/compression/Compressor;)[B+85
j org.apache.carbondata.core.datastore.page.encoding.adaptive.AdaptiveDeltaIntegralCodec$1.encodeData(Lorg/apache/carbondata/core/datastore/page/ColumnPage;)[B+45
j org.apache.carbondata.core.datastore.page.encoding.ColumnPageEncoder.encode(Lorg/apache/carbondata/core/datastore/page/ColumnPage;)Lorg/apache/carbondata/core/datastore/page/encoding/EncodedColumnPage;+2
j org.apache.carbondata.processing.store.TablePage.encodeAndCompressMeasures()[Lorg/apache/carbondata/core/datastore/page/encoding/EncodedColumnPage;+54
j org.apache.carbondata.processing.store.TablePage.encode()V+6
j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.processDataRows(Ljava/util/List;)Lorg/apache/carbondata/processing/store/TablePage;+86
j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.access$500(Lorg/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar;Ljava/util/List;)Lorg/apache/carbondata/processing/store/TablePage;+2
j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call()Ljava/lang/Void;+8
j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call()Ljava/lang/Object;+1
j java.util.concurrent.FutureTask.run()V+42
j java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub
> JVM is getting crashed during dataload while compressing in snappy
> ------------------------------------------------------------------
>
> Key: CARBONDATA-3233
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3233
> Project: CarbonData
> Issue Type: Bug
> Reporter: Akash R Nilugal
> Assignee: Akash R Nilugal
> Priority: Major
>
> when huge dataload is done, some times dataload is failed and jvm is crashed during snappy compression
>
> Below is the logs:
> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
> j org.xerial.snappy.SnappyNative.rawCompress(JJJ)J+0
> j org.apache.carbondata.core.datastore.compression.SnappyCompressor.rawCompress(JIJ)J+9
> j org.apache.carbondata.core.datastore.page.UnsafeFixLengthColumnPage.compress(Lorg/apache/carbondata/core/datastore/compression/Compressor[B+50
> j org.apache.carbondata.core.datastore.page.encoding.adaptive.AdaptiveCodec.encodeAndCompressPage(Lorg/apache/carbondata/core/datastore/page/ColumnPage;Lorg/apache/carbondata/core/datastore/page/ColumnPageValueConverter;Lorg/apache/carbondata/core/datastore/compression/Compressor[B+85
> j org.apache.carbondata.core.datastore.page.encoding.adaptive.AdaptiveDeltaIntegralCodec$1.encodeData(Lorg/apache/carbondata/core/datastore/page/ColumnPage[B+45
> j org.apache.carbondata.core.datastore.page.encoding.ColumnPageEncoder.encode(Lorg/apache/carbondata/core/datastore/page/ColumnPage;)Lorg/apache/carbondata/core/datastore/page/encoding/EncodedColumnPage;+2
> j org.apache.carbondata.processing.store.TablePage.encodeAndCompressMeasures()[Lorg/apache/carbondata/core/datastore/page/encoding/EncodedColumnPage;+54
> j org.apache.carbondata.processing.store.TablePage.encode()V+6
> j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.processDataRows(Ljava/util/List;)Lorg/apache/carbondata/processing/store/TablePage;+86
> j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.access$500(Lorg/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar;Ljava/util/List;)Lorg/apache/carbondata/processing/store/TablePage;+2
> j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call()Ljava/lang/Void;+8
> j org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call()Ljava/lang/Object;+1
> j java.util.concurrent.FutureTask.run()V+42
> j java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
> j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
> j java.lang.Thread.run()V+11
> v ~StubRoutines::call_stub
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)