You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@carbondata.apache.org by Sandeep N <fe...@gmail.com> on 2022/04/18 17:47:37 UTC

Newbie Question: Challenges with Getting Started

Hi all,

I ran into carbondata and started trying it out.  I am following this page
https://carbondata.apache.org/quick-start-guide.html.

So far I have downloaded
apache-carbondata-2.3.0-bin-spark3.1.1-hadoop2.7.2.jar. I am trying to use
it with Spark 3.1.3 (I imagine the micro version difference should not
matter).

So far I have tried this with OpenJDK 8 and OpenJDK 11 and in both
instances, create table works however when I attempt to load data from CSV
it fails with the exception below. This is a different csv from what is
called out on that quick-start page.

Here is how I am launching Carbondata
*spark-shell --conf
spark.sql.extensions=org.apache.spark.sql.CarbonExtensions --jars <location
to the above jar>*

I am getting failures on both OpenJDK 11 and OpenJDK 8. Open JDK 8 fails
with a segfault. I am running on a MacBook pro. The OpenJDK 11 errors seem
to indicate that is not supported but JDK 8 seems to crash and exit. Please
see the errors below, can someone point out what I am doing wrong?

Error on OpenJDK 8 when I try to load data from a CSV file
==================================================
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x0000000104cbd7bb, pid=17765, tid=0x000000000000bd03
#
# JRE version: OpenJDK Runtime Environment (8.0_282) (build
1.8.0_282-bre_2021_01_20_16_06-b00)
# Java VM: OpenJDK 64-Bit Server VM (25.282-b00 mixed mode bsd-amd64
compressed oops)
# Problematic frame:
# V  [libjvm.dylib+0x5667bb]

Error on OpenJDK 11 when I try to load data from a CSV file
==================================================
22/04/17 23:57:06 ERROR CarbonFactDataHandlerColumnar: Error in producer
java.lang.reflect.InaccessibleObjectException: Unable to make public void
jdk.internal.ref.Cleaner.clean() accessible: module java.base does not
"exports jdk.internal.ref" to unnamed module @5115e1e6
at
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:340)
at
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:198)
at java.base/java.lang.reflect.Method.setAccessible(Method.java:192)
at
org.apache.carbondata.core.memory.UnsafeMemoryManager.destroyDirectByteBuffer(UnsafeMemoryManager.java:232)
at
org.apache.carbondata.core.datastore.page.LVByteBufferColumnPage.ensureMemory(LVByteBufferColumnPage.java:125)
at
org.apache.carbondata.core.datastore.page.LVByteBufferColumnPage.putBytes(LVByteBufferColumnPage.java:97)
at
org.apache.carbondata.core.datastore.page.LocalDictColumnPage.putBytes(LocalDictColumnPage.java:139)
at
org.apache.carbondata.core.datastore.page.ColumnPage.putData(ColumnPage.java:413)
at
org.apache.carbondata.processing.store.TablePage.convertToColumnarAndAddToPages(TablePage.java:241)
at
org.apache.carbondata.processing.store.TablePage.addRow(TablePage.java:201)
at
org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.processDataRows(CarbonFactDataHandlerColumnar.java:397)
at
org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.access$500(CarbonFactDataHandlerColumnar.java:60)
at
org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call(CarbonFactDataHandlerColumnar.java:637)
at
org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call(CarbonFactDataHandlerColumnar.java:614)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)

Error on OpenJDK 11 when I try to insert a single record
==================================================
22/04/18 09:46:10 ERROR CarbonFactDataHandlerColumnar: Error in producer
java.lang.reflect.InaccessibleObjectException: Unable to make public void
jdk.internal.ref.Cleaner.clean() accessible: module java.base does not
"exports jdk.internal.ref" to unnamed module @cc4787f
at
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:340)
at
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:198)
at java.base/java.lang.reflect.Method.setAccessible(Method.java:192)
at
org.apache.carbondata.core.memory.UnsafeMemoryManager.destroyDirectByteBuffer(UnsafeMemoryManager.java:232)
at
org.apache.carbondata.core.datastore.page.LVByteBufferColumnPage.ensureMemory(LVByteBufferColumnPage.java:125)
at
org.apache.carbondata.core.datastore.page.LVByteBufferColumnPage.putBytes(LVByteBufferColumnPage.java:97)
at
org.apache.carbondata.core.datastore.page.LocalDictColumnPage.putBytes(LocalDictColumnPage.java:139)
at
org.apache.carbondata.core.datastore.page.ColumnPage.putData(ColumnPage.java:413)
at
org.apache.carbondata.processing.store.TablePage.convertToColumnarAndAddToPages(TablePage.java:241)
at
org.apache.carbondata.processing.store.TablePage.addRow(TablePage.java:201)
at
org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.processDataRows(CarbonFactDataHandlerColumnar.java:397)
at
org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.access$500(CarbonFactDataHandlerColumnar.java:60)
at
org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call(CarbonFactDataHandlerColumnar.java:637)
at
org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call(CarbonFactDataHandlerColumnar.java:614)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
22/04/18 09:46:10 ERROR CarbonFactDataHandlerColumnar: Problem while
writing the carbon data file

Thanks,

--Sandeep

Re: Newbie Question: Challenges with Getting Started

Posted by Liang Chen <ch...@apache.org>.

Hi

Should be ok, please try it again.

Regards


Sandeep N <fe...@gmail.com> 于2022年4月18日周一 19:47写道：

> Hi all,
>
> I ran into carbondata and started trying it out.  I am following this page
> https://carbondata.apache.org/quick-start-guide.html.
>
> So far I have downloaded
> apache-carbondata-2.3.0-bin-spark3.1.1-hadoop2.7.2.jar. I am trying to use
> it with Spark 3.1.3 (I imagine the micro version difference should not
> matter).
>
> So far I have tried this with OpenJDK 8 and OpenJDK 11 and in both
> instances, create table works however when I attempt to load data from CSV
> it fails with the exception below. This is a different csv from what is
> called out on that quick-start page.
>
> Here is how I am launching Carbondata
> *spark-shell --conf
> spark.sql.extensions=org.apache.spark.sql.CarbonExtensions --jars <location
> to the above jar>*
>
> I am getting failures on both OpenJDK 11 and OpenJDK 8. Open JDK 8 fails
> with a segfault. I am running on a MacBook pro. The OpenJDK 11 errors seem
> to indicate that is not supported but JDK 8 seems to crash and exit. Please
> see the errors below, can someone point out what I am doing wrong?
>
> Error on OpenJDK 8 when I try to load data from a CSV file
> ==================================================
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x0000000104cbd7bb, pid=17765,
> tid=0x000000000000bd03
> #
> # JRE version: OpenJDK Runtime Environment (8.0_282) (build
> 1.8.0_282-bre_2021_01_20_16_06-b00)
> # Java VM: OpenJDK 64-Bit Server VM (25.282-b00 mixed mode bsd-amd64
> compressed oops)
> # Problematic frame:
> # V  [libjvm.dylib+0x5667bb]
>
> Error on OpenJDK 11 when I try to load data from a CSV file
> ==================================================
> 22/04/17 23:57:06 ERROR CarbonFactDataHandlerColumnar: Error in producer
> java.lang.reflect.InaccessibleObjectException: Unable to make public void
> jdk.internal.ref.Cleaner.clean() accessible: module java.base does not
> "exports jdk.internal.ref" to unnamed module @5115e1e6
> at
>
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:340)
> at
>
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
> at
> java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:198)
> at java.base/java.lang.reflect.Method.setAccessible(Method.java:192)
> at
>
> org.apache.carbondata.core.memory.UnsafeMemoryManager.destroyDirectByteBuffer(UnsafeMemoryManager.java:232)
> at
> org.apache.carbondata.core.datastore.page
> .LVByteBufferColumnPage.ensureMemory(LVByteBufferColumnPage.java:125)
> at
> org.apache.carbondata.core.datastore.page
> .LVByteBufferColumnPage.putBytes(LVByteBufferColumnPage.java:97)
> at
> org.apache.carbondata.core.datastore.page
> .LocalDictColumnPage.putBytes(LocalDictColumnPage.java:139)
> at
>
> org.apache.carbondata.core.datastore.page.ColumnPage.putData(ColumnPage.java:413)
> at
>
> org.apache.carbondata.processing.store.TablePage.convertToColumnarAndAddToPages(TablePage.java:241)
> at
> org.apache.carbondata.processing.store.TablePage.addRow(TablePage.java:201)
> at
>
> org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.processDataRows(CarbonFactDataHandlerColumnar.java:397)
> at
>
> org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.access$500(CarbonFactDataHandlerColumnar.java:60)
> at
>
> org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call(CarbonFactDataHandlerColumnar.java:637)
> at
>
> org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call(CarbonFactDataHandlerColumnar.java:614)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
>
> Error on OpenJDK 11 when I try to insert a single record
> ==================================================
> 22/04/18 09:46:10 ERROR CarbonFactDataHandlerColumnar: Error in producer
> java.lang.reflect.InaccessibleObjectException: Unable to make public void
> jdk.internal.ref.Cleaner.clean() accessible: module java.base does not
> "exports jdk.internal.ref" to unnamed module @cc4787f
> at
>
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:340)
> at
>
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
> at
> java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:198)
> at java.base/java.lang.reflect.Method.setAccessible(Method.java:192)
> at
>
> org.apache.carbondata.core.memory.UnsafeMemoryManager.destroyDirectByteBuffer(UnsafeMemoryManager.java:232)
> at
> org.apache.carbondata.core.datastore.page
> .LVByteBufferColumnPage.ensureMemory(LVByteBufferColumnPage.java:125)
> at
> org.apache.carbondata.core.datastore.page
> .LVByteBufferColumnPage.putBytes(LVByteBufferColumnPage.java:97)
> at
> org.apache.carbondata.core.datastore.page
> .LocalDictColumnPage.putBytes(LocalDictColumnPage.java:139)
> at
>
> org.apache.carbondata.core.datastore.page.ColumnPage.putData(ColumnPage.java:413)
> at
>
> org.apache.carbondata.processing.store.TablePage.convertToColumnarAndAddToPages(TablePage.java:241)
> at
> org.apache.carbondata.processing.store.TablePage.addRow(TablePage.java:201)
> at
>
> org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.processDataRows(CarbonFactDataHandlerColumnar.java:397)
> at
>
> org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.access$500(CarbonFactDataHandlerColumnar.java:60)
> at
>
> org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call(CarbonFactDataHandlerColumnar.java:637)
> at
>
> org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar$Producer.call(CarbonFactDataHandlerColumnar.java:614)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> 22/04/18 09:46:10 ERROR CarbonFactDataHandlerColumnar: Problem while
> writing the carbon data file
>
> Thanks,
>
> --Sandeep
>