You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Guoliang Sun (Jira)" <ji...@apache.org> on 2025/02/21 03:19:00 UTC
[jira] [Closed] (KYLIN-5988) [Storage V3] Query hitting the model throws an error in the V3 project, and the built data is cleared
[ https://issues.apache.org/jira/browse/KYLIN-5988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Guoliang Sun closed KYLIN-5988.
-------------------------------
Resolution: Duplicate
> [Storage V3] Query hitting the model throws an error in the V3 project, and the built data is cleared
> -----------------------------------------------------------------------------------------------------
>
> Key: KYLIN-5988
> URL: https://issues.apache.org/jira/browse/KYLIN-5988
> Project: Kylin
> Issue Type: Bug
> Affects Versions: 5.0.0
> Reporter: Guoliang Sun
> Priority: Major
>
> 1. Create and build an incremental model.
> 2. Query hitting the model throws an error.
> 3. Error message is as follows:
> {code:java}
> Error while executing SQL "SELECT"LINEORDER"."LO_ORDERKEY" as "LINEORDER_LO_ORDERKEY", "LINEORDER"."LO_PARTKEY" as "LINEORDER_LO_PARTKEY", "LINEORDER"."LO_DISCOUNT" as "LINEORDER_LO_DISCOUNT", "LINEORDER"."LO_SUPPLYCOST" as "LINEORDER_LO_SUPPLYCOST", "LINEORDER"."LO_COMMITDATE" as "LINEORDER_LO_COMMITDATE", "LINEORDER"."LO_EXTENDEDPRICE" as "LINEORDER_LO_EXTENDEDPRICE", "LINEORDER"."LO_TAX" as "LINEORDER_LO_TAX", "LINEORDER"."LO_SUPPKEY" as "LINEORDER_LO_SUPPKEY", "LINEORDER"."LO_ORDTOTALPRICE" as "LINEORDER_LO_ORDTOTALPRICE", "LINEORDER"."LO_REVENUE" as "LINEORDER_LO_REVENUE", "LINEORDER"."LO_ORDERDATE" as "LINEORDER_LO_ORDERDATE", "LINEORDER"."LO_ORDERPRIOTITY" as "LINEORDER_LO_ORDERPRIOTITY", "LINEORDER"."LO_SHIPPRIOTITY" as "LINEORDER_LO_SHIPPRIOTITY", "LINEORDER"."LO_QUANTITY" as "LINEORDER_LO_QUANTITY", "LINEORDER"."LO_SHIPMODE" as "LINEORDER_LO_SHIPMODE", "LINEORDER"."LO_LINENUMBER" as "LINEORDER_LO_LINENUMBER", "LINEORDER"."LO_CUSTKEY" as "LINEORDER_LO_CUSTKEY", "CUSTOMER"."C_ADDRESS" as "CUSTOMER_C_ADDRESS", "CUSTOMER"."C_NATION" as "CUSTOMER_C_NATION", "CUSTOMER"."C_CITY" as "CUSTOMER_C_CITY", "CUSTOMER"."C_PHONE" as "CUSTOMER_C_PHONE", "CUSTOMER"."C_REGION" as "CUSTOMER_C_REGION", "CUSTOMER"."C_NAME" as "CUSTOMER_C_NAME", "CUSTOMER"."C_MKTSEGMENT" as "CUSTOMER_C_MKTSEGMENT", "CUSTOMER"."C_CUSTKEY" as "CUSTOMER_C_CUSTKEY"FROM "SSB"."LINEORDER" as "LINEORDER"INNER JOIN "SSB"."CUSTOMER" as "CUSTOMER"ON "LINEORDER"."LO_CUSTKEY" = "CUSTOMER"."C_CUSTKEY"LIMIT 500": Job aborted due to stage failure: Task 0 in stage 10106.0 failed 1 times, most recent failure: Lost task 0.0 in stage 10106.0 (TID 63159) (oliver-slave03.kylin.com executor 1): java.lang.NoClassDefFoundError: Could not initialize class org.xerial.snappy.Snappy at org.apache.parquet.hadoop.codec.SnappyDecompressor.decompress(SnappyDecompressor.java:62) at org.apache.parquet.hadoop.codec.NonBlockedDecompressorStream.read(NonBlockedDecompressorStream.java:51) at java.io.DataInputStream.readFully(DataInputStream.java:195) at java.io.DataInputStream.readFully(DataInputStream.java:169) at org.apache.parquet.bytes.BytesInput$StreamBytesInput.toByteArray(BytesInput.java:286) at org.apache.parquet.bytes.BytesInput.toByteBuffer(BytesInput.java:237) at org.apache.parquet.bytes.BytesInput.toInputStream(BytesInput.java:246) at org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainLongDictionary.<init>(PlainValuesDictionary.java:154) at org.apache.parquet.column.Encoding$1.initDictionary(Encoding.java:96) at org.apache.parquet.column.Encoding$5.initDictionary(Encoding.java:163) at org.apache.spark.sql.execution.datasources.parquet.VectorizedColumnReader.<init>(VectorizedColumnReader.java:123) at org.apache.spark.sql.execution.datasources.parquet.VectorizedParquetRecordReader.initColumnReader(VectorizedParquetRecordReader.java:426) at org.apache.spark.sql.execution.datasources.parquet.VectorizedParquetRecordReader.checkEndOfRowGroup(VectorizedParquetRecordReader.java:408) at org.apache.spark.sql.execution.datasources.parquet.VectorizedParquetRecordReader.nextBatch(VectorizedParquetRecordReader.java:323) at org.apache.spark.sql.execution.datasources.parquet.VectorizedParquetRecordReader.nextKeyValue(VectorizedParquetRecordReader.java:226) at org.apache.spark.sql.execution.datasources.RecordReaderIterator.hasNext(RecordReaderIterator.scala:42) at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.hasNext(FileScanRDD.scala:118) at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.nextIterator(FileScanRDD.scala:293) at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.hasNext(FileScanRDD.scala:118) at org.apache.spark.sql.execution.KylinStorageScanExec$$anon$1.hasNext(KylinStorageScanExec.scala:405) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.columnartorow_nextBatch_0$(Unknown Source) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source) at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:771) at org.apache.spark.sql.execution.RowToCHNativeColumnarExec.$anonfun$doExecuteColumnarInternal$3(RowToCHNativeColumnarExec.scala:54) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:856) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:856) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:366) at org.apache.spark.rdd.RDD.iterator(RDD.scala:330) at org.apache.gluten.execution.ColumnarInputRDDsWrapper.$anonfun$getIterators$1(WholeStageTransformer.scala:445) at scala.collection.immutable.List.flatMap(List.scala:366) at org.apache.gluten.execution.ColumnarInputRDDsWrapper.getIterators(WholeStageTransformer.scala:436) at org.apache.gluten.execution.WholeStageZippedPartitionsRDD.$anonfun$compute$1(WholeStageZippedPartitionsRDD.scala:48) at org.apache.gluten.utils.Arm$.withResource(Arm.scala:25) at org.apache.gluten.metrics.GlutenTimeMetric$.millis(GlutenTimeMetric.scala:37) at org.apache.gluten.execution.WholeStageZippedPartitionsRDD.compute(WholeStageZippedPartitionsRDD.scala:46) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:366) at org.apache.spark.rdd.RDD.iterator(RDD.scala:330) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:366) at org.apache.spark.rdd.RDD.iterator(RDD.scala:330) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:549) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1511) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:552) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
> Driver stacktrace:{code}
> SQL
> {code:java}
> SELECT
> "LINEORDER"."LO_ORDERKEY" as "LINEORDER_LO_ORDERKEY"
> , "LINEORDER"."LO_PARTKEY" as "LINEORDER_LO_PARTKEY"
> , "LINEORDER"."LO_DISCOUNT" as "LINEORDER_LO_DISCOUNT"
> , "LINEORDER"."LO_SUPPLYCOST" as "LINEORDER_LO_SUPPLYCOST"
> , "LINEORDER"."LO_COMMITDATE" as "LINEORDER_LO_COMMITDATE"
> , "LINEORDER"."LO_EXTENDEDPRICE" as "LINEORDER_LO_EXTENDEDPRICE"
> , "LINEORDER"."LO_TAX" as "LINEORDER_LO_TAX"
> , "LINEORDER"."LO_SUPPKEY" as "LINEORDER_LO_SUPPKEY"
> , "LINEORDER"."LO_ORDTOTALPRICE" as "LINEORDER_LO_ORDTOTALPRICE"
> , "LINEORDER"."LO_REVENUE" as "LINEORDER_LO_REVENUE"
> , "LINEORDER"."LO_ORDERDATE" as "LINEORDER_LO_ORDERDATE"
> , "LINEORDER"."LO_ORDERPRIOTITY" as "LINEORDER_LO_ORDERPRIOTITY"
> , "LINEORDER"."LO_SHIPPRIOTITY" as "LINEORDER_LO_SHIPPRIOTITY"
> , "LINEORDER"."LO_QUANTITY" as "LINEORDER_LO_QUANTITY"
> , "LINEORDER"."LO_SHIPMODE" as "LINEORDER_LO_SHIPMODE"
> , "LINEORDER"."LO_LINENUMBER" as "LINEORDER_LO_LINENUMBER"
> , "LINEORDER"."LO_CUSTKEY" as "LINEORDER_LO_CUSTKEY"
> , "CUSTOMER"."C_ADDRESS" as "CUSTOMER_C_ADDRESS"
> , "CUSTOMER"."C_NATION" as "CUSTOMER_C_NATION"
> , "CUSTOMER"."C_CITY" as "CUSTOMER_C_CITY"
> , "CUSTOMER"."C_PHONE" as "CUSTOMER_C_PHONE"
> , "CUSTOMER"."C_REGION" as "CUSTOMER_C_REGION"
> , "CUSTOMER"."C_NAME" as "CUSTOMER_C_NAME"
> , "CUSTOMER"."C_MKTSEGMENT" as "CUSTOMER_C_MKTSEGMENT"
> , "CUSTOMER"."C_CUSTKEY" as "CUSTOMER_C_CUSTKEY"
> FROM "SSB"."LINEORDER" as "LINEORDER"
> INNER JOIN "SSB"."CUSTOMER" as "CUSTOMER"
> ON "LINEORDER"."LO_CUSTKEY" = "CUSTOMER"."C_CUSTKEY" {code}
> 4. Meanwhile, the built data is also cleared.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)