You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "Liang Chen (JIRA)" <ji...@apache.org> on 2017/05/07 00:27:04 UTC
[jira] [Comment Edited] (CARBONDATA-1022) Get errors while do
"select" query in Spark-shell
[ https://issues.apache.org/jira/browse/CARBONDATA-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15999633#comment-15999633 ]
Liang Chen edited comment on CARBONDATA-1022 at 5/7/17 12:26 AM:
-----------------------------------------------------------------
Version : download "spark-1.6.1-bin-hadoop2.6" + CarbonData master jar(carbondata_2.10-1.1.0-SNAPSHOT-shade-hadoop2.2.0.jar)
Reproduced script as below:
$ ##set SPARK_HOME = spark1.6...##
$ carbondata_jar=./lib/$(ls -1 lib |grep "^carbondata_.*\.jar$")
$ ./bin/spark-shell --master local --jars ${carbondata_jar}
$ import org.apache.spark.sql.CarbonContext
$ val cc = new CarbonContext(sc, "./carbondata/store", "./carbondata/metastorelocation")
$ cc.sql("CREATE TABLE connectdemo1 (name String, gender String, province String, singler String, age Int) STORED BY 'org.apache.carbondata.format'")
$ cc.sql("LOAD DATA LOCAL INPATH '/Users/apple/DEMO/3Mtestdata.csv' INTO TABLE connectdemo1")
$ scala> cc.sql("select * from connectdemo1").show
17/05/07 08:20:03 ERROR SegmentUpdateStatusManager: pool-157-thread-1 Invalid tuple id op2.6/carbondata/store/default/connectdemo1/Fact/0/0/0-0_batchno0-0-1494116377555/0
+------+------+--------+-------+---+
| name|gender|province|singler|age|
+------+------+--------+-------+---+
|Abbott|Female| AB| false| 39|
|Abbott|Female| AB| false| 62|
|Abbott|Female| AB| false| 62|
|Abbott|Female| AB| false| 45|
|Abbott|Female| AB| false| 40|
|Abbott|Female| AB| false| 32|
|Abbott|Female| AB| false| 29|
|Abbott|Female| AB| false| 38|
was (Author: chenliang613):
Version : download "spark-1.6.1-bin-hadoop2.6" + CarbonData master jar(carbondata_2.10-1.1.0-SNAPSHOT-shade-hadoop2.2.0.jar)
Reproduced script as below:
$ import org.apache.spark.sql.CarbonContext
$ val cc = new CarbonContext(sc, "./carbondata/store", "./carbondata/metastorelocation")
$ cc.sql("CREATE TABLE connectdemo1 (name String, gender String, province String, singler String, age Int) STORED BY 'org.apache.carbondata.format'")
$ cc.sql("LOAD DATA LOCAL INPATH '/Users/apple/DEMO/3Mtestdata.csv' INTO TABLE connectdemo1")
$ scala> cc.sql("select * from connectdemo1").show
17/05/07 08:20:03 ERROR SegmentUpdateStatusManager: pool-157-thread-1 Invalid tuple id op2.6/carbondata/store/default/connectdemo1/Fact/0/0/0-0_batchno0-0-1494116377555/0
+------+------+--------+-------+---+
| name|gender|province|singler|age|
+------+------+--------+-------+---+
|Abbott|Female| AB| false| 39|
|Abbott|Female| AB| false| 62|
|Abbott|Female| AB| false| 62|
|Abbott|Female| AB| false| 45|
|Abbott|Female| AB| false| 40|
|Abbott|Female| AB| false| 32|
|Abbott|Female| AB| false| 29|
|Abbott|Female| AB| false| 38|
> Get errors while do "select" query in Spark-shell
> -------------------------------------------------
>
> Key: CARBONDATA-1022
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1022
> Project: CarbonData
> Issue Type: Bug
> Components: sql
> Reporter: Liang Chen
>
> Spark1.6's spark-shell, error info as below :
> scala> cc.sql("select * from connectdemo1").show
> 17/05/04 23:06:36 ERROR SegmentUpdateStatusManager: pool-178-thread-1 Invalid tuple id /carbondata/store/default/connectdemo1/Fact/0/0/0-0_batchno0-0-1493910126468/0
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)