You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by 喜之郎 <25...@qq.com> on 2018/05/18 03:46:59 UTC
after load data using SaveMode.Overwrite, query through beeline return all null field
hi dev.
carbon version :1.3.1
spark version:2.2.1
1) First I create a carbon table through beeline.
2) Then I use spark-submit and dataframe load data to carbon. Query is OK。
3) Then I use spark-submit and dataframe load data to carbon again, but query through thriftserver return all NULL field.
4) I restart thriftserver, and query through thriftserver return correct result.
This problem happens everytime. I think this problem may is about CarbonThriftServer.
is there anybody encountered this problem? can anybody give some advice,thanks.
you can use this code to reproduce problem.
===================
val carbonTableName = args(1)
val warehouse = new File("./warehouse").getCanonicalPath
val metastore = new File("./metastore").getCanonicalPath
val spark = SparkSession
.builder()
.appName("StreamExample")
.config("spark.sql.warehouse.dir", warehouse)
.getOrCreateCarbonSession(warehouse, metastore)
val df2 = spark.read.parquet(path)
df2.write
.format("carbondata")
.option("tableName", carbonTableName)
.option("compress", "true")
.option("tempCSV", "false")
.mode(SaveMode.Overwrite)
.save()=============
Re: after load data using SaveMode.Overwrite, query through beeline
return all null field
Posted by akashrn5 <ak...@gmail.com>.
Hi,
I have checked with the current version and the issue is not reproducing,
when I checked the code, there are code changes happened for the savemode
from 1.3 to 1.4 version.
You can check the PR #2186 for the changes done for that part and you can
check your issue again with that PR .
Regards,
Akash
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: after load data using SaveMode.Overwrite, query through beeline
return all null field
Posted by Liang Chen <ch...@gmail.com>.
Hi
Thank you reported this issue.
Let us check it and response to you asap.
Regards
Liang
喜之郎 wrote
> hi dev.
> carbon version :1.3.1
> spark version:2.2.1
> 1) First I create a carbon table through beeline.
> 2) Then I use spark-submit and dataframe load data to carbon. Query is OK。
> 3) Then I use spark-submit and dataframe load data to carbon again, but
> query through thriftserver return all NULL field.
> 4) I restart thriftserver, and query through thriftserver return correct
> result.
>
>
> This problem happens everytime. I think this problem may is about
> CarbonThriftServer.
> is there anybody encountered this problem? can anybody give some
> advice,thanks.
>
>
> you can use this code to reproduce problem.
> ===================
>
>
>
> val carbonTableName = args(1)
>
> val warehouse = new File("./warehouse").getCanonicalPath
> val metastore = new File("./metastore").getCanonicalPath
> val spark = SparkSession
> .builder()
> .appName("StreamExample")
> .config("spark.sql.warehouse.dir", warehouse)
> .getOrCreateCarbonSession(warehouse, metastore)
> val df2 = spark.read.parquet(path)
> df2.write
> .format("carbondata")
> .option("tableName", carbonTableName)
> .option("compress", "true")
> .option("tempCSV", "false")
> .mode(SaveMode.Overwrite)
> .save()=============
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/