You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@carbondata.apache.org by "f.l@neusoft.com" <f....@neusoft.com> on 2021/03/17 01:38:54 UTC

Failed to create table using the Thrift Server

Hi community:
In our project, we used spark-cli to operate carbondata,it is ok.
But we failed to create table using the Thrift Server when we are using the existing JDBC process provided by spark.
So I do need your help, please!

hadoop:CHD3.0.0-cdh6.3.2.
hive:2.1.1-cdh6.3.2
Spark : 2.4.7

carbondata :
    mvn -DskipTests -Pspark-2.4 -Pwindows clean package
    - assembly\target\scala-2.11\apache-carbondata-2.1.0-bin-spark2.4.5-hadoop2.7.2.jar
    - integration\hive\target\carbondata-hive-2.1.0.jar

We use carbondata as mentioned here <https://github.com/apache/carbondata/blob/master/docs/quick-start-guide.md#option-1-starting-thrift-server-with-carbonextensionssince-20>(Option1).

== SQL script==
create table cb_test_env6(id int, env_name string) STORED AS carbondata
-------------------------------------------------------------^^^
error:
21/03/16 16:24:01 INFO thriftserver.SparkExecuteStatementOperation: Running query 'create table cb_test_env6(id int, env_name string) STORED AS carbondata' with 0a5535f1-2fdb-4577-a4cc-7dfd1f09fb6c
21/03/16 16:24:01 ERROR thriftserver.SparkExecuteStatementOperation: Error executing query, currentState RUNNING,
org.apache.spark.sql.catalyst.parser.ParseException:
Operation not allowed: STORED AS with file format 'carbondata'(line 1, pos 61)
        at org.apache.spark.sql.catalyst.parser.ParserUtils$.operationNotAllowed(ParserUtils.scala:41)
        at org.apache.spark.sql.execution.SparkSqlAstBuilder$$anonfun$visitGenericFileFormat$1.apply(SparkSqlParser.scala:1280)
        at org.apache.spark.sql.execution.SparkSqlAstBuilder$$anonfun$visitGenericFileFormat$1.apply(SparkSqlParser.scala:1271)
        at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:108)
        at org.apache.spark.sql.execution.SparkSqlAstBuilder.visitGenericFileFormat(SparkSqlParser.scala:1271)
        at org.apache.spark.sql.execution.SparkSqlAstBuilder$$anonfun$visitCreateFileFormat$1.apply(SparkSqlParser.scala:1249)
        at org.apache.spark.sql.execution.SparkSqlAstBuilder$$anonfun$visitCreateFileFormat$1.apply(SparkSqlParser.scala:1243)
        at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:108)
        at org.apache.spark.sql.execution.SparkSqlAstBuilder.visitCreateFileFormat(SparkSqlParser.scala:1242)
        at org.apache.spark.sql.execution.SparkSqlAstBuilder$$anonfun$visitCreateHiveTable$1$$anonfun$34.apply(SparkSqlParser.scala:1146)
        at org.apache.spark.sql.execution.SparkSqlAstBuilder$$anonfun$visitCreateHiveTable$1$$anonfun$34.apply(SparkSqlParser.scala:1146)
        at scala.Option.map(Option.scala:146)
        at org.apache.spark.sql.execution.SparkSqlAstBuilder$$anonfun$visitCreateHiveTable$1.apply(SparkSqlParser.scala:1146)
        at org.apache.spark.sql.execution.SparkSqlAstBuilder$$anonfun$visitCreateHiveTable$1.apply(SparkSqlParser.scala:1113)
        at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:108)
        at org.apache.spark.sql.execution.SparkSqlAstBuilder.visitCreateHiveTable(SparkSqlParser.scala:1113)
        at org.apache.spark.sql.execution.SparkSqlAstBuilder.visitCreateHiveTable(SparkSqlParser.scala:55)
        at org.apache.spark.sql.catalyst.parser.SqlBaseParser$CreateHiveTableContext.accept(SqlBaseParser.java:1215)
        at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visit(AbstractParseTreeVisitor.java:18)
        at org.apache.spark.sql.catalyst.parser.AstBuilder$$anonfun$visitSingleStatement$1.apply(AstBuilder.scala:72)
        at org.apache.spark.sql.catalyst.parser.AstBuilder$$anonfun$visitSingleStatement$1.apply(AstBuilder.scala:72)
        at org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(ParserUtils.scala:108)
        at org.apache.spark.sql.catalyst.parser.AstBuilder.visitSingleStatement(AstBuilder.scala:71)
        at org.apache.spark.sql.catalyst.parser.AbstractSqlParser$$anonfun$parsePlan$1.apply(ParseDriver.scala:70)
        at org.apache.spark.sql.catalyst.parser.AbstractSqlParser$$anonfun$parsePlan$1.apply(ParseDriver.scala:69)
        at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:100)
        at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:48)
        at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:69)
        at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:643)
        at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:694)
        at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:232)
        at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:175)
        at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:171)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
        at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:185)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
________________________________

刘feng

www.neusoft.com

---------------------------------------------------------------------------------------------------
Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s)
is intended only for the use of the intended recipient and may be confidential and/or privileged of
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication
is not the intended recipient,unauthorized use,forwarding, printing, storing, disclosure or copying
is strictly prohibited, and may be unlawful.If you have received this communication in error,please
immediately notify the sender by return e-mail, and delete the original message and all copies from
your system. Thank you.
---------------------------------------------------------------------------------------------------

Re: Failed to create table using the Thrift Server

Posted by Karan-c980 <ka...@hotmail.com>.

Hi,

please make sure that carbon jars are present at target location.




--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/