You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kyuubi.apache.org by GitBox <gi...@apache.org> on 2021/07/19 12:44:21 UTC
[GitHub] [incubator-kyuubi] changhuyan opened a new issue #836: kyuubi读写orc和parquet格式的分区表报错
changhuyan opened a new issue #836:
URL: https://github.com/apache/incubator-kyuubi/issues/836
### 1. Describe the bug
当使用kyuubi-1.2.0-rc5分支读写6.3.2版本的CDH分区表格时报错；报错描述如下
ERROR ExceptionStackTrace:
com.hundsun.rdc.bdata.xsql.exception.XsqlException: Error in building connection and execption is
at com.hundsun.rdc.bdata.xsql.excute.XsqlExcute.tryExecuteSqlDirectly(XsqlExcute.java:246)
at com.hundsun.rdc.bdata.xsql.excute.XsqlExcute.submit(XsqlExcute.java:163)
at com.hundsun.rdc.bdata.compute.service.impl.XsqlServiceImpl$TaskThread.run(XsqlServiceImpl.java:570)
Caused by: java.lang.RuntimeException: sql execute exception is
at com.hundsun.rdc.bdata.xsql.excute.XsqlExcute.executeJdbcQuery(XsqlExcute.java:376)
at com.hundsun.rdc.bdata.xsql.excute.XsqlExcute.tryExecuteSqlDirectly(XsqlExcute.java:239)
... 2 more
Caused by: java.sql.SQLException: Error operating EXECUTE_STATEMENT: java.lang.NoClassDefFoundError: parquet/hadoop/ParquetOutputFormat org.apache.spark.sql.execution.FileSourceScanExec.dynamicallySelectedPartitions$lzycompute(DataSourceScanExec.scala:241)
org.apache.kyuubi.engine.spark.operation.ExecuteStatement.org$apache$kyuubi$engine$spark$operation$ExecuteStatement$$executeStatement(ExecuteStatement.scala:72)
Caused by: java.lang.ClassNotFoundException: parquet.hadoop.ParquetOutputFormat
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] changhuyan closed issue #836: kyuubi读写orc和parquet格式的分区表报错
Posted by GitBox <gi...@apache.org>.
changhuyan closed issue #836:
URL: https://github.com/apache/incubator-kyuubi/issues/836
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] changhuyan commented on issue #836: kyuubi读写orc和parquet格式的分区表报错
Posted by GitBox <gi...@apache.org>.
changhuyan commented on issue #836:
URL: https://github.com/apache/incubator-kyuubi/issues/836#issuecomment-882519192
经过排查发现，spark3.x的版本将parquet-hadoop-bundle-1.6.0.jar的功能合并到parquet-hadoop-1.10.1.jar中了，并且把类名其那面加了apache标识，导致该类型找不到，需要将parquet-hadoop-bundle-1.6.0.jar拷贝到spark的jar目录即可
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] changhuyan closed issue #836: kyuubi读写orc和parquet格式的分区表报错
Posted by GitBox <gi...@apache.org>.
changhuyan closed issue #836:
URL: https://github.com/apache/incubator-kyuubi/issues/836
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] changhuyan commented on issue #836: kyuubi读写orc和parquet格式的分区表报错
Posted by GitBox <gi...@apache.org>.
changhuyan commented on issue #836:
URL: https://github.com/apache/incubator-kyuubi/issues/836#issuecomment-882519192
经过排查发现，spark3.x的版本将parquet-hadoop-bundle-1.6.0.jar的功能合并到parquet-hadoop-1.10.1.jar中了，并且把类名其那面加了apache标识，导致该类型找不到，需要将parquet-hadoop-bundle-1.6.0.jar拷贝到spark的jar目录即可
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] changhuyan commented on issue #836: kyuubi读写orc和parquet格式的分区表报错
Posted by GitBox <gi...@apache.org>.
changhuyan commented on issue #836:
URL: https://github.com/apache/incubator-kyuubi/issues/836#issuecomment-882519192
经过排查发现，spark3.x的版本将parquet-hadoop-bundle-1.6.0.jar的功能合并到parquet-hadoop-1.10.1.jar中了，并且把类名其那面加了apache标识，导致该类型找不到，需要将parquet-hadoop-bundle-1.6.0.jar拷贝到spark的jar目录即可
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] changhuyan closed issue #836: kyuubi读写orc和parquet格式的分区表报错
Posted by GitBox <gi...@apache.org>.
changhuyan closed issue #836:
URL: https://github.com/apache/incubator-kyuubi/issues/836
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org