You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/10/22 01:59:47 UTC
[GitHub] [iceberg] cvasani opened a new issue #3346: Unable to create table in local environment
cvasani opened a new issue #3346:
URL: https://github.com/apache/iceberg/issues/3346
Hi Team,
I have tried to setup a iceberg on my local instance. Spark is local to my machine and tried executing given commands. Now command is throwing errors. Could you please advise ?
Commands :
```shell
spark-sql --packages org.apache.iceberg:iceberg-spark3-runtime:0.12.0\
--conf spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions \
--conf spark.sql.catalog.spark_catalog=org.apache.iceberg.spark.SparkSessionCatalog \
--conf spark.sql.catalog.spark_catalog.type=hive \
--conf spark.sql.catalog.local=org.apache.iceberg.spark.SparkCatalog \
--conf spark.sql.catalog.local.type=hadoop \
--conf spark.sql.catalog.local.warehouse=$PWD/warehouse
CREATE TABLE local.db.table (id bigint, data string) USING iceberg;
```
Error :
```shell
spark-sql> CREATE TABLE local.db.table (id bigint, data string) USING iceberg;
21/10/22 01:59:22 ERROR SparkSQLDriver: Failed in [CREATE TABLE local.db.table (id bigint, data string) USING iceberg]
java.lang.IncompatibleClassChangeError: class org.apache.spark.sql.catalyst.plans.logical.DynamicFileFilterWithCardinalityCheck has interface org.apache.spark.sql.catalyst.plans.logical.BinaryNode as super class
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:698)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions.$anonfun$apply$8(IcebergSparkSessionExtensions.scala:50)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] cvasani commented on issue #3346: Unable to create table in local environment
Posted by GitBox <gi...@apache.org>.
cvasani commented on issue #3346:
URL: https://github.com/apache/iceberg/issues/3346#issuecomment-949245250
> What version of spark are you running. We test with spark 3.0, 3.1, and 2.4.7 (different runtime jar)
>
> 3.2 is in progress
I am using spark 3.2. Thank you . let me test with 3.1. Could you please help me find documentation link for certification matrix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] cvasani commented on issue #3346: Unable to create table in local environment
Posted by GitBox <gi...@apache.org>.
cvasani commented on issue #3346:
URL: https://github.com/apache/iceberg/issues/3346#issuecomment-949248636
Thanks to RussellSpitzer . Tested with spark 3.0 and its working fine. Closing an issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] cvasani closed issue #3346: Unable to create table in local environment
Posted by GitBox <gi...@apache.org>.
cvasani closed issue #3346:
URL: https://github.com/apache/iceberg/issues/3346
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on issue #3346: Unable to create table in local environment
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on issue #3346:
URL: https://github.com/apache/iceberg/issues/3346#issuecomment-949243671
What version of spark are you running. We test with spark 3.0, 3.1, and 2.4.7 (different runtime jar)
3.2 is in progress
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on issue #3346: Unable to create table in local environment
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on issue #3346:
URL: https://github.com/apache/iceberg/issues/3346#issuecomment-949730228
We don't have a version matrix :( yet, but PR's are welcome!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org