You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2018/11/17 07:16:00 UTC
[jira] [Resolved] (SPARK-26093) Read Avro: ClassNotFoundException:
org.apache.spark.sql.avro.AvroFileFormat.DefaultSource
[ https://issues.apache.org/jira/browse/SPARK-26093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-26093.
----------------------------------
Resolution: Not A Problem
> Read Avro: ClassNotFoundException: org.apache.spark.sql.avro.AvroFileFormat.DefaultSource
> -----------------------------------------------------------------------------------------
>
> Key: SPARK-26093
> URL: https://issues.apache.org/jira/browse/SPARK-26093
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.4.0
> Environment: Spark 2.4.0
> Scala 2.11.12
> Java 1.8.0_181
> Reporter: Dagang Wei
> Priority: Major
>
> I downloaded and unpacked spark-2.4.0-bin-hadoop2.7.tgz to my Linux, then I followed [Read Avro files|https://docs.databricks.com/spark/latest/data-sources/read-avro.html] to read a local Avro file in spark-shell:
> $ bin/spark-shell --packages com.databricks:spark-avro_2.11:4.0.0
> ...
> version 2.4.0
> Using Scala version 2.11.12 (OpenJDK 64-Bit Server VM, Java 1.8.0_181)
> ...
> scala>
> import com.databricks.spark.avro._
> scala>
> val df = spark.read.avro("file:///.../foo.avro")
> java.lang.ClassNotFoundException: Failed to find data source: org.apache.spark.sql.avro.AvroFileFormat. Please find packages at http://spark.apache.org/third-party-projects.html
> at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:657)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:194)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:178)
> at com.databricks.spark.avro.package$AvroDataFrameReader$$anonfun$avro$2.apply(package.scala:34)
> at com.databricks.spark.avro.package$AvroDataFrameReader$$anonfun$avro$2.apply(package.scala:34)
> ... 51 elided
> Caused by: java.lang.ClassNotFoundException: org.apache.spark.sql.avro.AvroFileFormat.DefaultSource
> at scala.reflect.internal.util.AbstractFileClassLoader.findClass(AbstractFileClassLoader.scala:62)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20$$anonfun$apply$12.apply(DataSource.scala:634)
> at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20$$anonfun$apply$12.apply(DataSource.scala:634)
> at scala.util.Try$.apply(Try.scala:192)
> at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20.apply(DataSource.scala:634)
> at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20.apply(DataSource.scala:634)
> at scala.util.Try.orElse(Try.scala:84)
> at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:634)
> ... 55 more
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org