[jira] [Created] (SPARK-26093) Read Avro: ClassNotFoundException: org.apache.spark.sql.avro.AvroFileFormat.DefaultSource

["Dagang Wei (JIRA)" <ji...@apache.org>]

Dagang Wei created SPARK-26093:
----------------------------------

             Summary: Read Avro: ClassNotFoundException: org.apache.spark.sql.avro.AvroFileFormat.DefaultSource
                 Key: SPARK-26093
                 URL: https://issues.apache.org/jira/browse/SPARK-26093
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 2.4.0
         Environment: Spark 2.4.0

Scala 2.11.12

Java 1.8.0_181
            Reporter: Dagang Wei


I downloaded and unpacked spark-2.4.0-bin-hadoop2.7.tgz to my Linux, then I followed [Read Avro files|https://docs.databricks.com/spark/latest/data-sources/read-avro.html] to read a local Avro file in spark-shell:

$ bin/spark-shell --packages com.databricks:spark-avro_2.11:4.0.0

...

version 2.4.0
Using Scala version 2.11.12 (OpenJDK 64-Bit Server VM, Java 1.8.0_181)

...

scala> 

import com.databricks.spark.avro._

scala> 

val df = spark.read.avro("file:///.../foo.avro")

java.lang.ClassNotFoundException: Failed to find data source: org.apache.spark.sql.avro.AvroFileFormat. Please find packages at http://spark.apache.org/third-party-projects.html
 at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:657)
 at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:194)
 at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:178)
 at com.databricks.spark.avro.package$AvroDataFrameReader$$anonfun$avro$2.apply(package.scala:34)
 at com.databricks.spark.avro.package$AvroDataFrameReader$$anonfun$avro$2.apply(package.scala:34)
 ... 51 elided
Caused by: java.lang.ClassNotFoundException: org.apache.spark.sql.avro.AvroFileFormat.DefaultSource
 at scala.reflect.internal.util.AbstractFileClassLoader.findClass(AbstractFileClassLoader.scala:62)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20$$anonfun$apply$12.apply(DataSource.scala:634)
 at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20$$anonfun$apply$12.apply(DataSource.scala:634)
 at scala.util.Try$.apply(Try.scala:192)
 at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20.apply(DataSource.scala:634)
 at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20.apply(DataSource.scala:634)
 at scala.util.Try.orElse(Try.scala:84)
 at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:634)
 ... 55 more

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org