You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Ethan Guo (Jira)" <ji...@apache.org> on 2022/09/24 00:18:00 UTC
[jira] [Closed] (HUDI-4892) Fix hudi-spark3-bundle
[ https://issues.apache.org/jira/browse/HUDI-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ethan Guo closed HUDI-4892.
---------------------------
Resolution: Fixed
> Fix hudi-spark3-bundle
> ----------------------
>
> Key: HUDI-4892
> URL: https://issues.apache.org/jira/browse/HUDI-4892
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Ethan Guo
> Assignee: Ethan Guo
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.12.1
>
>
> Using hudi-spark3-bundle with Spark 3.3 shell, the following exception is thrown. Some classes are not packaged into the bundle.
> {code:java}
> scala> val df = spark.read.format("hudi").load("<table_path>")
> java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider org.apache.hudi.Spark32PlusDefaultSource not found
> at java.util.ServiceLoader.fail(ServiceLoader.java:239)
> at java.util.ServiceLoader.access$300(ServiceLoader.java:185)
> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:372)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:46)
> at scala.collection.Iterator.foreach(Iterator.scala:943)
> at scala.collection.Iterator.foreach$(Iterator.scala:943)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
> at scala.collection.IterableLike.foreach(IterableLike.scala:74)
> at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
> at scala.collection.TraversableLike.filterImpl(TraversableLike.scala:303)
> at scala.collection.TraversableLike.filterImpl$(TraversableLike.scala:297)
> at scala.collection.AbstractTraversable.filterImpl(Traversable.scala:108)
> at scala.collection.TraversableLike.filter(TraversableLike.scala:395)
> at scala.collection.TraversableLike.filter$(TraversableLike.scala:395)
> at scala.collection.AbstractTraversable.filter(Traversable.scala:108)
> at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:657)
> at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSourceV2(DataSource.scala:725)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:207)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:185)
> ... 47 elided {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)