[jira] [Assigned] (HUDI-3195) optimize spark3 pom and modify build command

["Yann Byron (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-3195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yann Byron reassigned HUDI-3195:
--------------------------------

    Assignee: Yann Byron

> optimize spark3 pom and modify build command
> --------------------------------------------
>
>                 Key: HUDI-3195
>                 URL: https://issues.apache.org/jira/browse/HUDI-3195
>             Project: Apache Hudi
>          Issue Type: Bug
>          Components: Spark Integration
>            Reporter: Yann Byron
>            Assignee: Yann Byron
>            Priority: Major
>
> when use `mvn clean install -P "scala-2.12,spark3,spark3.1.x"` to build spark and write data by spark, error is as follows: 
>  
>  
> {code:java}
> ERROR Executor: Exception in task 0.0 in stage 26.0 (TID 2005)
> java.lang.RuntimeException: org.apache.hudi.exception.HoodieException: org.apache.hudi.exception.HoodieException: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: org/apache/parquet/schema/LogicalTypeAnnotation$LogicalTypeAnnotationVisitor
>     at org.apache.hudi.client.utils.LazyIterableIterator.next(LazyIterableIterator.java:121)
>     at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:44)
>     at scala.collection.Iterator$$anon$11.nextCur(Iterator.scala:484)
>     at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:490)
>     at org.apache.spark.storage.memory.MemoryStore.putIterator(MemoryStore.scala:221) {code}
>  
>  
> That's due to the use of 1.12.1 parquet.
>  
> Also, we use this build command in CI, which will mislead users.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)