[jira] [Commented] (HUDI-5714) Hudi CLI Bundle Spark 2 Issues

["Y Ethan Guo (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HUDI-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17928558#comment-17928558 ] 

Y Ethan Guo commented on HUDI-5714:
-----------------------------------

This issue is no longer valide any more as we have deprecated support of Hudi on Spark 2.

> Hudi CLI Bundle Spark 2 Issues
> ------------------------------
>
>                 Key: HUDI-5714
>                 URL: https://issues.apache.org/jira/browse/HUDI-5714
>             Project: Apache Hudi
>          Issue Type: Task
>            Reporter: Rahil Chertara
>            Priority: Major
>             Fix For: 1.0.2
>
>
> Currently it seems that when trying the cli bundle with spark2, the cli faces the following issue.
>  
> ```
> is java.lang.NoSuchMethodError: javax.validation.BootstrapConfiguration.getClockProviderClassName()Ljava/lang/String;
> 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1804)
> 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
> 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
> 	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
> 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
> 	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
> 	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
> 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
> 	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
> 	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
> 	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:734)
> 	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408)
> 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:308)
> 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306)
> 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295)
> 	at org.apache.hudi.cli.Main.main(Main.java:35)
> ```
>  
> This issue is not present on spark3, since the cli bundle contains jakarta deps, and spark 3 jars contains the corresponding jakarta validation jar. The problem is spark2 does not provide the jakarta and instead provides the javax version of this jar which is incorrect. From what ive seen it seems both types of these jars should not be present on class path. [https://stackoverflow.com/questions/58878713/spring-boot-application-fails-to-start-with-javax-validation-bootstrapconfigurat] 
> The workaround is moving the javax validation api jar under spark2 jars to a temp folder. After this a gson exception occurs with spark2, the only unofficial "workaround" ive found is moving the spark2 gson also to a temp folder (which is not ideal) to get the hudi cli bundle to start with spark2 
> Well need to revisit this at later point For now spark 3.3. and spark 3.2 should work 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)