[jira] [Created] (HUDI-5714) Hudi CLI Bundle Spark 2 Issues

["Rahil Chertara (Jira)" <ji...@apache.org>]

Rahil Chertara created HUDI-5714:
------------------------------------

             Summary: Hudi CLI Bundle Spark 2 Issues
                 Key: HUDI-5714
                 URL: https://issues.apache.org/jira/browse/HUDI-5714
             Project: Apache Hudi
          Issue Type: Task
            Reporter: Rahil Chertara


Currently it seems that when trying the cli bundle with spark2, the cli faces the following issue.

 

```
is java.lang.NoSuchMethodError: javax.validation.BootstrapConfiguration.getClockProviderClassName()Ljava/lang/String;
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1804)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:734)
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:308)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295)
	at org.apache.hudi.cli.Main.main(Main.java:35)
```

 

This issue is not present on spark3, since the cli bundle contains jakarta deps, and spark 3 jars contains the corresponding jakarta validation jar. The problem is spark2 does not provide the jakarta and instead provides the javax version of this jar which is incorrect. From what ive seen it seems both types of these jars should not be present on class path. [https://stackoverflow.com/questions/58878713/spring-boot-application-fails-to-start-with-javax-validation-bootstrapconfigurat] 


The workaround is moving the javax validation api jar under spark2 jars to a temp folder. After this a gson exception occurs with spark2, the only unofficial "workaround" ive found is moving the spark2 gson also to a temp folder (which is not ideal) to get the hudi cli bundle to start with spark2 

Well need to revisit this at later point For now spark 3.3. and spark 3.2 should work 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)