[jira] [Created] (HUDI-6619) Fix hudi-integ-test-bundle dependency on jackson jsk310 package.

["Amrish Lal (Jira)" <ji...@apache.org>]

Amrish Lal created HUDI-6619:
--------------------------------

             Summary: Fix hudi-integ-test-bundle dependency on jackson jsk310 package.
                 Key: HUDI-6619
                 URL: https://issues.apache.org/jira/browse/HUDI-6619
             Project: Apache Hudi
          Issue Type: Bug
            Reporter: Amrish Lal


*Command being used to run integration test*
{code:java}
./bin/spark-submit \ --packages org.apache.spark:spark-avro_2.12:3.3.2 \ --jars file:////Users/amrish/jars/jackson-datatype-jsr310-2.10.0.jar \ --conf spark.task.cpus=1 \ --conf spark.executor.cores=1 \ --conf spark.task.maxFailures=100 \ --conf spark.memory.fraction=0.4 \ --conf spark.rdd.compress=true \ --conf spark.kryoserializer.buffer.max=2000m \ --conf spark.serializer=org.apache.spark.serializer.KryoSerializer \ --conf spark.kryo.registrator=org.apache.spark.HoodieSparkKryoRegistrar \ --conf spark.memory.storageFraction=0.1 \ --conf spark.shuffle.service.enabled=true \ --conf spark.sql.hive.convertMetastoreParquet=false \ --conf spark.driver.maxResultSize=12g \ --conf spark.executor.heartbeatInterval=120s \ --conf spark.network.timeout=600s \ --conf spark.yarn.max.executor.failures=10 \ --conf spark.sql.catalogImplementation=hive \ --class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob /Users/amrish/code/amrish-hudi-3/packaging/hudi-integ-test-bundle/target/hudi-integ-test-bundle-0.14\ .0-SNAPSHOT.jar \ --source-ordering-field test_suite_source_ordering_field \ --use-deltastreamer \ --target-base-path /tmp/hudi/output \ --input-base-path /tmp/hudi/input \ --target-table table1 \ --props file:///Users/amrish/junk/test-aggressive-clean-archival.properties \ --workload-yaml-path file:///Users/amrish/code/amrish-hudi-3/docker/demo/config/test-suite/simple-deltastreamer.yaml \ --schemaprovider-class org.apache.hudi.integ.testsuite.schema.TestSuiteFileBasedSchemaProvider \ --source-class org.apache.hudi.utilities.sources.AvroDFSSource \ --input-file-size 125829120 \ --workload-generator-classname org.apache.hudi.integ.testsuite.dag.WorkflowDagGenerator \ --table-type COPY_ON_WRITE \ --compact-scheduling-minshare 1 \ --clean-input \ --clean-output{code}
{{  }}
{*}Exception being thrown{*}{{{}{}}}
{code:java}
Caused by: java.lang.NoClassDefFoundError: org/apache/hudi/com/fasterxml/jackson/datatype/jsr310/JavaTimeModule at org.apache.hudi.common.util.JsonUtils.registerModules(JsonUtils.java:63) at org.apache.spark.sql.adapter.BaseSpark3Adapter.<init>(BaseSpark3Adapter.scala:51) at org.apache.spark.sql.adapter.Spark3_3Adapter.<init>(Spark3_3Adapter.scala:46) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at org.apache.hudi.SparkAdapterSupport$.sparkAdapter$lzycompute(SparkAdapterSupport.scala:49){code}
{{}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)