[jira] [Commented] (HUDI-1193) Upgrade http dependent version

["liujinhui (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HUDI-1193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17180301#comment-17180301 ] 

liujinhui commented on HUDI-1193:
---------------------------------

Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 4.0 failed 4 times, most recent failure: Lost task 0.3 in stage 4.0 (TID 77, t3-gray-alhzgz-ecs-dtpbu-data-lake-14, executor 12): org.apache.hudi.exception.HoodieException: Unable to instantiate class 
 at org.apache.hudi.common.util.ReflectionUtils.loadClass(ReflectionUtils.java:89)
 at org.apache.hudi.common.util.ReflectionUtils.loadClass(ReflectionUtils.java:98)
 at org.apache.hudi.common.bootstrap.index.BootstrapIndex.getBootstrapIndex(BootstrapIndex.java:159)
 at org.apache.hudi.common.table.view.AbstractTableFileSystemView.init(AbstractTableFileSystemView.java:100)
 at org.apache.hudi.common.table.view.HoodieTableFileSystemView.init(HoodieTableFileSystemView.java:95)
 at org.apache.hudi.common.table.view.HoodieTableFileSystemView.<init>(HoodieTableFileSystemView.java:89)
 at org.apache.hudi.common.table.view.FileSystemViewManager.createInMemoryFileSystemView(FileSystemViewManager.java:158)
 at org.apache.hudi.common.table.view.FileSystemViewManager.lambda$createViewManager$ab86fd6b$3(FileSystemViewManager.java:199)
 at org.apache.hudi.common.table.view.FileSystemViewManager.lambda$getFileSystemView$1(FileSystemViewManager.java:104)
 at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
 at org.apache.hudi.common.table.view.FileSystemViewManager.getFileSystemView(FileSystemViewManager.java:103)
 at org.apache.hudi.table.HoodieTable.getBaseFileOnlyView(HoodieTable.java:239)
 at org.apache.hudi.table.action.commit.UpsertPartitioner.getSmallFiles(UpsertPartitioner.java:234)
 at org.apache.hudi.table.action.commit.UpsertPartitioner.lambda$getSmallFilesForPartitions$d2bd4b49$1(UpsertPartitioner.java:216)
 at org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043)
 at org.apache.spark.api.java.JavaPairRDD$$anonfun$pairFunToScalaFun$1.apply(JavaPairRDD.scala:1043)
 at scala.collection.Iterator$$anon$11.next(Iterator.scala:410)
 at scala.collection.Iterator$class.foreach(Iterator.scala:891)
 at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
 at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
 at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
 at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
 at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
 at scala.collection.AbstractIterator.to(Iterator.scala:1334)
 at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
 at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1334)
 at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
 at scala.collection.AbstractIterator.toArray(Iterator.scala:1334)
 at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$13.apply(RDD.scala:945)
 at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$13.apply(RDD.scala:945)
 at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2121)
 at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2121)
 at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
 at org.apache.spark.scheduler.Task.run(Task.scala:121)
 at org.apache.spark.executor.Executor$TaskRunner$$anonfun$11.apply(Executor.scala:407)
 at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1408)
 at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:413)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
 at org.apache.hudi.common.util.ReflectionUtils.loadClass(ReflectionUtils.java:87)
 ... 39 more
Caused by: java.lang.NoSuchMethodError: org.apache.http.conn.ssl.SSLConnectionSocketFactory.<init>(Ljavax/net/ssl/SSLContext;Ljavax/net/ssl/HostnameVerifier;)V
 at com.aliyun.oss.common.comm.DefaultServiceClient.createHttpClientConnectionManager(DefaultServiceClient.java:249)
 at com.aliyun.oss.common.comm.DefaultServiceClient.<init>(DefaultServiceClient.java:87)
 at com.aliyun.oss.OSSClient.<init>(OSSClient.java:202)
 at org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystemStore.initialize(AliyunOSSFileSystemStore.java:155)
 at org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem.initialize(AliyunOSSFileSystem.java:344)
 at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3237)
 at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:123)
 at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3286)
 at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3254)
 at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:478)
 at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
 at org.apache.hudi.common.fs.FSUtils.getFs(FSUtils.java:95)
 at org.apache.hudi.common.table.HoodieTableMetaClient.getFs(HoodieTableMetaClient.java:261)
 at org.apache.hudi.common.bootstrap.index.HFileBootstrapIndex.<init>(HFileBootstrapIndex.java:101)
{code:java}
//代码占位符
{code}

> Upgrade http dependent version
> ------------------------------
>
>                 Key: HUDI-1193
>                 URL: https://issues.apache.org/jira/browse/HUDI-1193
>             Project: Apache Hudi
>          Issue Type: Bug
>    Affects Versions: 0.6.0
>            Reporter: liujinhui
>            Assignee: liujinhui
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 0.6.1
>
>
> Upgrade http dependent version。
>  
> Hudi currently uses
> <dependency>
>  <groupId>org.apache.httpcomponents</groupId>
>  <artifactId>httpcore</artifactId>
>  <version>4.3.2</version>
> </dependency>
> <dependency>
>  <groupId>org.apache.httpcomponents</groupId>
>  <artifactId>httpclient</artifactId>
>  <version>4.3.6</version>
> </dependency>
>  
> This will cause hui to be unable to write to oss, oss requires version 4.4.1



--
This message was sent by Atlassian Jira
(v8.3.4#803005)