[jira] [Commented] (HIVE-21374) Dependency conflicts on org.apache.httpcomponents:httpcore:jar, leading to invoking unexpected methods

["Hello CoCooo (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-21374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16782315#comment-16782315 ] 

Hello CoCooo commented on HIVE-21374:
-------------------------------------

The code snippet of *org.apache.http.impl.conn.CPool: boolean validate(org.apache.http.pool.PoolEntry)* in *org.apache.httpcomponents:httpcore:jar:4.4.1* ----

!validate4.4.1.png!

*org.apache.httpcomponents:httpcore:jar:4.4* does not contain *org.apache.http.impl.conn.CPool: boolean validate(org.apache.http.pool.PoolEntry)*.

The code snippet of *org.apache.http.impl.conn.AbstractConnPool: boolean validate(org.apache.http.pool.PoolEntry)* in *org.apache.httpcomponents:httpcore:jar:4.4* ----

!validate4.4.png!

The method *org.apache.http.impl.conn.CPool: boolean validate(org.apache.http.pool.PoolEntry)* in newer verison *org.apache.httpcomponents:httpcore:jar:4.4.1* deals with more cases, which changes the control flows and data flows. So referencing *AbstractConnPool**.**validate(org.apache.http.pool.PoolEntry)*** in version *org.apache.httpcomponents:httpcore:jar:4.4* by
dynamic binding, may lead to inconsisitent semantic behaviors.

 

Thanks again.

> Dependency conflicts on org.apache.httpcomponents:httpcore:jar, leading to invoking unexpected methods
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-21374
>                 URL: https://issues.apache.org/jira/browse/HIVE-21374
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>    Affects Versions: 2.3.4
>            Reporter: Hello CoCooo
>            Priority: Major
>             Fix For: 2.3.4
>
>         Attachments: validate4.4.1.png, validate4.4.png
>
>
> Hi! In *hive-rel-release-2.3.4\service-rpc,*  there are multiple versions of *org.apache.httpcomponents:httpcore:jar*. As shown in the following dependency tree, according to Maven's dependency management strategy, only *org.apache.httpcomponents:httpcore:jar:4.4* can be loaded, and *org.apache.httpcomponents:httpcore:jar:4.4.1* will be shadowed.
> Your project references the method {color:#d04437}*<org.apache.http.impl.conn.CPool.validate(org.apache.http.pool.PoolEntry)>* {color}via the following invocation path, which is included in the shadowed version *org.apache.httpcomponents:httpcore:jar:4.4.1*. However, this method is missing in the actual loaded version *org.apache.httpcomponents:httpcore:jar:4.4*. Surprisingly, it will not cause NoSuchMethodError at rumtime.
> {color:#59afe1}*Invocation path:*{color}
> {code:java}
> // code placeholder
> <org.apache.thrift.server.TThreadedSelectorServer: boolean requestInvoke(org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer)> C:\Users\Flipped\.m2\repository\org\apache\thrift\libthrift\0.9.3\libthrift-0.9.3.jar
> <org.junit.internal.runners.MethodRoadie$1: void run()> C:\Users\Flipped\.m2\repository\junit\junit\4.11\junit-4.11.jar
> <org.apache.http.pool.PoolEntryFuture: java.lang.Object get(long,java.util.concurrent.TimeUnit)> C:\Users\Flipped\.m2\repository\org\apache\httpcomponents\httpcore\4.4.1\httpcore-4.4.1.jar
> <org.apache.http.pool.AbstractConnPool$2: java.lang.Object getPoolEntry(long,java.util.concurrent.TimeUnit)> C:\Users\Flipped\.m2\repository\org\apache\httpcomponents\httpcore\4.4.1\httpcore-4.4.1.jar
> <org.apache.http.pool.AbstractConnPool$2: org.apache.http.pool.PoolEntry getPoolEntry(long,java.util.concurrent.TimeUnit)> C:\Users\Flipped\.m2\repository\org\apache\httpcomponents\httpcore\4.4.1\httpcore-4.4.1.jar
> <org.apache.http.pool.AbstractConnPool: org.apache.http.pool.PoolEntry access$000(org.apache.http.pool.AbstractConnPool,java.lang.Object,java.lang.Object,long,java.util.concurrent.TimeUnit,org.apache.http.pool.PoolEntryFuture)> C:\Users\Flipped\.m2\repository\org\apache\httpcomponents\httpcore\4.4.1\httpcore-4.4.1.jar
> <org.apache.http.pool.AbstractConnPool: org.apache.http.pool.PoolEntry getPoolEntryBlocking(java.lang.Object,java.lang.Object,long,java.util.concurrent.TimeUnit,org.apache.http.pool.PoolEntryFuture)> C:\Users\Flipped\.m2\repository\org\apache\httpcomponents\httpcore\4.4.1\httpcore-4.4.1.jar
> <org.apache.http.impl.conn.CPool: boolean validate(org.apache.http.pool.PoolEntry)>{code}
> By further analyzing, I found that the caller *org.apache.thrift.server.TThreadedSelectorServer.requestInvoke(AbstractNonblockingServer$FrameBuffer)* would invoke the method *{color:#d04437}AbstractConnPool.validate(PoolEntry){color}* defined in the *superclass of org.apache.http.impl.conn.CPool (**CPool* *extends AbstractConnPool)* with the same signature of the expected callee, due to dynamic binding mechanism.
> Although the actual invoked method belonging to *{color:#d04437}AbstractConnPool{color}* has the same method name, same parameter types and return type as the expected method defined in its subclass {color:#d04437}*CPool*{color}, but it has different control flows and different behaviors. Maybe it is buggy behavior.
> *Solution:*
> Use the newer version *org.apache.httpcomponents:httpcore:jar:4.4.1* in parent pom file to keep the version consistency.
>  
> *Dependency tree----*
> [INFO] org.apache.hive:hive-service-rpc:jar:2.3.4
> [INFO] +- commons-codec:commons-codec:jar:1.4:compile
> [INFO] +- commons-cli:commons-cli:jar:1.2:compile
> [INFO] +- tomcat:jasper-compiler:jar:5.5.23:compile
> [INFO] |  +- javax.servlet:jsp-api:jar:2.0:compile
> [INFO] |  |  \- (javax.servlet:servlet-api:jar:2.4:compile - omitted for duplicate)
> [INFO] |  \- ant:ant:jar:1.6.5:compile
> [INFO] +- tomcat:jasper-runtime:jar:5.5.23:compile
> [INFO] |  +- javax.servlet:servlet-api:jar:2.4:compile
> [INFO] |  \- commons-el:commons-el:jar:1.0:compile
> [INFO] |     \- commons-logging:commons-logging:jar:1.0.3:compile
> [INFO] +- org.apache.thrift:libfb303:jar:0.9.3:compile
> [INFO] |  \- (org.apache.thrift:libthrift:jar:0.9.3:compile - omitted for duplicate)
> [INFO] +- org.apache.thrift:libthrift:jar:0.9.3:compile
> [INFO] |  +- (org.slf4j:slf4j-api:jar:1.7.10:compile - version managed from 1.7.12; omitted for duplicate)
> [INFO] |  +- org.apache.httpcomponents:httpclient:jar:4.4:compile (version managed from 4.4.1)
> [INFO] |  |  +- *(org.apache.httpcomponents:httpcore:jar:4.4:compile - version managed from 4.4.1; omitted for duplicate)*
> [INFO] |  |  +- (commons-logging:commons-logging:jar:1.2:compile - omitted for conflict with 1.0.3)
> [INFO] |  |  \- (commons-codec:commons-codec:jar:1.4:compile - version managed from 1.9; omitted for duplicate)
> [INFO] |  \- *org.apache.httpcomponents:httpcore:jar:4.4:compile*
> [INFO] +- junit:junit:jar:4.11:test
> [INFO] |  \- org.hamcrest:hamcrest-core:jar:1.3:test
> [INFO] +- org.slf4j:slf4j-api:jar:1.7.10:compile
> [INFO] \- org.mockito:mockito-all:jar:1.9.5:test
>  
>  
> Thanks!
> Best regards,
> Coco



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)