[jira] [Commented] (YARN-9777) hadoop-yarn-server-common pom.xml has runtime dependency on mssql-jdbc

["Dénes Bodó (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-9777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17540039#comment-17540039 ] 

Dénes Bodó commented on YARN-9777:
----------------------------------

Hey there,

I am here because of an orphan-ish mssql-jdbc jar living in mapreduce.tar.gz and in hadoop-yarn lib dir.

One of the Apache components, Sqoop (I know it is in the attic now) is for moving data between RDBMS and Hadoop cluster by running MapReduce jobs to parallely execute data movement.

In case a user wants to move data from/to MSSQL Server and they use their proper JDBC driver jar in Sqoop lib for their use case then jar conflict usually happens and customer gets a failed data import.

 

This can happen in any Apache Oozie action too which executes its action as a Yarn application which also has the mssql jdbc jar on the class path from mapreduce tar.gz file.

 

 

Can we remove this mssql jdbc jar from Hadoop's dependencies?

> hadoop-yarn-server-common pom.xml has runtime dependency on mssql-jdbc
> ----------------------------------------------------------------------
>
>                 Key: YARN-9777
>                 URL: https://issues.apache.org/jira/browse/YARN-9777
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn
>    Affects Versions: 3.1.0
>            Reporter: Naveen Gangam
>            Priority: Minor
>
> I dont know the history behind this or what these classes are used for in YARN but having a runtime dependency on vendor specific JDBC driver seems wrong.
>     <dependency>
>       <groupId>com.microsoft.sqlserver</groupId>
>       <artifactId>mssql-jdbc</artifactId>
>       <scope>runtime</scope>
>        <exclusions>
>         <exclusion>
>           <groupId>com.microsoft.azure</groupId>
>           <artifactId>azure-keyvault</artifactId>
>         </exclusion>
>       </exclusions>
>     </dependency>
> This causes uber jars in dependent modules like hive-jdbc-standalone.jar to include all "runtime" scoped jars to be automatically bundled. 
> Instead the scope of the above dependency to be "provided" rather than "runtime".
> Also is this scoped correctly? or should this be a "runtime" dependency instead of the default "compile"
>     <dependency>
>       <groupId>com.zaxxer</groupId>
>       <artifactId>HikariCP-java7</artifactId>
>     </dependency>
> Thanks



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org