[jira] [Commented] (HIVE-9664) Hive "add jar" command should be able to download and add jars from a repository

["Anant Nag (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14338279#comment-14338279 ] 

Anant Nag commented on HIVE-9664:
---------------------------------

[~appodictic], [~leftylev]: Thank you for your input. I'm making use of the grapes internally to download resources from the artifactory. 

Following are the changes that will be made to the commands: 

Add resource command:

Adding resources from hdfs and local filesystem works as before. Resources can be added from the artifactory using the following: 

Syntax: 
{code}
add jar <ivy://org:module:version?query_string> <jar ivy://org:module:version?query_string>*
{code}

group - Which module group the module comes from. Translates directly to a Maven groupId or an Ivy Organization. 
module - The name of the module to load. Translated directly to a Maven artifactId or an Ivy artifact.
version - The version of the module to use. Any version or * (for latest) or an Ivy Range '[2.2.1,)' meaning 2.2.1 or any greater version) can be used.

Various parameters can be passed in the url to configure how and which jars are added to the artifactory. The parameters are in the form of key value pairs where key is the name of the paramter and the value is the value of the parameter. These key-value pairs should be separated by &.

example: 
{code}
add jar ivy://org:module:version?key=value&key1=value1&key2=value2;
{code}



The different parameters that can be passed are:

1) exclude: Takes a comma separated value of the form (org:module)
Sometimes you will want to exclude transitive dependencies as you might be already using a slightly different but compatible version of some artifact.  exclude can be defined in the query string. It takes a comma(,) separated values of the form org:module, all these dependencies won't be downloaded from the artifactory.

Usage:
{code}
add jar ivy://org:module:version?exclude=org1:module1,org2:module2,org3:module3;
{code}
Example:
{code}
add jar ivy://org.apache.pig:pig:0.10.0?exclude=org.apache.hadoop:avro;
{code}
2) transitive: Takes values true or false. Defaults to true.
When transitive = true, all the transitive dependencies are downloaded and added to the classpath.

Usage:
{code}
add jar ivy://org:module:version?transitive=true&exclude=org1:module1
add jar ivy://org.module:version?transitive=false;
{code}

example:
{code}
add jar ivy://org.apache.pig:pig:0.10.0?exclude=org.apache.hadoop:avro&transitive=true;
add jar ivy://org.apache.pig:pig:0.10.0?transitive=false
{code}

3) ext: The extension of the file to add. jar by default.

example:
{code}
add jar ivy://org.apache.pig:pig:0.10.0?ext=jar
add jar ivy://com.linkedin.informed-bridge-jira:informed-bridge-jira:0.0.3?classifier=docs-json&ext=tar.gz;
{code}

4) classifier: The maven classifier to resolve by

example:
{code}
add jar ivy://com.linkedin.informed-bridge-jira:informed-bridge-jira:0.0.3?classifier=docs-json&ext=tar.gz;
{code}

Delete resource command:

Delete resource works as before for the resources added from hdfs and local file system.
Resources added from the artifactory can be deleted using the following command.

{code}
delete jar ivy://org:module:version
{code}

The delete jar command will delete all the transitive dependencies of the jar which was added using the same org:module:version. 

If two jars share a set of transitive dependencies and one of the jars is deleted using the above syntax, then all the transitive dependencies will be deleted for the jar except the ones which are shared. 

Example: 
{code}
add jar ivy://org.apache.pig:pig:0.10.0
add jar ivy://org.apache.pig:pig:0.11.1.15
delete jar ivy://org.apache.pig:pig:0.10.0
{code}

if A is the set containing the transitive dependencies of pig-0.10.0 and B is the set containing the transitive dependencies of pig-0.11.1.15, then after executing the above commands, A-(A intersection B) will be deleted. 

Now on executing
{code}
delete jar ivy://org.apache.pig:pig:0.11.1.15
{code}
all the remaining dependencies will be deleted. 


List resource: works as before. 


> Hive "add jar" command should be able to download and add jars from a repository
> --------------------------------------------------------------------------------
>
>                 Key: HIVE-9664
>                 URL: https://issues.apache.org/jira/browse/HIVE-9664
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Anant Nag
>              Labels: hive
>
> Currently Hive's "add jar" command takes a local path to the dependency jar. This clutters the local file-system as users may forget to remove this jar later
> It would be nice if Hive supported a Gradle like notation to download the jar from a repository.
> Example:  add jar org:module:version
>         
> It should also be backward compatible and should take jar from the local file-system as well. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)