[jira] [Created] (HADOOP-13016) reinstate hadoop-hdfs as dependency of hadoop-client, create hadoop-lean-client for minimal deployments

["Steve Loughran (JIRA)" <ji...@apache.org>]

Steve Loughran created HADOOP-13016:
---------------------------------------

             Summary: reinstate hadoop-hdfs as dependency of hadoop-client, create hadoop-lean-client for minimal deployments
                 Key: HADOOP-13016
                 URL: https://issues.apache.org/jira/browse/HADOOP-13016
             Project: Hadoop Common
          Issue Type: Improvement
          Components: build
    Affects Versions: 2.8.0
            Reporter: Steve Loughran


the split of hadoop-hdfs and hadoop-hdfs-client is breaking code of mine whose builds declared a dependency on hadoop-client and expected all of HDFS to make it in.

I'm finding this first, because I'm building and testing downstream code against branch-2; I find myself having to explicitly declare a dependency on hadoop-hdfs to make things work again.

We've also seen problems downstream (e.g. spark) where the move of s3n classes to hadoop-aws has broken code which expects it to be there.

At the same time, I see the merits in a lean, low-dependency client, which hadoop-client and its dependencies is not today.

I propose

# reinstate hadoop-hdfs as dependency of hadoop-client
# add hadoop-aws as a dependency of hadoop-client —but excluding adding any amazon-aws JARs.
# create hadoop-lean-client for minimal deployments, stripping out all extraneous dependencies,
# for hadoop-lean-client, have a compatibility statement of "we will strip out anything we can from this, even over point releases". That is, anything that can be dropped in future, will.

This will give downstream projects a choice: the old POM with everything, the lean POM for new apps.

And, by reinstating hadoop-hdfs, things will build again




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)