[Discuss] Transitive dependencies and internal .pom changes

[Patrick Rhomberg <pr...@apache.org>]

Hello all!

# As a one paragraph elevator pitch:

  A module should declare its own dependencies and not expose those
dependencies to a consumer unless explicitly intending to do so.  As part
of working towards better decoupling between Geode's modules, we should
eliminate our reliance on transitive dependencies.  Dependencies required
by the top-level project should remain unchanged.  Please review these
changes [1] and discuss possible impact.


# Verbosely:

As part of general build improvements, becoming Gradle 5 ready, and
positioning the build to be more modular, Robert Houghton and I have been
investigating reducing our (internal) reliance on transitive dependencies,
explicitly declaring those dependencies a given module has, and removing
dependencies that are no longer consumed by a given module.

Leaking dependencies has the potential to effect user experience.  For
instance, if a user would like to use a different implementation of logging
than we use internally, doing so should not ruin the classpath.  Leaking
our own implementations are dangerous in these sorts of situation, and this
has been a pain-point of interaction between Geode and Spring in the past.
For instance, GEODE-5001 found at its crux exactly this logging version
mismatch as its issue.  Conversely, a great deal of preparation for this PR
was spent tracking down which plugins were leaking conflicting versions
JGit.  Let's do better by the consumers of Geode!

Much of our investigation has been driven by the Nebula dependency linter
plugin, which I would additionally propose to be a valuable addition in
verifying and maintaining the quality of our build.

Note all dependency and POM changes effect only each module.  The
dependences for the product as a whole should and will remain unchanged by
this.  The core driving force will be to allow for better modularity for
Geode and extension developers interacting with individual pieces of Geode.

See pull request 2532 [1] for comparison in explicit dependency declaration
and potential changes to existing POMs.

Regards.

Imagination is Change.
~Patrick Rhomberg

[1] https://github.com/apache/geode/pull/2532

Re: [Discuss] Transitive dependencies and internal .pom changes

[Anthony Baker <ab...@pivotal.io>]

I think it would be easier to review and discuss these changes with smaller, more focused PR’s.  Is there a way to split up these changes?  That would help me a lot and I would be less concerned about introducing a problem with our POM’s / dependencies.

Thanks,
Anthony


> On Oct 1, 2018, at 2:04 PM, Dan Smith <ds...@pivotal.io> wrote:
> 
> If what we are doing is just making sure that each geode module is
> explicitly declaring all of its first order dependencies (things that
> directly show up in import statements) in its build.gradle, then this seems
> good to me.
> 
> I don't think we want to stop using gradle's ability to pull in all of the
> transitive implementation dependencies that we don't directly use. We don't
> want to get into the business of maintaining those manually.
> 
> Removing unused dependencies and pulling in this lint plugin also seem like
> good improvements.
> 
> -Dan
> 
> On Mon, Oct 1, 2018 at 1:44 PM Patrick Rhomberg <pr...@apache.org>
> wrote:
> 
>>> Patrick, you just mentioned the java-library plugin's ability to
>>> distinguish between api and implementation dependencies. Geode doesn't
>> use
>>> that plugin yet AFAIK, nor does the PR introduce that plugin right?
>> 
>> That is correct, we do not yet make use of the `java-library` plugin (which
>> has the `implementation` and `api` configurations) and are still using the
>> `java` plugin (and rely on the `compile` configurations).  I brought it up
>> only to mention that there will be a great deal of inter-module build
>> breakages if we were to try to transition to using `implementation` while
>> our actual dependencies of each module were not yet explicit.
>> 
>> And yeah, all of this is on the road to Java9 modules.
>> 
>> As for accepting this PR, there are still multiple issues with it as it
>> stands -- I mostly opened it in its current state to spur this discussion.
>> I'm was going to go open a handful of tickets so that each individual PR
>> and commit are more bite-sized.
>> 
>> On Fri, Sep 28, 2018 at 4:26 PM, Bill Burcham <bb...@pivotal.io> wrote:
>> 
>>> So it sounds like (per Robert) we use the gradle-lint-plugin's
>>> unused-dependency rule to warn us of unused dependencies. That handles
>> one
>>> side of the equation (only list *necessary* dependencies).
>>> 
>>> I haven't heard anyone mention tool support for the other side of the
>>> equation: ensuring that we list the *sufficient* list of dependencies per
>>> build.gradle.
>>> 
>>> Patrick, you just mentioned the java-library plugin's ability to
>>> distinguish between api and implementation dependencies. Geode doesn't
>> use
>>> that plugin yet AFAIK, nor does the PR introduce that plugin right?
>>> 
>>> All this seems of a piece with the march toward modularity, exemplified
>> by
>>> Java 9 modules. Patrick, is your vision something like:
>>> 
>>> 1. accept this PR
>>> …
>>> 2. introduce the java-library plugin and segregate dependencies into api
>>> vs. implementation ones
>>> …
>>> N. define Java modules
>>> 
>>> 
>>> On Fri, Sep 28, 2018 at 10:39 AM Patrick Rhomberg <pr...@apache.org>
>>> wrote:
>>> 
>>>> Bill has the heart of it, yes.
>>>> 
>>>> I should have also mentioned that this ties into java-library plugin
>>>> configuration, notably that the `compile` configuration is deprecated.
>>>> For dependences that we do not wish to leak, we will need to use
>>>> `implementation`.  For dependencies which we intentionally elect to
>> share
>>>> with our consumers, we should use `api`.
>>>> 
>>>> For modularity, it should not break one module's ability to build for
>>>> another module to declare a `compile` configuration to be
>>> `implementation`,
>>>> unless of course that dependency is an active and necessary component
>>> part
>>>> of that module's consumption.  In that case, it should be declared part
>>> of
>>>> the modules `api`.
>>>> In this vein and as Bill pointed out, a module should have an accurate
>>>> representation of its own build-time dependencies.
>>>> 
>>>> In time, this will allow a much improved build experience, since
>>>> `implementation` and `api` are positioned for better granularity of
>>>> incremental building.
>>>> 
>>>> On Fri, Sep 28, 2018 at 10:23 AM, Robert Houghton <
>> rhoughton@pivotal.io>
>>>> wrote:
>>>> 
>>>>> Hi Bill,
>>>>> 
>>>>> We are using a Gradle plug-in to identify dependencies that are
>> unused,
>>>> or
>>>>> are declared in the wrong module or scope.
>>>>> This is called out by the Gradle documents on improving build [
>>>>> https://guides.gradle.org/performance/#avoid_unnecessary_and_unused_
>>>>> dependencies].
>>>>> The plug-in documentation is here [
>>>>> https://github.com/nebula-plugins/gradle-lint-plugin/
>>>>> wiki/Unused-Dependency-Rule
>>>>> ]
>>>>> 
>>>>> Thank you,
>>>>> -Robert Houghton
>>>>> 
>>>>> On Fri, Sep 28, 2018 at 10:18 AM Bill Burcham <bb...@pivotal.io>
>>>> wrote:
>>>>> 
>>>>>> From the PR, Anthony, it seems to me that Patrick is proposing that
>>>> each
>>>>>> build.gradle be explicit about mentioning the "things" it depends
>> on.
>>>> For
>>>>>> example:
>>>>>> 
>>>>>> [image: image.png]
>>>>>> 
>>>>>> Look at how geode-connectors goes from mentioning a few
>> dependencies
>>> to
>>>>>> mentioning many more. The value of this is that it ostensibly shows
>>> us
>>>>> all
>>>>>> the things that geode-connectors actually depends on.
>>>>>> 
>>>>>> The challenge I see is two-fold:
>>>>>> 
>>>>>> • as with Java imports and "unused imports", there is the risk of
>>>> listing
>>>>>> more dependencies than we actually need
>>>>>> • there's also the risk that we still don't have a complete list of
>>>>>> dependencies
>>>>>> 
>>>>>> Patrick: do you have tool support for this? Is there a tool that
>> can
>>>>>> identify or even remove unused dependencies? What process do you
>> use
>>>> for
>>>>>> finding these heretofore hidden dependencies? Did you run gradle
>>>>>> app:dependencies?
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 

Re: [Discuss] Transitive dependencies and internal .pom changes

[Dan Smith <ds...@pivotal.io>]

If what we are doing is just making sure that each geode module is
explicitly declaring all of its first order dependencies (things that
directly show up in import statements) in its build.gradle, then this seems
good to me.

I don't think we want to stop using gradle's ability to pull in all of the
transitive implementation dependencies that we don't directly use. We don't
want to get into the business of maintaining those manually.

Removing unused dependencies and pulling in this lint plugin also seem like
good improvements.

-Dan

On Mon, Oct 1, 2018 at 1:44 PM Patrick Rhomberg <pr...@apache.org>
wrote:

> > Patrick, you just mentioned the java-library plugin's ability to
> > distinguish between api and implementation dependencies. Geode doesn't
> use
> > that plugin yet AFAIK, nor does the PR introduce that plugin right?
>
> That is correct, we do not yet make use of the `java-library` plugin (which
> has the `implementation` and `api` configurations) and are still using the
> `java` plugin (and rely on the `compile` configurations).  I brought it up
> only to mention that there will be a great deal of inter-module build
> breakages if we were to try to transition to using `implementation` while
> our actual dependencies of each module were not yet explicit.
>
> And yeah, all of this is on the road to Java9 modules.
>
> As for accepting this PR, there are still multiple issues with it as it
> stands -- I mostly opened it in its current state to spur this discussion.
> I'm was going to go open a handful of tickets so that each individual PR
> and commit are more bite-sized.
>
> On Fri, Sep 28, 2018 at 4:26 PM, Bill Burcham <bb...@pivotal.io> wrote:
>
> > So it sounds like (per Robert) we use the gradle-lint-plugin's
> > unused-dependency rule to warn us of unused dependencies. That handles
> one
> > side of the equation (only list *necessary* dependencies).
> >
> > I haven't heard anyone mention tool support for the other side of the
> > equation: ensuring that we list the *sufficient* list of dependencies per
> > build.gradle.
> >
> > Patrick, you just mentioned the java-library plugin's ability to
> > distinguish between api and implementation dependencies. Geode doesn't
> use
> > that plugin yet AFAIK, nor does the PR introduce that plugin right?
> >
> > All this seems of a piece with the march toward modularity, exemplified
> by
> > Java 9 modules. Patrick, is your vision something like:
> >
> > 1. accept this PR
> > …
> > 2. introduce the java-library plugin and segregate dependencies into api
> > vs. implementation ones
> > …
> > N. define Java modules
> >
> >
> > On Fri, Sep 28, 2018 at 10:39 AM Patrick Rhomberg <pr...@apache.org>
> > wrote:
> >
> > > Bill has the heart of it, yes.
> > >
> > > I should have also mentioned that this ties into java-library plugin
> > > configuration, notably that the `compile` configuration is deprecated.
> > > For dependences that we do not wish to leak, we will need to use
> > > `implementation`.  For dependencies which we intentionally elect to
> share
> > > with our consumers, we should use `api`.
> > >
> > > For modularity, it should not break one module's ability to build for
> > > another module to declare a `compile` configuration to be
> > `implementation`,
> > > unless of course that dependency is an active and necessary component
> > part
> > > of that module's consumption.  In that case, it should be declared part
> > of
> > > the modules `api`.
> > > In this vein and as Bill pointed out, a module should have an accurate
> > > representation of its own build-time dependencies.
> > >
> > > In time, this will allow a much improved build experience, since
> > > `implementation` and `api` are positioned for better granularity of
> > > incremental building.
> > >
> > > On Fri, Sep 28, 2018 at 10:23 AM, Robert Houghton <
> rhoughton@pivotal.io>
> > > wrote:
> > >
> > > > Hi Bill,
> > > >
> > > > We are using a Gradle plug-in to identify dependencies that are
> unused,
> > > or
> > > > are declared in the wrong module or scope.
> > > > This is called out by the Gradle documents on improving build [
> > > > https://guides.gradle.org/performance/#avoid_unnecessary_and_unused_
> > > > dependencies].
> > > > The plug-in documentation is here [
> > > > https://github.com/nebula-plugins/gradle-lint-plugin/
> > > > wiki/Unused-Dependency-Rule
> > > > ]
> > > >
> > > > Thank you,
> > > > -Robert Houghton
> > > >
> > > > On Fri, Sep 28, 2018 at 10:18 AM Bill Burcham <bb...@pivotal.io>
> > > wrote:
> > > >
> > > > > From the PR, Anthony, it seems to me that Patrick is proposing that
> > > each
> > > > > build.gradle be explicit about mentioning the "things" it depends
> on.
> > > For
> > > > > example:
> > > > >
> > > > > [image: image.png]
> > > > >
> > > > > Look at how geode-connectors goes from mentioning a few
> dependencies
> > to
> > > > > mentioning many more. The value of this is that it ostensibly shows
> > us
> > > > all
> > > > > the things that geode-connectors actually depends on.
> > > > >
> > > > > The challenge I see is two-fold:
> > > > >
> > > > > • as with Java imports and "unused imports", there is the risk of
> > > listing
> > > > > more dependencies than we actually need
> > > > > • there's also the risk that we still don't have a complete list of
> > > > > dependencies
> > > > >
> > > > > Patrick: do you have tool support for this? Is there a tool that
> can
> > > > > identify or even remove unused dependencies? What process do you
> use
> > > for
> > > > > finding these heretofore hidden dependencies? Did you run gradle
> > > > > app:dependencies?
> > > > >
> > > >
> > >
> >
>

Re: [Discuss] Transitive dependencies and internal .pom changes

[Patrick Rhomberg <pr...@apache.org>]

> Patrick, you just mentioned the java-library plugin's ability to
> distinguish between api and implementation dependencies. Geode doesn't use
> that plugin yet AFAIK, nor does the PR introduce that plugin right?

That is correct, we do not yet make use of the `java-library` plugin (which
has the `implementation` and `api` configurations) and are still using the
`java` plugin (and rely on the `compile` configurations).  I brought it up
only to mention that there will be a great deal of inter-module build
breakages if we were to try to transition to using `implementation` while
our actual dependencies of each module were not yet explicit.

And yeah, all of this is on the road to Java9 modules.

As for accepting this PR, there are still multiple issues with it as it
stands -- I mostly opened it in its current state to spur this discussion.
I'm was going to go open a handful of tickets so that each individual PR
and commit are more bite-sized.

On Fri, Sep 28, 2018 at 4:26 PM, Bill Burcham <bb...@pivotal.io> wrote:

> So it sounds like (per Robert) we use the gradle-lint-plugin's
> unused-dependency rule to warn us of unused dependencies. That handles one
> side of the equation (only list *necessary* dependencies).
>
> I haven't heard anyone mention tool support for the other side of the
> equation: ensuring that we list the *sufficient* list of dependencies per
> build.gradle.
>
> Patrick, you just mentioned the java-library plugin's ability to
> distinguish between api and implementation dependencies. Geode doesn't use
> that plugin yet AFAIK, nor does the PR introduce that plugin right?
>
> All this seems of a piece with the march toward modularity, exemplified by
> Java 9 modules. Patrick, is your vision something like:
>
> 1. accept this PR
> …
> 2. introduce the java-library plugin and segregate dependencies into api
> vs. implementation ones
> …
> N. define Java modules
>
>
> On Fri, Sep 28, 2018 at 10:39 AM Patrick Rhomberg <pr...@apache.org>
> wrote:
>
> > Bill has the heart of it, yes.
> >
> > I should have also mentioned that this ties into java-library plugin
> > configuration, notably that the `compile` configuration is deprecated.
> > For dependences that we do not wish to leak, we will need to use
> > `implementation`.  For dependencies which we intentionally elect to share
> > with our consumers, we should use `api`.
> >
> > For modularity, it should not break one module's ability to build for
> > another module to declare a `compile` configuration to be
> `implementation`,
> > unless of course that dependency is an active and necessary component
> part
> > of that module's consumption.  In that case, it should be declared part
> of
> > the modules `api`.
> > In this vein and as Bill pointed out, a module should have an accurate
> > representation of its own build-time dependencies.
> >
> > In time, this will allow a much improved build experience, since
> > `implementation` and `api` are positioned for better granularity of
> > incremental building.
> >
> > On Fri, Sep 28, 2018 at 10:23 AM, Robert Houghton <rh...@pivotal.io>
> > wrote:
> >
> > > Hi Bill,
> > >
> > > We are using a Gradle plug-in to identify dependencies that are unused,
> > or
> > > are declared in the wrong module or scope.
> > > This is called out by the Gradle documents on improving build [
> > > https://guides.gradle.org/performance/#avoid_unnecessary_and_unused_
> > > dependencies].
> > > The plug-in documentation is here [
> > > https://github.com/nebula-plugins/gradle-lint-plugin/
> > > wiki/Unused-Dependency-Rule
> > > ]
> > >
> > > Thank you,
> > > -Robert Houghton
> > >
> > > On Fri, Sep 28, 2018 at 10:18 AM Bill Burcham <bb...@pivotal.io>
> > wrote:
> > >
> > > > From the PR, Anthony, it seems to me that Patrick is proposing that
> > each
> > > > build.gradle be explicit about mentioning the "things" it depends on.
> > For
> > > > example:
> > > >
> > > > [image: image.png]
> > > >
> > > > Look at how geode-connectors goes from mentioning a few dependencies
> to
> > > > mentioning many more. The value of this is that it ostensibly shows
> us
> > > all
> > > > the things that geode-connectors actually depends on.
> > > >
> > > > The challenge I see is two-fold:
> > > >
> > > > • as with Java imports and "unused imports", there is the risk of
> > listing
> > > > more dependencies than we actually need
> > > > • there's also the risk that we still don't have a complete list of
> > > > dependencies
> > > >
> > > > Patrick: do you have tool support for this? Is there a tool that can
> > > > identify or even remove unused dependencies? What process do you use
> > for
> > > > finding these heretofore hidden dependencies? Did you run gradle
> > > > app:dependencies?
> > > >
> > >
> >
>

Re: [Discuss] Transitive dependencies and internal .pom changes

[Bill Burcham <bb...@pivotal.io>]

So it sounds like (per Robert) we use the gradle-lint-plugin's
unused-dependency rule to warn us of unused dependencies. That handles one
side of the equation (only list *necessary* dependencies).

I haven't heard anyone mention tool support for the other side of the
equation: ensuring that we list the *sufficient* list of dependencies per
build.gradle.

Patrick, you just mentioned the java-library plugin's ability to
distinguish between api and implementation dependencies. Geode doesn't use
that plugin yet AFAIK, nor does the PR introduce that plugin right?

All this seems of a piece with the march toward modularity, exemplified by
Java 9 modules. Patrick, is your vision something like:

1. accept this PR
…
2. introduce the java-library plugin and segregate dependencies into api
vs. implementation ones
…
N. define Java modules


On Fri, Sep 28, 2018 at 10:39 AM Patrick Rhomberg <pr...@apache.org>
wrote:

> Bill has the heart of it, yes.
>
> I should have also mentioned that this ties into java-library plugin
> configuration, notably that the `compile` configuration is deprecated.
> For dependences that we do not wish to leak, we will need to use
> `implementation`.  For dependencies which we intentionally elect to share
> with our consumers, we should use `api`.
>
> For modularity, it should not break one module's ability to build for
> another module to declare a `compile` configuration to be `implementation`,
> unless of course that dependency is an active and necessary component part
> of that module's consumption.  In that case, it should be declared part of
> the modules `api`.
> In this vein and as Bill pointed out, a module should have an accurate
> representation of its own build-time dependencies.
>
> In time, this will allow a much improved build experience, since
> `implementation` and `api` are positioned for better granularity of
> incremental building.
>
> On Fri, Sep 28, 2018 at 10:23 AM, Robert Houghton <rh...@pivotal.io>
> wrote:
>
> > Hi Bill,
> >
> > We are using a Gradle plug-in to identify dependencies that are unused,
> or
> > are declared in the wrong module or scope.
> > This is called out by the Gradle documents on improving build [
> > https://guides.gradle.org/performance/#avoid_unnecessary_and_unused_
> > dependencies].
> > The plug-in documentation is here [
> > https://github.com/nebula-plugins/gradle-lint-plugin/
> > wiki/Unused-Dependency-Rule
> > ]
> >
> > Thank you,
> > -Robert Houghton
> >
> > On Fri, Sep 28, 2018 at 10:18 AM Bill Burcham <bb...@pivotal.io>
> wrote:
> >
> > > From the PR, Anthony, it seems to me that Patrick is proposing that
> each
> > > build.gradle be explicit about mentioning the "things" it depends on.
> For
> > > example:
> > >
> > > [image: image.png]
> > >
> > > Look at how geode-connectors goes from mentioning a few dependencies to
> > > mentioning many more. The value of this is that it ostensibly shows us
> > all
> > > the things that geode-connectors actually depends on.
> > >
> > > The challenge I see is two-fold:
> > >
> > > • as with Java imports and "unused imports", there is the risk of
> listing
> > > more dependencies than we actually need
> > > • there's also the risk that we still don't have a complete list of
> > > dependencies
> > >
> > > Patrick: do you have tool support for this? Is there a tool that can
> > > identify or even remove unused dependencies? What process do you use
> for
> > > finding these heretofore hidden dependencies? Did you run gradle
> > > app:dependencies?
> > >
> >
>

Re: [Discuss] Transitive dependencies and internal .pom changes

[Patrick Rhomberg <pr...@apache.org>]

Bill has the heart of it, yes.

I should have also mentioned that this ties into java-library plugin
configuration, notably that the `compile` configuration is deprecated.
For dependences that we do not wish to leak, we will need to use
`implementation`.  For dependencies which we intentionally elect to share
with our consumers, we should use `api`.

For modularity, it should not break one module's ability to build for
another module to declare a `compile` configuration to be `implementation`,
unless of course that dependency is an active and necessary component part
of that module's consumption.  In that case, it should be declared part of
the modules `api`.
In this vein and as Bill pointed out, a module should have an accurate
representation of its own build-time dependencies.

In time, this will allow a much improved build experience, since
`implementation` and `api` are positioned for better granularity of
incremental building.

On Fri, Sep 28, 2018 at 10:23 AM, Robert Houghton <rh...@pivotal.io>
wrote:

> Hi Bill,
>
> We are using a Gradle plug-in to identify dependencies that are unused, or
> are declared in the wrong module or scope.
> This is called out by the Gradle documents on improving build [
> https://guides.gradle.org/performance/#avoid_unnecessary_and_unused_
> dependencies].
> The plug-in documentation is here [
> https://github.com/nebula-plugins/gradle-lint-plugin/
> wiki/Unused-Dependency-Rule
> ]
>
> Thank you,
> -Robert Houghton
>
> On Fri, Sep 28, 2018 at 10:18 AM Bill Burcham <bb...@pivotal.io> wrote:
>
> > From the PR, Anthony, it seems to me that Patrick is proposing that each
> > build.gradle be explicit about mentioning the "things" it depends on. For
> > example:
> >
> > [image: image.png]
> >
> > Look at how geode-connectors goes from mentioning a few dependencies to
> > mentioning many more. The value of this is that it ostensibly shows us
> all
> > the things that geode-connectors actually depends on.
> >
> > The challenge I see is two-fold:
> >
> > • as with Java imports and "unused imports", there is the risk of listing
> > more dependencies than we actually need
> > • there's also the risk that we still don't have a complete list of
> > dependencies
> >
> > Patrick: do you have tool support for this? Is there a tool that can
> > identify or even remove unused dependencies? What process do you use for
> > finding these heretofore hidden dependencies? Did you run gradle
> > app:dependencies?
> >
>

Re: [Discuss] Transitive dependencies and internal .pom changes

[Robert Houghton <rh...@pivotal.io>]

Hi Bill,

We are using a Gradle plug-in to identify dependencies that are unused, or
are declared in the wrong module or scope.
This is called out by the Gradle documents on improving build [
https://guides.gradle.org/performance/#avoid_unnecessary_and_unused_dependencies].
The plug-in documentation is here [
https://github.com/nebula-plugins/gradle-lint-plugin/wiki/Unused-Dependency-Rule
]

Thank you,
-Robert Houghton

On Fri, Sep 28, 2018 at 10:18 AM Bill Burcham <bb...@pivotal.io> wrote:

> From the PR, Anthony, it seems to me that Patrick is proposing that each
> build.gradle be explicit about mentioning the "things" it depends on. For
> example:
>
> [image: image.png]
>
> Look at how geode-connectors goes from mentioning a few dependencies to
> mentioning many more. The value of this is that it ostensibly shows us all
> the things that geode-connectors actually depends on.
>
> The challenge I see is two-fold:
>
> • as with Java imports and "unused imports", there is the risk of listing
> more dependencies than we actually need
> • there's also the risk that we still don't have a complete list of
> dependencies
>
> Patrick: do you have tool support for this? Is there a tool that can
> identify or even remove unused dependencies? What process do you use for
> finding these heretofore hidden dependencies? Did you run gradle
> app:dependencies?
>

Re: [Discuss] Transitive dependencies and internal .pom changes

[Bill Burcham <bb...@pivotal.io>]

From the PR, Anthony, it seems to me that Patrick is proposing that each
build.gradle be explicit about mentioning the "things" it depends on. For
example:

[image: image.png]

Look at how geode-connectors goes from mentioning a few dependencies to
mentioning many more. The value of this is that it ostensibly shows us all
the things that geode-connectors actually depends on.

The challenge I see is two-fold:

• as with Java imports and "unused imports", there is the risk of listing
more dependencies than we actually need
• there's also the risk that we still don't have a complete list of
dependencies

Patrick: do you have tool support for this? Is there a tool that can
identify or even remove unused dependencies? What process do you use for
finding these heretofore hidden dependencies? Did you run gradle
app:dependencies?

Re: [Discuss] Transitive dependencies and internal .pom changes

[John Blum <jb...@pivotal.io>]

Agreed, plus many times you can declare that a dependency is either
(appropriately) "test" scope (for test dependencies only), "optional", or
(in certain cases) "provided", which will (should) have no impact to end
users, e.g. like conflicting dependencies.

However, I am in favor of reducing dependencies that are no long needed, if
possible.

On Thu, Sep 27, 2018 at 10:07 PM, Anthony Baker <ab...@pivotal.io> wrote:

> I don’t follow why we should get rid of transitive dependencies.  Can you
> help me understand?  How does this help with decoupling modules?
>
> The whole java ecosystem is based on declaring and consuming transitive
> dependencies via maven pom’s.  I get the api/implementation dependency
> separation, but that’s different.
>
> Anthony
>
>
> > On Sep 27, 2018, at 7:04 PM, Patrick Rhomberg <pr...@apache.org>
> wrote:
> >
> > Hello all!
> >
> > # As a one paragraph elevator pitch:
> >
> >  A module should declare its own dependencies and not expose those
> > dependencies to a consumer unless explicitly intending to do so.  As part
> > of working towards better decoupling between Geode's modules, we should
> > eliminate our reliance on transitive dependencies.  Dependencies required
> > by the top-level project should remain unchanged.  Please review these
> > changes [1] and discuss possible impact.
> >
> >
> > # Verbosely:
> >
> > As part of general build improvements, becoming Gradle 5 ready, and
> > positioning the build to be more modular, Robert Houghton and I have been
> > investigating reducing our (internal) reliance on transitive
> dependencies,
> > explicitly declaring those dependencies a given module has, and removing
> > dependencies that are no longer consumed by a given module.
> >
> > Leaking dependencies has the potential to effect user experience.  For
> > instance, if a user would like to use a different implementation of
> logging
> > than we use internally, doing so should not ruin the classpath.  Leaking
> > our own implementations are dangerous in these sorts of situation, and
> this
> > has been a pain-point of interaction between Geode and Spring in the
> past.
> > For instance, GEODE-5001 found at its crux exactly this logging version
> > mismatch as its issue.  Conversely, a great deal of preparation for this
> PR
> > was spent tracking down which plugins were leaking conflicting versions
> > JGit.  Let's do better by the consumers of Geode!
> >
> > Much of our investigation has been driven by the Nebula dependency linter
> > plugin, which I would additionally propose to be a valuable addition in
> > verifying and maintaining the quality of our build.
> >
> > Note all dependency and POM changes effect only each module.  The
> > dependences for the product as a whole should and will remain unchanged
> by
> > this.  The core driving force will be to allow for better modularity for
> > Geode and extension developers interacting with individual pieces of
> Geode.
> >
> > See pull request 2532 [1] for comparison in explicit dependency
> declaration
> > and potential changes to existing POMs.
> >
> > Regards.
> >
> > Imagination is Change.
> > ~Patrick Rhomberg
> >
> > [1] https://github.com/apache/geode/pull/2532
>
>


-- 
-John
john.blum10101 (skype)

Re: [Discuss] Transitive dependencies and internal .pom changes

[Anthony Baker <ab...@pivotal.io>]

I don’t follow why we should get rid of transitive dependencies.  Can you help me understand?  How does this help with decoupling modules?  

The whole java ecosystem is based on declaring and consuming transitive dependencies via maven pom’s.  I get the api/implementation dependency separation, but that’s different.

Anthony


> On Sep 27, 2018, at 7:04 PM, Patrick Rhomberg <pr...@apache.org> wrote:
> 
> Hello all!
> 
> # As a one paragraph elevator pitch:
> 
>  A module should declare its own dependencies and not expose those
> dependencies to a consumer unless explicitly intending to do so.  As part
> of working towards better decoupling between Geode's modules, we should
> eliminate our reliance on transitive dependencies.  Dependencies required
> by the top-level project should remain unchanged.  Please review these
> changes [1] and discuss possible impact.
> 
> 
> # Verbosely:
> 
> As part of general build improvements, becoming Gradle 5 ready, and
> positioning the build to be more modular, Robert Houghton and I have been
> investigating reducing our (internal) reliance on transitive dependencies,
> explicitly declaring those dependencies a given module has, and removing
> dependencies that are no longer consumed by a given module.
> 
> Leaking dependencies has the potential to effect user experience.  For
> instance, if a user would like to use a different implementation of logging
> than we use internally, doing so should not ruin the classpath.  Leaking
> our own implementations are dangerous in these sorts of situation, and this
> has been a pain-point of interaction between Geode and Spring in the past.
> For instance, GEODE-5001 found at its crux exactly this logging version
> mismatch as its issue.  Conversely, a great deal of preparation for this PR
> was spent tracking down which plugins were leaking conflicting versions
> JGit.  Let's do better by the consumers of Geode!
> 
> Much of our investigation has been driven by the Nebula dependency linter
> plugin, which I would additionally propose to be a valuable addition in
> verifying and maintaining the quality of our build.
> 
> Note all dependency and POM changes effect only each module.  The
> dependences for the product as a whole should and will remain unchanged by
> this.  The core driving force will be to allow for better modularity for
> Geode and extension developers interacting with individual pieces of Geode.
> 
> See pull request 2532 [1] for comparison in explicit dependency declaration
> and potential changes to existing POMs.
> 
> Regards.
> 
> Imagination is Change.
> ~Patrick Rhomberg
> 
> [1] https://github.com/apache/geode/pull/2532