You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@netbeans.apache.org by Svata Dedic <ga...@volny.cz> on 2023/11/29 19:22:38 UTC
Project dependencies - add/remove ability
Hi,
I am working (finishing) on an extension of (still private) Project
Dependency API that would allow to *add* dependency to a project (and
also remove). This is a change that will span both Maven and Gradle
modules, so I thought I'll put a note in advance, but PR will be landing
shortly in Github.
The reason is the ability to add technology libraries, i.e. add
monitoring to an application, add JDBC driver etc, which all requires
(among other things) to add a set of artifact(s) to a project. I would
like to have that ability for our company product, primarily, but the
API to do _basic_ dependency changes in both major supported build
systems. Only basic operations (add, remove) and only projects that use
some artifact repositories are considered (maven, gradle, possibly mx),
not ant-based projects.
In order to get a reasonable API outcome, I decided to work jointly with
Refactoring API and LSP API: the changes are best described as text
changes to project files - and in case of our LSP client use-case,
perform them. And rectoring API allows to commit() them in NetBeans
case. Note that while *java* refactoring API describes text
modifications in its result, the generic refactoring API does not. It
does not seem beneficial to invent yet another text-changes description
now, when LSP APIs are here, and basically do what's needed (even for Java).
Gradle implementation: I do not want to pollute basic gradle modules
more, so I decided to create a small java.gradle.dependencies module; I
am thinking about migrating the dependencies part out of java.gradle
module to the new one, which should clean up java.gradle module a bit.
Maven implementation: For Maven, I'd work this into maven.hints. All
dependencies are there except refactoring.api.
Some more steps are IMHO needed until the dependency API is mature
enough to get out of private:
- the Scope is not designed well; it was anticipated (and that's why the
API is not public) the introduction of modifications shed a little more
light on it. I'll be working on that, keeping Micronaut support module
as a early adopter (with necessary updates).
- the Dependency children ought to be generated lazily; this gives the
client the option not to traverse / expand subtrees for dependencies
already seen. But some clients need exhaustive traversal.
- improve the interaction with BOMs
Cheers,
-Svata
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Project dependencies - add/remove ability
Posted by Svata Dedic <ga...@volny.cz>.
On 01. 12. 23 16:22, Michael Bien wrote:
>
> is this about a convenience action which:
>
Possibly, but it's mainly about a convenience API that allows (other)
modules to do that.
Currently a "technology" module (micronaut, spring, oracle cloud) that
would like to introduce an artifact to the build, must check
- project type (say gradle/maven)
- based on project type:
maven: use Maven API (ModelUtils.checkModelDependency) to add a
<dependency> child
gradle: do textual edit of build.gradle
It's rather peculiar to make such actions from a "technology" module
that does not directly depend on Maven and Gradle internals, and of
course does not want to reimplement it.
In addition, I'd like to conform to refactoring API that computes the
chane first, then allows to apply it, where the Maven API operation will
directly mutate the POM document.
The result should be nothing more (from API perspective) than add/remove
(artifact) dependency, later maybe update (version change, simple excludes).
Help would be appreciated in thinking out how to possibly integrate sets
of maven/gradle artifact references into Library concept of Java project
common API ... it seems the current impl assumes .jars are present.
-S.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Project dependencies - add/remove ability
Posted by Michael Bien <mb...@gmail.com>.
On 01.12.23 16:00, Svata Dedic wrote:
> On 01. 12. 23 15:53, Michael Bien wrote:
>>
>> Sounds like a useful feature in some situations. The impl details
>> will be interesting though, since I am sure someone will try to add a
>> maven project to a gradle project or vise versa.
>>
> The feature should be primarily used to add artifact (e.g.
> io-micronaut:micronaut-router:4.1.2) dependencies. But yes, it's
> possible to describe a project on local hard disk ... which will most
> probably cause an "unsupported" exception.
since micronaut-router is on maven central, the existing add-dependency
dialog could be used which would find and add the dep to the pom. Staged
dev-time dependencies could be probably solved via project profiles?
>
> It is only IMHO safe to add a subproject in the same multiproject in
> gradle (not in maven) as a dependency. Maven relies entirely on
> artifacts (?).
this is what makes it confusing to me. I still don't think I 100%
understood what a project dependency is, when the build system is all
about artifacts. Thats why I feared it might be something "on top" of
the build system - but you clarified that already.
is this about a convenience action which:
- can add an opened project as "dependency", e.g for devs which have
micronaut-router open
- this will find the artifact by looking at the project and simply paste:
<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-router</artifactId>
<version>4.2.1-SNAPSHOT</version>
</dependency>
to the pom?
-mbien
> I could imagine that the implementation would attempt to extract
> output artifact's GAV from the identified project and use that.
>
> -S.
>
>> best regards && looking forward to try the feature,
>>
>> -mbien
>>
>>
>>>
>>> -S.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
>>> For additional commands, e-mail: dev-help@netbeans.apache.org
>>>
>>> For further information about the NetBeans mailing lists, visit:
>>> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>>>
>>>
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Project dependencies - add/remove ability
Posted by Svata Dedic <ga...@volny.cz>.
On 01. 12. 23 15:53, Michael Bien wrote:
>
> Sounds like a useful feature in some situations. The impl details will
> be interesting though, since I am sure someone will try to add a maven
> project to a gradle project or vise versa.
>
The feature should be primarily used to add artifact (e.g.
io-micronaut:micronaut-router:4.1.2) dependencies. But yes, it's
possible to describe a project on local hard disk ... which will most
probably cause an "unsupported" exception.
It is only IMHO safe to add a subproject in the same multiproject in
gradle (not in maven) as a dependency. Maven relies entirely on
artifacts (?).
I could imagine that the implementation would attempt to extract output
artifact's GAV from the identified project and use that.
-S.
> best regards && looking forward to try the feature,
>
> -mbien
>
>
>>
>> -S.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
>> For additional commands, e-mail: dev-help@netbeans.apache.org
>>
>> For further information about the NetBeans mailing lists, visit:
>> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>>
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Project dependencies - add/remove ability
Posted by Michael Bien <mb...@gmail.com>.
On 01.12.23 08:19, Svata Dedic wrote:
> On 01. 12. 23 3:29, Michael Bien wrote:
>> And why is it not sufficient to (mvn) install the dependency into the
>> local repo, and then depend on it the regular way?
>>
>
> The API is just an abstraction implemented by project system. So with
> Maven project, it will be implemented as an edit into pom.xml. For
> gradle, build.gradle will be changed. Similar to current (private)
> Dependency API query, which just extracts project dependencies from
> pom/build.gradle and presents it. No parallel build system or
> additional metadata.
thanks for clarification Svata - I hoped it would work like that, I just
wanted to be sure.
Sounds like a useful feature in some situations. The impl details will
be interesting though, since I am sure someone will try to add a maven
project to a gradle project or vise versa.
best regards && looking forward to try the feature,
-mbien
>
> -S.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Project dependencies - add/remove ability
Posted by Svata Dedic <ga...@volny.cz>.
On 01. 12. 23 3:29, Michael Bien wrote:
> And why is it not sufficient to (mvn) install the dependency into the
> local repo, and then depend on it the regular way?
>
The API is just an abstraction implemented by project system. So with
Maven project, it will be implemented as an edit into pom.xml. For
gradle, build.gradle will be changed. Similar to current (private)
Dependency API query, which just extracts project dependencies from
pom/build.gradle and presents it. No parallel build system or additional
metadata.
-S.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Project dependencies - add/remove ability
Posted by Michael Bien <mb...@gmail.com>.
Hi Svata,
lets say project A sets a dependency on project B and project
dependencies are an IDE feature. How will project A build outside of the
IDE?
And why is it not sufficient to (mvn) install the dependency into the
local repo, and then depend on it the regular way?
I don't doubt that this can be useful but NB has a long tradition of not
creating IDE dependent project structures. Other IDEs became less
invasive too which makes picking an IDE no longer a team decision.
(I likely don't fully understand what the feature is since
refactoring+hints were also mentioned)
best regards,
michael
On 29.11.23 20:22, Svata Dedic wrote:
> Hi,
>
> I am working (finishing) on an extension of (still private) Project
> Dependency API that would allow to *add* dependency to a project (and
> also remove). This is a change that will span both Maven and Gradle
> modules, so I thought I'll put a note in advance, but PR will be
> landing shortly in Github.
>
> The reason is the ability to add technology libraries, i.e. add
> monitoring to an application, add JDBC driver etc, which all requires
> (among other things) to add a set of artifact(s) to a project. I would
> like to have that ability for our company product, primarily, but the
> API to do _basic_ dependency changes in both major supported build
> systems. Only basic operations (add, remove) and only projects that
> use some artifact repositories are considered (maven, gradle, possibly
> mx), not ant-based projects.
>
> In order to get a reasonable API outcome, I decided to work jointly
> with Refactoring API and LSP API: the changes are best described as
> text changes to project files - and in case of our LSP client
> use-case, perform them. And rectoring API allows to commit() them in
> NetBeans case. Note that while *java* refactoring API describes text
> modifications in its result, the generic refactoring API does not. It
> does not seem beneficial to invent yet another text-changes
> description now, when LSP APIs are here, and basically do what's
> needed (even for Java).
>
> Gradle implementation: I do not want to pollute basic gradle modules
> more, so I decided to create a small java.gradle.dependencies module;
> I am thinking about migrating the dependencies part out of java.gradle
> module to the new one, which should clean up java.gradle module a bit.
>
> Maven implementation: For Maven, I'd work this into maven.hints. All
> dependencies are there except refactoring.api.
>
> Some more steps are IMHO needed until the dependency API is mature
> enough to get out of private:
> - the Scope is not designed well; it was anticipated (and that's why
> the API is not public) the introduction of modifications shed a little
> more light on it. I'll be working on that, keeping Micronaut support
> module as a early adopter (with necessary updates).
> - the Dependency children ought to be generated lazily; this gives the
> client the option not to traverse / expand subtrees for dependencies
> already seen. But some clients need exhaustive traversal.
> - improve the interaction with BOMs
>
> Cheers,
> -Svata
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Project dependencies - add/remove ability
Posted by Neil C Smith <ne...@apache.org>.
On Mon, 18 Dec 2023 at 12:25, Svata Dedic <sv...@gmail.com> wrote:
>
> Will check, thanks.
>
> Anything is possible until the API matures and goes public, tentative
> timeframe is NB23. I am willing to adapt 'early adopter' code, we now
> need for Oracle exensions, of course.
Restarting this conversation as requested at
https://github.com/apache/netbeans/pull/7651#discussion_r1711213464
> Re.: concerns about 'too specific abstraction': could you please continue the
> e-mail thread ideally with concrete ideas what is missing in the dependency/artifact
> spec structures -- or what is too specific (+ examples from rust/npm system where
> the proposed API falls short) ?
My concern with the API, in particular with it in the IDE cluster, is
that it just seems a little Maven-centric, mainly in the artifact
spec. That may just be a terminology thing - eg. use of groupID,
classifier, type, snapshot, etc. But potentially highlights issues
for use with other languages, where some things may not map cleanly or
at all. It also seems to miss the possibility to specify multiple
sources of dependencies, which might be of consideration in some
languages / projects?
I was suggesting you might look at a preexisting specification used
for identifying packages in a language neutral way, in particular
PackageURL.
It's already used in a bunch of places, including various SBOM specs.
You can see package URLs for Maven at the top of all Maven central
pages - eg. https://central.sonatype.com/artifact/org.netbeans.api/org-openide-util
- pkg:maven/org.netbeans.api/org-openide-util@RELEASE220
The spec is at https://github.com/package-url/purl-spec/blob/master/PURL-SPECIFICATION.rst
and there is information on particular type mappings at
https://github.com/package-url/purl-spec/blob/master/PURL-TYPES.rst
There's an MIT-licensed Java parser. I'm using a fork of that (must
remind myself why I modified) along with MiMa (Minimal Maven) to do
dependency resolution in an application, and I find it maps nicely.
I'd also be interested in seeing review from a Rust, PHP, etc.
perspective before the API becomes exposed.
Best wishes,
Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Project dependencies - add/remove ability
Posted by Svata Dedic <sv...@gmail.com>.
Will check, thanks.
Anything is possible until the API matures and goes public, tentative
timeframe is NB23. I am willing to adapt 'early adopter' code, we now
need for Oracle exensions, of course.
-S.
On 18. 12. 23 13:04, Neil C Smith wrote:
> On Wed, 29 Nov 2023 at 19:24, Svata Dedic <ga...@volny.cz> wrote:
>> I am working (finishing) on an extension of (still private) Project
>> Dependency API that would allow to *add* dependency to a project (and
>> also remove). This is a change that will span both Maven and Gradle
>> modules, so I thought I'll put a note in advance, but PR will be landing
>> shortly in Github.
>
> Just catching up on emails having been away.
>
> I like the sound of this API, but if the base API is to be in the IDE
> cluster, it might be nice to build around an artifact specification
> that is less tied to Gradle / Maven conventions and usable for other
> languages including JS and Rust?
>
> I've mentioned before in another context, but I do like PackageURL,
> and it's getting more and more adoption.
> https://github.com/package-url/purl-spec
>
> You can see PackageURLs for Maven at the top of every page in Maven
> Central search - eg. yellow highlighted at
> https://central.sonatype.com/artifact/org.netbeans.api/org-openide-util-lookup
>
> Best wishes,
>
> Neil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Project dependencies - add/remove ability
Posted by Neil C Smith <ne...@apache.org>.
On Wed, 29 Nov 2023 at 19:24, Svata Dedic <ga...@volny.cz> wrote:
> I am working (finishing) on an extension of (still private) Project
> Dependency API that would allow to *add* dependency to a project (and
> also remove). This is a change that will span both Maven and Gradle
> modules, so I thought I'll put a note in advance, but PR will be landing
> shortly in Github.
Just catching up on emails having been away.
I like the sound of this API, but if the base API is to be in the IDE
cluster, it might be nice to build around an artifact specification
that is less tied to Gradle / Maven conventions and usable for other
languages including JS and Rust?
I've mentioned before in another context, but I do like PackageURL,
and it's getting more and more adoption.
https://github.com/package-url/purl-spec
You can see PackageURLs for Maven at the top of every page in Maven
Central search - eg. yellow highlighted at
https://central.sonatype.com/artifact/org.netbeans.api/org-openide-util-lookup
Best wishes,
Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists