You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Tamás Cservenák <ta...@cservenak.net> on 2022/12/07 11:59:48 UTC
Maven 4 incoming changes (w/ resolver)
Howdy,
To remove current impediments for the most anticipated Maven 4 feature, the
"consumer POM", there was some discussion between Guillaume and me, and I'd
like to share what we came up with.
Current problem with (already working) feature is that it relies on
FileTransformer API in resolver, but that API has several issues:
- is OOM prone, hence is deprecated without replacement since Resolver
1.8.0, see https://issues.apache.org/jira/browse/MRESOLVER-244
- another issue is artifact signing, as that API above renders
maven-pgp-plugin unusable (as what should be signed does not exists yet)
Hence, to overcome these limitations, we agreed to:
1) Provide a replacement for FileTransformer API that does the same but is
not OOM prone. This solves the OOM/deprecation side of things, but signing
is not solved with it. As we know the signing issue is that transformation
(with old but also with new API) happens "just in time" (just before
install or deploy) in Resolver, outside of Maven realm, so is even further
out of reach for any plugin. There were solutions like Slawek's sign plugin
(https://github.com/s4u/sign-maven-plugin) that works with FileTransformer
API, but from technical aspect (redoes what resolver already does) is
fragile and mixes concerns IMHO.
2) As signing is somewhat very similar to checksumming (calculated and
deployed by resolver on its own), we decided to take a stab on extending
resolver with "Signature API" that is extensible with similar design as
Checksum SPI (since 1.8.0 checksums have extensible SPI, one can easily
introduce algorithm like xxhash to resolver). The OOTB signing "provider"
will be based on existing maven-gpg-plugin code, essentially providing 100%
the same feature that today's Maven 3.x + maven-gpg-plugin provides. Right
now, the immediate goal is to achieve Maven 3 + m-gpg-p functionality (sign
goal).
Regarding "Signature SPI" (and provided GPG implementation) and it's use
cases:
- as signing would be from now provided by resolver, one could use "knobs" (
https://maven.apache.org/resolver/configuration.html) directly (ie
`-Daether...`) to enable and configure it. Its effect would be that
WHATEVER is uploaded thru Resolver Connector during the session, will be
signed as well.
- as next, the idea came up: we could modify m-gpg-p to provide
"configuration" (knobs) via session into resolver, so "Signature API" would
become active based on presence of these in session, hence, we would end up
with literally same functionality and behaviour as today maven3 + m-gpg-p
has (except that signing happens elsewhere), signing could be controlled by
plugin per module basis.
Related PRs for review (does not implement all this above, going "baby
steps"):
https://github.com/apache/maven-resolver/pull/227
https://github.com/apache/maven-resolver/pull/226
Example WIP PR to change Maven 4 to use new ArtifactTransform API:
https://github.com/apache/maven/pull/904 (Remains in Robert's spirit while
it does SAVE transformed POM, but into randomized temp file, and
immediately is deleted once install/deploy is done).
Any review, opinion and comment is welcome!
Thanks
T
Re: Maven 4 incoming changes (w/ resolver)
Posted by Christoph Läubrich <ma...@laeubi-soft.de>.
> But all these problems are gone if there is a P2 connector, right? :)
not really (even though I have not investigated in full details) as this
is more on the producing than on the consumer site and P2 sites are
usually packed as an own artifact than deployed to a folder/webserver
directly.
> In the first mail I explained the "fate" of m-gpg-p, it will possibly
> just "configure" things, and will not sign anything.
In the end one might just see it how it works out, but probably one
could have just an option to do
Path artifact = ...
Signature sig = myCoolNewSigningService.sign(artifact, 'pgp');
Probably just a dream... ;-)
Am 07.12.22 um 15:44 schrieb Tamás Cservenák:
> On Wed, Dec 7, 2022 at 3:18 PM Christoph Läubrich <ma...@laeubi-soft.de>
> wrote:
>
>> Another point is that currently we reuse signatures attached by m-pgp-p
>> for embedding them in other meta-data as well (p2 repository) so if only
>> the deploy will generate the signature and do not make it accessible we
>> would need to generate the signature again.
>>
>>
> But all these problems are gone if there is a P2 connector, right? :)
>
>
>> > so I guess a plugin could make use of signer as well...
>>
>>
>
>> I thought so because how would m-pgp-p make use of it then?
>>
>
> In the first mail I explained the "fate" of m-gpg-p, it will possibly just
> "configure" things, and will not sign anything.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Maven 4 incoming changes (w/ resolver)
Posted by Tamás Cservenák <ta...@cservenak.net>.
On Wed, Dec 7, 2022 at 3:18 PM Christoph Läubrich <ma...@laeubi-soft.de>
wrote:
> Another point is that currently we reuse signatures attached by m-pgp-p
> for embedding them in other meta-data as well (p2 repository) so if only
> the deploy will generate the signature and do not make it accessible we
> would need to generate the signature again.
>
>
But all these problems are gone if there is a P2 connector, right? :)
> > so I guess a plugin could make use of signer as well...
>
>
> I thought so because how would m-pgp-p make use of it then?
>
In the first mail I explained the "fate" of m-gpg-p, it will possibly just
"configure" things, and will not sign anything.
Re: Maven 4 incoming changes (w/ resolver)
Posted by Christoph Läubrich <ma...@laeubi-soft.de>.
> There are Java based solutions
I know, but if you "bake something in" it seems valid to ask for such a
solution in the first place.
> as signing is really tied to deployment
Not everything is deployed to a maven repository though... currently we
enhance the m-gpg-p with another one overriding methods and stuff but
this is even more bad as using an "partly official" SPI I think :-)
Another point is that currently we reuse signatures attached by m-pgp-p
for embedding them in other meta-data as well (p2 repository) so if only
the deploy will generate the signature and do not make it accessible we
would need to generate the signature again.
> so I guess a plugin could make use of signer as well...
I thought so because how would m-pgp-p make use of it then?
Am 07.12.22 um 15:05 schrieb Tamás Cservenák:
> Howdy,
>
> There are Java based solutions, and I agree about all, but our current goal
> is to provide 100% of what maven 3 + maven-gpg-plugin provides.
>
> Hopefully others will chime in and will provide "java native" SPI
> implementations as well.
>
> And no: "sign other artifacts without need of resolver" is not in our
> current use case, as signing is really tied to deployment,
> In other words, if you sign other artifacts, what will you do with them?
>
> But a bit milder answer is: according to this page
> https://maven.apache.org/resolver/api-compatibility.html SPI is part of
> "public API" (unlike transport for example), so I guess a plugin could make
> use of signer as well...
>
>
>
> Danke
> T
>
> On Wed, Dec 7, 2022 at 2:55 PM Christoph Läubrich <ma...@laeubi-soft.de>
> wrote:
>
>> AFAIK m-gpg-p requires native gpg executable to work and is quite slow.
>>
>> I think it would be better to have a java based default (e.g.
>> bouncy-castle based).
>>
>> Beside this as I recently came across this, will Signature SPI be
>> accessible by other mojos so the can sign other artifacts as well
>> without need of resolver? And will it be possible to konfirue this still
>> in pom.xml?
>>
>> I think your message indicates this but I'm not sure if I understand all
>> this "knobs" things...
>>
>>
>>
>> Am 07.12.22 um 12:59 schrieb Tamás Cservenák:
>>> Howdy,
>>>
>>> To remove current impediments for the most anticipated Maven 4 feature,
>> the
>>> "consumer POM", there was some discussion between Guillaume and me, and
>> I'd
>>> like to share what we came up with.
>>>
>>> Current problem with (already working) feature is that it relies on
>>> FileTransformer API in resolver, but that API has several issues:
>>> - is OOM prone, hence is deprecated without replacement since Resolver
>>> 1.8.0, see https://issues.apache.org/jira/browse/MRESOLVER-244
>>> - another issue is artifact signing, as that API above renders
>>> maven-pgp-plugin unusable (as what should be signed does not exists yet)
>>>
>>> Hence, to overcome these limitations, we agreed to:
>>>
>>> 1) Provide a replacement for FileTransformer API that does the same but
>> is
>>> not OOM prone. This solves the OOM/deprecation side of things, but
>> signing
>>> is not solved with it. As we know the signing issue is that
>> transformation
>>> (with old but also with new API) happens "just in time" (just before
>>> install or deploy) in Resolver, outside of Maven realm, so is even
>> further
>>> out of reach for any plugin. There were solutions like Slawek's sign
>> plugin
>>> (https://github.com/s4u/sign-maven-plugin) that works with
>> FileTransformer
>>> API, but from technical aspect (redoes what resolver already does) is
>>> fragile and mixes concerns IMHO.
>>>
>>> 2) As signing is somewhat very similar to checksumming (calculated and
>>> deployed by resolver on its own), we decided to take a stab on extending
>>> resolver with "Signature API" that is extensible with similar design as
>>> Checksum SPI (since 1.8.0 checksums have extensible SPI, one can easily
>>> introduce algorithm like xxhash to resolver). The OOTB signing "provider"
>>> will be based on existing maven-gpg-plugin code, essentially providing
>> 100%
>>> the same feature that today's Maven 3.x + maven-gpg-plugin provides.
>> Right
>>> now, the immediate goal is to achieve Maven 3 + m-gpg-p functionality
>> (sign
>>> goal).
>>>
>>> Regarding "Signature SPI" (and provided GPG implementation) and it's use
>>> cases:
>>> - as signing would be from now provided by resolver, one could use
>> "knobs" (
>>> https://maven.apache.org/resolver/configuration.html) directly (ie
>>> `-Daether...`) to enable and configure it. Its effect would be that
>>> WHATEVER is uploaded thru Resolver Connector during the session, will be
>>> signed as well.
>>> - as next, the idea came up: we could modify m-gpg-p to provide
>>> "configuration" (knobs) via session into resolver, so "Signature API"
>> would
>>> become active based on presence of these in session, hence, we would end
>> up
>>> with literally same functionality and behaviour as today maven3 + m-gpg-p
>>> has (except that signing happens elsewhere), signing could be controlled
>> by
>>> plugin per module basis.
>>>
>>> Related PRs for review (does not implement all this above, going "baby
>>> steps"):
>>> https://github.com/apache/maven-resolver/pull/227
>>> https://github.com/apache/maven-resolver/pull/226
>>>
>>> Example WIP PR to change Maven 4 to use new ArtifactTransform API:
>>> https://github.com/apache/maven/pull/904 (Remains in Robert's spirit
>> while
>>> it does SAVE transformed POM, but into randomized temp file, and
>>> immediately is deleted once install/deploy is done).
>>>
>>> Any review, opinion and comment is welcome!
>>>
>>> Thanks
>>> T
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Maven 4 incoming changes (w/ resolver)
Posted by Tamás Cservenák <ta...@cservenak.net>.
Howdy,
There are Java based solutions, and I agree about all, but our current goal
is to provide 100% of what maven 3 + maven-gpg-plugin provides.
Hopefully others will chime in and will provide "java native" SPI
implementations as well.
And no: "sign other artifacts without need of resolver" is not in our
current use case, as signing is really tied to deployment,
In other words, if you sign other artifacts, what will you do with them?
But a bit milder answer is: according to this page
https://maven.apache.org/resolver/api-compatibility.html SPI is part of
"public API" (unlike transport for example), so I guess a plugin could make
use of signer as well...
Danke
T
On Wed, Dec 7, 2022 at 2:55 PM Christoph Läubrich <ma...@laeubi-soft.de>
wrote:
> AFAIK m-gpg-p requires native gpg executable to work and is quite slow.
>
> I think it would be better to have a java based default (e.g.
> bouncy-castle based).
>
> Beside this as I recently came across this, will Signature SPI be
> accessible by other mojos so the can sign other artifacts as well
> without need of resolver? And will it be possible to konfirue this still
> in pom.xml?
>
> I think your message indicates this but I'm not sure if I understand all
> this "knobs" things...
>
>
>
> Am 07.12.22 um 12:59 schrieb Tamás Cservenák:
> > Howdy,
> >
> > To remove current impediments for the most anticipated Maven 4 feature,
> the
> > "consumer POM", there was some discussion between Guillaume and me, and
> I'd
> > like to share what we came up with.
> >
> > Current problem with (already working) feature is that it relies on
> > FileTransformer API in resolver, but that API has several issues:
> > - is OOM prone, hence is deprecated without replacement since Resolver
> > 1.8.0, see https://issues.apache.org/jira/browse/MRESOLVER-244
> > - another issue is artifact signing, as that API above renders
> > maven-pgp-plugin unusable (as what should be signed does not exists yet)
> >
> > Hence, to overcome these limitations, we agreed to:
> >
> > 1) Provide a replacement for FileTransformer API that does the same but
> is
> > not OOM prone. This solves the OOM/deprecation side of things, but
> signing
> > is not solved with it. As we know the signing issue is that
> transformation
> > (with old but also with new API) happens "just in time" (just before
> > install or deploy) in Resolver, outside of Maven realm, so is even
> further
> > out of reach for any plugin. There were solutions like Slawek's sign
> plugin
> > (https://github.com/s4u/sign-maven-plugin) that works with
> FileTransformer
> > API, but from technical aspect (redoes what resolver already does) is
> > fragile and mixes concerns IMHO.
> >
> > 2) As signing is somewhat very similar to checksumming (calculated and
> > deployed by resolver on its own), we decided to take a stab on extending
> > resolver with "Signature API" that is extensible with similar design as
> > Checksum SPI (since 1.8.0 checksums have extensible SPI, one can easily
> > introduce algorithm like xxhash to resolver). The OOTB signing "provider"
> > will be based on existing maven-gpg-plugin code, essentially providing
> 100%
> > the same feature that today's Maven 3.x + maven-gpg-plugin provides.
> Right
> > now, the immediate goal is to achieve Maven 3 + m-gpg-p functionality
> (sign
> > goal).
> >
> > Regarding "Signature SPI" (and provided GPG implementation) and it's use
> > cases:
> > - as signing would be from now provided by resolver, one could use
> "knobs" (
> > https://maven.apache.org/resolver/configuration.html) directly (ie
> > `-Daether...`) to enable and configure it. Its effect would be that
> > WHATEVER is uploaded thru Resolver Connector during the session, will be
> > signed as well.
> > - as next, the idea came up: we could modify m-gpg-p to provide
> > "configuration" (knobs) via session into resolver, so "Signature API"
> would
> > become active based on presence of these in session, hence, we would end
> up
> > with literally same functionality and behaviour as today maven3 + m-gpg-p
> > has (except that signing happens elsewhere), signing could be controlled
> by
> > plugin per module basis.
> >
> > Related PRs for review (does not implement all this above, going "baby
> > steps"):
> > https://github.com/apache/maven-resolver/pull/227
> > https://github.com/apache/maven-resolver/pull/226
> >
> > Example WIP PR to change Maven 4 to use new ArtifactTransform API:
> > https://github.com/apache/maven/pull/904 (Remains in Robert's spirit
> while
> > it does SAVE transformed POM, but into randomized temp file, and
> > immediately is deleted once install/deploy is done).
> >
> > Any review, opinion and comment is welcome!
> >
> > Thanks
> > T
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: Maven 4 incoming changes (w/ resolver)
Posted by Christoph Läubrich <ma...@laeubi-soft.de>.
AFAIK m-gpg-p requires native gpg executable to work and is quite slow.
I think it would be better to have a java based default (e.g.
bouncy-castle based).
Beside this as I recently came across this, will Signature SPI be
accessible by other mojos so the can sign other artifacts as well
without need of resolver? And will it be possible to konfirue this still
in pom.xml?
I think your message indicates this but I'm not sure if I understand all
this "knobs" things...
Am 07.12.22 um 12:59 schrieb Tamás Cservenák:
> Howdy,
>
> To remove current impediments for the most anticipated Maven 4 feature, the
> "consumer POM", there was some discussion between Guillaume and me, and I'd
> like to share what we came up with.
>
> Current problem with (already working) feature is that it relies on
> FileTransformer API in resolver, but that API has several issues:
> - is OOM prone, hence is deprecated without replacement since Resolver
> 1.8.0, see https://issues.apache.org/jira/browse/MRESOLVER-244
> - another issue is artifact signing, as that API above renders
> maven-pgp-plugin unusable (as what should be signed does not exists yet)
>
> Hence, to overcome these limitations, we agreed to:
>
> 1) Provide a replacement for FileTransformer API that does the same but is
> not OOM prone. This solves the OOM/deprecation side of things, but signing
> is not solved with it. As we know the signing issue is that transformation
> (with old but also with new API) happens "just in time" (just before
> install or deploy) in Resolver, outside of Maven realm, so is even further
> out of reach for any plugin. There were solutions like Slawek's sign plugin
> (https://github.com/s4u/sign-maven-plugin) that works with FileTransformer
> API, but from technical aspect (redoes what resolver already does) is
> fragile and mixes concerns IMHO.
>
> 2) As signing is somewhat very similar to checksumming (calculated and
> deployed by resolver on its own), we decided to take a stab on extending
> resolver with "Signature API" that is extensible with similar design as
> Checksum SPI (since 1.8.0 checksums have extensible SPI, one can easily
> introduce algorithm like xxhash to resolver). The OOTB signing "provider"
> will be based on existing maven-gpg-plugin code, essentially providing 100%
> the same feature that today's Maven 3.x + maven-gpg-plugin provides. Right
> now, the immediate goal is to achieve Maven 3 + m-gpg-p functionality (sign
> goal).
>
> Regarding "Signature SPI" (and provided GPG implementation) and it's use
> cases:
> - as signing would be from now provided by resolver, one could use "knobs" (
> https://maven.apache.org/resolver/configuration.html) directly (ie
> `-Daether...`) to enable and configure it. Its effect would be that
> WHATEVER is uploaded thru Resolver Connector during the session, will be
> signed as well.
> - as next, the idea came up: we could modify m-gpg-p to provide
> "configuration" (knobs) via session into resolver, so "Signature API" would
> become active based on presence of these in session, hence, we would end up
> with literally same functionality and behaviour as today maven3 + m-gpg-p
> has (except that signing happens elsewhere), signing could be controlled by
> plugin per module basis.
>
> Related PRs for review (does not implement all this above, going "baby
> steps"):
> https://github.com/apache/maven-resolver/pull/227
> https://github.com/apache/maven-resolver/pull/226
>
> Example WIP PR to change Maven 4 to use new ArtifactTransform API:
> https://github.com/apache/maven/pull/904 (Remains in Robert's spirit while
> it does SAVE transformed POM, but into randomized temp file, and
> immediately is deleted once install/deploy is done).
>
> Any review, opinion and comment is welcome!
>
> Thanks
> T
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org