You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Michael Osipov <mi...@apache.org> on 2020/07/10 07:46:12 UTC
Second MRESOLVER-123
Folks,
MRESOLVER-123 [1] solves a very long standing issue in Resolver: Thread
concurrency in a single JVM while downloading artifacts/metadata. It has
been verified to work with several usecases provided by reporters. This
approach has one drawback: Since it is a global lock it introduces
somwhat of a lock contention. The slower your connection link the longer
the lock is held. Ideally, folks use repo managers in private networks
to solve this issue. Also, read/write reeentrancy seems to work fine
based on the logs provided. At the end I prefer build stability over
performance.
Please have a look and raise objects if you have any. If you don't hear
any I will merge next week and start release process of Resolver 1.5.0.
A midterm goal would be to provide a group id based locking which should
drastically reduce contention, but no promises here for an ETA. As for
the multi JVM safety, Takari local repository failed to provide a
working solution, project seems to be abandoned. I have considered to
investigate a Redisson-based approach for this. Though, this solely
depends on the community how necessary this really is.
Michael
[1] https://github.com/apache/maven-resolver/pull/65
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Second MRESOLVER-123
Posted by Michael Osipov <mi...@apache.org>.
Am 2020-07-11 um 15:52 schrieb Elliotte Rusty Harold:
> I don't think we can safely or should assume devs use private repo
> managers, or that those that do improve performance.
Why not? This is actually what we are recommending in tickets, SO and
our website.
> Instead of locking, would it be possible to implement some sort of
> queue system for artifact downloads or use asynchronous futures?
Everything is possible as long as someone implements it. The queue would
lock until the artifact has been consumed. What difference would that
make? The artifact download is already async in terms of that parallel
threads with tasks are spawned.
I do not intend to put anymore to time to something more sophisticated
like a queue for now, I did already invest a lot. This issue has plagued
many users.
I am open to new issues until someone can implement them in time.
Michael
> On Fri, Jul 10, 2020 at 3:46 AM Michael Osipov <mi...@apache.org> wrote:
>>
>> Folks,
>>
>> MRESOLVER-123 [1] solves a very long standing issue in Resolver: Thread
>> concurrency in a single JVM while downloading artifacts/metadata. It has
>> been verified to work with several usecases provided by reporters. This
>> approach has one drawback: Since it is a global lock it introduces
>> somwhat of a lock contention. The slower your connection link the longer
>> the lock is held. Ideally, folks use repo managers in private networks
>> to solve this issue. Also, read/write reeentrancy seems to work fine
>> based on the logs provided. At the end I prefer build stability over
>> performance.
>>
>> Please have a look and raise objects if you have any. If you don't hear
>> any I will merge next week and start release process of Resolver 1.5.0.
>>
>> A midterm goal would be to provide a group id based locking which should
>> drastically reduce contention, but no promises here for an ETA. As for
>> the multi JVM safety, Takari local repository failed to provide a
>> working solution, project seems to be abandoned. I have considered to
>> investigate a Redisson-based approach for this. Though, this solely
>> depends on the community how necessary this really is.
>>
>> Michael
>>
>> [1] https://github.com/apache/maven-resolver/pull/65
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Second MRESOLVER-123
Posted by Elliotte Rusty Harold <el...@ibiblio.org>.
I don't think we can safely or should assume devs use private repo
managers, or that those that do improve performance.
Instead of locking, would it be possible to implement some sort of
queue system for artifact downloads or use asynchronous futures?
On Fri, Jul 10, 2020 at 3:46 AM Michael Osipov <mi...@apache.org> wrote:
>
> Folks,
>
> MRESOLVER-123 [1] solves a very long standing issue in Resolver: Thread
> concurrency in a single JVM while downloading artifacts/metadata. It has
> been verified to work with several usecases provided by reporters. This
> approach has one drawback: Since it is a global lock it introduces
> somwhat of a lock contention. The slower your connection link the longer
> the lock is held. Ideally, folks use repo managers in private networks
> to solve this issue. Also, read/write reeentrancy seems to work fine
> based on the logs provided. At the end I prefer build stability over
> performance.
>
> Please have a look and raise objects if you have any. If you don't hear
> any I will merge next week and start release process of Resolver 1.5.0.
>
> A midterm goal would be to provide a group id based locking which should
> drastically reduce contention, but no promises here for an ETA. As for
> the multi JVM safety, Takari local repository failed to provide a
> working solution, project seems to be abandoned. I have considered to
> investigate a Redisson-based approach for this. Though, this solely
> depends on the community how necessary this really is.
>
> Michael
>
> [1] https://github.com/apache/maven-resolver/pull/65
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
--
Elliotte Rusty Harold
elharo@ibiblio.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org