You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Dan Tran <da...@gmail.com> on 2016/12/24 03:07:22 UTC

[VOTE] Releasing Wagon 2.11

Hi,

We have fixed 18 issues:
*https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318122&version=12333396
<https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318122&version=12333396>*

There are still issues left in JIRA:
*https://issues.apache.org/jira/issues/?jql=project%20%3D%20WAGON%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20created%20DESC%2C%20priority%20DESC
<https://issues.apache.org/jira/issues/?jql=project%20%3D%20WAGON%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20created%20DESC%2C%20priority%20DESC>*

Staging repository:
https://repository.apache.org/content/repositories/maven-1313
https://repository.apache.org/content/repositories/maven-
1313/org/apache/maven/wagon/wagon/2.11/wagon-2.11-source-release.zip

Source release checksum(s):
wagon-2.11-source-release.zip sha1: 98bc0b031880b175e144c55447a62000c58abfa5

Staging site:
*http://maven.apache.org/components/wagon-archives/wagon-LATEST/
<http://maven.apache.org/components/wagon-archives/wagon-LATEST/>*

Guide to testing staged releases:
https://maven.apache.org/guides/development/guide-testing-releases.html
<https://maven.apache.org/guides/development/guide-testing-releases.html>
Vote open for 72H + 48H to accomodate for holidays

[+1]
[0]
[-1]

Thanks,

The Maven Team

Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
>> Right now I see Wagon as a reference project for Maven 3.4.0:

I disagree. It relies on overriding management althought that correctly
is not supported when consumed.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Hervé BOUTEMY <he...@free.fr>.
Le mercredi 28 décembre 2016, 22:50:52 CET Robert Scholte a écrit :
> Which makes me wonder if this really is a fix. Wagon can be built with a
> wide range of Maven version covering a lot of years AND the
> maven-dependency-plugin shows what you would expect: junit is available
> with test-scope.
> 
> "That's always been that way."
> Well, apparently not. Maybe it was documented that way, but I expect that
> users did their dependency management which felt natural and which worked.
> 
> I'm afraid that this is exactly where Jason was warning you for.
> Right now I see Wagon as a reference project for Maven 3.4.0: it should
> still work without the pom adjustments, because its original setup made
> sense.
+1

> 
> Robert
> 
> On Wed, 28 Dec 2016 22:31:52 +0100, Christian Schulte <cs...@schulte.it>
> 
> wrote:
> > Am 12/28/16 um 22:21 schrieb Guillaume Boué:
> >> How come the tests compile fine with Maven 2.2.1, 3.0.5 and 3.3.9 then?
> > 
> > Because it does not come with MRESOLVER-9 fixed.
> > 
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
To be even more clear: It's bullshit you can override the management in
the POM when those overrides disappear transitively. Do not override
management and be done with it.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
Am 12/28/16 um 22:50 schrieb Robert Scholte:
> Which makes me wonder if this really is a fix. Wagon can be built with a  
> wide range of Maven version covering a lot of years AND the  
> maven-dependency-plugin shows what you would expect: junit is available  
> with test-scope.
> 
> "That's always been that way."
> Well, apparently not. Maybe it was documented that way, but I expect that  
> users did their dependency management which felt natural and which worked.

The scope from management has been overridden in the POM and the
resolver is overriding that override with the management. That's what
makes no sense but there is no way to make the resolver not override
those overrides. It cannot detect them. And that really has always been
that way.

> Right now I see Wagon as a reference project for Maven 3.4.0: it should  
> still work without the pom adjustments, because its original setup made  
> sense.

I am not the person designing the test scope to be non-transitive. I
just made it behave that way consistently.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mi...@apache.org>.
Am 2016-12-28 um 22:50 schrieb Robert Scholte:
> Which makes me wonder if this really is a fix. Wagon can be built with a
> wide range of Maven version covering a lot of years AND the
> maven-dependency-plugin shows what you would expect: junit is available
> with test-scope.
>
> "That's always been that way."
> Well, apparently not. Maybe it was documented that way, but I expect
> that users did their dependency management which felt natural and which
> worked.
>
> I'm afraid that this is exactly where Jason was warning you for.
> Right now I see Wagon as a reference project for Maven 3.4.0: it should
> still work without the pom adjustments, because its original setup made
> sense.

Just a side note: I clean up a lot of dependency issues with a separate 
ticket (WAGON-471) because deps were transitive but never declared in 
the POM. Right now, with Maven 3.4.0-SNAPSHOT, it does not work because 
resolution is different. I have obstained to test with Maven 
3.4.0-SNAPSHOT because I wanted to focus on the Wagon issues.


Michael

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
Am 12/28/16 um 22:50 schrieb Robert Scholte:
> "That's always been that way."
> Well, apparently not. Maybe it was documented that way, but I expect that  
> users did their dependency management which felt natural and which worked.

I created an issue in JIRA to track this:
<https://issues.apache.org/jira/browse/MNG-6141>

It has an example project attached demonstrating what is going on using
versions and not scopes or the optional flag. I think that's the real
issue at hand. It just happens that managing something transitive to
something non-transitive starts to work correctly in Maven 3.4+ and this
produces compilation errors in Wagon. No-one is noticing those
dependency management overrides are only in effect locally during
building the module declaring them an nowhere else.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Robert Scholte <rf...@apache.org>.
Which makes me wonder if this really is a fix. Wagon can be built with a  
wide range of Maven version covering a lot of years AND the  
maven-dependency-plugin shows what you would expect: junit is available  
with test-scope.

"That's always been that way."
Well, apparently not. Maybe it was documented that way, but I expect that  
users did their dependency management which felt natural and which worked.

I'm afraid that this is exactly where Jason was warning you for.
Right now I see Wagon as a reference project for Maven 3.4.0: it should  
still work without the pom adjustments, because its original setup made  
sense.

Robert

On Wed, 28 Dec 2016 22:31:52 +0100, Christian Schulte <cs...@schulte.it>  
wrote:

> Am 12/28/16 um 22:21 schrieb Guillaume Boué:
>>
>> How come the tests compile fine with Maven 2.2.1, 3.0.5 and 3.3.9 then?
>
> Because it does not come with MRESOLVER-9 fixed.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
Am 12/28/16 um 22:21 schrieb Guillaume Bou:
> 
> How come the tests compile fine with Maven 2.2.1, 3.0.5 and 3.3.9 then?

Because it does not come with MRESOLVER-9 fixed.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
Am 12/28/16 um 22:21 schrieb Guillaume Bou:
> This is the tree with Maven 3.3.9:
> 
> [DEBUG] org.apache.maven.wagon:wagon-file:jar:2.12-SNAPSHOT
> [DEBUG]    org.codehaus.plexus:plexus-utils:jar:3.0.24:compile
> [DEBUG]    org.slf4j:slf4j-simple:jar:1.7.19:test
> [DEBUG]       org.slf4j:slf4j-api:jar:1.7.19:test
> [DEBUG]    org.apache.maven.wagon:wagon-provider-api:jar:2.12-SNAPSHOT:compile
> [DEBUG]    org.apache.maven.wagon:wagon-provider-test:jar:2.12-SNAPSHOT:test
> [DEBUG]       org.codehaus.plexus:plexus-container-default:jar:1.5.5:test
> [DEBUG]          org.codehaus.plexus:plexus-classworlds:jar:2.2.2:test
> [DEBUG]          org.apache.xbean:xbean-reflect:jar:3.4:test
> [DEBUG]             log4j:log4j:jar:1.2.12:test
> [DEBUG]             commons-logging:commons-logging-api:jar:1.1:test
> [DEBUG]          com.google.collections:google-collections:jar:1.0:test
> [DEBUG]       org.easymock:easymock:jar:3.2:test
> [DEBUG]          cglib:cglib-nodep:jar:2.2.2:test
> [DEBUG]          org.objenesis:objenesis:jar:1.3:test
> [DEBUG]       org.eclipse.jetty.aggregate:jetty-all:jar:8.1.22.v20160922:test
> [DEBUG]          org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:test
> [DEBUG]       javax.servlet:javax.servlet-api:jar:3.0.1:test
> [DEBUG]       junit:junit:jar:4.11:test (scope managed from compile by org.apache.maven.wagon:wagon:2.12-SNAPSHOT)
> [DEBUG]          org.hamcrest:hamcrest-core:jar:1.3:test

You have overridden the test scope from management in the POM to
compile. Why would you expect Maven to override that from the management
transitively?

I mean this:

(scope managed from compile by org.apache.maven.wagon:wagon:2.12-SNAPSHOT)

That compile is the override in the POM. In the POM you told Maven to
ignore the test scope from management and override that with compile.
Why are you expecting Maven to ignore your override?


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
Am 12/29/16 um 03:19 schrieb Christian Schulte:
> Am 12/29/16 um 02:41 schrieb Christian Schulte:
>> Am 12/29/16 um 02:36 schrieb Christian Schulte:
>>> Am 12/29/16 um 00:41 schrieb Michael Osipov:
>>>> If this is how it should (I am neither pro nor cons) work, we should 
>>>> deprecate this element or at least put a big WARNING on it.
>>>
>>> We should spit out a big fat warning whenever someone overrides hers/his
>>> own management. It shouldn't even be possible to do so. That's not a
>>> feature :-)
>>
>> Ask a manager about that. Management is dominant. It should not be
>> possible to override that anywhere. The resolver should really override
>> direct dependencies with the management as well and not just transitive
>> ones. It currently does not only to not break this "feature".
> 
> Maybe more technically: Suppose the maven-plugins parent would manage
> versions of various components. Maven core, plexus archiver,
> plexus-utils, etc. This must be applied to all plugins using that parent
> no matter what a plugin declares itself. The author of that parent and
> the author of a plugin's POM are different persons. Maybe the parent can
> only be released by a PMC member and plugins can be released by
> committers etc. Allowing to override the management really is a bad idea.

So for model version 5.0.0 I would like to request to implement "final"
and maybe "override" semantics (for parents, if they still exist and for
mixins). There at least needs to be a way to express "final" declarations.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
Am 12/29/16 um 02:41 schrieb Christian Schulte:
> Am 12/29/16 um 02:36 schrieb Christian Schulte:
>> Am 12/29/16 um 00:41 schrieb Michael Osipov:
>>> If this is how it should (I am neither pro nor cons) work, we should 
>>> deprecate this element or at least put a big WARNING on it.
>>
>> We should spit out a big fat warning whenever someone overrides hers/his
>> own management. It shouldn't even be possible to do so. That's not a
>> feature :-)
> 
> Ask a manager about that. Management is dominant. It should not be
> possible to override that anywhere. The resolver should really override
> direct dependencies with the management as well and not just transitive
> ones. It currently does not only to not break this "feature".

Maybe more technically: Suppose the maven-plugins parent would manage
versions of various components. Maven core, plexus archiver,
plexus-utils, etc. This must be applied to all plugins using that parent
no matter what a plugin declares itself. The author of that parent and
the author of a plugin's POM are different persons. Maybe the parent can
only be released by a PMC member and plugins can be released by
committers etc. Allowing to override the management really is a bad idea.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
Am 12/29/16 um 02:36 schrieb Christian Schulte:
> Am 12/29/16 um 00:41 schrieb Michael Osipov:
>> If this is how it should (I am neither pro nor cons) work, we should 
>> deprecate this element or at least put a big WARNING on it.
> 
> We should spit out a big fat warning whenever someone overrides hers/his
> own management. It shouldn't even be possible to do so. That's not a
> feature :-)

Ask a manager about that. Management is dominant. It should not be
possible to override that anywhere. The resolver should really override
direct dependencies with the management as well and not just transitive
ones. It currently does not only to not break this "feature".


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
Am 12/29/16 um 00:41 schrieb Michael Osipov:
> If this is how it should (I am neither pro nor cons) work, we should 
> deprecate this element or at least put a big WARNING on it.

We should spit out a big fat warning whenever someone overrides hers/his
own management. It shouldn't even be possible to do so. That's not a
feature :-)


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
Am 12/29/16 um 02:12 schrieb Christian Schulte:
> Am 12/29/16 um 00:41 schrieb Michael Osipov:
>> Am 2016-12-28 um 22:51 schrieb Christian Schulte:
>>> I just pushed a fix for this. I could also have made that transitive
>>> dependency a direct one, where it is used, and could have left the scope
>>> management in.
>>>
>>> "Dependency management overrides are not transitive."
>>
>> Just checked this commit. It ultimately means that <scope> is useless in 
>> dependency management unless you know all transitive dependencies in 
>> order not to break them.
> 
> Maven and the resolver have a different way of applying management. In
> Maven, the management is used as a source of default values to be used
> when those values are not provided otherwise (POM authoring - direct -
> you can declare everything yourself). The resolver uses it as a global
> override (POM consumption - transitive - you cannot do anything about
> those consumed POMs but need to able to do so -> management). It both
> makes sense somehow. What is inconsistent is to override management in a
> module and then consume that module transitively in another module as if
> it were something outside the reactor. All of the issues presented on
> dev@ so far were due to this (overridden management).
> 
>> If this is how it should (I am neither pro nor cons) work, we should 
>> deprecate this element or at least put a big WARNING on it.
> 
> We need to distinguish between the management applied when authoring a
> POM and the management applied when consuming a POM. There is two
> stories about it. In a multi-module project you are the author so you
> just use the dependency management to get rid of redundant dependency
> declarations (versions, scopes, etc. all in one place and left out
> everywhere).  When consuming a project, you want to control the
> dependencies the same way so your management is used to apply your
> declarations onto the consumed projects (global override).
> 
> Your POMs - you are the author in full control of everything:
> Do not override anything you have put into the management anywhere
> because that is what the resolver does as soon as a module is consumed
> transitively - either in the same reactor, or somewhere else.

Adding to this: The resolver does not know anything about the reactor or
modules. It resolves dependencies no matter what. Maybe that way: There
are no transitive modules and no special handling in resolving a
transitive module or a transitive dependency. That's where the
difference between direct and transitive is coming from.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
Am 12/29/16 um 00:41 schrieb Michael Osipov:
> Am 2016-12-28 um 22:51 schrieb Christian Schulte:
>> I just pushed a fix for this. I could also have made that transitive
>> dependency a direct one, where it is used, and could have left the scope
>> management in.
>>
>> "Dependency management overrides are not transitive."
> 
> Just checked this commit. It ultimately means that <scope> is useless in 
> dependency management unless you know all transitive dependencies in 
> order not to break them.

Maven and the resolver have a different way of applying management. In
Maven, the management is used as a source of default values to be used
when those values are not provided otherwise (POM authoring - direct -
you can declare everything yourself). The resolver uses it as a global
override (POM consumption - transitive - you cannot do anything about
those consumed POMs but need to able to do so -> management). It both
makes sense somehow. What is inconsistent is to override management in a
module and then consume that module transitively in another module as if
it were something outside the reactor. All of the issues presented on
dev@ so far were due to this (overridden management).

> If this is how it should (I am neither pro nor cons) work, we should 
> deprecate this element or at least put a big WARNING on it.

We need to distinguish between the management applied when authoring a
POM and the management applied when consuming a POM. There is two
stories about it. In a multi-module project you are the author so you
just use the dependency management to get rid of redundant dependency
declarations (versions, scopes, etc. all in one place and left out
everywhere).  When consuming a project, you want to control the
dependencies the same way so your management is used to apply your
declarations onto the consumed projects (global override).

Your POMs - you are the author in full control of everything:
Do not override anything you have put into the management anywhere
because that is what the resolver does as soon as a module is consumed
transitively - either in the same reactor, or somewhere else.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mi...@apache.org>.
Am 2016-12-28 um 22:51 schrieb Christian Schulte:
> I just pushed a fix for this. I could also have made that transitive
> dependency a direct one, where it is used, and could have left the scope
> management in.
>
> "Dependency management overrides are not transitive."

Just checked this commit. It ultimately means that <scope> is useless in 
dependency management unless you know all transitive dependencies in 
order not to break them.

If this is how it should (I am neither pro nor cons) work, we should 
deprecate this element or at least put a big WARNING on it.

Michael


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
I just pushed a fix for this. I could also have made that transitive
dependency a direct one, where it is used, and could have left the scope
management in.

"Dependency management overrides are not transitive."


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
Am 12/28/16 um 22:21 schrieb Guillaume Bou:
> 
> How come the tests compile fine with Maven 2.2.1, 3.0.5 and 3.3.9 then?
> 

I'd say this is the root cause of nearly all issues we are having a hard
time fixing and shipping. It does not make sense to compare some recent
behaviour to some former behaviour when that former behaviour is a
result of patches trying to make things behave in a way users expect it.
Using Maven 2.x or 3.x as the specification of how things /should/
behave when even the documentation says things really are meant to be
different than what we ship is rediculous, IMHO. What we now have on
master is consistent with itself and matches the documentation. That's
how Maven 2.0.0 should have behaved from day one but did not. The
documentation in this area has not been changed for more than nearly a
decade. Maven never behaved that way. I am not kean on endless
discussions about things. I do not have enough experience with GIT. Just
create a branch of current master and then reset master to 3.3.9 so that
I can squash and merge issues for the next release one by one into
master and get master into a state it would pass a release vote. That's
what I was refering to with "release branch". Passes a release build,
passes all core and plugin ITs and passes a release vote. Master clearly
does not pass a release vote. Hopefully what will pass a release vote is
not just coloured output.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Guillaume Boué <gb...@apache.org>.
Le 28/12/2016  21:49, Christian Schulte a crit :
> Am 12/28/16 um 21:34 schrieb Guillaume Bou:
>> I have the same results as Herv, both on Windows and Ubuntu. This is
>> what I have with Maven 3.3.9:
>>
>> - Windows 10 64bit, OpenJDK 1.8.0_102, Test failure: Timeout in
>> HugeFileDownloadTest (perhaps the timeout should be increased?)
>> - Ubuntu 16.04 32bit, OpenJDK 1.8.0_111, Maven 3.3.9: All OK
>> - Ubuntu 16.04 32bit, OpenJDK 1.7.0_95, Maven 3.3.9: All OK
>>
>> With Maven 3.4.0-SNAPSHOT (65960ac18c8121648fe163612375a2ba5f4535c2), I
>> have compilation errors in the test class FileWagonTest (both on Windows
>> and Ubuntu)
>>
>> [INFO] -------------------------------------------------------------
>> [ERROR] COMPILATION ERROR :
>> [INFO] -------------------------------------------------------------
>> [ERROR] /home/guillaume/workspace-maven/maven-wagon/wagon-providers/wagon-file/src/test/java/org/apache/maven/wagon/providers/file/FileWagonTest.java:[37,8] cannot access junit.framework.TestCase
>>     class file for junit.framework.TestCase not found
>> [ERROR] /home/guillaume/workspace-maven/maven-wagon/wagon-providers/wagon-file/src/test/java/org/apache/maven/wagon/providers/file/FileWagonTest.java:[48,52] cannot find symbol
>>     symbol:   method getName()
>>     location: class org.apache.maven.wagon.providers.file.FileWagonTest
>>
>> The JUnit dependency isn't getting pulled. This is the scenario at hand:
>>
>> - wagon-file has as parent wagon-providers which has a dependency on
>> wagon-provider-test with scope test.
>> - wagon-provider-test has a compile scoped dependency on junit
>> (<scope>compile</scope> is explicitly written).
>> - wagon-provider-test has as parent wagon, which has a test scoped
>> dependency management on junit.
>>
>> What would be the problem?
> Overriding the dependency management is only possible in the direct
> dependencies (in the POM). When resolving a dependency, those overrides
> are beeing ignored and the management gets applied. That's always been
> that way. If your test classes need junit, you should declare junit in
> the POM and not rely on it to be there transitively. The dependency
> plugin's analyze goal would warn about this for the main artifact
> (src/main) but not for the test artifact (src/test).
>

How come the tests compile fine with Maven 2.2.1, 3.0.5 and 3.3.9 then?

This is the tree with Maven 3.3.9:

[DEBUG] org.apache.maven.wagon:wagon-file:jar:2.12-SNAPSHOT
[DEBUG]    org.codehaus.plexus:plexus-utils:jar:3.0.24:compile
[DEBUG]    org.slf4j:slf4j-simple:jar:1.7.19:test
[DEBUG]       org.slf4j:slf4j-api:jar:1.7.19:test
[DEBUG]    org.apache.maven.wagon:wagon-provider-api:jar:2.12-SNAPSHOT:compile
[DEBUG]    org.apache.maven.wagon:wagon-provider-test:jar:2.12-SNAPSHOT:test
[DEBUG]       org.codehaus.plexus:plexus-container-default:jar:1.5.5:test
[DEBUG]          org.codehaus.plexus:plexus-classworlds:jar:2.2.2:test
[DEBUG]          org.apache.xbean:xbean-reflect:jar:3.4:test
[DEBUG]             log4j:log4j:jar:1.2.12:test
[DEBUG]             commons-logging:commons-logging-api:jar:1.1:test
[DEBUG]          com.google.collections:google-collections:jar:1.0:test
[DEBUG]       org.easymock:easymock:jar:3.2:test
[DEBUG]          cglib:cglib-nodep:jar:2.2.2:test
[DEBUG]          org.objenesis:objenesis:jar:1.3:test
[DEBUG]       org.eclipse.jetty.aggregate:jetty-all:jar:8.1.22.v20160922:test
[DEBUG]          org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:test
[DEBUG]       javax.servlet:javax.servlet-api:jar:3.0.1:test
[DEBUG]       junit:junit:jar:4.11:test (scope managed from compile by org.apache.maven.wagon:wagon:2.12-SNAPSHOT)
[DEBUG]          org.hamcrest:hamcrest-core:jar:1.3:test



>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>


---
L'absence de virus dans ce courrier lectronique a t vrifie par le logiciel antivirus Avast.
https://www.avast.com/antivirus


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Christian Schulte <cs...@schulte.it>.
Am 12/28/16 um 21:34 schrieb Guillaume Bou:
> I have the same results as Herv, both on Windows and Ubuntu. This is 
> what I have with Maven 3.3.9:
> 
> - Windows 10 64bit, OpenJDK 1.8.0_102, Test failure: Timeout in 
> HugeFileDownloadTest (perhaps the timeout should be increased?)
> - Ubuntu 16.04 32bit, OpenJDK 1.8.0_111, Maven 3.3.9: All OK
> - Ubuntu 16.04 32bit, OpenJDK 1.7.0_95, Maven 3.3.9: All OK
> 
> With Maven 3.4.0-SNAPSHOT (65960ac18c8121648fe163612375a2ba5f4535c2), I 
> have compilation errors in the test class FileWagonTest (both on Windows 
> and Ubuntu)
> 
> [INFO] -------------------------------------------------------------
> [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR] /home/guillaume/workspace-maven/maven-wagon/wagon-providers/wagon-file/src/test/java/org/apache/maven/wagon/providers/file/FileWagonTest.java:[37,8] cannot access junit.framework.TestCase
>    class file for junit.framework.TestCase not found
> [ERROR] /home/guillaume/workspace-maven/maven-wagon/wagon-providers/wagon-file/src/test/java/org/apache/maven/wagon/providers/file/FileWagonTest.java:[48,52] cannot find symbol
>    symbol:   method getName()
>    location: class org.apache.maven.wagon.providers.file.FileWagonTest
> 
> The JUnit dependency isn't getting pulled. This is the scenario at hand:
> 
> - wagon-file has as parent wagon-providers which has a dependency on 
> wagon-provider-test with scope test.
> - wagon-provider-test has a compile scoped dependency on junit 
> (<scope>compile</scope> is explicitly written).
> - wagon-provider-test has as parent wagon, which has a test scoped 
> dependency management on junit.
> 
> What would be the problem?

Overriding the dependency management is only possible in the direct
dependencies (in the POM). When resolving a dependency, those overrides
are beeing ignored and the management gets applied. That's always been
that way. If your test classes need junit, you should declare junit in
the POM and not rely on it to be there transitively. The dependency
plugin's analyze goal would warn about this for the main artifact
(src/main) but not for the test artifact (src/test).



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mi...@apache.org>.
I have made some concluding tests on my machine at work because my home 
machine is so fast that you don't see a difference if you change something.
My tests are two-fold: improve HugeFileDownloadTest and manually 
reconstruct with pure Jetty and HttpClient (same versions).

First of all sparse files work with pure Java within tens of milliseconds:

     Path tempDirectory = Files.createTempDirectory("jetty");
     final ByteBuffer buf = ByteBuffer.allocate(4).putInt(2);
     buf.rewind();

     final OpenOption[] options = { StandardOpenOption.WRITE, 
StandardOpenOption.CREATE_NEW , StandardOpenOption.SPARSE };
     final Path hugeFile = tempDirectory.resolve("hugefile.txt");

     try (final SeekableByteChannel channel = 
Files.newByteChannel(hugeFile, options);) {
         channel.position(HUGE_FILE_SIZE);
         channel.write(buf);
     }

Handmade tests are always faster than Wagon tests (all times approx.):
Wagon:
Create file: 80 s to 90 s
Download w/ content length: 3,5 min (optimized)
Download w/ chunks: 2,5 min


Handmade:
Create file: 60 s to 70 s (with NIO.2 and without sparse)
Download w/ content length: 2,5 min
Download w/ chunks: < 1,5 min

There is a similar issue: http://stackoverflow.com/q/9031311/696632

I started to play with buffer sizes in Wagon, the anonymous inner class 
and DefaultServlet. It seems to me that Jetty's IO class
buffer size of 64 KiB -- which is used by the DefaultServlet -- makes 
downloads slower in this use case.
I have turned the anonymous inner HttpServlet to 8 KiB as well as 
Wagon's internal buffer to 8 KiB (default 4 KiB has been tried too).
Unfortunately, the results sometimes vary but the smaller buffer most of 
the time wins against Jetty's IO. I will commit a changed version to 
that test and raise to 800 seconds. This should do it, in my opinion. 
There is nothing we can do anymore with spinning disks and the 
non-available sparse files on Windows. Note that RandomAccessFile 
creates them on Unix.

Additionally, I tried to download with curl from Jetty, it is a bit 
faster than the handmade version. Personally, there must be some 
misconfiguration in the HttpClient if it performs slower than my cheap 
hand-written code.

To finalize this. I ran the tests on an old Pentium 4 2,4 GHz, 2 GiB RAM 
with FreeBSD 11.0-STABLE:
2017-01-02T21:39:53.542 [main] INFO 
org.apache.maven.wagon.providers.http.HugeFileDownloadTest - Creating 
test file
2017-01-02T21:39:53.549 [main] INFO 
org.apache.maven.wagon.providers.http.HugeFileDownloadTest - Test file 
created
2017-01-02T21:39:53.552 [main] INFO org.eclipse.jetty.server.Server - 
jetty-8.1.22.v20160922
2017-01-02T21:39:53.556 [main] INFO 
org.eclipse.jetty.server.AbstractConnector - Started 
SelectChannelConnector@0.0.0.0:20435
http://localhost:20435 - Session: Opened
2017-01-02T21:39:53.569 [main] INFO 
org.apache.maven.wagon.providers.http.HugeFileDownloadTest - Fetching 
'hugefile.txt' in chunks
2017-01-02T21:44:51.315 [main] INFO 
org.apache.maven.wagon.providers.http.HugeFileDownloadTest - The file 
was successfully fetched
http://localhost:20435 - Session: Disconnecting
http://localhost:20435 - Session: Disconnected
2017-01-02T21:44:51.329 [main] INFO 
org.eclipse.jetty.server.handler.ContextHandler - stopped 
o.e.j.s.ServletContextHandler{/,file:/usr/home/mosipov/Projekte/maven-wagon/wagon-providers/wagon-http/target/}
2017-01-02T21:44:51.868 [main] INFO 
org.apache.maven.wagon.providers.http.HugeFileDownloadTest - Creating 
test file
2017-01-02T21:44:51.893 [main] INFO 
org.apache.maven.wagon.providers.http.HugeFileDownloadTest - Test file 
created
2017-01-02T21:44:51.894 [main] INFO org.eclipse.jetty.server.Server - 
jetty-8.1.22.v20160922
2017-01-02T21:44:51.905 [main] INFO 
org.eclipse.jetty.server.AbstractConnector - Started 
SelectChannelConnector@0.0.0.0:23105
http://localhost:23105 - Session: Opened
2017-01-02T21:44:52.019 [main] INFO 
org.apache.maven.wagon.providers.http.HugeFileDownloadTest - Fetching 
'hugefile.txt' with content length
2017-01-02T21:48:03.017 [main] INFO 
org.apache.maven.wagon.providers.http.HugeFileDownloadTest - The file 
was successfully fetched
http://localhost:23105 - Session: Disconnecting
http://localhost:23105 - Session: Disconnected
2017-01-02T21:48:03.030 [main] INFO 
org.eclipse.jetty.server.handler.ContextHandler - stopped 
o.e.j.s.ServletContextHandler{/,file:/usr/home/mosipov/Projekte/maven-wagon/wagon-providers/wagon-http/target/}

For such an old hardware, this is still decent.

Michael

Am 2016-12-31 um 21:20 schrieb Guillaume Boué:
> Thanks for the analysis! Agree with dropping fsutil then; I committed
> the addition of the logs with it just so that we can have concrete
> numbers for comparison (the last build indicates there was no permission
> issues in using it, otherwise it wouldn't have timed out but just failed
> to find the file). NIO.2 requires Java 7 though, so looks like a good
> reason to try and update to it (and Jetty 9 in the process, as Olivier
> suggested), but maybe not for this version. That also makes me wonder if
> the NIO.2 SPARSE option doesn't implictly require special privileges on
> Windows (like it does for creating symbolic links).
>
> In any case, the result of the Jenkins build with regard to the commit
> are here
> https://builds.apache.org/job/maven-wagon-jetty8-windows/org.apache.maven.wagon$wagon-http/3/consoleFull.
> The 4 Go test file was created instantly (30 milliseconds), and it timed
> out performing the first download; so the creation of the file isn't an
> issue. I was following the growing size of the download in the target
> directory as it happened and things didn't appear to work slowly.
>
> The tests for wagon-http started at 11:47:28 and timed out after
> 11:53:04 when doing the first download for HugeFileDownloadTest; this
> makes up for the 400 seconds (roughly 6-7 minutes) of the Surefire
> timeout. Another breakdown is seen here (oddly without
> HugeFileDownloadTest):
>
> https://builds.apache.org/job/maven-wagon-jetty8-windows/org.apache.maven.wagon$wagon-http/3/testReport/org.apache.maven.wagon.providers.http/
>
>
> All of this is actually caused by HttpsWagonTest,
> HttpsWagonPreemptiveTest and HugeFileDownloadTest taking each a bit more
> than 2 minutes. I don't think we can cut
> <https://builds.apache.org/job/maven-wagon-jetty8-windows/org.apache.maven.wagon$wagon-http/3/testReport/org.apache.maven.wagon.providers.http/HttpsWagonTest/><https://builds.apache.org/job/maven-wagon-jetty8-windows/org.apache.maven.wagon$wagon-http/3/testReport/org.apache.maven.wagon.providers.http/HttpsWagonPreemptiveTest/>HugeFileDownloadTest
> below 2 minutes (with the sparse file created instantly, I see no room
> for improvements here), so focus should actually be made on those 2
> other tests.
>
> Guillaume
>
>
> Le 31/12/2016 à 16:51, Michael Osipov a écrit :
>> Am 2016-12-31 um 12:13 schrieb Guillaume Boué:
>>> Do you think I can add a dummy log before the creation of the test file
>>> (and add the timestamps to the logs of wagon-http), to see how much time
>>> it takes on the Windows Server 2012? I'd like to see the breakdown of
>>> what takes time on the Jenkins machine, perhaps there is nothing we can
>>> do better (except increase the timeout to 500 or so...).
>>
>> Let me share new findings before I get to the answers. I tested this
>> branch on my machine at work which is part of an Active Directory
>> domain, I am local admin. It is a regular, 2-core i5 machine with 12
>> GiB RAM and medium-size HDD:
>>
>> 1. fsutil failed, domain policy requires elevated permissions to run
>> fsutil in command prompt and you even see it because stderr/stdout of
>> fsutil is swalled. I tried manually.
>> 2. I have even raised the timeout to 800 seconds, it still fails.
>>
>> Given this, I'd wouldn't really use fsutil. Rather NIO2 with
>> StandardOption.SPARSE. This might work faster.
>>
>> To your questions: go head and use SLF4J, it is already available.
>> Print the log messages and you'll see how much creation really consumes.
>> I'd test this ony my machine at home and work.
>>
>> Michael
>>
>>
>>> Le 30/12/2016 à 22:46, Michael Osipov a écrit :
>>>> I just pushed another commit to the branch with your changes.
>>>> The job does not really work on Windows [1], it simply takes too long
>>>> to complete on the Windows Server 2012.
>>>>
>>>> [1]
>>>> https://builds.apache.org/job/maven-wagon-jetty8-windows/2/org.apache.maven.wagon$wagon-http/console
>>>>
>>>>
>>>>
>>>> Am 2016-12-29 um 19:10 schrieb Guillaume Boué:
>>>>> I have a Toshiba SSHD (MQ02ABD100H). I think the issue is that the
>>>>> Java
>>>>> code should create a sparse file to have things faster. Using
>>>>> setLength
>>>>> on a random access file probably does it depending on the OS and
>>>>> type of
>>>>> drive, but it isn't creating one in my situation.
>>>>>
>>>>> When run on Ubuntu, creating the test file is done practically
>>>>> instantly
>>>>> whereas it takes 60 seconds on my Windows machine. Downloading takes
>>>>> around 30 seconds in Ubuntu, whereas it takes 150 seconds in Windows.
>>>>>
>>>>> I changed the method creating the test file (makeHugeFile) to:
>>>>>
>>>>>         if ( Os.isFamily( Os.FAMILY_WINDOWS ) )
>>>>>         {
>>>>>             Process p = new ProcessBuilder( "fsutil", "file",
>>>>> "createnew", hugeFile.getAbsolutePath(),
>>>>>                                             String.valueOf(
>>>>> HUGE_FILE_SIZE ) ).start();
>>>>>             p.waitFor();
>>>>>         }
>>>>>         else
>>>>>         {
>>>>>             RandomAccessFile ra = new RandomAccessFile(
>>>>> hugeFile.getPath(), "rw" );
>>>>>             ra.setLength( HUGE_FILE_SIZE + 1 );
>>>>>             ra.seek( HUGE_FILE_SIZE );
>>>>>             ra.write( 1 );
>>>>>             ra.close();
>>>>>         }
>>>>>
>>>>> This matches with the timings I get on Ubuntu (both for the
>>>>> creation of
>>>>> the file, and the downloading via Wagon). I ran the build several
>>>>> times
>>>>> with this change without any timeout issues. Can you test this
>>>>> change on
>>>>> your side?
>>>>>
>>>>> Side-note: I added "ra.close();" in the code above, because the random
>>>>> access file wasn't closed otherwise, and then the file wasn't getting
>>>>> deleted properly.
>>>>>
>>>>> Guillaume
>>>>>
>>>>> Le 29/12/2016 à 15:50, Michael Osipov a écrit :
>>>>>> Am 2016-12-29 um 12:24 schrieb Guillaume Boué:
>>>>>>> I ran them at least 10 times, and there was the timeout issue each
>>>>>>> time.
>>>>>>> Yes, the timeout is Surefire waiting for the forked VM. I applied
>>>>>>> the
>>>>>>> patch (I had done similar tries also) but I still have the issue on
>>>>>>> Windows 10.
>>>>>>>
>>>>>>> You'll find the logs attached. It seems that the download is really
>>>>>>> advancing (inside target, I can see the file huge<numbers>txt slowly
>>>>>>> growing), but is just taking a long time. Additionally, the test
>>>>>>> class
>>>>>>> HugeFileDownloadTest is composed of 2 tests, downloading the same
>>>>>>> file 2
>>>>>>> times with different parameters: from the logs, it seems one of
>>>>>>> them is
>>>>>>> succeeding (taking 140 seconds of the 400 in total), and then it
>>>>>>> times
>>>>>>> out performing the second.
>>>>>>>
>>>>>>> Also, part of the issue is probably with the time it takes to
>>>>>>> create the
>>>>>>> 4 Go test file that is later downloaded through Wagon (maybe it
>>>>>>> should
>>>>>>> be done by calling the fsutil command when the test runs on
>>>>>>> Windows).
>>>>>>> I'll try to do more timings when running the tests with Ubuntu
>>>>>>> and see
>>>>>>> where the difference is.
>>>>>>
>>>>>> Thanks for the analysis. I think the cheapest improvement we can
>>>>>> do is
>>>>>> to create the file only once (setUp(), tearDown(). It is created for
>>>>>> each test again. I do not think that this is really necessary.
>>>>>>
>>>>>> What type of disks do you have? This test passes on my machine within
>>>>>> 70 seconds on a Samsung SSD.
>>>>>>
>>>>>> I will apply the patch to the branch because those fixes are
>>>>>> necessary
>>>>>> anyway.
>>>>>>
>>>>>> Please keep me updated.
>>>>>>
>>>>>> Michael
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>>>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>>>>
>>>>>
>>>>>
>>>>> ---
>>>>> L'absence de virus dans ce courrier électronique a été vérifiée par le
>>>>> logiciel antivirus Avast.
>>>>> https://www.avast.com/antivirus
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>>
>>>
>>>
>>> ---
>>> L'absence de virus dans ce courrier électronique a été vérifiée par le
>>> logiciel antivirus Avast.
>>> https://www.avast.com/antivirus
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>
>>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>
>
>
> ---
> L'absence de virus dans ce courrier électronique a été vérifiée par le
> logiciel antivirus Avast.
> https://www.avast.com/antivirus
>



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Guillaume Boué <gb...@apache.org>.
Thanks for the analysis! Agree with dropping fsutil then; I committed 
the addition of the logs with it just so that we can have concrete 
numbers for comparison (the last build indicates there was no permission 
issues in using it, otherwise it wouldn't have timed out but just failed 
to find the file). NIO.2 requires Java 7 though, so looks like a good 
reason to try and update to it (and Jetty 9 in the process, as Olivier 
suggested), but maybe not for this version. That also makes me wonder if 
the NIO.2 SPARSE option doesn't implictly require special privileges on 
Windows (like it does for creating symbolic links).

In any case, the result of the Jenkins build with regard to the commit 
are here 
https://builds.apache.org/job/maven-wagon-jetty8-windows/org.apache.maven.wagon$wagon-http/3/consoleFull. 
The 4 Go test file was created instantly (30 milliseconds), and it timed 
out performing the first download; so the creation of the file isn't an 
issue. I was following the growing size of the download in the target 
directory as it happened and things didn't appear to work slowly.

The tests for wagon-http started at 11:47:28 and timed out after 
11:53:04 when doing the first download for HugeFileDownloadTest; this 
makes up for the 400 seconds (roughly 6-7 minutes) of the Surefire 
timeout. Another breakdown is seen here (oddly without 
HugeFileDownloadTest):

https://builds.apache.org/job/maven-wagon-jetty8-windows/org.apache.maven.wagon$wagon-http/3/testReport/org.apache.maven.wagon.providers.http/

All of this is actually caused by HttpsWagonTest, 
HttpsWagonPreemptiveTest and HugeFileDownloadTest taking each a bit more 
than 2 minutes. I don't think we can cut 
<https://builds.apache.org/job/maven-wagon-jetty8-windows/org.apache.maven.wagon$wagon-http/3/testReport/org.apache.maven.wagon.providers.http/HttpsWagonTest/><https://builds.apache.org/job/maven-wagon-jetty8-windows/org.apache.maven.wagon$wagon-http/3/testReport/org.apache.maven.wagon.providers.http/HttpsWagonPreemptiveTest/>HugeFileDownloadTest 
below 2 minutes (with the sparse file created instantly, I see no room 
for improvements here), so focus should actually be made on those 2 
other tests.

Guillaume


Le 31/12/2016  16:51, Michael Osipov a crit :
> Am 2016-12-31 um 12:13 schrieb Guillaume Bou:
>> Do you think I can add a dummy log before the creation of the test file
>> (and add the timestamps to the logs of wagon-http), to see how much time
>> it takes on the Windows Server 2012? I'd like to see the breakdown of
>> what takes time on the Jenkins machine, perhaps there is nothing we can
>> do better (except increase the timeout to 500 or so...).
>
> Let me share new findings before I get to the answers. I tested this 
> branch on my machine at work which is part of an Active Directory 
> domain, I am local admin. It is a regular, 2-core i5 machine with 12 
> GiB RAM and medium-size HDD:
>
> 1. fsutil failed, domain policy requires elevated permissions to run 
> fsutil in command prompt and you even see it because stderr/stdout of 
> fsutil is swalled. I tried manually.
> 2. I have even raised the timeout to 800 seconds, it still fails.
>
> Given this, I'd wouldn't really use fsutil. Rather NIO2 with 
> StandardOption.SPARSE. This might work faster.
>
> To your questions: go head and use SLF4J, it is already available. 
> Print the log messages and you'll see how much creation really consumes.
> I'd test this ony my machine at home and work.
>
> Michael
>
>
>> Le 30/12/2016  22:46, Michael Osipov a crit :
>>> I just pushed another commit to the branch with your changes.
>>> The job does not really work on Windows [1], it simply takes too long
>>> to complete on the Windows Server 2012.
>>>
>>> [1]
>>> https://builds.apache.org/job/maven-wagon-jetty8-windows/2/org.apache.maven.wagon$wagon-http/console 
>>>
>>>
>>>
>>> Am 2016-12-29 um 19:10 schrieb Guillaume Bou:
>>>> I have a Toshiba SSHD (MQ02ABD100H). I think the issue is that the 
>>>> Java
>>>> code should create a sparse file to have things faster. Using 
>>>> setLength
>>>> on a random access file probably does it depending on the OS and 
>>>> type of
>>>> drive, but it isn't creating one in my situation.
>>>>
>>>> When run on Ubuntu, creating the test file is done practically 
>>>> instantly
>>>> whereas it takes 60 seconds on my Windows machine. Downloading takes
>>>> around 30 seconds in Ubuntu, whereas it takes 150 seconds in Windows.
>>>>
>>>> I changed the method creating the test file (makeHugeFile) to:
>>>>
>>>>         if ( Os.isFamily( Os.FAMILY_WINDOWS ) )
>>>>         {
>>>>             Process p = new ProcessBuilder( "fsutil", "file",
>>>> "createnew", hugeFile.getAbsolutePath(),
>>>>                                             String.valueOf(
>>>> HUGE_FILE_SIZE ) ).start();
>>>>             p.waitFor();
>>>>         }
>>>>         else
>>>>         {
>>>>             RandomAccessFile ra = new RandomAccessFile(
>>>> hugeFile.getPath(), "rw" );
>>>>             ra.setLength( HUGE_FILE_SIZE + 1 );
>>>>             ra.seek( HUGE_FILE_SIZE );
>>>>             ra.write( 1 );
>>>>             ra.close();
>>>>         }
>>>>
>>>> This matches with the timings I get on Ubuntu (both for the 
>>>> creation of
>>>> the file, and the downloading via Wagon). I ran the build several 
>>>> times
>>>> with this change without any timeout issues. Can you test this 
>>>> change on
>>>> your side?
>>>>
>>>> Side-note: I added "ra.close();" in the code above, because the random
>>>> access file wasn't closed otherwise, and then the file wasn't getting
>>>> deleted properly.
>>>>
>>>> Guillaume
>>>>
>>>> Le 29/12/2016  15:50, Michael Osipov a crit :
>>>>> Am 2016-12-29 um 12:24 schrieb Guillaume Bou:
>>>>>> I ran them at least 10 times, and there was the timeout issue each
>>>>>> time.
>>>>>> Yes, the timeout is Surefire waiting for the forked VM. I applied 
>>>>>> the
>>>>>> patch (I had done similar tries also) but I still have the issue on
>>>>>> Windows 10.
>>>>>>
>>>>>> You'll find the logs attached. It seems that the download is really
>>>>>> advancing (inside target, I can see the file huge<numbers>txt slowly
>>>>>> growing), but is just taking a long time. Additionally, the test 
>>>>>> class
>>>>>> HugeFileDownloadTest is composed of 2 tests, downloading the same
>>>>>> file 2
>>>>>> times with different parameters: from the logs, it seems one of
>>>>>> them is
>>>>>> succeeding (taking 140 seconds of the 400 in total), and then it 
>>>>>> times
>>>>>> out performing the second.
>>>>>>
>>>>>> Also, part of the issue is probably with the time it takes to
>>>>>> create the
>>>>>> 4 Go test file that is later downloaded through Wagon (maybe it 
>>>>>> should
>>>>>> be done by calling the fsutil command when the test runs on 
>>>>>> Windows).
>>>>>> I'll try to do more timings when running the tests with Ubuntu 
>>>>>> and see
>>>>>> where the difference is.
>>>>>
>>>>> Thanks for the analysis. I think the cheapest improvement we can 
>>>>> do is
>>>>> to create the file only once (setUp(), tearDown(). It is created for
>>>>> each test again. I do not think that this is really necessary.
>>>>>
>>>>> What type of disks do you have? This test passes on my machine within
>>>>> 70 seconds on a Samsung SSD.
>>>>>
>>>>> I will apply the patch to the branch because those fixes are 
>>>>> necessary
>>>>> anyway.
>>>>>
>>>>> Please keep me updated.
>>>>>
>>>>> Michael
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>>>
>>>>
>>>>
>>>> ---
>>>> L'absence de virus dans ce courrier lectronique a t vrifie par le
>>>> logiciel antivirus Avast.
>>>> https://www.avast.com/antivirus
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>
>>
>>
>> ---
>> L'absence de virus dans ce courrier lectronique a t vrifie par le
>> logiciel antivirus Avast.
>> https://www.avast.com/antivirus
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>



---
L'absence de virus dans ce courrier lectronique a t vrifie par le logiciel antivirus Avast.
https://www.avast.com/antivirus

Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mi...@apache.org>.
Am 2016-12-31 um 12:13 schrieb Guillaume Boué:
> Do you think I can add a dummy log before the creation of the test file
> (and add the timestamps to the logs of wagon-http), to see how much time
> it takes on the Windows Server 2012? I'd like to see the breakdown of
> what takes time on the Jenkins machine, perhaps there is nothing we can
> do better (except increase the timeout to 500 or so...).

Let me share new findings before I get to the answers. I tested this 
branch on my machine at work which is part of an Active Directory 
domain, I am local admin. It is a regular, 2-core i5 machine with 12 GiB 
RAM and medium-size HDD:

1. fsutil failed, domain policy requires elevated permissions to run 
fsutil in command prompt and you even see it because stderr/stdout of 
fsutil is swalled. I tried manually.
2. I have even raised the timeout to 800 seconds, it still fails.

Given this, I'd wouldn't really use fsutil. Rather NIO2 with 
StandardOption.SPARSE. This might work faster.

To your questions: go head and use SLF4J, it is already available. Print 
the log messages and you'll see how much creation really consumes.
I'd test this ony my machine at home and work.

Michael


> Le 30/12/2016 à 22:46, Michael Osipov a écrit :
>> I just pushed another commit to the branch with your changes.
>> The job does not really work on Windows [1], it simply takes too long
>> to complete on the Windows Server 2012.
>>
>> [1]
>> https://builds.apache.org/job/maven-wagon-jetty8-windows/2/org.apache.maven.wagon$wagon-http/console
>>
>>
>> Am 2016-12-29 um 19:10 schrieb Guillaume Boué:
>>> I have a Toshiba SSHD (MQ02ABD100H). I think the issue is that the Java
>>> code should create a sparse file to have things faster. Using setLength
>>> on a random access file probably does it depending on the OS and type of
>>> drive, but it isn't creating one in my situation.
>>>
>>> When run on Ubuntu, creating the test file is done practically instantly
>>> whereas it takes 60 seconds on my Windows machine. Downloading takes
>>> around 30 seconds in Ubuntu, whereas it takes 150 seconds in Windows.
>>>
>>> I changed the method creating the test file (makeHugeFile) to:
>>>
>>>         if ( Os.isFamily( Os.FAMILY_WINDOWS ) )
>>>         {
>>>             Process p = new ProcessBuilder( "fsutil", "file",
>>> "createnew", hugeFile.getAbsolutePath(),
>>>                                             String.valueOf(
>>> HUGE_FILE_SIZE ) ).start();
>>>             p.waitFor();
>>>         }
>>>         else
>>>         {
>>>             RandomAccessFile ra = new RandomAccessFile(
>>> hugeFile.getPath(), "rw" );
>>>             ra.setLength( HUGE_FILE_SIZE + 1 );
>>>             ra.seek( HUGE_FILE_SIZE );
>>>             ra.write( 1 );
>>>             ra.close();
>>>         }
>>>
>>> This matches with the timings I get on Ubuntu (both for the creation of
>>> the file, and the downloading via Wagon). I ran the build several times
>>> with this change without any timeout issues. Can you test this change on
>>> your side?
>>>
>>> Side-note: I added "ra.close();" in the code above, because the random
>>> access file wasn't closed otherwise, and then the file wasn't getting
>>> deleted properly.
>>>
>>> Guillaume
>>>
>>> Le 29/12/2016 à 15:50, Michael Osipov a écrit :
>>>> Am 2016-12-29 um 12:24 schrieb Guillaume Boué:
>>>>> I ran them at least 10 times, and there was the timeout issue each
>>>>> time.
>>>>> Yes, the timeout is Surefire waiting for the forked VM. I applied the
>>>>> patch (I had done similar tries also) but I still have the issue on
>>>>> Windows 10.
>>>>>
>>>>> You'll find the logs attached. It seems that the download is really
>>>>> advancing (inside target, I can see the file huge<numbers>txt slowly
>>>>> growing), but is just taking a long time. Additionally, the test class
>>>>> HugeFileDownloadTest is composed of 2 tests, downloading the same
>>>>> file 2
>>>>> times with different parameters: from the logs, it seems one of
>>>>> them is
>>>>> succeeding (taking 140 seconds of the 400 in total), and then it times
>>>>> out performing the second.
>>>>>
>>>>> Also, part of the issue is probably with the time it takes to
>>>>> create the
>>>>> 4 Go test file that is later downloaded through Wagon (maybe it should
>>>>> be done by calling the fsutil command when the test runs on Windows).
>>>>> I'll try to do more timings when running the tests with Ubuntu and see
>>>>> where the difference is.
>>>>
>>>> Thanks for the analysis. I think the cheapest improvement we can do is
>>>> to create the file only once (setUp(), tearDown(). It is created for
>>>> each test again. I do not think that this is really necessary.
>>>>
>>>> What type of disks do you have? This test passes on my machine within
>>>> 70 seconds on a Samsung SSD.
>>>>
>>>> I will apply the patch to the branch because those fixes are necessary
>>>> anyway.
>>>>
>>>> Please keep me updated.
>>>>
>>>> Michael
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>>
>>>
>>>
>>> ---
>>> L'absence de virus dans ce courrier électronique a été vérifiée par le
>>> logiciel antivirus Avast.
>>> https://www.avast.com/antivirus
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>
>>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>
>
> ---
> L'absence de virus dans ce courrier électronique a été vérifiée par le
> logiciel antivirus Avast.
> https://www.avast.com/antivirus
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Guillaume Boué <gb...@apache.org>.
Do you think I can add a dummy log before the creation of the test file 
(and add the timestamps to the logs of wagon-http), to see how much time 
it takes on the Windows Server 2012? I'd like to see the breakdown of 
what takes time on the Jenkins machine, perhaps there is nothing we can 
do better (except increase the timeout to 500 or so...).

Guillaume


Le 30/12/2016  22:46, Michael Osipov a crit :
> I just pushed another commit to the branch with your changes.
> The job does not really work on Windows [1], it simply takes too long 
> to complete on the Windows Server 2012.
>
> [1] 
> https://builds.apache.org/job/maven-wagon-jetty8-windows/2/org.apache.maven.wagon$wagon-http/console
>
> Am 2016-12-29 um 19:10 schrieb Guillaume Bou:
>> I have a Toshiba SSHD (MQ02ABD100H). I think the issue is that the Java
>> code should create a sparse file to have things faster. Using setLength
>> on a random access file probably does it depending on the OS and type of
>> drive, but it isn't creating one in my situation.
>>
>> When run on Ubuntu, creating the test file is done practically instantly
>> whereas it takes 60 seconds on my Windows machine. Downloading takes
>> around 30 seconds in Ubuntu, whereas it takes 150 seconds in Windows.
>>
>> I changed the method creating the test file (makeHugeFile) to:
>>
>>         if ( Os.isFamily( Os.FAMILY_WINDOWS ) )
>>         {
>>             Process p = new ProcessBuilder( "fsutil", "file",
>> "createnew", hugeFile.getAbsolutePath(),
>>                                             String.valueOf(
>> HUGE_FILE_SIZE ) ).start();
>>             p.waitFor();
>>         }
>>         else
>>         {
>>             RandomAccessFile ra = new RandomAccessFile(
>> hugeFile.getPath(), "rw" );
>>             ra.setLength( HUGE_FILE_SIZE + 1 );
>>             ra.seek( HUGE_FILE_SIZE );
>>             ra.write( 1 );
>>             ra.close();
>>         }
>>
>> This matches with the timings I get on Ubuntu (both for the creation of
>> the file, and the downloading via Wagon). I ran the build several times
>> with this change without any timeout issues. Can you test this change on
>> your side?
>>
>> Side-note: I added "ra.close();" in the code above, because the random
>> access file wasn't closed otherwise, and then the file wasn't getting
>> deleted properly.
>>
>> Guillaume
>>
>> Le 29/12/2016  15:50, Michael Osipov a crit :
>>> Am 2016-12-29 um 12:24 schrieb Guillaume Bou:
>>>> I ran them at least 10 times, and there was the timeout issue each 
>>>> time.
>>>> Yes, the timeout is Surefire waiting for the forked VM. I applied the
>>>> patch (I had done similar tries also) but I still have the issue on
>>>> Windows 10.
>>>>
>>>> You'll find the logs attached. It seems that the download is really
>>>> advancing (inside target, I can see the file huge<numbers>txt slowly
>>>> growing), but is just taking a long time. Additionally, the test class
>>>> HugeFileDownloadTest is composed of 2 tests, downloading the same 
>>>> file 2
>>>> times with different parameters: from the logs, it seems one of 
>>>> them is
>>>> succeeding (taking 140 seconds of the 400 in total), and then it times
>>>> out performing the second.
>>>>
>>>> Also, part of the issue is probably with the time it takes to 
>>>> create the
>>>> 4 Go test file that is later downloaded through Wagon (maybe it should
>>>> be done by calling the fsutil command when the test runs on Windows).
>>>> I'll try to do more timings when running the tests with Ubuntu and see
>>>> where the difference is.
>>>
>>> Thanks for the analysis. I think the cheapest improvement we can do is
>>> to create the file only once (setUp(), tearDown(). It is created for
>>> each test again. I do not think that this is really necessary.
>>>
>>> What type of disks do you have? This test passes on my machine within
>>> 70 seconds on a Samsung SSD.
>>>
>>> I will apply the patch to the branch because those fixes are necessary
>>> anyway.
>>>
>>> Please keep me updated.
>>>
>>> Michael
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>
>>
>>
>> ---
>> L'absence de virus dans ce courrier lectronique a t vrifie par le
>> logiciel antivirus Avast.
>> https://www.avast.com/antivirus
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>


---
L'absence de virus dans ce courrier lectronique a t vrifie par le logiciel antivirus Avast.
https://www.avast.com/antivirus


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mi...@apache.org>.
I just pushed another commit to the branch with your changes.
The job does not really work on Windows [1], it simply takes too long to 
complete on the Windows Server 2012.

[1] 
https://builds.apache.org/job/maven-wagon-jetty8-windows/2/org.apache.maven.wagon$wagon-http/console

Am 2016-12-29 um 19:10 schrieb Guillaume Boué:
> I have a Toshiba SSHD (MQ02ABD100H). I think the issue is that the Java
> code should create a sparse file to have things faster. Using setLength
> on a random access file probably does it depending on the OS and type of
> drive, but it isn't creating one in my situation.
>
> When run on Ubuntu, creating the test file is done practically instantly
> whereas it takes 60 seconds on my Windows machine. Downloading takes
> around 30 seconds in Ubuntu, whereas it takes 150 seconds in Windows.
>
> I changed the method creating the test file (makeHugeFile) to:
>
>         if ( Os.isFamily( Os.FAMILY_WINDOWS ) )
>         {
>             Process p = new ProcessBuilder( "fsutil", "file",
> "createnew", hugeFile.getAbsolutePath(),
>                                             String.valueOf(
> HUGE_FILE_SIZE ) ).start();
>             p.waitFor();
>         }
>         else
>         {
>             RandomAccessFile ra = new RandomAccessFile(
> hugeFile.getPath(), "rw" );
>             ra.setLength( HUGE_FILE_SIZE + 1 );
>             ra.seek( HUGE_FILE_SIZE );
>             ra.write( 1 );
>             ra.close();
>         }
>
> This matches with the timings I get on Ubuntu (both for the creation of
> the file, and the downloading via Wagon). I ran the build several times
> with this change without any timeout issues. Can you test this change on
> your side?
>
> Side-note: I added "ra.close();" in the code above, because the random
> access file wasn't closed otherwise, and then the file wasn't getting
> deleted properly.
>
> Guillaume
>
> Le 29/12/2016 à 15:50, Michael Osipov a écrit :
>> Am 2016-12-29 um 12:24 schrieb Guillaume Boué:
>>> I ran them at least 10 times, and there was the timeout issue each time.
>>> Yes, the timeout is Surefire waiting for the forked VM. I applied the
>>> patch (I had done similar tries also) but I still have the issue on
>>> Windows 10.
>>>
>>> You'll find the logs attached. It seems that the download is really
>>> advancing (inside target, I can see the file huge<numbers>txt slowly
>>> growing), but is just taking a long time. Additionally, the test class
>>> HugeFileDownloadTest is composed of 2 tests, downloading the same file 2
>>> times with different parameters: from the logs, it seems one of them is
>>> succeeding (taking 140 seconds of the 400 in total), and then it times
>>> out performing the second.
>>>
>>> Also, part of the issue is probably with the time it takes to create the
>>> 4 Go test file that is later downloaded through Wagon (maybe it should
>>> be done by calling the fsutil command when the test runs on Windows).
>>> I'll try to do more timings when running the tests with Ubuntu and see
>>> where the difference is.
>>
>> Thanks for the analysis. I think the cheapest improvement we can do is
>> to create the file only once (setUp(), tearDown(). It is created for
>> each test again. I do not think that this is really necessary.
>>
>> What type of disks do you have? This test passes on my machine within
>> 70 seconds on a Samsung SSD.
>>
>> I will apply the patch to the branch because those fixes are necessary
>> anyway.
>>
>> Please keep me updated.
>>
>> Michael
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>
>
> ---
> L'absence de virus dans ce courrier électronique a été vérifiée par le
> logiciel antivirus Avast.
> https://www.avast.com/antivirus
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mi...@apache.org>.
I have tested that code and read a bit about sparse files.

Tests first: I had on average 65 s on Windows 10, it is down now to 50 
to 55 seconds. The huge problem is that we cannot use @BeforeClass to 
create the file once because JUnit pre-annotations does not support hat.
To the sparse stuff: according the some Stackoverflow answers, 
RandomAccessFile does create a sparse file automatically on Linux, 
Solaris, etc. where as on Windows you need to set a bit in native code 
for NTFS. The only options for sparse files is either 'fsutil' or NIO2. 
I'd stick to fsutil the way it is right now.

Do you see any room for improvement without huge effort? Otherwise I'd 
go on and commit your improvement.
I still do not undertand why it runs so slow on your Windows box.

Michael

Am 2016-12-29 um 19:10 schrieb Guillaume Boué:
> I have a Toshiba SSHD (MQ02ABD100H). I think the issue is that the Java
> code should create a sparse file to have things faster. Using setLength
> on a random access file probably does it depending on the OS and type of
> drive, but it isn't creating one in my situation.
>
> When run on Ubuntu, creating the test file is done practically instantly
> whereas it takes 60 seconds on my Windows machine. Downloading takes
> around 30 seconds in Ubuntu, whereas it takes 150 seconds in Windows.
>
> I changed the method creating the test file (makeHugeFile) to:
>
>         if ( Os.isFamily( Os.FAMILY_WINDOWS ) )
>         {
>             Process p = new ProcessBuilder( "fsutil", "file",
> "createnew", hugeFile.getAbsolutePath(),
>                                             String.valueOf(
> HUGE_FILE_SIZE ) ).start();
>             p.waitFor();
>         }
>         else
>         {
>             RandomAccessFile ra = new RandomAccessFile(
> hugeFile.getPath(), "rw" );
>             ra.setLength( HUGE_FILE_SIZE + 1 );
>             ra.seek( HUGE_FILE_SIZE );
>             ra.write( 1 );
>             ra.close();
>         }
>
> This matches with the timings I get on Ubuntu (both for the creation of
> the file, and the downloading via Wagon). I ran the build several times
> with this change without any timeout issues. Can you test this change on
> your side?
>
> Side-note: I added "ra.close();" in the code above, because the random
> access file wasn't closed otherwise, and then the file wasn't getting
> deleted properly.
>
> Guillaume
>
> Le 29/12/2016 à 15:50, Michael Osipov a écrit :
>> Am 2016-12-29 um 12:24 schrieb Guillaume Boué:
>>> I ran them at least 10 times, and there was the timeout issue each time.
>>> Yes, the timeout is Surefire waiting for the forked VM. I applied the
>>> patch (I had done similar tries also) but I still have the issue on
>>> Windows 10.
>>>
>>> You'll find the logs attached. It seems that the download is really
>>> advancing (inside target, I can see the file huge<numbers>txt slowly
>>> growing), but is just taking a long time. Additionally, the test class
>>> HugeFileDownloadTest is composed of 2 tests, downloading the same file 2
>>> times with different parameters: from the logs, it seems one of them is
>>> succeeding (taking 140 seconds of the 400 in total), and then it times
>>> out performing the second.
>>>
>>> Also, part of the issue is probably with the time it takes to create the
>>> 4 Go test file that is later downloaded through Wagon (maybe it should
>>> be done by calling the fsutil command when the test runs on Windows).
>>> I'll try to do more timings when running the tests with Ubuntu and see
>>> where the difference is.
>>
>> Thanks for the analysis. I think the cheapest improvement we can do is
>> to create the file only once (setUp(), tearDown(). It is created for
>> each test again. I do not think that this is really necessary.
>>
>> What type of disks do you have? This test passes on my machine within
>> 70 seconds on a Samsung SSD.
>>
>> I will apply the patch to the branch because those fixes are necessary
>> anyway.
>>
>> Please keep me updated.
>>
>> Michael
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>
>
> ---
> L'absence de virus dans ce courrier électronique a été vérifiée par le
> logiciel antivirus Avast.
> https://www.avast.com/antivirus
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Guillaume Boué <gb...@apache.org>.
I have a Toshiba SSHD (MQ02ABD100H). I think the issue is that the Java 
code should create a sparse file to have things faster. Using setLength 
on a random access file probably does it depending on the OS and type of 
drive, but it isn't creating one in my situation.

When run on Ubuntu, creating the test file is done practically instantly 
whereas it takes 60 seconds on my Windows machine. Downloading takes 
around 30 seconds in Ubuntu, whereas it takes 150 seconds in Windows.

I changed the method creating the test file (makeHugeFile) to:

         if ( Os.isFamily( Os.FAMILY_WINDOWS ) )
         {
             Process p = new ProcessBuilder( "fsutil", "file", "createnew", hugeFile.getAbsolutePath(),
                                             String.valueOf( HUGE_FILE_SIZE ) ).start();
             p.waitFor();
         }
         else
         {
             RandomAccessFile ra = new RandomAccessFile( hugeFile.getPath(), "rw" );
             ra.setLength( HUGE_FILE_SIZE + 1 );
             ra.seek( HUGE_FILE_SIZE );
             ra.write( 1 );
             ra.close();
         }

This matches with the timings I get on Ubuntu (both for the creation of 
the file, and the downloading via Wagon). I ran the build several times 
with this change without any timeout issues. Can you test this change on 
your side?

Side-note: I added "ra.close();" in the code above, because the random 
access file wasn't closed otherwise, and then the file wasn't getting 
deleted properly.

Guillaume

Le 29/12/2016  15:50, Michael Osipov a crit :
> Am 2016-12-29 um 12:24 schrieb Guillaume Bou:
>> I ran them at least 10 times, and there was the timeout issue each time.
>> Yes, the timeout is Surefire waiting for the forked VM. I applied the
>> patch (I had done similar tries also) but I still have the issue on
>> Windows 10.
>>
>> You'll find the logs attached. It seems that the download is really
>> advancing (inside target, I can see the file huge<numbers>txt slowly
>> growing), but is just taking a long time. Additionally, the test class
>> HugeFileDownloadTest is composed of 2 tests, downloading the same file 2
>> times with different parameters: from the logs, it seems one of them is
>> succeeding (taking 140 seconds of the 400 in total), and then it times
>> out performing the second.
>>
>> Also, part of the issue is probably with the time it takes to create the
>> 4 Go test file that is later downloaded through Wagon (maybe it should
>> be done by calling the fsutil command when the test runs on Windows).
>> I'll try to do more timings when running the tests with Ubuntu and see
>> where the difference is.
>
> Thanks for the analysis. I think the cheapest improvement we can do is 
> to create the file only once (setUp(), tearDown(). It is created for 
> each test again. I do not think that this is really necessary.
>
> What type of disks do you have? This test passes on my machine within 
> 70 seconds on a Samsung SSD.
>
> I will apply the patch to the branch because those fixes are necessary 
> anyway.
>
> Please keep me updated.
>
> Michael
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>


---
L'absence de virus dans ce courrier lectronique a t vrifie par le logiciel antivirus Avast.
https://www.avast.com/antivirus


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mi...@apache.org>.
Am 2016-12-29 um 12:24 schrieb Guillaume Boué:
> I ran them at least 10 times, and there was the timeout issue each time.
> Yes, the timeout is Surefire waiting for the forked VM. I applied the
> patch (I had done similar tries also) but I still have the issue on
> Windows 10.
>
> You'll find the logs attached. It seems that the download is really
> advancing (inside target, I can see the file huge<numbers>txt slowly
> growing), but is just taking a long time. Additionally, the test class
> HugeFileDownloadTest is composed of 2 tests, downloading the same file 2
> times with different parameters: from the logs, it seems one of them is
> succeeding (taking 140 seconds of the 400 in total), and then it times
> out performing the second.
>
> Also, part of the issue is probably with the time it takes to create the
> 4 Go test file that is later downloaded through Wagon (maybe it should
> be done by calling the fsutil command when the test runs on Windows).
> I'll try to do more timings when running the tests with Ubuntu and see
> where the difference is.

Thanks for the analysis. I think the cheapest improvement we can do is 
to create the file only once (setUp(), tearDown(). It is created for 
each test again. I do not think that this is really necessary.

What type of disks do you have? This test passes on my machine within 70 
seconds on a Samsung SSD.

I will apply the patch to the branch because those fixes are necessary 
anyway.

Please keep me updated.

Michael


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Guillaume Boué <gb...@apache.org>.
I ran them at least 10 times, and there was the timeout issue each time. 
Yes, the timeout is Surefire waiting for the forked VM. I applied the 
patch (I had done similar tries also) but I still have the issue on 
Windows 10.

You'll find the logs attached. It seems that the download is really 
advancing (inside target, I can see the file huge<numbers>txt slowly 
growing), but is just taking a long time. Additionally, the test class 
HugeFileDownloadTest is composed of 2 tests, downloading the same file 2 
times with different parameters: from the logs, it seems one of them is 
succeeding (taking 140 seconds of the 400 in total), and then it times 
out performing the second.

Also, part of the issue is probably with the time it takes to create the 
4 Go test file that is later downloaded through Wagon (maybe it should 
be done by calling the fsutil command when the test runs on Windows). 
I'll try to do more timings when running the tests with Ubuntu and see 
where the difference is.


Le 29/12/2016  01:18, Michael Osipov a crit :
> Am 2016-12-28 um 21:34 schrieb Guillaume Bou:
>> I have the same results as Herv, both on Windows and Ubuntu. This is
>> what I have with Maven 3.3.9:
>>
>> - Windows 10 64bit, OpenJDK 1.8.0_102, Test failure: Timeout in
>> HugeFileDownloadTest (perhaps the timeout should be increased?)
>
> How often did you run the tests to get this result? I tried at least 
> 20 times or more.
> Can you enable the logging I asked Dan for? You might see something else.
>
> By timeout do you mean the timeout Surefire waits for the forked VM?
>
> Can you try the attached patch? I have noticed that in this 
> HugeFileDownloadTest the server is never shutdown as well as in other 
> cases.
>
> Michael
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org



---
L'absence de virus dans ce courrier lectronique a t vrifie par le logiciel antivirus Avast.
https://www.avast.com/antivirus

Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mi...@apache.org>.
Am 2016-12-28 um 21:34 schrieb Guillaume Boué:
> I have the same results as Hervé, both on Windows and Ubuntu. This is
> what I have with Maven 3.3.9:
>
> - Windows 10 64bit, OpenJDK 1.8.0_102, Test failure: Timeout in
> HugeFileDownloadTest (perhaps the timeout should be increased?)

How often did you run the tests to get this result? I tried at least 20 
times or more.
Can you enable the logging I asked Dan for? You might see something else.

By timeout do you mean the timeout Surefire waits for the forked VM?

Can you try the attached patch? I have noticed that in this 
HugeFileDownloadTest the server is never shutdown as well as in other cases.

Michael


Re: [VOTE] Releasing Wagon 2.11

Posted by Guillaume Boué <gb...@apache.org>.
I have the same results as Herv, both on Windows and Ubuntu. This is 
what I have with Maven 3.3.9:

- Windows 10 64bit, OpenJDK 1.8.0_102, Test failure: Timeout in 
HugeFileDownloadTest (perhaps the timeout should be increased?)
- Ubuntu 16.04 32bit, OpenJDK 1.8.0_111, Maven 3.3.9: All OK
- Ubuntu 16.04 32bit, OpenJDK 1.7.0_95, Maven 3.3.9: All OK

With Maven 3.4.0-SNAPSHOT (65960ac18c8121648fe163612375a2ba5f4535c2), I 
have compilation errors in the test class FileWagonTest (both on Windows 
and Ubuntu)

[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/guillaume/workspace-maven/maven-wagon/wagon-providers/wagon-file/src/test/java/org/apache/maven/wagon/providers/file/FileWagonTest.java:[37,8] cannot access junit.framework.TestCase
   class file for junit.framework.TestCase not found
[ERROR] /home/guillaume/workspace-maven/maven-wagon/wagon-providers/wagon-file/src/test/java/org/apache/maven/wagon/providers/file/FileWagonTest.java:[48,52] cannot find symbol
   symbol:   method getName()
   location: class org.apache.maven.wagon.providers.file.FileWagonTest

The JUnit dependency isn't getting pulled. This is the scenario at hand:

- wagon-file has as parent wagon-providers which has a dependency on 
wagon-provider-test with scope test.
- wagon-provider-test has a compile scoped dependency on junit 
(<scope>compile</scope> is explicitly written).
- wagon-provider-test has as parent wagon, which has a test scoped 
dependency management on junit.

What would be the problem?

The tree displayed in the logs when building wagon-provider-test 
correctly lists JUnit with scope compile:

[DEBUG] org.apache.maven.wagon:wagon-provider-test:jar:2.12-SNAPSHOT
[DEBUG]    org.apache.maven.wagon:wagon-provider-api:jar:2.12-SNAPSHOT:compile
[DEBUG]    org.codehaus.plexus:plexus-container-default:jar:1.5.5:compile
[DEBUG]       org.codehaus.plexus:plexus-classworlds:jar:2.2.2:compile
[DEBUG]       org.apache.xbean:xbean-reflect:jar:3.4:compile
[DEBUG]          log4j:log4j:jar:1.2.12:compile
[DEBUG]          commons-logging:commons-logging-api:jar:1.1:compile
[DEBUG]       com.google.collections:google-collections:jar:1.0:compile
[DEBUG]    org.codehaus.plexus:plexus-utils:jar:3.0.24:compile
[DEBUG]    org.easymock:easymock:jar:3.2:compile
[DEBUG]       cglib:cglib-nodep:jar:2.2.2:compile
[DEBUG]       org.objenesis:objenesis:jar:1.3:compile
[DEBUG]    org.eclipse.jetty.aggregate:jetty-all:jar:8.1.22.v20160922:compile
[DEBUG]       org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:compile
[DEBUG]    javax.servlet:javax.servlet-api:jar:3.0.1:compile
[DEBUG]    org.slf4j:slf4j-api:jar:1.7.19:compile
[DEBUG]    junit:junit:jar:4.11:compile
[DEBUG]       org.hamcrest:hamcrest-core:jar:1.3:compile

But when building wagon-file, the tree is:

[DEBUG] org.apache.maven.wagon:wagon-file:jar:2.12-SNAPSHOT
[DEBUG]    org.codehaus.plexus:plexus-utils:jar:3.0.24:compile
[DEBUG]    org.slf4j:slf4j-simple:jar:1.7.19:test
[DEBUG]       org.slf4j:slf4j-api:jar:1.7.19:test
[DEBUG]    org.apache.maven.wagon:wagon-provider-api:jar:2.12-SNAPSHOT:compile
[DEBUG]    org.apache.maven.wagon:wagon-provider-test:jar:2.12-SNAPSHOT:test
[DEBUG]       org.codehaus.plexus:plexus-container-default:jar:1.5.5:test
[DEBUG]          org.codehaus.plexus:plexus-classworlds:jar:2.2.2:test
[DEBUG]          org.apache.xbean:xbean-reflect:jar:3.4:test
[DEBUG]             log4j:log4j:jar:1.2.12:test
[DEBUG]             commons-logging:commons-logging-api:jar:1.1:test
[DEBUG]          com.google.collections:google-collections:jar:1.0:test
[DEBUG]       org.easymock:easymock:jar:3.2:test
[DEBUG]          cglib:cglib-nodep:jar:2.2.2:test
[DEBUG]          org.objenesis:objenesis:jar:1.3:test
[DEBUG]       org.eclipse.jetty.aggregate:jetty-all:jar:8.1.22.v20160922:test
[DEBUG]          org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:test
[DEBUG]       javax.servlet:javax.servlet-api:jar:3.0.1:test


Is the problem that, during resolving the dependencies for wagon-file, 
the compiled scope JUnit dependency of the direct wagon-provider-test 
dependency is getting managed to scope test (due to the parent), and 
therefore that this now test-scoped dependency of wagon-provider-test 
isn't retrieved transitively (because it is a test dependency of a 
direct test dependency)?

Guillaume

Le 28/12/2016  12:00, Herv BOUTEMY a crit :
> the branch runs without issue on my machine now:
> mvn: 3.3.9, jdk: Oracle 1.7.0_71, OS: Linux
>
> the proposed release was consistently failing on HugeFileDownloadTest
>
> Notice that with Maven 3.4.0-SNAPSHOT, there is a compilation failure:
> testCompile (default-testCompile) on project wagon-file
>
> Using "-ldm" makes the build happy again, until a Surefire run fails with a
> timeout...
>
> Regards,
>
> Herv
>
> Le mercredi 28 dcembre 2016, 02:05:23 CET Michael Osipov a crit :
>> Hi Dan,
>>
>> Am 2016-12-25 um 19:51 schrieb Dan Tran:
>>> Just want to confirm,  your FreeBSD and build.apache.org's ubuntu see the
>>> same test failure??
>>>
>>> Do you think we should cancel the vote and wait for your fix?
>> I was finally able to complete my coding and testings. After several
>> hours of testing and debugging, I've found the reasons why several tests
>> were failing sporadically, some were bad test conditions
>> (testSecuredGet(), testSecuredGetToStream()) and some rooted in bugs in
>> HttpWagon. I'll go in detail:
>>
>> 1. Flaky testSecuredGet(ToStream):
>> This failed once in a while due to race conditions between Jetty's qtp
>> threads and the main thread running the tests and the assertions. Jetty
>> did not fully complete the process chain while Surefire continued to
>> verify the result. This yielded to: expected <2> requests found <1> and
>> so forth. I added a simple sleep() and it did work. I avoided to have
>> complex sync code with CountDownLatch or alike. Some logging added to
>>
>> the code shows the following in Surefire's XML output:
>>> <failure message="not 2 security handler use
>>> [HandlerRequestResponse{method=&apos;GET&apos;, responseCode=401,
>>> requestUri=&apos;/test-secured-resource&apos;}] expected:&lt;2&gt; but
>>> was:&lt;1&gt;"
>>> type="junit.framework.AssertionFailedError"><![CDATA[junit.framework.Asse
>>> rtionFailedError: not 2 security handler use
>>> [HandlerRequestResponse{method='GET', responseCode=401,
>>> requestUri='/test-secured-resource'}] expected:<2> but was:<1>>
>>>          at junit.framework.Assert.fail(Assert.java:57)
>>>          at junit.framework.Assert.failNotEquals(Assert.java:329)
>>>          at junit.framework.Assert.assertEquals(Assert.java:78)
>>>          at junit.framework.Assert.assertEquals(Assert.java:234)
>>>          at junit.framework.TestCase.assertEquals(TestCase.java:401)
>>>          at
>>>          org.apache.maven.wagon.http.HttpWagonTestCase.testPreemptiveAuthe
>>>          ntication(HttpWagonTestCase.java:1910) at
>>>          org.apache.maven.wagon.http.HttpWagonTestCase.testPreemptiveAuthe
>>>          nticationGet(HttpWagonTestCase.java:1891) at
>>>          org.apache.maven.wagon.http.HttpWagonTestCase.runTestSecuredGetTo
>>>          Stream(HttpWagonTestCase.java:1409) at
>>>          org.apache.maven.wagon.http.HttpWagonTestCase.testSecuredGetToStr
>>>          eam(HttpWagonTestCase.java:1374) at
>>>          sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>          at
>>>          sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorI
>>>          mpl.java:57) at
>>>          sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodA
>>>          ccessorImpl.java:43) at
>>>          java.lang.reflect.Method.invoke(Method.java:606)
>>>          at junit.framework.TestCase.runTest(TestCase.java:176)
>>>          at junit.framework.TestCase.runBare(TestCase.java:141)
>>>          at junit.framework.TestResult$1.protect(TestResult.java:122)
>>>          at junit.framework.TestResult.runProtected(TestResult.java:142)
>>>          at junit.framework.TestResult.run(TestResult.java:125)
>>>          at junit.framework.TestCase.run(TestCase.java:129)
>>>          at junit.framework.TestSuite.runTest(TestSuite.java:255)
>>>          at junit.framework.TestSuite.run(TestSuite.java:250)
>>>          at
>>>          org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRun
>>>          ner.java:84) at
>>>          org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Pro
>>>          vider.java:283) at
>>>          org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(
>>>          JUnit4Provider.java:173) at
>>>          org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JU
>>>          nit4Provider.java:153) at
>>>          org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Prov
>>>          ider.java:128) at
>>>          org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSam
>>>          eClassLoader(ForkedBooter.java:203) at
>>>          org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(
>>>          ForkedBooter.java:155) at
>>>          org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.j
>>>          ava:103)>
>>> ]]></failure>
>> now the logging:
>>> <system-err><![CDATA[[main] INFO org.eclipse.jetty.server.Server -
>>> jetty-8.1.22.v20160922 [main] INFO
>>> org.eclipse.jetty.server.AbstractConnector - Started
>>> SelectChannelConnector@0.0.0.0:45658 [main] DEBUG
>>> org.apache.http.client.protocol.RequestAddCookies - CookieSpec selected:
>>> compatibility [main] DEBUG
>>> org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection
>>> request: [route: {}->http://localhost:45658][total kept alive: 39; route
>>> allocated: 0 of 20; total allocated: 39 of 40] [main] DEBUG
>>> org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection
>>> leased: [id: 86][route: {}->http://localhost:45658][total kept alive: 39;
>>> route allocated: 1 of 20; total allocated: 40 of 40] [main] DEBUG
>>> org.apache.http.impl.execchain.MainClientExec - Opening connection
>>> {}->http://localhost:45658 [main] DEBUG
>>> org.apache.http.impl.conn.DefaultHttpClientConnectionOperator -
>>> Connecting to localhost/127.0.0.1:45658 [main] DEBUG
>>> org.apache.http.impl.conn.DefaultHttpClientConnectionOperator -
>>> Connection established 127.0.0.1:45659<->127.0.0.1:45658 [main] DEBUG
>>> org.apache.http.impl.conn.DefaultManagedHttpClientConnection -
>>> http-outgoing-86: set socket timeout to 1800000 [main] DEBUG
>>> org.apache.http.impl.execchain.MainClientExec - Executing request GET
>>> /test-secured-resource HTTP/1.1 [main] DEBUG
>>> org.apache.http.impl.execchain.MainClientExec - Target auth state:
>>> UNCHALLENGED [main] DEBUG org.apache.http.impl.execchain.MainClientExec -
>>> Proxy auth state: UNCHALLENGED [main] DEBUG org.apache.http.headers -
>>> http-outgoing-86 >> GET /test-secured-resource HTTP/1.1 [main] DEBUG
>>> org.apache.http.headers - http-outgoing-86 >> Cache-control: no-cache
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-store:
>>> no-store [main] DEBUG org.apache.http.headers - http-outgoing-86 >>
>>> Pragma: no-cache [main] DEBUG org.apache.http.headers - http-outgoing-86
>>>>> Expires: 0 [main] DEBUG org.apache.http.headers - http-outgoing-86 >>
>>> Accept-Encoding: gzip [main] DEBUG org.apache.http.headers -
>>> http-outgoing-86 >> Host: localhost:45658 [main] DEBUG
>>> org.apache.http.headers - http-outgoing-86 >> Connection: Keep-Alive
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> User-Agent:
>>> Apache-HttpClient/4.5.2 (Java/1.7.0_111) [main] DEBUG
>>> org.apache.http.headers - http-outgoing-86 << HTTP/1.1 401 Unauthorized
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 <<
>>> WWW-Authenticate: basic realm="MyRealm" [main] DEBUG
>>> org.apache.http.headers - http-outgoing-86 << Cache-Control:
>>> must-revalidate,no-cache,no-store [main] DEBUG org.apache.http.headers -
>>> http-outgoing-86 << Content-Type: text/html;charset=ISO-8859-1 [main]
>>> DEBUG org.apache.http.headers - http-outgoing-86 << Content-Length: 1311
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Server:
>>> Jetty(8.1.22.v20160922) [main] DEBUG
>>> org.apache.http.impl.execchain.MainClientExec - Connection can be kept
>>> alive indefinitely [main] DEBUG
>>> org.apache.http.impl.auth.HttpAuthenticator - Authentication required
>>> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator -
>>> localhost:45658 requested authentication [main] DEBUG
>>> org.apache.http.impl.client.TargetAuthenticationStrategy - Authentication
>>> schemes in the order of preference: [Negotiate, Kerberos, NTLM, Digest,
>>> Basic] [main] DEBUG
>>> org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for
>>> Negotiate authentication scheme not available [main] DEBUG
>>> org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for
>>> Kerberos authentication scheme not available [main] DEBUG
>>> org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for
>>> NTLM authentication scheme not available [main] DEBUG
>>> org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for
>>> Digest authentication scheme not available [main] DEBUG
>>> org.apache.http.impl.auth.HttpAuthenticator - Selected authentication
>>> options: [BASIC [complete=true]] [main] DEBUG
>>> org.apache.http.impl.conn.DefaultManagedHttpClientConnection -
>>> http-outgoing-86: set socket timeout to 1800000 [main] DEBUG
>>> org.apache.http.impl.execchain.MainClientExec - Executing request GET
>>> /test-secured-resource HTTP/1.1 [main] DEBUG
>>> org.apache.http.impl.execchain.MainClientExec - Target auth state:
>>> CHALLENGED [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator -
>>> Generating response to an authentication challenge using basic scheme
>>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Proxy auth
>>> state: UNCHALLENGED [main] DEBUG org.apache.http.headers -
>>> http-outgoing-86 >> GET /test-secured-resource HTTP/1.1 [main] DEBUG
>>> org.apache.http.headers - http-outgoing-86 >> Cache-control: no-cache
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-store:
>>> no-store [main] DEBUG org.apache.http.headers - http-outgoing-86 >>
>>> Pragma: no-cache [main] DEBUG org.apache.http.headers - http-outgoing-86
>>>>> Expires: 0 [main] DEBUG org.apache.http.headers - http-outgoing-86 >>
>>> Accept-Encoding: gzip [main] DEBUG org.apache.http.headers -
>>> http-outgoing-86 >> Host: localhost:45658 [main] DEBUG
>>> org.apache.http.headers - http-outgoing-86 >> Connection: Keep-Alive
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> User-Agent:
>>> Apache-HttpClient/4.5.2 (Java/1.7.0_111) [main] DEBUG
>>> org.apache.http.headers - http-outgoing-86 >> Authorization: Basic
>>> dXNlcjpzZWNyZXQ=
>>> org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
>>> (count 1): handled GET /test-secured-resource with status 401 [main]
>>> DEBUG org.apache.http.headers - http-outgoing-86 << HTTP/1.1 200 OK
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Accept-Ranges:
>>> bytes [main] DEBUG org.apache.http.headers - http-outgoing-86 <<
>>> Content-Length: 10 [main] DEBUG org.apache.http.headers -
>>> http-outgoing-86 << Last-Modified: Mon, 26 Dec 2016 23:07:55 GMT [main]
>>> DEBUG org.apache.http.headers - http-outgoing-86 << Server:
>>> Jetty(8.1.22.v20160922) [main] DEBUG
>>> org.apache.http.impl.execchain.MainClientExec - Connection can be kept
>>> alive indefinitely [main] DEBUG
>>> org.apache.http.impl.auth.HttpAuthenticator - Authentication succeeded
>>> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy -
>>> Caching 'basic' auth scheme for http://localhost:45658 [main] DEBUG
>>> org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection
>>> [id: 86][route: {}->http://localhost:45658] can be kept alive
>>> indefinitely [main] DEBUG
>>> org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection
>>> released: [id: 86][route: {}->http://localhost:45658][total kept alive:
>>> 40; route allocated: 1 of 20; total allocated: 40 of 40]
>>> org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
>>> (1)
>>> org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
>>> (count 2): handled GET /test-secured-resource with status 200 [main] INFO
>>> org.eclipse.jetty.server.handler.ContextHandler - stopped
>>> o.e.j.s.ServletContextHandler{/,file:/usr/home/mosipov/Projekte/maven-wag
>>> on/wagon-providers/wagon-http/target/test-output/} ]]></system-err>
>> now take a look at TestSecurityHandler output. I added it after
>> super.handle() called by Jetty. As you can see,
>> "org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
>> (1)" is printed where the assertions kick in, after Wagon has completed,
>> but the second request is handled afterwards. A traditional race condition.
>>
>> 2. Bugs found:
>> Several auth tests fail for no reason too once in a while. Logging
>> showed me that HttpClient simply did not send any Basic data, regardless
>> if preemptive or after a roundtrip. The basic problem was that
>> non-threadsafe classes have been used in a non-threadsafe manner as well
>> as auth caches have not beed released properly, and preemptive auth was
>> configured incorrectly. After fixing the issues one by one, the tests
>> never failed again.
>>
>> I have tested the code on three platforms and six JDK versions tens of
>> times, they never failed again. After we have resolved your
>> HugeDowloadTest issue and no one else objects, I'd like to merge this
>> into master and like you to reroll 2.11 with those changes.
>>
>> Meanwhile, I'll test this branch with Maven Core IT Suite too.
>>
>> Michael
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>


---
L'absence de virus dans ce courrier lectronique a t vrifie par le logiciel antivirus Avast.
https://www.avast.com/antivirus


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Hervé BOUTEMY <he...@free.fr>.
the branch runs without issue on my machine now:
mvn: 3.3.9, jdk: Oracle 1.7.0_71, OS: Linux

the proposed release was consistently failing on HugeFileDownloadTest

Notice that with Maven 3.4.0-SNAPSHOT, there is a compilation failure:
testCompile (default-testCompile) on project wagon-file

Using "-ldm" makes the build happy again, until a Surefire run fails with a 
timeout...

Regards,

Hervé

Le mercredi 28 décembre 2016, 02:05:23 CET Michael Osipov a écrit :
> Hi Dan,
> 
> Am 2016-12-25 um 19:51 schrieb Dan Tran:
> > Just want to confirm,  your FreeBSD and build.apache.org's ubuntu see the
> > same test failure??
> > 
> > Do you think we should cancel the vote and wait for your fix?
> 
> I was finally able to complete my coding and testings. After several
> hours of testing and debugging, I've found the reasons why several tests
> were failing sporadically, some were bad test conditions
> (testSecuredGet(), testSecuredGetToStream()) and some rooted in bugs in
> HttpWagon. I'll go in detail:
> 
> 1. Flaky testSecuredGet(ToStream):
> This failed once in a while due to race conditions between Jetty's qtp
> threads and the main thread running the tests and the assertions. Jetty
> did not fully complete the process chain while Surefire continued to
> verify the result. This yielded to: expected <2> requests found <1> and
> so forth. I added a simple sleep() and it did work. I avoided to have
> complex sync code with CountDownLatch or alike. Some logging added to
> 
> the code shows the following in Surefire's XML output:
> > <failure message="not 2 security handler use
> > [HandlerRequestResponse{method=&apos;GET&apos;, responseCode=401,
> > requestUri=&apos;/test-secured-resource&apos;}] expected:&lt;2&gt; but
> > was:&lt;1&gt;"
> > type="junit.framework.AssertionFailedError"><![CDATA[junit.framework.Asse
> > rtionFailedError: not 2 security handler use
> > [HandlerRequestResponse{method='GET', responseCode=401,
> > requestUri='/test-secured-resource'}] expected:<2> but was:<1>> 
> >         at junit.framework.Assert.fail(Assert.java:57)
> >         at junit.framework.Assert.failNotEquals(Assert.java:329)
> >         at junit.framework.Assert.assertEquals(Assert.java:78)
> >         at junit.framework.Assert.assertEquals(Assert.java:234)
> >         at junit.framework.TestCase.assertEquals(TestCase.java:401)
> >         at
> >         org.apache.maven.wagon.http.HttpWagonTestCase.testPreemptiveAuthe
> >         ntication(HttpWagonTestCase.java:1910) at
> >         org.apache.maven.wagon.http.HttpWagonTestCase.testPreemptiveAuthe
> >         nticationGet(HttpWagonTestCase.java:1891) at
> >         org.apache.maven.wagon.http.HttpWagonTestCase.runTestSecuredGetTo
> >         Stream(HttpWagonTestCase.java:1409) at
> >         org.apache.maven.wagon.http.HttpWagonTestCase.testSecuredGetToStr
> >         eam(HttpWagonTestCase.java:1374) at
> >         sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at
> >         sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorI
> >         mpl.java:57) at
> >         sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodA
> >         ccessorImpl.java:43) at
> >         java.lang.reflect.Method.invoke(Method.java:606)
> >         at junit.framework.TestCase.runTest(TestCase.java:176)
> >         at junit.framework.TestCase.runBare(TestCase.java:141)
> >         at junit.framework.TestResult$1.protect(TestResult.java:122)
> >         at junit.framework.TestResult.runProtected(TestResult.java:142)
> >         at junit.framework.TestResult.run(TestResult.java:125)
> >         at junit.framework.TestCase.run(TestCase.java:129)
> >         at junit.framework.TestSuite.runTest(TestSuite.java:255)
> >         at junit.framework.TestSuite.run(TestSuite.java:250)
> >         at
> >         org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRun
> >         ner.java:84) at
> >         org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Pro
> >         vider.java:283) at
> >         org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(
> >         JUnit4Provider.java:173) at
> >         org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JU
> >         nit4Provider.java:153) at
> >         org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Prov
> >         ider.java:128) at
> >         org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSam
> >         eClassLoader(ForkedBooter.java:203) at
> >         org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(
> >         ForkedBooter.java:155) at
> >         org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.j
> >         ava:103)> 
> > ]]></failure>
> 
> now the logging:
> > <system-err><![CDATA[[main] INFO org.eclipse.jetty.server.Server -
> > jetty-8.1.22.v20160922 [main] INFO
> > org.eclipse.jetty.server.AbstractConnector - Started
> > SelectChannelConnector@0.0.0.0:45658 [main] DEBUG
> > org.apache.http.client.protocol.RequestAddCookies - CookieSpec selected:
> > compatibility [main] DEBUG
> > org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection
> > request: [route: {}->http://localhost:45658][total kept alive: 39; route
> > allocated: 0 of 20; total allocated: 39 of 40] [main] DEBUG
> > org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection
> > leased: [id: 86][route: {}->http://localhost:45658][total kept alive: 39;
> > route allocated: 1 of 20; total allocated: 40 of 40] [main] DEBUG
> > org.apache.http.impl.execchain.MainClientExec - Opening connection
> > {}->http://localhost:45658 [main] DEBUG
> > org.apache.http.impl.conn.DefaultHttpClientConnectionOperator -
> > Connecting to localhost/127.0.0.1:45658 [main] DEBUG
> > org.apache.http.impl.conn.DefaultHttpClientConnectionOperator -
> > Connection established 127.0.0.1:45659<->127.0.0.1:45658 [main] DEBUG
> > org.apache.http.impl.conn.DefaultManagedHttpClientConnection -
> > http-outgoing-86: set socket timeout to 1800000 [main] DEBUG
> > org.apache.http.impl.execchain.MainClientExec - Executing request GET
> > /test-secured-resource HTTP/1.1 [main] DEBUG
> > org.apache.http.impl.execchain.MainClientExec - Target auth state:
> > UNCHALLENGED [main] DEBUG org.apache.http.impl.execchain.MainClientExec -
> > Proxy auth state: UNCHALLENGED [main] DEBUG org.apache.http.headers -
> > http-outgoing-86 >> GET /test-secured-resource HTTP/1.1 [main] DEBUG
> > org.apache.http.headers - http-outgoing-86 >> Cache-control: no-cache
> > [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-store:
> > no-store [main] DEBUG org.apache.http.headers - http-outgoing-86 >>
> > Pragma: no-cache [main] DEBUG org.apache.http.headers - http-outgoing-86
> > >> Expires: 0 [main] DEBUG org.apache.http.headers - http-outgoing-86 >>
> > Accept-Encoding: gzip [main] DEBUG org.apache.http.headers -
> > http-outgoing-86 >> Host: localhost:45658 [main] DEBUG
> > org.apache.http.headers - http-outgoing-86 >> Connection: Keep-Alive
> > [main] DEBUG org.apache.http.headers - http-outgoing-86 >> User-Agent:
> > Apache-HttpClient/4.5.2 (Java/1.7.0_111) [main] DEBUG
> > org.apache.http.headers - http-outgoing-86 << HTTP/1.1 401 Unauthorized
> > [main] DEBUG org.apache.http.headers - http-outgoing-86 <<
> > WWW-Authenticate: basic realm="MyRealm" [main] DEBUG
> > org.apache.http.headers - http-outgoing-86 << Cache-Control:
> > must-revalidate,no-cache,no-store [main] DEBUG org.apache.http.headers -
> > http-outgoing-86 << Content-Type: text/html;charset=ISO-8859-1 [main]
> > DEBUG org.apache.http.headers - http-outgoing-86 << Content-Length: 1311
> > [main] DEBUG org.apache.http.headers - http-outgoing-86 << Server:
> > Jetty(8.1.22.v20160922) [main] DEBUG
> > org.apache.http.impl.execchain.MainClientExec - Connection can be kept
> > alive indefinitely [main] DEBUG
> > org.apache.http.impl.auth.HttpAuthenticator - Authentication required
> > [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator -
> > localhost:45658 requested authentication [main] DEBUG
> > org.apache.http.impl.client.TargetAuthenticationStrategy - Authentication
> > schemes in the order of preference: [Negotiate, Kerberos, NTLM, Digest,
> > Basic] [main] DEBUG
> > org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for
> > Negotiate authentication scheme not available [main] DEBUG
> > org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for
> > Kerberos authentication scheme not available [main] DEBUG
> > org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for
> > NTLM authentication scheme not available [main] DEBUG
> > org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for
> > Digest authentication scheme not available [main] DEBUG
> > org.apache.http.impl.auth.HttpAuthenticator - Selected authentication
> > options: [BASIC [complete=true]] [main] DEBUG
> > org.apache.http.impl.conn.DefaultManagedHttpClientConnection -
> > http-outgoing-86: set socket timeout to 1800000 [main] DEBUG
> > org.apache.http.impl.execchain.MainClientExec - Executing request GET
> > /test-secured-resource HTTP/1.1 [main] DEBUG
> > org.apache.http.impl.execchain.MainClientExec - Target auth state:
> > CHALLENGED [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator -
> > Generating response to an authentication challenge using basic scheme
> > [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Proxy auth
> > state: UNCHALLENGED [main] DEBUG org.apache.http.headers -
> > http-outgoing-86 >> GET /test-secured-resource HTTP/1.1 [main] DEBUG
> > org.apache.http.headers - http-outgoing-86 >> Cache-control: no-cache
> > [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-store:
> > no-store [main] DEBUG org.apache.http.headers - http-outgoing-86 >>
> > Pragma: no-cache [main] DEBUG org.apache.http.headers - http-outgoing-86
> > >> Expires: 0 [main] DEBUG org.apache.http.headers - http-outgoing-86 >>
> > Accept-Encoding: gzip [main] DEBUG org.apache.http.headers -
> > http-outgoing-86 >> Host: localhost:45658 [main] DEBUG
> > org.apache.http.headers - http-outgoing-86 >> Connection: Keep-Alive
> > [main] DEBUG org.apache.http.headers - http-outgoing-86 >> User-Agent:
> > Apache-HttpClient/4.5.2 (Java/1.7.0_111) [main] DEBUG
> > org.apache.http.headers - http-outgoing-86 >> Authorization: Basic
> > dXNlcjpzZWNyZXQ=
> > org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
> > (count 1): handled GET /test-secured-resource with status 401 [main]
> > DEBUG org.apache.http.headers - http-outgoing-86 << HTTP/1.1 200 OK
> > [main] DEBUG org.apache.http.headers - http-outgoing-86 << Accept-Ranges:
> > bytes [main] DEBUG org.apache.http.headers - http-outgoing-86 <<
> > Content-Length: 10 [main] DEBUG org.apache.http.headers -
> > http-outgoing-86 << Last-Modified: Mon, 26 Dec 2016 23:07:55 GMT [main]
> > DEBUG org.apache.http.headers - http-outgoing-86 << Server:
> > Jetty(8.1.22.v20160922) [main] DEBUG
> > org.apache.http.impl.execchain.MainClientExec - Connection can be kept
> > alive indefinitely [main] DEBUG
> > org.apache.http.impl.auth.HttpAuthenticator - Authentication succeeded
> > [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy -
> > Caching 'basic' auth scheme for http://localhost:45658 [main] DEBUG
> > org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection
> > [id: 86][route: {}->http://localhost:45658] can be kept alive
> > indefinitely [main] DEBUG
> > org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection
> > released: [id: 86][route: {}->http://localhost:45658][total kept alive:
> > 40; route allocated: 1 of 20; total allocated: 40 of 40]
> > org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
> > (1)
> > org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
> > (count 2): handled GET /test-secured-resource with status 200 [main] INFO
> > org.eclipse.jetty.server.handler.ContextHandler - stopped
> > o.e.j.s.ServletContextHandler{/,file:/usr/home/mosipov/Projekte/maven-wag
> > on/wagon-providers/wagon-http/target/test-output/} ]]></system-err>
> 
> now take a look at TestSecurityHandler output. I added it after
> super.handle() called by Jetty. As you can see,
> "org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
> (1)" is printed where the assertions kick in, after Wagon has completed,
> but the second request is handled afterwards. A traditional race condition.
> 
> 2. Bugs found:
> Several auth tests fail for no reason too once in a while. Logging
> showed me that HttpClient simply did not send any Basic data, regardless
> if preemptive or after a roundtrip. The basic problem was that
> non-threadsafe classes have been used in a non-threadsafe manner as well
> as auth caches have not beed released properly, and preemptive auth was
> configured incorrectly. After fixing the issues one by one, the tests
> never failed again.
> 
> I have tested the code on three platforms and six JDK versions tens of
> times, they never failed again. After we have resolved your
> HugeDowloadTest issue and no one else objects, I'd like to merge this
> into master and like you to reroll 2.11 with those changes.
> 
> Meanwhile, I'll test this branch with Maven Core IT Suite too.
> 
> Michael
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mi...@apache.org>.
Am 2016-12-28 um 06:05 schrieb Dan Tran:
> i still see the same timeout error on the jetty8 branch. No issue at master
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test
> (default-test) on project wagon-http: There was a timeout or other
> error in the fork -> [Help
> 1]org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
> execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test
> (default-test) on project wagon-http: There was a timeout or other
> error in the fork
> 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>
>

I still would like to see your Surefire XML. Please provide! Can you 
raise forkedProcessTimeoutInSeconds?
This could also be a race condition where one thread does not wait for 
another. Please also note: 
https://mail-archives.apache.org/mod_mbox/maven-dev/201612.mbox/%3C81e68f7b-b68f-ce5d-2529-12d604d42c98%40apache.org%3E

Can you set up a job on Jenkins for this branch? I'd like to see wether 
the Ubuntu slave fails too.

 > It is your call now. since I cant release it

Do you want me to roll the release?

> On Tue, Dec 27, 2016 at 5:05 PM, Michael Osipov <mi...@apache.org> wrote:
>
>> Hi Dan,
>>
>> Am 2016-12-25 um 19:51 schrieb Dan Tran:
>>
>>> Just want to confirm,  your FreeBSD and build.apache.org's ubuntu see the
>>> same test failure??
>>>
>>> Do you think we should cancel the vote and wait for your fix?
>>>
>>
>> I was finally able to complete my coding and testings. After several hours
>> of testing and debugging, I've found the reasons why several tests were
>> failing sporadically, some were bad test conditions (testSecuredGet(),
>> testSecuredGetToStream()) and some rooted in bugs in HttpWagon. I'll go in
>> detail:
>>
>> 1. Flaky testSecuredGet(ToStream):
>> This failed once in a while due to race conditions between Jetty's qtp
>> threads and the main thread running the tests and the assertions. Jetty did
>> not fully complete the process chain while Surefire continued to verify the
>> result. This yielded to: expected <2> requests found <1> and so forth. I
>> added a simple sleep() and it did work. I avoided to have complex sync code
>> with CountDownLatch or alike. Some logging added to the code shows the
>> following in Surefire's XML output:
>>
>> <failure message="not 2 security handler use [HandlerRequestResponse{method=&apos;GET&apos;,
>>> responseCode=401, requestUri=&apos;/test-secured-resource&apos;}]
>>> expected:&lt;2&gt; but was:&lt;1&gt;" type="junit.framework.Assertio
>>> nFailedError"><![CDATA[junit.framework.AssertionFailedError: not 2
>>> security handler use [HandlerRequestResponse{method='GET',
>>> responseCode=401, requestUri='/test-secured-resource'}] expected:<2> but
>>> was:<1>
>>>         at junit.framework.Assert.fail(Assert.java:57)
>>>         at junit.framework.Assert.failNotEquals(Assert.java:329)
>>>         at junit.framework.Assert.assertEquals(Assert.java:78)
>>>         at junit.framework.Assert.assertEquals(Assert.java:234)
>>>         at junit.framework.TestCase.assertEquals(TestCase.java:401)
>>>         at org.apache.maven.wagon.http.HttpWagonTestCase.testPreemptive
>>> Authentication(HttpWagonTestCase.java:1910)
>>>         at org.apache.maven.wagon.http.HttpWagonTestCase.testPreemptive
>>> AuthenticationGet(HttpWagonTestCase.java:1891)
>>>         at org.apache.maven.wagon.http.HttpWagonTestCase.runTestSecured
>>> GetToStream(HttpWagonTestCase.java:1409)
>>>         at org.apache.maven.wagon.http.HttpWagonTestCase.testSecuredGet
>>> ToStream(HttpWagonTestCase.java:1374)
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>>> ssorImpl.java:57)
>>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>>> thodAccessorImpl.java:43)
>>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>>         at junit.framework.TestCase.runTest(TestCase.java:176)
>>>         at junit.framework.TestCase.runBare(TestCase.java:141)
>>>         at junit.framework.TestResult$1.protect(TestResult.java:122)
>>>         at junit.framework.TestResult.runProtected(TestResult.java:142)
>>>         at junit.framework.TestResult.run(TestResult.java:125)
>>>         at junit.framework.TestCase.run(TestCase.java:129)
>>>         at junit.framework.TestSuite.runTest(TestSuite.java:255)
>>>         at junit.framework.TestSuite.run(TestSuite.java:250)
>>>         at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38Cla
>>> ssRunner.java:84)
>>>         at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUni
>>> t4Provider.java:283)
>>>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithR
>>> erun(JUnit4Provider.java:173)
>>>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestS
>>> et(JUnit4Provider.java:153)
>>>         at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit
>>> 4Provider.java:128)
>>>         at org.apache.maven.surefire.booter.ForkedBooter.invokeProvider
>>> InSameClassLoader(ForkedBooter.java:203)
>>>         at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInPro
>>> cess(ForkedBooter.java:155)
>>>         at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBoo
>>> ter.java:103)
>>> ]]></failure>
>>>
>>
>> now the logging:
>>
>> <system-err><![CDATA[[main] INFO org.eclipse.jetty.server.Server -
>>> jetty-8.1.22.v20160922
>>> [main] INFO org.eclipse.jetty.server.AbstractConnector - Started
>>> SelectChannelConnector@0.0.0.0:45658
>>> [main] DEBUG org.apache.http.client.protocol.RequestAddCookies -
>>> CookieSpec selected: compatibility
>>> [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager
>>> - Connection request: [route: {}->http://localhost:45658][total kept
>>> alive: 39; route allocated: 0 of 20; total allocated: 39 of 40]
>>> [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager
>>> - Connection leased: [id: 86][route: {}->http://localhost:45658][total
>>> kept alive: 39; route allocated: 1 of 20; total allocated: 40 of 40]
>>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Opening
>>> connection {}->http://localhost:45658
>>> [main] DEBUG org.apache.http.impl.conn.DefaultHttpClientConnectionOperator
>>> - Connecting to localhost/127.0.0.1:45658
>>> [main] DEBUG org.apache.http.impl.conn.DefaultHttpClientConnectionOperator
>>> - Connection established 127.0.0.1:45659<->127.0.0.1:45658
>>> [main] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection
>>> - http-outgoing-86: set socket timeout to 1800000
>>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Executing
>>> request GET /test-secured-resource HTTP/1.1
>>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Target auth
>>> state: UNCHALLENGED
>>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Proxy auth
>>> state: UNCHALLENGED
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> GET
>>> /test-secured-resource HTTP/1.1
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-control:
>>> no-cache
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-store:
>>> no-store
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Pragma:
>>> no-cache
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Expires: 0
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >>
>>> Accept-Encoding: gzip
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Host:
>>> localhost:45658
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Connection:
>>> Keep-Alive
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> User-Agent:
>>> Apache-HttpClient/4.5.2 (Java/1.7.0_111)
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << HTTP/1.1 401
>>> Unauthorized
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 <<
>>> WWW-Authenticate: basic realm="MyRealm"
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Cache-Control:
>>> must-revalidate,no-cache,no-store
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Content-Type:
>>> text/html;charset=ISO-8859-1
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 <<
>>> Content-Length: 1311
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Server:
>>> Jetty(8.1.22.v20160922)
>>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Connection
>>> can be kept alive indefinitely
>>> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator -
>>> Authentication required
>>> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator -
>>> localhost:45658 requested authentication
>>> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy -
>>> Authentication schemes in the order of preference: [Negotiate, Kerberos,
>>> NTLM, Digest, Basic]
>>> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy -
>>> Challenge for Negotiate authentication scheme not available
>>> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy -
>>> Challenge for Kerberos authentication scheme not available
>>> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy -
>>> Challenge for NTLM authentication scheme not available
>>> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy -
>>> Challenge for Digest authentication scheme not available
>>> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator - Selected
>>> authentication options: [BASIC [complete=true]]
>>> [main] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection
>>> - http-outgoing-86: set socket timeout to 1800000
>>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Executing
>>> request GET /test-secured-resource HTTP/1.1
>>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Target auth
>>> state: CHALLENGED
>>> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator - Generating
>>> response to an authentication challenge using basic scheme
>>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Proxy auth
>>> state: UNCHALLENGED
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> GET
>>> /test-secured-resource HTTP/1.1
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-control:
>>> no-cache
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-store:
>>> no-store
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Pragma:
>>> no-cache
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Expires: 0
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >>
>>> Accept-Encoding: gzip
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Host:
>>> localhost:45658
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Connection:
>>> Keep-Alive
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> User-Agent:
>>> Apache-HttpClient/4.5.2 (Java/1.7.0_111)
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Authorization:
>>> Basic dXNlcjpzZWNyZXQ=
>>> org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
>>> (count 1): handled GET /test-secured-resource with status 401
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << HTTP/1.1 200 OK
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Accept-Ranges:
>>> bytes
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 <<
>>> Content-Length: 10
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Last-Modified:
>>> Mon, 26 Dec 2016 23:07:55 GMT
>>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Server:
>>> Jetty(8.1.22.v20160922)
>>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Connection
>>> can be kept alive indefinitely
>>> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator -
>>> Authentication succeeded
>>> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy -
>>> Caching 'basic' auth scheme for http://localhost:45658
>>> [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager
>>> - Connection [id: 86][route: {}->http://localhost:45658] can be kept
>>> alive indefinitely
>>> [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager
>>> - Connection released: [id: 86][route: {}->http://localhost:45658][total
>>> kept alive: 40; route allocated: 1 of 20; total allocated: 40 of 40]
>>> org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
>>> (1)
>>> org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
>>> (count 2): handled GET /test-secured-resource with status 200
>>> [main] INFO org.eclipse.jetty.server.handler.ContextHandler - stopped
>>> o.e.j.s.ServletContextHandler{/,file:/usr/home/mosipov/Proje
>>> kte/maven-wagon/wagon-providers/wagon-http/target/test-output/}
>>> ]]></system-err>
>>>
>>
>> now take a look at TestSecurityHandler output. I added it after
>> super.handle() called by Jetty. As you can see,
>> "org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
>> (1)" is printed where the assertions kick in, after Wagon has completed,
>> but the second request is handled afterwards. A traditional race condition.
>>
>> 2. Bugs found:
>> Several auth tests fail for no reason too once in a while. Logging showed
>> me that HttpClient simply did not send any Basic data, regardless if
>> preemptive or after a roundtrip. The basic problem was that non-threadsafe
>> classes have been used in a non-threadsafe manner as well as auth caches
>> have not beed released properly, and preemptive auth was configured
>> incorrectly. After fixing the issues one by one, the tests never failed
>> again.
>>
>> I have tested the code on three platforms and six JDK versions tens of
>> times, they never failed again. After we have resolved your HugeDowloadTest
>> issue and no one else objects, I'd like to merge this into master and like
>> you to reroll 2.11 with those changes.
>>
>> Meanwhile, I'll test this branch with Maven Core IT Suite too.
>>
>>
>> Michael
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mo...@gmx.de>.
Am 2016-12-28 um 06:05 schrieb Dan Tran:
> i still see the same timeout error on the jetty8 branch. No issue at master
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test
> (default-test) on project wagon-http: There was a timeout or other
> error in the fork -> [Help
> 1]org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
> execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test
> (default-test) on project wagon-http: There was a timeout or other
> error in the fork
> 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)

I have now extended tests to other Linux distros:

Fedora 25, 64 bit:
OpenJDK 8 Update 111: pass

CentOS 7, 64 bit:
OpenJDK 8 Update 111: pass
OpenJDK 7 Update 121: pass*

*This is actually your combo which works for me, but not for you...

Additionally, I added this version to Maven 3.4.0-SNAPSHOT and ran ITs 
on Windows 10, Fedora 25 and FreeBSD 10.3-RELEASE. All passed.


Michael

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Dan Tran <da...@gmail.com>.
i still see the same timeout error on the jetty8 branch. No issue at master

[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test
(default-test) on project wagon-http: There was a timeout or other
error in the fork -> [Help
1]org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test
(default-test) on project wagon-http: There was a timeout or other
error in the fork
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)



It is your call now. since I cant release it

-Dan

On Tue, Dec 27, 2016 at 5:05 PM, Michael Osipov <mi...@apache.org> wrote:

> Hi Dan,
>
> Am 2016-12-25 um 19:51 schrieb Dan Tran:
>
>> Just want to confirm,  your FreeBSD and build.apache.org's ubuntu see the
>> same test failure??
>>
>> Do you think we should cancel the vote and wait for your fix?
>>
>
> I was finally able to complete my coding and testings. After several hours
> of testing and debugging, I've found the reasons why several tests were
> failing sporadically, some were bad test conditions (testSecuredGet(),
> testSecuredGetToStream()) and some rooted in bugs in HttpWagon. I'll go in
> detail:
>
> 1. Flaky testSecuredGet(ToStream):
> This failed once in a while due to race conditions between Jetty's qtp
> threads and the main thread running the tests and the assertions. Jetty did
> not fully complete the process chain while Surefire continued to verify the
> result. This yielded to: expected <2> requests found <1> and so forth. I
> added a simple sleep() and it did work. I avoided to have complex sync code
> with CountDownLatch or alike. Some logging added to the code shows the
> following in Surefire's XML output:
>
> <failure message="not 2 security handler use [HandlerRequestResponse{method=&apos;GET&apos;,
>> responseCode=401, requestUri=&apos;/test-secured-resource&apos;}]
>> expected:&lt;2&gt; but was:&lt;1&gt;" type="junit.framework.Assertio
>> nFailedError"><![CDATA[junit.framework.AssertionFailedError: not 2
>> security handler use [HandlerRequestResponse{method='GET',
>> responseCode=401, requestUri='/test-secured-resource'}] expected:<2> but
>> was:<1>
>>         at junit.framework.Assert.fail(Assert.java:57)
>>         at junit.framework.Assert.failNotEquals(Assert.java:329)
>>         at junit.framework.Assert.assertEquals(Assert.java:78)
>>         at junit.framework.Assert.assertEquals(Assert.java:234)
>>         at junit.framework.TestCase.assertEquals(TestCase.java:401)
>>         at org.apache.maven.wagon.http.HttpWagonTestCase.testPreemptive
>> Authentication(HttpWagonTestCase.java:1910)
>>         at org.apache.maven.wagon.http.HttpWagonTestCase.testPreemptive
>> AuthenticationGet(HttpWagonTestCase.java:1891)
>>         at org.apache.maven.wagon.http.HttpWagonTestCase.runTestSecured
>> GetToStream(HttpWagonTestCase.java:1409)
>>         at org.apache.maven.wagon.http.HttpWagonTestCase.testSecuredGet
>> ToStream(HttpWagonTestCase.java:1374)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:57)
>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> thodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>         at junit.framework.TestCase.runTest(TestCase.java:176)
>>         at junit.framework.TestCase.runBare(TestCase.java:141)
>>         at junit.framework.TestResult$1.protect(TestResult.java:122)
>>         at junit.framework.TestResult.runProtected(TestResult.java:142)
>>         at junit.framework.TestResult.run(TestResult.java:125)
>>         at junit.framework.TestCase.run(TestCase.java:129)
>>         at junit.framework.TestSuite.runTest(TestSuite.java:255)
>>         at junit.framework.TestSuite.run(TestSuite.java:250)
>>         at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38Cla
>> ssRunner.java:84)
>>         at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUni
>> t4Provider.java:283)
>>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithR
>> erun(JUnit4Provider.java:173)
>>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestS
>> et(JUnit4Provider.java:153)
>>         at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit
>> 4Provider.java:128)
>>         at org.apache.maven.surefire.booter.ForkedBooter.invokeProvider
>> InSameClassLoader(ForkedBooter.java:203)
>>         at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInPro
>> cess(ForkedBooter.java:155)
>>         at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBoo
>> ter.java:103)
>> ]]></failure>
>>
>
> now the logging:
>
> <system-err><![CDATA[[main] INFO org.eclipse.jetty.server.Server -
>> jetty-8.1.22.v20160922
>> [main] INFO org.eclipse.jetty.server.AbstractConnector - Started
>> SelectChannelConnector@0.0.0.0:45658
>> [main] DEBUG org.apache.http.client.protocol.RequestAddCookies -
>> CookieSpec selected: compatibility
>> [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager
>> - Connection request: [route: {}->http://localhost:45658][total kept
>> alive: 39; route allocated: 0 of 20; total allocated: 39 of 40]
>> [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager
>> - Connection leased: [id: 86][route: {}->http://localhost:45658][total
>> kept alive: 39; route allocated: 1 of 20; total allocated: 40 of 40]
>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Opening
>> connection {}->http://localhost:45658
>> [main] DEBUG org.apache.http.impl.conn.DefaultHttpClientConnectionOperator
>> - Connecting to localhost/127.0.0.1:45658
>> [main] DEBUG org.apache.http.impl.conn.DefaultHttpClientConnectionOperator
>> - Connection established 127.0.0.1:45659<->127.0.0.1:45658
>> [main] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection
>> - http-outgoing-86: set socket timeout to 1800000
>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Executing
>> request GET /test-secured-resource HTTP/1.1
>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Target auth
>> state: UNCHALLENGED
>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Proxy auth
>> state: UNCHALLENGED
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> GET
>> /test-secured-resource HTTP/1.1
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-control:
>> no-cache
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-store:
>> no-store
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Pragma:
>> no-cache
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Expires: 0
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >>
>> Accept-Encoding: gzip
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Host:
>> localhost:45658
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Connection:
>> Keep-Alive
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> User-Agent:
>> Apache-HttpClient/4.5.2 (Java/1.7.0_111)
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << HTTP/1.1 401
>> Unauthorized
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 <<
>> WWW-Authenticate: basic realm="MyRealm"
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Cache-Control:
>> must-revalidate,no-cache,no-store
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Content-Type:
>> text/html;charset=ISO-8859-1
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 <<
>> Content-Length: 1311
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Server:
>> Jetty(8.1.22.v20160922)
>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Connection
>> can be kept alive indefinitely
>> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator -
>> Authentication required
>> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator -
>> localhost:45658 requested authentication
>> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy -
>> Authentication schemes in the order of preference: [Negotiate, Kerberos,
>> NTLM, Digest, Basic]
>> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy -
>> Challenge for Negotiate authentication scheme not available
>> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy -
>> Challenge for Kerberos authentication scheme not available
>> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy -
>> Challenge for NTLM authentication scheme not available
>> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy -
>> Challenge for Digest authentication scheme not available
>> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator - Selected
>> authentication options: [BASIC [complete=true]]
>> [main] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection
>> - http-outgoing-86: set socket timeout to 1800000
>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Executing
>> request GET /test-secured-resource HTTP/1.1
>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Target auth
>> state: CHALLENGED
>> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator - Generating
>> response to an authentication challenge using basic scheme
>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Proxy auth
>> state: UNCHALLENGED
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> GET
>> /test-secured-resource HTTP/1.1
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-control:
>> no-cache
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-store:
>> no-store
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Pragma:
>> no-cache
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Expires: 0
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >>
>> Accept-Encoding: gzip
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Host:
>> localhost:45658
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Connection:
>> Keep-Alive
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> User-Agent:
>> Apache-HttpClient/4.5.2 (Java/1.7.0_111)
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Authorization:
>> Basic dXNlcjpzZWNyZXQ=
>> org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
>> (count 1): handled GET /test-secured-resource with status 401
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << HTTP/1.1 200 OK
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Accept-Ranges:
>> bytes
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 <<
>> Content-Length: 10
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Last-Modified:
>> Mon, 26 Dec 2016 23:07:55 GMT
>> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Server:
>> Jetty(8.1.22.v20160922)
>> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Connection
>> can be kept alive indefinitely
>> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator -
>> Authentication succeeded
>> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy -
>> Caching 'basic' auth scheme for http://localhost:45658
>> [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager
>> - Connection [id: 86][route: {}->http://localhost:45658] can be kept
>> alive indefinitely
>> [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager
>> - Connection released: [id: 86][route: {}->http://localhost:45658][total
>> kept alive: 40; route allocated: 1 of 20; total allocated: 40 of 40]
>> org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
>> (1)
>> org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
>> (count 2): handled GET /test-secured-resource with status 200
>> [main] INFO org.eclipse.jetty.server.handler.ContextHandler - stopped
>> o.e.j.s.ServletContextHandler{/,file:/usr/home/mosipov/Proje
>> kte/maven-wagon/wagon-providers/wagon-http/target/test-output/}
>> ]]></system-err>
>>
>
> now take a look at TestSecurityHandler output. I added it after
> super.handle() called by Jetty. As you can see,
> "org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167
> (1)" is printed where the assertions kick in, after Wagon has completed,
> but the second request is handled afterwards. A traditional race condition.
>
> 2. Bugs found:
> Several auth tests fail for no reason too once in a while. Logging showed
> me that HttpClient simply did not send any Basic data, regardless if
> preemptive or after a roundtrip. The basic problem was that non-threadsafe
> classes have been used in a non-threadsafe manner as well as auth caches
> have not beed released properly, and preemptive auth was configured
> incorrectly. After fixing the issues one by one, the tests never failed
> again.
>
> I have tested the code on three platforms and six JDK versions tens of
> times, they never failed again. After we have resolved your HugeDowloadTest
> issue and no one else objects, I'd like to merge this into master and like
> you to reroll 2.11 with those changes.
>
> Meanwhile, I'll test this branch with Maven Core IT Suite too.
>
>
> Michael
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>

Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mi...@apache.org>.
Hi Dan,

Am 2016-12-25 um 19:51 schrieb Dan Tran:
> Just want to confirm,  your FreeBSD and build.apache.org's ubuntu see the
> same test failure??
>
> Do you think we should cancel the vote and wait for your fix?

I was finally able to complete my coding and testings. After several 
hours of testing and debugging, I've found the reasons why several tests 
were failing sporadically, some were bad test conditions 
(testSecuredGet(), testSecuredGetToStream()) and some rooted in bugs in 
HttpWagon. I'll go in detail:

1. Flaky testSecuredGet(ToStream):
This failed once in a while due to race conditions between Jetty's qtp 
threads and the main thread running the tests and the assertions. Jetty 
did not fully complete the process chain while Surefire continued to 
verify the result. This yielded to: expected <2> requests found <1> and 
so forth. I added a simple sleep() and it did work. I avoided to have 
complex sync code with CountDownLatch or alike. Some logging added to 
the code shows the following in Surefire's XML output:

> <failure message="not 2 security handler use [HandlerRequestResponse{method=&apos;GET&apos;, responseCode=401, requestUri=&apos;/test-secured-resource&apos;}] expected:&lt;2&gt; but was:&lt;1&gt;" type="junit.framework.AssertionFailedError"><![CDATA[junit.framework.AssertionFailedError: not 2 security handler use [HandlerRequestResponse{method='GET', responseCode=401, requestUri='/test-secured-resource'}] expected:<2> but was:<1>
>         at junit.framework.Assert.fail(Assert.java:57)
>         at junit.framework.Assert.failNotEquals(Assert.java:329)
>         at junit.framework.Assert.assertEquals(Assert.java:78)
>         at junit.framework.Assert.assertEquals(Assert.java:234)
>         at junit.framework.TestCase.assertEquals(TestCase.java:401)
>         at org.apache.maven.wagon.http.HttpWagonTestCase.testPreemptiveAuthentication(HttpWagonTestCase.java:1910)
>         at org.apache.maven.wagon.http.HttpWagonTestCase.testPreemptiveAuthenticationGet(HttpWagonTestCase.java:1891)
>         at org.apache.maven.wagon.http.HttpWagonTestCase.runTestSecuredGetToStream(HttpWagonTestCase.java:1409)
>         at org.apache.maven.wagon.http.HttpWagonTestCase.testSecuredGetToStream(HttpWagonTestCase.java:1374)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at junit.framework.TestCase.runTest(TestCase.java:176)
>         at junit.framework.TestCase.runBare(TestCase.java:141)
>         at junit.framework.TestResult$1.protect(TestResult.java:122)
>         at junit.framework.TestResult.runProtected(TestResult.java:142)
>         at junit.framework.TestResult.run(TestResult.java:125)
>         at junit.framework.TestCase.run(TestCase.java:129)
>         at junit.framework.TestSuite.runTest(TestSuite.java:255)
>         at junit.framework.TestSuite.run(TestSuite.java:250)
>         at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
>         at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
>         at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
>         at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
> ]]></failure>

now the logging:

> <system-err><![CDATA[[main] INFO org.eclipse.jetty.server.Server - jetty-8.1.22.v20160922
> [main] INFO org.eclipse.jetty.server.AbstractConnector - Started SelectChannelConnector@0.0.0.0:45658
> [main] DEBUG org.apache.http.client.protocol.RequestAddCookies - CookieSpec selected: compatibility
> [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection request: [route: {}->http://localhost:45658][total kept alive: 39; route allocated: 0 of 20; total allocated: 39 of 40]
> [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection leased: [id: 86][route: {}->http://localhost:45658][total kept alive: 39; route allocated: 1 of 20; total allocated: 40 of 40]
> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Opening connection {}->http://localhost:45658
> [main] DEBUG org.apache.http.impl.conn.DefaultHttpClientConnectionOperator - Connecting to localhost/127.0.0.1:45658
> [main] DEBUG org.apache.http.impl.conn.DefaultHttpClientConnectionOperator - Connection established 127.0.0.1:45659<->127.0.0.1:45658
> [main] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-86: set socket timeout to 1800000
> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Executing request GET /test-secured-resource HTTP/1.1
> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Target auth state: UNCHALLENGED
> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Proxy auth state: UNCHALLENGED
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> GET /test-secured-resource HTTP/1.1
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-control: no-cache
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-store: no-store
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Pragma: no-cache
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Expires: 0
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Accept-Encoding: gzip
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Host: localhost:45658
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Connection: Keep-Alive
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> User-Agent: Apache-HttpClient/4.5.2 (Java/1.7.0_111)
> [main] DEBUG org.apache.http.headers - http-outgoing-86 << HTTP/1.1 401 Unauthorized
> [main] DEBUG org.apache.http.headers - http-outgoing-86 << WWW-Authenticate: basic realm="MyRealm"
> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Cache-Control: must-revalidate,no-cache,no-store
> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Content-Type: text/html;charset=ISO-8859-1
> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Content-Length: 1311
> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Server: Jetty(8.1.22.v20160922)
> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Connection can be kept alive indefinitely
> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator - Authentication required
> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator - localhost:45658 requested authentication
> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy - Authentication schemes in the order of preference: [Negotiate, Kerberos, NTLM, Digest, Basic]
> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for Negotiate authentication scheme not available
> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for Kerberos authentication scheme not available
> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for NTLM authentication scheme not available
> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for Digest authentication scheme not available
> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator - Selected authentication options: [BASIC [complete=true]]
> [main] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-86: set socket timeout to 1800000
> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Executing request GET /test-secured-resource HTTP/1.1
> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Target auth state: CHALLENGED
> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator - Generating response to an authentication challenge using basic scheme
> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Proxy auth state: UNCHALLENGED
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> GET /test-secured-resource HTTP/1.1
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-control: no-cache
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Cache-store: no-store
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Pragma: no-cache
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Expires: 0
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Accept-Encoding: gzip
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Host: localhost:45658
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Connection: Keep-Alive
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> User-Agent: Apache-HttpClient/4.5.2 (Java/1.7.0_111)
> [main] DEBUG org.apache.http.headers - http-outgoing-86 >> Authorization: Basic dXNlcjpzZWNyZXQ=
> org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167 (count 1): handled GET /test-secured-resource with status 401
> [main] DEBUG org.apache.http.headers - http-outgoing-86 << HTTP/1.1 200 OK
> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Accept-Ranges: bytes
> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Content-Length: 10
> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Last-Modified: Mon, 26 Dec 2016 23:07:55 GMT
> [main] DEBUG org.apache.http.headers - http-outgoing-86 << Server: Jetty(8.1.22.v20160922)
> [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Connection can be kept alive indefinitely
> [main] DEBUG org.apache.http.impl.auth.HttpAuthenticator - Authentication succeeded
> [main] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy - Caching 'basic' auth scheme for http://localhost:45658
> [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection [id: 86][route: {}->http://localhost:45658] can be kept alive indefinitely
> [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection released: [id: 86][route: {}->http://localhost:45658][total kept alive: 40; route allocated: 1 of 20; total allocated: 40 of 40]
> org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167 (1)
> org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167 (count 2): handled GET /test-secured-resource with status 200
> [main] INFO org.eclipse.jetty.server.handler.ContextHandler - stopped o.e.j.s.ServletContextHandler{/,file:/usr/home/mosipov/Projekte/maven-wagon/wagon-providers/wagon-http/target/test-output/}
> ]]></system-err>

now take a look at TestSecurityHandler output. I added it after 
super.handle() called by Jetty. As you can see, 
"org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler@9187167 
(1)" is printed where the assertions kick in, after Wagon has completed, 
but the second request is handled afterwards. A traditional race condition.

2. Bugs found:
Several auth tests fail for no reason too once in a while. Logging 
showed me that HttpClient simply did not send any Basic data, regardless 
if preemptive or after a roundtrip. The basic problem was that 
non-threadsafe classes have been used in a non-threadsafe manner as well 
as auth caches have not beed released properly, and preemptive auth was 
configured incorrectly. After fixing the issues one by one, the tests 
never failed again.

I have tested the code on three platforms and six JDK versions tens of 
times, they never failed again. After we have resolved your 
HugeDowloadTest issue and no one else objects, I'd like to merge this 
into master and like you to reroll 2.11 with those changes.

Meanwhile, I'll test this branch with Maven Core IT Suite too.

Michael

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mi...@apache.org>.
Am 2016-12-25 um 19:51 schrieb Dan Tran:
> Just want to confirm,  your FreeBSD and build.apache.org's ubuntu see the
> same test failure??

As you have already noticed, the branch is up. It is preliminary work, 
but should be quite complete.

The runTestSecuredGet() and friends failures might be due to:
> [main] INFO org.eclipse.jetty.server.handler.ContextHandler - stopped o.e.j.s.ServletContextHandler{/,file:/usr/home/mosipov/Projekte/maven-wagon/wagon-providers/wagon-http/target/test-output/}
> [qtp1948835427-2572] WARN org.eclipse.jetty.server.Response - Committed before 500 org.eclipse.jetty.io.EofException
> [qtp1948835427-2572] WARN org.eclipse.jetty.server.AbstractHttpConnection - /test-secured-resource
> java.lang.IllegalStateException: Committed
>         at org.eclipse.jetty.server.Response.resetBuffer(Response.java:1145)
>         at org.eclipse.jetty.server.Response.sendError(Response.java:315)
>         at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:566)
>         at org.apache.maven.wagon.http.HttpWagonTestCase$TestSecurityHandler.handle(HttpWagonTestCase.java:2148)
>         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
>         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:427)
>         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
>         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
>         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>         at org.eclipse.jetty.server.Server.handle(Server.java:370)
>         at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
>         at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
>         at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:973)
>         at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1035)
>         at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:641)
>         at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:231)
>         at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
>         at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
>         at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:670)
>         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>         at java.lang.Thread.run(Thread.java:745)

I have them sporadically. According to Stackoverflow, the client closes 
the connection in flight, the server does not expect that and fails. 
Though the Java code is the same, it might be related to different 
socket approaches in Windows and Linux/FreeBSD.

Jetty's BasicAuthenticator sends 401 which fails with IOExeption, 
SecurityHandler catches and sends 500 which fails again. This is clearly 
a socket issue for me.
This requires tweaking the HttpClientConnectionManager.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mi...@apache.org>.
Am 2016-12-25 um 19:51 schrieb Dan Tran:
> Just want to confirm,  your FreeBSD and build.apache.org's ubuntu see the
> same test failure??

Yes, I had always the same two build failures as the Ubuntu box in 
contrast to Windows.
I am currently testing the same code on different JDKs also to see 
wether this is a JDK issue or test test setup issue.

Windows 10 Pro, 64 bit:
* Oracle JDK 7 Update 80: pass
* Oracle JDK 8 Update 112: pass (1)
* OpenJDK 7 (Azul Zulu) Update 121: pass

FreeBSD 10.3-RELEASE-p11, 64 bit:
* OpenJDK 7 Update 111: pass
* OpenJDK 8 Update 112: pass

All ran with Maven 3.3.9.

(1) Some HTTP Provider (preemptive) tests are flaky, I do not know why 
yet, strangely they always pass when run from inside Eclipse.

> Do you think we should cancel the vote and wait for your fix?

We could, yes. This would also include other bugfixes in the code, 
especially with HTTP PUT.

I'll try to push a feature branch on this today and will have you a look 
at it. If this is fine, we can merge into master and reroll the vote.

Michael

> On Sun, Dec 25, 2016 at 10:34 AM, Michael Osipov <mi...@apache.org>
> wrote:
>
>> Am 2016-12-25 um 18:45 schrieb Dan Tran:
>>
>>> Thank Michael
>>>
>>>   * Test failures at windows are intermittent.  It is a known issue since
>>> the last few versions
>>>   * No issue at my local redhat 7/java7 build
>>>   * I am seeing test failures at
>>> https://builds.apache.org/view/All/job/maven-wagon/1323/    starting with
>>> your change for WAGON-472. Dont think it is related, could be from build
>>> env
>>>
>>
>> This is build env. I have tested Wagon on Windows and FreeBSD before any
>> patching. Windows was fine, FreeBSD failed for those two cases. It also
>> passed on Windows with OpenJDK 7 Update 121 from Azul Systems. Though, I
>> have good news: I have been working on the unit tests in genereal the last
>> two days by migrating to Jetty 8 (last Java 1.6 supported) version and
>> found several bugs in Wagon HTTP Provider and the unit test setup itself. I
>> will report them shortly.
>>
>> Michael
>>
>>
>> On Sat, Dec 24, 2016 at 5:07 AM, Michael Osipov <mi...@apache.org>
>>> wrote:
>>>
>>> Am 2016-12-24 um 04:07 schrieb Dan Tran:
>>>>
>>>> Hi,
>>>>>
>>>>> We have fixed 18 issues:
>>>>> *https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>>>>> ectId=12318122&version=12333396
>>>>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>>>>> ectId=12318122&version=12333396>*
>>>>>
>>>>> There are still issues left in JIRA:
>>>>> *https://issues.apache.org/jira/issues/?jql=project%20%3D%
>>>>> 20WAGON%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%
>>>>> 20created%20DESC%2C%20priority%20DESC
>>>>> <https://issues.apache.org/jira/issues/?jql=project%20%3D%
>>>>> 20WAGON%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%
>>>>> 20created%20DESC%2C%20priority%20DESC>*
>>>>>
>>>>> Staging repository:
>>>>> https://repository.apache.org/content/repositories/maven-1313
>>>>> https://repository.apache.org/content/repositories/maven-
>>>>> 1313/org/apache/maven/wagon/wagon/2.11/wagon-2.11-source-release.zip
>>>>>
>>>>> Source release checksum(s):
>>>>> wagon-2.11-source-release.zip sha1: 98bc0b031880b175e144c55447a620
>>>>> 00c58abfa5
>>>>>
>>>>> Staging site:
>>>>> *http://maven.apache.org/components/wagon-archives/wagon-LATEST/
>>>>> <http://maven.apache.org/components/wagon-archives/wagon-LATEST/>*
>>>>>
>>>>> Guide to testing staged releases:
>>>>> https://maven.apache.org/guides/development/guide-testing-releases.html
>>>>> <https://maven.apache.org/guides/development/guide-testing-
>>>>> releases.html>
>>>>> Vote open for 72H + 48H to accomodate for holidays
>>>>>
>>>>>
>>>> +1
>>>>
>>>> SHA1 is fine, site is fine. Builds passes on Windows but constantly fails
>>>> on non-Windows, here on my FreeBSD machine as well as on our Jenkins
>>>> Ubuntu
>>>> slave. It should probably investigated for the 3.0 release.
>>>>
>>>> Michael
>>>>
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>>
>>>>
>>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Dan Tran <da...@gmail.com>.
Just want to confirm,  your FreeBSD and build.apache.org's ubuntu see the
same test failure??

Do you think we should cancel the vote and wait for your fix?

Thanks

-Dan

On Sun, Dec 25, 2016 at 10:34 AM, Michael Osipov <mi...@apache.org>
wrote:

> Am 2016-12-25 um 18:45 schrieb Dan Tran:
>
>> Thank Michael
>>
>>   * Test failures at windows are intermittent.  It is a known issue since
>> the last few versions
>>   * No issue at my local redhat 7/java7 build
>>   * I am seeing test failures at
>> https://builds.apache.org/view/All/job/maven-wagon/1323/    starting with
>> your change for WAGON-472. Dont think it is related, could be from build
>> env
>>
>
> This is build env. I have tested Wagon on Windows and FreeBSD before any
> patching. Windows was fine, FreeBSD failed for those two cases. It also
> passed on Windows with OpenJDK 7 Update 121 from Azul Systems. Though, I
> have good news: I have been working on the unit tests in genereal the last
> two days by migrating to Jetty 8 (last Java 1.6 supported) version and
> found several bugs in Wagon HTTP Provider and the unit test setup itself. I
> will report them shortly.
>
> Michael
>
>
> On Sat, Dec 24, 2016 at 5:07 AM, Michael Osipov <mi...@apache.org>
>> wrote:
>>
>> Am 2016-12-24 um 04:07 schrieb Dan Tran:
>>>
>>> Hi,
>>>>
>>>> We have fixed 18 issues:
>>>> *https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>>>> ectId=12318122&version=12333396
>>>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>>>> ectId=12318122&version=12333396>*
>>>>
>>>> There are still issues left in JIRA:
>>>> *https://issues.apache.org/jira/issues/?jql=project%20%3D%
>>>> 20WAGON%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%
>>>> 20created%20DESC%2C%20priority%20DESC
>>>> <https://issues.apache.org/jira/issues/?jql=project%20%3D%
>>>> 20WAGON%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%
>>>> 20created%20DESC%2C%20priority%20DESC>*
>>>>
>>>> Staging repository:
>>>> https://repository.apache.org/content/repositories/maven-1313
>>>> https://repository.apache.org/content/repositories/maven-
>>>> 1313/org/apache/maven/wagon/wagon/2.11/wagon-2.11-source-release.zip
>>>>
>>>> Source release checksum(s):
>>>> wagon-2.11-source-release.zip sha1: 98bc0b031880b175e144c55447a620
>>>> 00c58abfa5
>>>>
>>>> Staging site:
>>>> *http://maven.apache.org/components/wagon-archives/wagon-LATEST/
>>>> <http://maven.apache.org/components/wagon-archives/wagon-LATEST/>*
>>>>
>>>> Guide to testing staged releases:
>>>> https://maven.apache.org/guides/development/guide-testing-releases.html
>>>> <https://maven.apache.org/guides/development/guide-testing-
>>>> releases.html>
>>>> Vote open for 72H + 48H to accomodate for holidays
>>>>
>>>>
>>> +1
>>>
>>> SHA1 is fine, site is fine. Builds passes on Windows but constantly fails
>>> on non-Windows, here on my FreeBSD machine as well as on our Jenkins
>>> Ubuntu
>>> slave. It should probably investigated for the 3.0 release.
>>>
>>> Michael
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>>> For additional commands, e-mail: dev-help@maven.apache.org
>>>
>>>
>>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>

Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mi...@apache.org>.
Am 2016-12-25 um 18:45 schrieb Dan Tran:
> Thank Michael
>
>   * Test failures at windows are intermittent.  It is a known issue since
> the last few versions
>   * No issue at my local redhat 7/java7 build
>   * I am seeing test failures at
> https://builds.apache.org/view/All/job/maven-wagon/1323/    starting with
> your change for WAGON-472. Dont think it is related, could be from build env

This is build env. I have tested Wagon on Windows and FreeBSD before any 
patching. Windows was fine, FreeBSD failed for those two cases. It also 
passed on Windows with OpenJDK 7 Update 121 from Azul Systems. Though, I 
have good news: I have been working on the unit tests in genereal the 
last two days by migrating to Jetty 8 (last Java 1.6 supported) version 
and found several bugs in Wagon HTTP Provider and the unit test setup 
itself. I will report them shortly.

Michael

> On Sat, Dec 24, 2016 at 5:07 AM, Michael Osipov <mi...@apache.org> wrote:
>
>> Am 2016-12-24 um 04:07 schrieb Dan Tran:
>>
>>> Hi,
>>>
>>> We have fixed 18 issues:
>>> *https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>>> ectId=12318122&version=12333396
>>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>>> ectId=12318122&version=12333396>*
>>>
>>> There are still issues left in JIRA:
>>> *https://issues.apache.org/jira/issues/?jql=project%20%3D%
>>> 20WAGON%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%
>>> 20created%20DESC%2C%20priority%20DESC
>>> <https://issues.apache.org/jira/issues/?jql=project%20%3D%
>>> 20WAGON%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%
>>> 20created%20DESC%2C%20priority%20DESC>*
>>>
>>> Staging repository:
>>> https://repository.apache.org/content/repositories/maven-1313
>>> https://repository.apache.org/content/repositories/maven-
>>> 1313/org/apache/maven/wagon/wagon/2.11/wagon-2.11-source-release.zip
>>>
>>> Source release checksum(s):
>>> wagon-2.11-source-release.zip sha1: 98bc0b031880b175e144c55447a620
>>> 00c58abfa5
>>>
>>> Staging site:
>>> *http://maven.apache.org/components/wagon-archives/wagon-LATEST/
>>> <http://maven.apache.org/components/wagon-archives/wagon-LATEST/>*
>>>
>>> Guide to testing staged releases:
>>> https://maven.apache.org/guides/development/guide-testing-releases.html
>>> <https://maven.apache.org/guides/development/guide-testing-releases.html>
>>> Vote open for 72H + 48H to accomodate for holidays
>>>
>>
>> +1
>>
>> SHA1 is fine, site is fine. Builds passes on Windows but constantly fails
>> on non-Windows, here on my FreeBSD machine as well as on our Jenkins Ubuntu
>> slave. It should probably investigated for the 3.0 release.
>>
>> Michael
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org


Re: [VOTE] Releasing Wagon 2.11

Posted by Dan Tran <da...@gmail.com>.
Thank Michael

  * Test failures at windows are intermittent.  It is a known issue since
the last few versions
  * No issue at my local redhat 7/java7 build
  * I am seeing test failures at
https://builds.apache.org/view/All/job/maven-wagon/1323/    starting with
your change for WAGON-472. Dont think it is related, could be from build env


Thanks

On Sat, Dec 24, 2016 at 5:07 AM, Michael Osipov <mi...@apache.org> wrote:

> Am 2016-12-24 um 04:07 schrieb Dan Tran:
>
>> Hi,
>>
>> We have fixed 18 issues:
>> *https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12318122&version=12333396
>> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?proj
>> ectId=12318122&version=12333396>*
>>
>> There are still issues left in JIRA:
>> *https://issues.apache.org/jira/issues/?jql=project%20%3D%
>> 20WAGON%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%
>> 20created%20DESC%2C%20priority%20DESC
>> <https://issues.apache.org/jira/issues/?jql=project%20%3D%
>> 20WAGON%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%
>> 20created%20DESC%2C%20priority%20DESC>*
>>
>> Staging repository:
>> https://repository.apache.org/content/repositories/maven-1313
>> https://repository.apache.org/content/repositories/maven-
>> 1313/org/apache/maven/wagon/wagon/2.11/wagon-2.11-source-release.zip
>>
>> Source release checksum(s):
>> wagon-2.11-source-release.zip sha1: 98bc0b031880b175e144c55447a620
>> 00c58abfa5
>>
>> Staging site:
>> *http://maven.apache.org/components/wagon-archives/wagon-LATEST/
>> <http://maven.apache.org/components/wagon-archives/wagon-LATEST/>*
>>
>> Guide to testing staged releases:
>> https://maven.apache.org/guides/development/guide-testing-releases.html
>> <https://maven.apache.org/guides/development/guide-testing-releases.html>
>> Vote open for 72H + 48H to accomodate for holidays
>>
>
> +1
>
> SHA1 is fine, site is fine. Builds passes on Windows but constantly fails
> on non-Windows, here on my FreeBSD machine as well as on our Jenkins Ubuntu
> slave. It should probably investigated for the 3.0 release.
>
> Michael
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>

Re: [VOTE] Releasing Wagon 2.11

Posted by Michael Osipov <mi...@apache.org>.
Am 2016-12-24 um 04:07 schrieb Dan Tran:
> Hi,
>
> We have fixed 18 issues:
> *https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318122&version=12333396
> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318122&version=12333396>*
>
> There are still issues left in JIRA:
> *https://issues.apache.org/jira/issues/?jql=project%20%3D%20WAGON%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20created%20DESC%2C%20priority%20DESC
> <https://issues.apache.org/jira/issues/?jql=project%20%3D%20WAGON%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20created%20DESC%2C%20priority%20DESC>*
>
> Staging repository:
> https://repository.apache.org/content/repositories/maven-1313
> https://repository.apache.org/content/repositories/maven-
> 1313/org/apache/maven/wagon/wagon/2.11/wagon-2.11-source-release.zip
>
> Source release checksum(s):
> wagon-2.11-source-release.zip sha1: 98bc0b031880b175e144c55447a62000c58abfa5
>
> Staging site:
> *http://maven.apache.org/components/wagon-archives/wagon-LATEST/
> <http://maven.apache.org/components/wagon-archives/wagon-LATEST/>*
>
> Guide to testing staged releases:
> https://maven.apache.org/guides/development/guide-testing-releases.html
> <https://maven.apache.org/guides/development/guide-testing-releases.html>
> Vote open for 72H + 48H to accomodate for holidays

+1

SHA1 is fine, site is fine. Builds passes on Windows but constantly 
fails on non-Windows, here on my FreeBSD machine as well as on our 
Jenkins Ubuntu slave. It should probably investigated for the 3.0 release.

Michael




---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org