You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by GitBox <gi...@apache.org> on 2021/12/17 11:00:58 UTC
[GitHub] [maven-mvnd] gnodet commented on pull request #538: Fastclean extension
gnodet commented on pull request #538:
URL: https://github.com/apache/maven-mvnd/pull/538#issuecomment-996632492
> Wow! You... are... quick!! :) Thank you so much for taking the time to test out the idea.
>
> So I gave it a shot and strangely, despite you spinning up a new thread, the maven clean goal didn't finish until the thread completed. I don't really understand why. I made some changes to the code by using a ExecutorService (might be good to serialize the deletion as well):
>
> `private static ExecutorService executorService = Executors.newSingleThreadExecutor();`
>
> and then
>
> `executorService.submit(() -> deleteDir(delDir, retryOnError));`
>
> and then it seem to finish the maven-goal right away (and process the delete in the background)
>
> To summarize the results a bit for my specific case (13 projects in build):
>
> **Time spent when using the fastclean plugin:** clean goal: 174 ms total build time: 01:51 min (Wall Clock)
>
> **Time spent when using the stock maven clean plugin:** clean goal: 62263 ms total build time: 02:15 min (Wall Clock)
>
> So approx. 20% reduction in clean&build time! I think that's definitely worth something :) Could possibly be sped up further by delaying the deletes (or lowering priority of it)
>
> (Using the fastclean plugin in mvn with -T1 it took 536 ms)
>
> So I'm really hopeful this could go somewhere. However, it feels like this has to be run within the maven daemon, as opposed to just maven, in order to be able to run this properly. Otherwise we'll take the hit during maven shutdown.
>
> Once again, thanks a lot! Atli
>
> ps. btw, I also skipped the tmpDir and renamed directly (instead of moving the delDir to the target folder). I did that because if you start a new build while the delete is still ongoing, you get into problems:
>
> ```
> String name = directory.getFileName().toString() + "-" + random;
> Path delDir = directory.resolveSibling(name);
> try {
> Files.move(directory, delDir);
> Files.createDirectory(directory);
> ```
>
> Some additional numbers from my tests:
>
> [INFO] Plugins in lifecycle Phases: [INFO] [INFO] clean: [INFO] 206 ms: org.apache.maven.plugins:maven-clean-plugin:2.5:clean:default-clean [INFO] 2 ms: org.apache.maven.plugins:maven-clean-plugin:2.5:clean:default-clean [INFO] 470 ms: org.apache.maven.plugins:maven-clean-plugin:3.1.0:clean:default-clean [INFO] 11531 ms: org.apache.maven.plugins:maven-clean-plugin:2.5:clean:default-clean [INFO] 9513 ms: org.apache.maven.plugins:maven-clean-plugin:2.5:clean:default-clean [INFO] 2 ms: org.apache.maven.plugins:maven-clean-plugin:2.5:clean:default-clean [INFO] 12009 ms: org.apache.maven.plugins:maven-clean-plugin:2.5:clean:default-clean [INFO] 1926 ms: org.apache.maven.plugins:maven-clean-plugin:2.5:clean:default-clean [INFO] 2 ms: org.apache.maven.plugins:maven-clean-plugin:2.5:clean:default-clean [INFO] 2377 ms: org.apache.maven.plugins:maven-clean-plugin:2.5:clean:default-clean [INFO] 4097 ms: org.apache.maven.plugins:maven-clean-plugin:2.5:clean:default-clean [INFO] 2432 ms: org
.apache.maven.plugins:maven-clean-plugin:2.5:clean:default-clean [INFO] 17696 ms: org.apache.maven.plugins:maven-clean-plugin:2.5:clean:default-clean
>
> **vs fastclean:**
>
> [INFO] Plugins in lifecycle Phases: [INFO] [INFO] clean: [INFO] 23 ms: org.mvndaemon.mvnd:mvnd-fastclean-maven-plugin:0.7.2-SNAPSHOT:clean:default-clean [INFO] 19 ms: org.mvndaemon.mvnd:mvnd-fastclean-maven-plugin:0.7.2-SNAPSHOT:clean:default-clean [INFO] 2 ms: org.mvndaemon.mvnd:mvnd-fastclean-maven-plugin:0.7.2-SNAPSHOT:clean:default-clean [INFO] 12 ms: org.mvndaemon.mvnd:mvnd-fastclean-maven-plugin:0.7.2-SNAPSHOT:clean:default-clean [INFO] 20 ms: org.mvndaemon.mvnd:mvnd-fastclean-maven-plugin:0.7.2-SNAPSHOT:clean:default-clean [INFO] 10 ms: org.mvndaemon.mvnd:mvnd-fastclean-maven-plugin:0.7.2-SNAPSHOT:clean:default-clean [INFO] 15 ms: org.mvndaemon.mvnd:mvnd-fastclean-maven-plugin:0.7.2-SNAPSHOT:clean:default-clean [INFO] 13 ms: org.mvndaemon.mvnd:mvnd-fastclean-maven-plugin:0.7.2-SNAPSHOT:clean:default-clean [INFO] 16 ms: org.mvndaemon.mvnd:mvnd-fastclean-maven-plugin:0.7.2-SNAPSHOT:clean:default-clean [INFO] 9 ms: org.mvndaemon.mvnd:mvnd-fastclean-maven-plugin:0.7.2-SNAPSHO
T:clean:default-clean [INFO] 2 ms: org.mvndaemon.mvnd:mvnd-fastclean-maven-plugin:0.7.2-SNAPSHOT:clean:default-clean [INFO] 16 ms: org.mvndaemon.mvnd:mvnd-fastclean-maven-plugin:0.7.2-SNAPSHOT:clean:default-clean [INFO] 17 ms: org.mvndaemon.mvnd:mvnd-fastclean-maven-plugin:0.7.2-SNAPSHOT:clean:default-clean
It would be worth trying with the modified [maven-clean-plugin](https://github.com/apache/maven-clean-plugin/pull/6) .
I've added a join which cause maven to wait until the deletion is finished, but this should only happen at the end of the maven execution. My assumption is that if it's waiting, you can either kill maven (and only the `[root]/target/.clean` directory will remain), or if you used `mvn clean verify` for example, the deletion should occur in parallel with the actual build.
I do encourage you to continue the discussion in https://github.com/apache/maven-clean-plugin/pull/6, in particular, if you could provide a project where the problem can be reproduced to further test, it would be awesome.
I'm closing this one as I don't see any pushback to improve the `maven-clean-plugin`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org