You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ant.apache.org by "Oliver Jowett (Created) (JIRA)" <ji...@apache.org> on 2011/10/03 04:36:33 UTC
[jira] [Created] (IVY-1318) Faster symlink creation (avoid a JVM
fork per symlink)
Faster symlink creation (avoid a JVM fork per symlink)
------------------------------------------------------
Key: IVY-1318
URL: https://issues.apache.org/jira/browse/IVY-1318
Project: Ivy
Issue Type: New Feature
Components: Core
Affects Versions: 2.2.0
Reporter: Oliver Jowett
Priority: Minor
We have a large project using Ivy with many large artifacts.
In theory we should benefit from symlink="true" (space savings and reduced I/O), but in practice the overhead of forking the Ant JVM for each individual symlink means that a build using symlink="true" actually runs a lot (30%+) slower, which is too much of a cost to justify using it.
To work around this I modified FileUtils to run a single long-lived shell process that reads instructions from stdin and performs symlinks accordingly.
This makes symlink="true" no slower than symlink="false".
I will attach a patch against 2.2.0. Tested only in a Linux environment, but in theory it should work on anything POSIX-y, and should fall back to file copy much as the existing code does if anything goes wrong.
Any interest in getting this included in an official version?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (IVY-1318) Faster symlink creation (avoid a JVM
fork per symlink)
Posted by "Oliver Jowett (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oliver Jowett updated IVY-1318:
-------------------------------
Attachment: ivy-2.2.0-symlink.txt
Attached ivy-2.2.0-symlink.txt, a unified diff against Ivy 2.2.0.
> Faster symlink creation (avoid a JVM fork per symlink)
> ------------------------------------------------------
>
> Key: IVY-1318
> URL: https://issues.apache.org/jira/browse/IVY-1318
> Project: Ivy
> Issue Type: New Feature
> Components: Core
> Affects Versions: 2.2.0
> Reporter: Oliver Jowett
> Priority: Minor
> Attachments: ivy-2.2.0-symlink.txt
>
>
> We have a large project using Ivy with many large artifacts.
> In theory we should benefit from symlink="true" (space savings and reduced I/O), but in practice the overhead of forking the Ant JVM for each individual symlink means that a build using symlink="true" actually runs a lot (30%+) slower, which is too much of a cost to justify using it.
> To work around this I modified FileUtils to run a single long-lived shell process that reads instructions from stdin and performs symlinks accordingly.
> This makes symlink="true" no slower than symlink="false".
> I will attach a patch against 2.2.0. Tested only in a Linux environment, but in theory it should work on anything POSIX-y, and should fall back to file copy much as the existing code does if anything goes wrong.
> Any interest in getting this included in an official version?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (IVY-1318) Faster symlink creation (avoid a JVM
fork per symlink)
Posted by "Nicolas Lalevée (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nicolas Lalevée updated IVY-1318:
---------------------------------
Attachment: IVY-1318-r1373520.patch
patch rebased on the current trunk.
> Faster symlink creation (avoid a JVM fork per symlink)
> ------------------------------------------------------
>
> Key: IVY-1318
> URL: https://issues.apache.org/jira/browse/IVY-1318
> Project: Ivy
> Issue Type: New Feature
> Components: Core
> Affects Versions: 2.2.0
> Reporter: Oliver Jowett
> Priority: Minor
> Labels: patch
> Attachments: IVY-1318-r1373520.patch, ivy-2.2.0-symlink.txt
>
>
> We have a large project using Ivy with many large artifacts.
> In theory we should benefit from symlink="true" (space savings and reduced I/O), but in practice the overhead of forking the Ant JVM for each individual symlink means that a build using symlink="true" actually runs a lot (30%+) slower, which is too much of a cost to justify using it.
> To work around this I modified FileUtils to run a single long-lived shell process that reads instructions from stdin and performs symlinks accordingly.
> This makes symlink="true" no slower than symlink="false".
> I will attach a patch against 2.2.0. Tested only in a Linux environment, but in theory it should work on anything POSIX-y, and should fall back to file copy much as the existing code does if anything goes wrong.
> Any interest in getting this included in an official version?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IVY-1318) Faster symlink creation (avoid a JVM
fork per symlink)
Posted by "Nicolas Lalevée (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435276#comment-13435276 ]
Nicolas Lalevée commented on IVY-1318:
--------------------------------------
You patch is interesting but it doesn't seem thread safe. It also use Java 1.5 feature, and now Ivy is still requiring to be Java 1.4 compatible.
And as far as I understand, on long running jvm, it means another process should live as long as Ivy does. Ivy is used in long running jvm like Eclipse, for the plugin IvyDE. Not sure it is best.
I think I'll prefer IVY-1252.
> Faster symlink creation (avoid a JVM fork per symlink)
> ------------------------------------------------------
>
> Key: IVY-1318
> URL: https://issues.apache.org/jira/browse/IVY-1318
> Project: Ivy
> Issue Type: New Feature
> Components: Core
> Affects Versions: 2.2.0
> Reporter: Oliver Jowett
> Priority: Minor
> Labels: patch
> Attachments: IVY-1318-r1373520.patch, ivy-2.2.0-symlink.txt
>
>
> We have a large project using Ivy with many large artifacts.
> In theory we should benefit from symlink="true" (space savings and reduced I/O), but in practice the overhead of forking the Ant JVM for each individual symlink means that a build using symlink="true" actually runs a lot (30%+) slower, which is too much of a cost to justify using it.
> To work around this I modified FileUtils to run a single long-lived shell process that reads instructions from stdin and performs symlinks accordingly.
> This makes symlink="true" no slower than symlink="false".
> I will attach a patch against 2.2.0. Tested only in a Linux environment, but in theory it should work on anything POSIX-y, and should fall back to file copy much as the existing code does if anything goes wrong.
> Any interest in getting this included in an official version?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IVY-1318) Faster symlink creation (avoid a JVM
fork per symlink)
Posted by "Oliver Jowett (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119129#comment-13119129 ]
Oliver Jowett commented on IVY-1318:
------------------------------------
IVY-1252 seems to be similar in intent (but a totally different implementation)
> Faster symlink creation (avoid a JVM fork per symlink)
> ------------------------------------------------------
>
> Key: IVY-1318
> URL: https://issues.apache.org/jira/browse/IVY-1318
> Project: Ivy
> Issue Type: New Feature
> Components: Core
> Affects Versions: 2.2.0
> Reporter: Oliver Jowett
> Priority: Minor
> Attachments: ivy-2.2.0-symlink.txt
>
>
> We have a large project using Ivy with many large artifacts.
> In theory we should benefit from symlink="true" (space savings and reduced I/O), but in practice the overhead of forking the Ant JVM for each individual symlink means that a build using symlink="true" actually runs a lot (30%+) slower, which is too much of a cost to justify using it.
> To work around this I modified FileUtils to run a single long-lived shell process that reads instructions from stdin and performs symlinks accordingly.
> This makes symlink="true" no slower than symlink="false".
> I will attach a patch against 2.2.0. Tested only in a Linux environment, but in theory it should work on anything POSIX-y, and should fall back to file copy much as the existing code does if anything goes wrong.
> Any interest in getting this included in an official version?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira