You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geronimo.apache.org by "Gao Yong Pan (JIRA)" <de...@geronimo.apache.org> on 2005/11/23 08:59:36 UTC
[jira] Created: (GERONIMO-1223) Ho
Ho
--
Key: GERONIMO-1223
URL: http://issues.apache.org/jira/browse/GERONIMO-1223
Project: Geronimo
Type: Improvement
Reporter: Gao Yong Pan
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Closed: (GERONIMO-1223) Hot Deploy performance improvement
Just check directories -patch available
Posted by "Vamsavardhana Reddy (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/GERONIMO-1223?page=all ]
Vamsavardhana Reddy closed GERONIMO-1223.
-----------------------------------------
> Hot Deploy performance improvement Just check directories -patch available
> --------------------------------------------------------------------------
>
> Key: GERONIMO-1223
> URL: http://issues.apache.org/jira/browse/GERONIMO-1223
> Project: Geronimo
> Issue Type: Improvement
> Security Level: public(Regular issues)
> Components: deployment
> Affects Versions: 1.0
> Environment: linux or windows
> Reporter: Gao Yong Pan
> Assigned To: Aaron Mulder
> Fix For: 1.0
>
> Attachments: DirectoryMonitor.java.patch
>
>
> The method getLastModifiedInDir in the hot-deploy/src/java/org/apache/geronimo/deployment/hot/DirectoryMonitor.java is using the algorithm by calculating highest modified time of any files or directories, this could be improved.
> In fact, we only need to check every directory is enough, because any changes, for example, add, remove, edit any files in a directory, the change can be reflected by its current directory's modification time, so we don't need to check files.
> I also write a patch for the issue. A simple performance testing shows that the patch will improve 58% of the performance.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (GERONIMO-1223) Hot Deploy performance improvement Just check directories -patch available
Posted by "Gao Yong Pan (JIRA)" <de...@geronimo.apache.org>.
[ http://issues.apache.org/jira/browse/GERONIMO-1223?page=all ]
Gao Yong Pan updated GERONIMO-1223:
-----------------------------------
Description:
The method getLastModifiedInDir in the hot-deploy/src/java/org/apache/geronimo/deployment/hot/DirectoryMonitor.java is using the algorithm by calculating highest modified time of any files or directories, this could be improved.
In fact, we only need to check every directory is enough, because any changes, for example, add, remove, edit any files in a directory, the change can be reflected by its current directory's modification time, so we don't need to check files.
I also write a patch for the issue. A simple performance testing shows that the patch will improve 58% of the performance.
was:
The method getLastModifiedInDir in the hot-deploy/src/java/org/apache/geronimo/deployment/hot/DirectoryMonitor.java is using the algorithm by calculating highest modified time of any files or directories, this could be improved.
In fact, we only need to check every directory is enough, because any changes, for example, add, remove, edit any files in a directory, the change can be reflected by its parent directory's modification time, so we don't need to check files.
I also write a patch for the issue. A simple performance testing shows that the patch will improve 58% of the performance.
> Hot Deploy performance improvement Just check directories -patch available
> --------------------------------------------------------------------------
>
> Key: GERONIMO-1223
> URL: http://issues.apache.org/jira/browse/GERONIMO-1223
> Project: Geronimo
> Type: Improvement
> Components: deployment
> Versions: 1.0
> Environment: linux or windows
> Reporter: Gao Yong Pan
> Attachments: DirectoryMonitor.java.patch
>
> The method getLastModifiedInDir in the hot-deploy/src/java/org/apache/geronimo/deployment/hot/DirectoryMonitor.java is using the algorithm by calculating highest modified time of any files or directories, this could be improved.
> In fact, we only need to check every directory is enough, because any changes, for example, add, remove, edit any files in a directory, the change can be reflected by its current directory's modification time, so we don't need to check files.
> I also write a patch for the issue. A simple performance testing shows that the patch will improve 58% of the performance.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Resolved: (GERONIMO-1223) Hot Deploy performance improvement Just check directories -patch available
Posted by "Aaron Mulder (JIRA)" <de...@geronimo.apache.org>.
[ http://issues.apache.org/jira/browse/GERONIMO-1223?page=all ]
Aaron Mulder resolved GERONIMO-1223:
------------------------------------
Fix Version: 1.0
Resolution: Won't Fix
Assign To: Aaron Mulder
Given that the patch won't work, I'm putting this issue on ice for now. Please reopen if you find another workaround.
> Hot Deploy performance improvement Just check directories -patch available
> --------------------------------------------------------------------------
>
> Key: GERONIMO-1223
> URL: http://issues.apache.org/jira/browse/GERONIMO-1223
> Project: Geronimo
> Type: Improvement
> Components: deployment
> Versions: 1.0
> Environment: linux or windows
> Reporter: Gao Yong Pan
> Assignee: Aaron Mulder
> Fix For: 1.0
> Attachments: DirectoryMonitor.java.patch
>
> The method getLastModifiedInDir in the hot-deploy/src/java/org/apache/geronimo/deployment/hot/DirectoryMonitor.java is using the algorithm by calculating highest modified time of any files or directories, this could be improved.
> In fact, we only need to check every directory is enough, because any changes, for example, add, remove, edit any files in a directory, the change can be reflected by its current directory's modification time, so we don't need to check files.
> I also write a patch for the issue. A simple performance testing shows that the patch will improve 58% of the performance.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (GERONIMO-1223) Hot Deploy performance improvement Just check directories -patch available
Posted by "Aaron Mulder (JIRA)" <de...@geronimo.apache.org>.
[ http://issues.apache.org/jira/browse/GERONIMO-1223?page=comments#action_12358377 ]
Aaron Mulder commented on GERONIMO-1223:
----------------------------------------
The intended design was that if you put in only JAR files it will never resort to scanning through directories, but if you put in an exploded JAR, it will be sure to always notice when that is changed, even if you only edit or copy in a single new JSP. So before implement the behavior described here and in the patch, we should make sure it notices updates where you:
1) copy over the existing directory with a new copy with a file added or changed
2) unzip/untar a new copy of the directory with a file added or changed
3) copy in a new version of one file
4) copy in an updated version of one file
5) create a new file in place with an editor
6) update a new file in place with an editor
7) "touch" a file
I can't confirm or deny whether the patch affects that, so I hope someone can test for sure on Windows, Mac, and Linux.
I am curious, as to the 58%, what is the actual time taken and for what size unpacked directory structure (in terms of file count and MB)? If it's 58% of 50 milliseconds I'm not so concerned. If it's 58% of 35 seconds, that's a little disturbing.
I wouldn't be opposed to adding a GBean property to make it work like the patch describes and just disabling that by default until we can confirm whether the conditions abover are still met.
I also wonder whether there's any faster way to scan a directory structure than File.listFiles and so on -- I didn't see anything obvious in NIO for instance, but I didn't look too hard either.
> Hot Deploy performance improvement Just check directories -patch available
> --------------------------------------------------------------------------
>
> Key: GERONIMO-1223
> URL: http://issues.apache.org/jira/browse/GERONIMO-1223
> Project: Geronimo
> Type: Improvement
> Components: deployment
> Versions: 1.0
> Environment: linux or windows
> Reporter: Gao Yong Pan
> Attachments: DirectoryMonitor.java.patch
>
> The method getLastModifiedInDir in the hot-deploy/src/java/org/apache/geronimo/deployment/hot/DirectoryMonitor.java is using the algorithm by calculating highest modified time of any files or directories, this could be improved.
> In fact, we only need to check every directory is enough, because any changes, for example, add, remove, edit any files in a directory, the change can be reflected by its parent directory's modification time, so we don't need to check files.
> I also write a patch for the issue. A simple performance testing shows that the patch will improve 58% of the performance.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (GERONIMO-1223) Hot Deploy performance improvement Just check directories -patch available
Posted by "Gao Yong Pan (JIRA)" <de...@geronimo.apache.org>.
[ http://issues.apache.org/jira/browse/GERONIMO-1223?page=all ]
Gao Yong Pan updated GERONIMO-1223:
-----------------------------------
Attachment: DirectoryMonitor.java.patch
> Hot Deploy performance improvement Just check directories -patch available
> --------------------------------------------------------------------------
>
> Key: GERONIMO-1223
> URL: http://issues.apache.org/jira/browse/GERONIMO-1223
> Project: Geronimo
> Type: Improvement
> Components: deployment
> Versions: 1.0
> Environment: linux or windows
> Reporter: Gao Yong Pan
> Attachments: DirectoryMonitor.java.patch
>
> The method getLastModifiedInDir in the hot-deploy/src/java/org/apache/geronimo/deployment/hot/DirectoryMonitor.java is using the algorithm by calculating highest modified time of any files or directories, this could be improved.
> In fact, we only need to check every directory is enough, because any changes, for example, add, remove, edit any files in a directory, the change can be reflected by its parent directory's modification time, so we don't need to check files.
> I also write a patch for the issue. A simple performance testing shows that the patch will improve 58% of the performance.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (GERONIMO-1223) Hot Deploy performance improvement Just check directories -patch available
Posted by "Gao Yong Pan (JIRA)" <de...@geronimo.apache.org>.
[ http://issues.apache.org/jira/browse/GERONIMO-1223?page=comments#action_12358489 ]
Gao Yong Pan commented on GERONIMO-1223:
----------------------------------------
I found one situation that the patch won't work.
[pgy@localhost sample]$ cp file2 dir1
[pgy@localhost sample]$ stat -c %y dir1/file2 dir1
2005-11-25 16:29:20.522243936 +0800
2005-11-25 16:28:27.028376240 +0800
Please discard the patch.
> Hot Deploy performance improvement Just check directories -patch available
> --------------------------------------------------------------------------
>
> Key: GERONIMO-1223
> URL: http://issues.apache.org/jira/browse/GERONIMO-1223
> Project: Geronimo
> Type: Improvement
> Components: deployment
> Versions: 1.0
> Environment: linux or windows
> Reporter: Gao Yong Pan
> Attachments: DirectoryMonitor.java.patch
>
> The method getLastModifiedInDir in the hot-deploy/src/java/org/apache/geronimo/deployment/hot/DirectoryMonitor.java is using the algorithm by calculating highest modified time of any files or directories, this could be improved.
> In fact, we only need to check every directory is enough, because any changes, for example, add, remove, edit any files in a directory, the change can be reflected by its current directory's modification time, so we don't need to check files.
> I also write a patch for the issue. A simple performance testing shows that the patch will improve 58% of the performance.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (GERONIMO-1223) Hot Deploy performance improvement Just check directories -patch available
Posted by "Jeff Genender (JIRA)" <de...@geronimo.apache.org>.
[ http://issues.apache.org/jira/browse/GERONIMO-1223?page=comments#action_12358373 ]
Jeff Genender commented on GERONIMO-1223:
-----------------------------------------
The 58% can only depend on the number of files below the directory. That number would obviously change on the number of deployables.
Also, this patch will not work when you "touch" a file as it does not update the directory stats.
> Hot Deploy performance improvement Just check directories -patch available
> --------------------------------------------------------------------------
>
> Key: GERONIMO-1223
> URL: http://issues.apache.org/jira/browse/GERONIMO-1223
> Project: Geronimo
> Type: Improvement
> Components: deployment
> Versions: 1.0
> Environment: linux or windows
> Reporter: Gao Yong Pan
> Attachments: DirectoryMonitor.java.patch
>
> The method getLastModifiedInDir in the hot-deploy/src/java/org/apache/geronimo/deployment/hot/DirectoryMonitor.java is using the algorithm by calculating highest modified time of any files or directories, this could be improved.
> In fact, we only need to check every directory is enough, because any changes, for example, add, remove, edit any files in a directory, the change can be reflected by its parent directory's modification time, so we don't need to check files.
> I also write a patch for the issue. A simple performance testing shows that the patch will improve 58% of the performance.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (GERONIMO-1223) Hot Deploy performance improvement Just check directories -patch available
Posted by "Kevan Miller (JIRA)" <de...@geronimo.apache.org>.
[ http://issues.apache.org/jira/browse/GERONIMO-1223?page=comments#action_12358368 ]
Kevan Miller commented on GERONIMO-1223:
----------------------------------------
Have you done some performance measurements of Hot Deploy? If so, could you share them? I had assumed that any overhead would be minimal. 58% of a significant overhead is still pretty significant -- something we should know about...
> Hot Deploy performance improvement Just check directories -patch available
> --------------------------------------------------------------------------
>
> Key: GERONIMO-1223
> URL: http://issues.apache.org/jira/browse/GERONIMO-1223
> Project: Geronimo
> Type: Improvement
> Components: deployment
> Versions: 1.0
> Environment: linux or windows
> Reporter: Gao Yong Pan
> Attachments: DirectoryMonitor.java.patch
>
> The method getLastModifiedInDir in the hot-deploy/src/java/org/apache/geronimo/deployment/hot/DirectoryMonitor.java is using the algorithm by calculating highest modified time of any files or directories, this could be improved.
> In fact, we only need to check every directory is enough, because any changes, for example, add, remove, edit any files in a directory, the change can be reflected by its parent directory's modification time, so we don't need to check files.
> I also write a patch for the issue. A simple performance testing shows that the patch will improve 58% of the performance.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (GERONIMO-1223) Hot Deploy performance improvement Just check directories -patch available
Posted by "Gao Yong Pan (JIRA)" <de...@geronimo.apache.org>.
[ http://issues.apache.org/jira/browse/GERONIMO-1223?page=all ]
Gao Yong Pan updated GERONIMO-1223:
-----------------------------------
Geronimo Info: [Patch Available]
Component: deployment
Summary: Hot Deploy performance improvement Just check directories -patch available (was: Ho)
Description:
The method getLastModifiedInDir in the hot-deploy/src/java/org/apache/geronimo/deployment/hot/DirectoryMonitor.java is using the algorithm by calculating highest modified time of any files or directories, this could be improved.
In fact, we only need to check every directory is enough, because any changes, for example, add, remove, edit any files in a directory, the change can be reflected by its parent directory's modification time, so we don't need to check files.
I also write a patch for the issue. A simple performance testing shows that the patch will improve 58% of the performance.
was:
Version: 1.0
Environment: linux or windows (was: )
> Hot Deploy performance improvement Just check directories -patch available
> --------------------------------------------------------------------------
>
> Key: GERONIMO-1223
> URL: http://issues.apache.org/jira/browse/GERONIMO-1223
> Project: Geronimo
> Type: Improvement
> Components: deployment
> Versions: 1.0
> Environment: linux or windows
> Reporter: Gao Yong Pan
>
> The method getLastModifiedInDir in the hot-deploy/src/java/org/apache/geronimo/deployment/hot/DirectoryMonitor.java is using the algorithm by calculating highest modified time of any files or directories, this could be improved.
> In fact, we only need to check every directory is enough, because any changes, for example, add, remove, edit any files in a directory, the change can be reflected by its parent directory's modification time, so we don't need to check files.
> I also write a patch for the issue. A simple performance testing shows that the patch will improve 58% of the performance.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (GERONIMO-1223) Hot Deploy performance improvement Just check directories -patch available
Posted by "Gao Yong Pan (JIRA)" <de...@geronimo.apache.org>.
[ http://issues.apache.org/jira/browse/GERONIMO-1223?page=comments#action_12358453 ]
Gao Yong Pan commented on GERONIMO-1223:
----------------------------------------
My performance measurement test is not for Hot Deploy, only for getLastModifiedInDir on an existing directory, the percent number will have greate different and I think it depends on how many files in the directory.
1) and 2) can't be tested due to the sub-task1( http://issues.apache.org/jira/browse/GERONIMO-1228 ).
Let's do a simple test now.
Create a directory named dir1 in the current directory sample like this,
[pgy@localhost sample]$ mkdir dir1
[pgy@localhost sample]$ stat dir1
File: `dir1'
Size: 48 Blocks: 0 IO Block: 131072 directory
Device: 303h/771d Inode: 590636 Links: 2
Access: (0775/drwxrwxr-x) Uid: ( 512/ pgy) Gid: ( 512/ pgy)
Access: 2005-11-24 16:11:55.987833792 +0800
Modify: 2005-11-24 16:11:55.987833792 +0800
Change: 2005-11-24 16:11:55.987833792 +0800
[pgy@localhost sample]$ stat `pwd`
File: `/home/pgy/sample'
Size: 72 Blocks: 0 IO Block: 131072 directory
Device: 303h/771d Inode: 590627 Links: 3
Access: (0775/drwxrwxr-x) Uid: ( 512/ pgy) Gid: ( 512/ pgy)
Access: 2005-11-24 16:11:53.233252552 +0800
Modify: 2005-11-24 16:11:55.987833792 +0800
Change: 2005-11-24 16:11:55.987833792 +0800
Note: the modification time of dir1 and the current directory 'sample' is complete same.
Create a file under the directory dir1 with the name file1 now,
[pgy@localhost sample]$ touch dir1/file1
[pgy@localhost sample]$ stat dir1/file1
File: `dir1/file1'
Size: 0 Blocks: 0 IO Block: 131072 regular empty file
Device: 303h/771d Inode: 590648 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 512/ pgy) Gid: ( 512/ pgy)
Access: 2005-11-24 16:13:20.822936896 +0800
Modify: 2005-11-24 16:13:20.822936896 +0800
Change: 2005-11-24 16:13:20.822936896 +0800
[pgy@localhost sample]$ stat dir1
File: `dir1'
Size: 72 Blocks: 0 IO Block: 131072 directory
Device: 303h/771d Inode: 590636 Links: 2
Access: (0775/drwxrwxr-x) Uid: ( 512/ pgy) Gid: ( 512/ pgy)
Access: 2005-11-24 16:11:55.987833792 +0800
Modify: 2005-11-24 16:13:20.822936896 +0800
Change: 2005-11-24 16:13:20.822936896 +0800
Note: the modification time of file1 is complete same as its current directory dir1, of course the modification time of the directory sample doesn't be updated.
So, we only need to check the modified time of files in the directory ${GeronimoHome}/deploy and its sub directories, don' check the modification time of the files stored in the subdirectories.
> Hot Deploy performance improvement Just check directories -patch available
> --------------------------------------------------------------------------
>
> Key: GERONIMO-1223
> URL: http://issues.apache.org/jira/browse/GERONIMO-1223
> Project: Geronimo
> Type: Improvement
> Components: deployment
> Versions: 1.0
> Environment: linux or windows
> Reporter: Gao Yong Pan
> Attachments: DirectoryMonitor.java.patch
>
> The method getLastModifiedInDir in the hot-deploy/src/java/org/apache/geronimo/deployment/hot/DirectoryMonitor.java is using the algorithm by calculating highest modified time of any files or directories, this could be improved.
> In fact, we only need to check every directory is enough, because any changes, for example, add, remove, edit any files in a directory, the change can be reflected by its current directory's modification time, so we don't need to check files.
> I also write a patch for the issue. A simple performance testing shows that the patch will improve 58% of the performance.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira