You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Robin Anil (JIRA)" <ji...@apache.org> on 2009/09/04 13:07:57 UTC
[jira] Created: (MAHOUT-170) Enable Java compile optimize flag
during build
Enable Java compile optimize flag during build
----------------------------------------------
Key: MAHOUT-170
URL: https://issues.apache.org/jira/browse/MAHOUT-170
Project: Mahout
Issue Type: Improvement
Affects Versions: 0.2
Reporter: Robin Anil
in maven compile plugin enable optimize=true flag
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAHOUT-170) Enable Java compile optimize flag
during build
Posted by "Robin Anil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robin Anil resolved MAHOUT-170.
-------------------------------
Resolution: Fixed
Issue closed. jvm launch parameters would be tackled in a separate jira issue
> Enable Java compile optimize flag during build
> ----------------------------------------------
>
> Key: MAHOUT-170
> URL: https://issues.apache.org/jira/browse/MAHOUT-170
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.2
> Reporter: Robin Anil
> Fix For: 0.2
>
> Attachments: optimize.patch
>
>
> in maven compile plugin enable optimize=true flag
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-170) Enable Java compile optimize flag
during build
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751518#action_12751518 ]
Sean Owen commented on MAHOUT-170:
----------------------------------
These are JVM args, no? not javac. You could probably set them to javac but it would be enabling JIT optimizations when running javac :)
Truly there is no optimization magic in javac besides this nearly-no-op -O flag, which we should set anyway. Compile-time optimization comes from stuff like ProGuard; but in Java it's the JVM + JIT that does optimization. And yeah it's important stuff, though almost everything you can control concerns garbage collection. Which is important.
> Enable Java compile optimize flag during build
> ----------------------------------------------
>
> Key: MAHOUT-170
> URL: https://issues.apache.org/jira/browse/MAHOUT-170
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.2
> Reporter: Robin Anil
> Attachments: optimize.patch
>
>
> in maven compile plugin enable optimize=true flag
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-170) Enable Java compile optimize flag
during build
Posted by "Robin Anil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763085#action_12763085 ]
Robin Anil commented on MAHOUT-170:
-----------------------------------
HBase does jvm tuning out of the by enabling Concurrent GC Sweep in the hbase-env.sh
For Sequential Versions we can enable it from the Shell Script
For Hadoop jobs to get the benefit, it has to be put in hadoop-env.sh or in mapred.child.java.opts conf parameter
> Enable Java compile optimize flag during build
> ----------------------------------------------
>
> Key: MAHOUT-170
> URL: https://issues.apache.org/jira/browse/MAHOUT-170
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.2
> Reporter: Robin Anil
> Fix For: 0.2
>
> Attachments: optimize.patch
>
>
> in maven compile plugin enable optimize=true flag
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-170) Enable Java compile optimize flag
during build
Posted by "Robin Anil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robin Anil updated MAHOUT-170:
------------------------------
Attachment: optimize.patch
> Enable Java compile optimize flag during build
> ----------------------------------------------
>
> Key: MAHOUT-170
> URL: https://issues.apache.org/jira/browse/MAHOUT-170
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.2
> Reporter: Robin Anil
> Attachments: optimize.patch
>
>
> in maven compile plugin enable optimize=true flag
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-170) Enable Java compile optimize flag
during build
Posted by "Robin Anil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760890#action_12760890 ]
Robin Anil commented on MAHOUT-170:
-----------------------------------
A Quick Test, uses some parameters which i felt would improve performance.
Running FPGrowth on 340K transactions trying to fetch top 50 closed patterns each of the 360 features
Max Heap set at 2GB, Heap used 400-500MB
run with -server option
Here is the log of the test in order
1. adding -XX:+AggressiveOpts: decreased total running time by about 30s (3:10 v 3:40) and Yourkit reports cpu time decreased from 2:48 to 2:38
2. adding -XX:+UseFastAccessorMethods Runtime 2:54, YourKit reports CPU time as 2:36
3. adding -Xmn1024M (increased the heap size for young generation to improve GC for shortlived objects) Thrashing occurred
-Xmn128M No thrashing this time runtime increased to 3:45 CPU time 3:10
removed -Xmn option
4. adding -Xss 128K reduced the execution stack size. Runtime and cpu time increased slightly (5-7 sec) from (2)
5. adding -XX:ParallelGCThreads=20 Again Runtime and cpu time increased by about 10-15 secs from (2)
Moral 1: Dont mess around unless you know what you are doing
Moral 2: -server -XX:+AggressiveOpts -XX:+UseFastAccessorMethods gives significant boost to the runtime
> Enable Java compile optimize flag during build
> ----------------------------------------------
>
> Key: MAHOUT-170
> URL: https://issues.apache.org/jira/browse/MAHOUT-170
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.2
> Reporter: Robin Anil
> Attachments: optimize.patch
>
>
> in maven compile plugin enable optimize=true flag
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-170) Enable Java compile optimize flag
during build
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751516#action_12751516 ]
Ted Dunning commented on MAHOUT-170:
------------------------------------
I will be interested to hear if this helps.
Other options of possible interest include:
- optimized pointers (if we are running >4GB heaps)
- -XX+AggressiveOpts
> Enable Java compile optimize flag during build
> ----------------------------------------------
>
> Key: MAHOUT-170
> URL: https://issues.apache.org/jira/browse/MAHOUT-170
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.2
> Reporter: Robin Anil
> Attachments: optimize.patch
>
>
> in maven compile plugin enable optimize=true flag
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-170) Enable Java compile optimize flag
during build
Posted by "Robin Anil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robin Anil updated MAHOUT-170:
------------------------------
Fix Version/s: 0.2
> Enable Java compile optimize flag during build
> ----------------------------------------------
>
> Key: MAHOUT-170
> URL: https://issues.apache.org/jira/browse/MAHOUT-170
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.2
> Reporter: Robin Anil
> Fix For: 0.2
>
> Attachments: optimize.patch
>
>
> in maven compile plugin enable optimize=true flag
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.