You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Deneche A. Hakim (JIRA)" <ji...@apache.org> on 2009/06/29 13:30:48 UTC
[jira] Created: (MAHOUT-140) In-memory mapreduce Random Forests
In-memory mapreduce Random Forests
----------------------------------
Key: MAHOUT-140
URL: https://issues.apache.org/jira/browse/MAHOUT-140
Project: Mahout
Issue Type: New Feature
Components: Classification
Affects Versions: 0.2
Reporter: Deneche A. Hakim
Priority: Minor
Each mapper is responsible for growing a number of trees with a whole copy of the dataset loaded in memory, it uses the reference implementation's code to build each tree and estimate the oob error.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (MAHOUT-140) In-memory mapreduce Random Forests
Posted by deneche abdelhakim <a_...@yahoo.fr>.
Actually, I'm not used any reducer at all, the output of the mappers is collected and handled by the main program after the end of the job.
Running the job with 10 map tasks in a 10 instances (c1.medium) cluster takes 0h 11m 39s 209, speculative execution is on so 12 map tasks have been launched.
running the same job with 5x10 map tasks takes 0h 11m 54s 962, 59 map tasks have been launched.
And running the same job again with 5x10 map tasks with job parameter mapred.job.reuse.jvm.num.tasks=-1 (no limit how many tasks to run per jvm) takes 0h 11m 57s 115
--- En date de : Sam 18.7.09, Ted Dunning <te...@gmail.com> a écrit :
> De: Ted Dunning <te...@gmail.com>
> Objet: Re: [jira] Commented: (MAHOUT-140) In-memory mapreduce Random Forests
> À: mahout-dev@lucene.apache.org
> Date: Samedi 18 Juillet 2009, 20h36
> This is interesting.
>
> Is the reduce trivial here? (if so, then and shuffling
> isn't the problem and
> you may have demonstrated this with your no output
> version)
>
> WHat happens if you increase the number of maps to 5x the
> number of nodes?
>
>
>
> On Sat, Jul 18, 2009 at 11:11 AM, Deneche A. Hakim (JIRA)
> <ji...@apache.org>wrote:
>
> > It looks like building a single tree in a sequential
> manner is 2x faster
> > than building the same tree with the cluster !!! I
> don't have a lot of
> > experience with clusters, is it normal ??? may be 10
> instances is just too
> > small to get a good speedup, or may be there is a bug
> hiding somewhere (I
> > can hear it walking in the code when the moon...)
> >
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>
Re: [jira] Commented: (MAHOUT-140) In-memory mapreduce Random Forests
Posted by Ted Dunning <te...@gmail.com>.
This is interesting.
Is the reduce trivial here? (if so, then and shuffling isn't the problem and
you may have demonstrated this with your no output version)
WHat happens if you increase the number of maps to 5x the number of nodes?
On Sat, Jul 18, 2009 at 11:11 AM, Deneche A. Hakim (JIRA)
<ji...@apache.org>wrote:
> It looks like building a single tree in a sequential manner is 2x faster
> than building the same tree with the cluster !!! I don't have a lot of
> experience with clusters, is it normal ??? may be 10 instances is just too
> small to get a good speedup, or may be there is a bug hiding somewhere (I
> can hear it walking in the code when the moon...)
>
--
Ted Dunning, CTO
DeepDyve
[jira] Updated: (MAHOUT-140) In-memory mapreduce Random Forests
Posted by "Deneche A. Hakim (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Deneche A. Hakim updated MAHOUT-140:
------------------------------------
Fix Version/s: 0.2
* Will be committed as part of [MAHOUT-145|https://issues.apache.org/jira/browse/MAHOUT-145]
* For now two implementations are available, one that uses Hadoop 0.20.0 API and one that don't use it. Later only one implementation should remain
> In-memory mapreduce Random Forests
> ----------------------------------
>
> Key: MAHOUT-140
> URL: https://issues.apache.org/jira/browse/MAHOUT-140
> Project: Mahout
> Issue Type: New Feature
> Components: Classification
> Affects Versions: 0.2
> Reporter: Deneche A. Hakim
> Priority: Minor
> Fix For: 0.2
>
> Attachments: inmem_July19_patch.diff, mapred_jul12.diff, mapred_patch.diff
>
>
> Each mapper is responsible for growing a number of trees with a whole copy of the dataset loaded in memory, it uses the reference implementation's code to build each tree and estimate the oob error.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-140) In-memory mapreduce Random Forests
Posted by "Deneche A. Hakim (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12732922#action_12732922 ]
Deneche A. Hakim commented on MAHOUT-140:
-----------------------------------------
* First of all I implemented the *in-mem-sequential* builder which simulates the execution of many mappers in a sequential manner. Passing the same seed to the *in-mem-mapred* and *in-mem-sequential* implementations generates the same trees with the same output, this should make the comparison easier.
* on a 10 instances cluster (ec2 c1.medium) building 200 trees with KDD10% with a seed=1 gives :
|| in-mem-sequential || in-mem-mapred ||
| 0h 52m 38s 665 | 0h 13m 3s 691 |
its a 4x speedup, I don't know if I should expect a higher speedup, so I run some more tests to try and find what takes most of the time.
* I noticed that speculative execution was turned on, passing (-Dmapred.map.tasks.speculative.execution=false) gives:
|| in-mem-mapred||
| 0h 12m 46s 150 |
it doesn't seem to be the cause of the slowdown.
* How much time does the output takes, this includes computing the oob estimate and outputing the trees and the oob predictions ? I added a special job parameter (debug.mahout.rf.output) when false the mappers don't compute the oob estimates and don't output anything, they just prepare the bags and build the trees. The result is:
|| in-mem-mapred||
| 0h 12m 35s 557 |
actually the output doesn't seem to make much time
* How much time does launching and configuring the MR take, this includes loading the data in all the nodes ? running the *in-mem-mapred* with just 10 trees, thus 1 tree per map, gives:
|| in-mem-mapred||
| 0h 1m 36s 335 |
It seem that building the trees *is* what's taking most of the time
* Because I'm running a number of maps equal to the number of cluster-nodes, if one maps takes 100 minutes and all other maps take only 1 minute, the job still takes 100 minutes to finish. I added a special job parameter (debug.mahout.rf.single.seed), when true all mappers use the same seed thus they all behave similarly. The results are:
|| in-mem-sequential || in-mem-mapred ||
| 0h 40m 39s 829 | 0h 9m 30s 577 |
It looks like building a single tree in a sequential manner is 2x faster than building the same tree with the cluster !!! I don't have a lot of experience with clusters, is it normal ??? may be 10 instances is just too small to get a good speedup, or may be there is a bug hiding somewhere (I can hear it walking in the code when the moon...)
> In-memory mapreduce Random Forests
> ----------------------------------
>
> Key: MAHOUT-140
> URL: https://issues.apache.org/jira/browse/MAHOUT-140
> Project: Mahout
> Issue Type: New Feature
> Components: Classification
> Affects Versions: 0.2
> Reporter: Deneche A. Hakim
> Priority: Minor
> Attachments: mapred_jul12.diff, mapred_patch.diff
>
>
> Each mapper is responsible for growing a number of trees with a whole copy of the dataset loaded in memory, it uses the reference implementation's code to build each tree and estimate the oob error.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-140) In-memory mapreduce Random Forests
Posted by "Deneche A. Hakim (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Deneche A. Hakim updated MAHOUT-140:
------------------------------------
Status: Patch Available (was: Open)
Work in progress...
A working implementation, I tested it on a two-ubuntu-cluster but more tests are needed. There is a known issue when the number of maps is high, I'll try to solve it in the next patch.
The main limitation of this implementation is that each mapper loads a copy of the data in memory, and because I don't know how to actually share the data between the mappers of the same slave node, this means that is you launch simultaneously N maps per cluster node, you'll get N copies of the data in each node's memory !!!
> In-memory mapreduce Random Forests
> ----------------------------------
>
> Key: MAHOUT-140
> URL: https://issues.apache.org/jira/browse/MAHOUT-140
> Project: Mahout
> Issue Type: New Feature
> Components: Classification
> Affects Versions: 0.2
> Reporter: Deneche A. Hakim
> Priority: Minor
>
> Each mapper is responsible for growing a number of trees with a whole copy of the dataset loaded in memory, it uses the reference implementation's code to build each tree and estimate the oob error.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (MAHOUT-140) In-memory mapreduce Random Forests
Posted by "Deneche A. Hakim (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Deneche A. Hakim reassigned MAHOUT-140:
---------------------------------------
Assignee: Deneche A. Hakim
> In-memory mapreduce Random Forests
> ----------------------------------
>
> Key: MAHOUT-140
> URL: https://issues.apache.org/jira/browse/MAHOUT-140
> Project: Mahout
> Issue Type: New Feature
> Components: Classification
> Affects Versions: 0.2
> Reporter: Deneche A. Hakim
> Assignee: Deneche A. Hakim
> Priority: Minor
> Fix For: 0.2
>
> Attachments: inmem_July19_patch.diff, mapred_jul12.diff, mapred_patch.diff
>
>
> Each mapper is responsible for growing a number of trees with a whole copy of the dataset loaded in memory, it uses the reference implementation's code to build each tree and estimate the oob error.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-140) In-memory mapreduce Random Forests
Posted by "Deneche A. Hakim (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Deneche A. Hakim updated MAHOUT-140:
------------------------------------
Attachment: mapred_patch.diff
org.apache.mahout.rf.mapred
To make it simple, MAHOUT-122 (ref. implementation) is also included in this patch.
In-memory mapreduce implementation RandomForests, each mapper is responsible for growing a number of trees with a whole copy of the dataset loaded in memory, it uses the reference implementation's code to build each tree and estimate the oob error.
There is no need for input data, the dataset is distributed to the slave nodes using the DistributedCache. A custom InputFormat (InMemInputFormat) is configured with desired number of trees and generates a number of InputSplits (InMemInputSplit) equal to the configured number of maps (mapred.map.tasks).
There is no need for a reducer, each map outputs (InMemOutput) the trees it built and, for each tree, the labels the tree predicted for each oob instance. This step has to be done in the mapper because only there we know which instances are oob.
The main program (InMemBuilder) is responsible for configuring and launching the job. At the end of the job it parses the output files and builds the corresponding RandomForest, and for each tree predictions it calls (if available) a PredictionCallback that allows the caller to compute any error needed.
To test this implementation I added BuildForest that takes simple parameters and can build a forest, with the Kdd dataset, using either the sequential (reference) or mapreduce implementation. The basic usage is as follows:
hadoop jar mahout-core-....job org.apache.mahout.rf.mapred.examples.BuildForest [MR] path m nbtrees
* MR : use the mapreduce implementation
* path : path to the Kdd dataset
* m : number of variables to select at each tree-node
* nbtrees : size of the forest
BuildForest implements the Tool interface, so you'll be able to pass Hadoop parameters.
I did a small experiment on my two-nodes (ubuntu) cluster and got a x2 speedup, but there is a lot of random going around, and my cluster is weird (1 node is 2x faster than the other), I shall do more tests this coming week.
PS: I also added a package.htm in org.apache.mahout.rf.mapred that contains this description
> In-memory mapreduce Random Forests
> ----------------------------------
>
> Key: MAHOUT-140
> URL: https://issues.apache.org/jira/browse/MAHOUT-140
> Project: Mahout
> Issue Type: New Feature
> Components: Classification
> Affects Versions: 0.2
> Reporter: Deneche A. Hakim
> Priority: Minor
> Attachments: mapred_patch.diff
>
>
> Each mapper is responsible for growing a number of trees with a whole copy of the dataset loaded in memory, it uses the reference implementation's code to build each tree and estimate the oob error.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-140) In-memory mapreduce Random Forests
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730128#action_12730128 ]
Ted Dunning commented on MAHOUT-140:
------------------------------------
These results look *really* promising.
But I am curious about how to interpret these numbers. It appears that you get decent speed-up with a larger cluster (5x speedup with 10x nodes).
But these numbers don't seem to show speedup over the results that you gave in MAHOUT-122 where a single node seemed to be able to build 50 trees on 5% of the data in <10m.
My guess is that the comparison I am making is invalid.
Can you clarify how things look so far? This seems like it ought to be much more promising than what I am saying. I don't understand how your small cluster here could be slower than the reference implementation.
A second question is why you don't see perfect speedup with an increasing cluster. Do you have any insight into how the time breaks down between hadoop MR startup, data cache loading, tree building, oob error estimation and storing output?
> In-memory mapreduce Random Forests
> ----------------------------------
>
> Key: MAHOUT-140
> URL: https://issues.apache.org/jira/browse/MAHOUT-140
> Project: Mahout
> Issue Type: New Feature
> Components: Classification
> Affects Versions: 0.2
> Reporter: Deneche A. Hakim
> Priority: Minor
> Attachments: mapred_jul12.diff, mapred_patch.diff
>
>
> Each mapper is responsible for growing a number of trees with a whole copy of the dataset loaded in memory, it uses the reference implementation's code to build each tree and estimate the oob error.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-140) In-memory mapreduce Random Forests
Posted by "Deneche A. Hakim (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730292#action_12730292 ]
Deneche A. Hakim commented on MAHOUT-140:
-----------------------------------------
bq. But these numbers don't seem to show speedup over the results that you gave in MAHOUT-122 where a single node seemed to be able to build 50 trees on 5% of the data in <10m.
bq. My guess is that the comparison I am making is invalid.
bq. Can you clarify how things look so far? This seems like it ought to be much more promising than what I am saying. I don't understand how your small cluster here could be slower than the reference implementation.
I noticed too that running BuildForest on my laptop is faster than EC2, I suspect that my laptop's CPU is faster than the EC2 instance that I used (m1.small). To be sure I run the sequential version, which allow the use of a specific seed thus being repeatable, and got the following results:
the program uses the reference implementation to build 50 trees with Kdd10%, selecting 1 random variable at each tree-node, starting with seed=1, estimating the o-o-b error and using the optimized IG code
|| Instance || build time ||
| my laptop | 9m 14s 978 |
| 1 m1.small | 28m 59s 510 |
| 1 c1.medium | 11m 35s 286 |
The m1.small is, indeed, slower than my laptop but still the reference implementation running on this instance takes only 29m compared to 45m when using the mapred implementation. But because the mapred implementation does not accept seed values, for now, the comparison between the sequential and sequential implementations will be difficult.
I'm thinking of a way to make the mapred implementation use specific seeds: the main program passes a specific seed value (user parameter) to InMemInputFormat, this seed is used to instantiate a Random object used to generate a different seed for each InputSplit (mapper). This way I can make the reference implementation use the same scheme, given desired the number of mappers, and thus be able to compare between the two implementations. What do you think of this scheme ?
bq. A second question is why you don't see perfect speedup with an increasing cluster. Do you have any insight into how the time breaks down between hadoop MR startup, data cache loading, tree building, oob error estimation and storing output?
I noticed that loading the data can take some time and because all the mappers do the loading, the loading time is always the same wherever you use a small or a large cluster. I also noticed that the compression is activated when using hadoop on EC2 and it also takes some time to initialize after the mappers finish their work. But I need to run more tests and collect more info to be able to answer your question.
> In-memory mapreduce Random Forests
> ----------------------------------
>
> Key: MAHOUT-140
> URL: https://issues.apache.org/jira/browse/MAHOUT-140
> Project: Mahout
> Issue Type: New Feature
> Components: Classification
> Affects Versions: 0.2
> Reporter: Deneche A. Hakim
> Priority: Minor
> Attachments: mapred_jul12.diff, mapred_patch.diff
>
>
> Each mapper is responsible for growing a number of trees with a whole copy of the dataset loaded in memory, it uses the reference implementation's code to build each tree and estimate the oob error.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-140) In-memory mapreduce Random Forests
Posted by "Deneche A. Hakim (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Deneche A. Hakim updated MAHOUT-140:
------------------------------------
Attachment: mapred_jul12.diff
*Changes*
* The oob error estimation has been rewritten to become much more faster
* BuildForest has an optional argument '-o' to use the optimized IG calculations
I tested the implementation on Amazon EC2:
* on a 1 small instance cluster (1 master + 1 slave), building 50 trees with KDD10% takes 44m 45s
* on a 10 small instances cluster (1 master + 10 slaves), building 50 trees with KDD10% takes 7m 50s
*what's next*
* Although many improvements are possible, the actual InMem implementation does a good job. I shall start coding the other mapreduce variant where each mapper uses only the subset of data available to grow the trees
> In-memory mapreduce Random Forests
> ----------------------------------
>
> Key: MAHOUT-140
> URL: https://issues.apache.org/jira/browse/MAHOUT-140
> Project: Mahout
> Issue Type: New Feature
> Components: Classification
> Affects Versions: 0.2
> Reporter: Deneche A. Hakim
> Priority: Minor
> Attachments: mapred_jul12.diff, mapred_patch.diff
>
>
> Each mapper is responsible for growing a number of trees with a whole copy of the dataset loaded in memory, it uses the reference implementation's code to build each tree and estimate the oob error.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-140) In-memory mapreduce Random Forests
Posted by "Deneche A. Hakim (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Deneche A. Hakim updated MAHOUT-140:
------------------------------------
Attachment: inmem_Sep29.patch
committed patch
> In-memory mapreduce Random Forests
> ----------------------------------
>
> Key: MAHOUT-140
> URL: https://issues.apache.org/jira/browse/MAHOUT-140
> Project: Mahout
> Issue Type: New Feature
> Components: Classification
> Affects Versions: 0.2
> Reporter: Deneche A. Hakim
> Assignee: Deneche A. Hakim
> Priority: Minor
> Fix For: 0.2
>
> Attachments: inmem_July19_patch.diff, inmem_Sep29.patch, mapred_jul12.diff, mapred_patch.diff
>
>
> Each mapper is responsible for growing a number of trees with a whole copy of the dataset loaded in memory, it uses the reference implementation's code to build each tree and estimate the oob error.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (MAHOUT-140) In-memory mapreduce
Random Forests
Posted by "Deneche A. Hakim (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12732922#action_12732922 ]
Deneche A. Hakim edited comment on MAHOUT-140 at 7/18/09 11:20 AM:
-------------------------------------------------------------------
* First of all I implemented the *in-mem-sequential* builder which simulates the execution of many mappers in a sequential manner. I also implemented the seed generation scheme for the *in-mem-mapred* implementation, passing the same seed to the *in-mem-mapred* and *in-mem-sequential* implementations generates the same trees with the same output, this should make the comparison easier.
* on a 10 instances cluster (ec2 c1.medium) building 200 trees with KDD10% with a seed=1 gives :
|| in-mem-sequential || in-mem-mapred ||
| 0h 52m 38s 665 | 0h 13m 3s 691 |
its a 4x speedup, I don't know if I should expect a higher speedup, so I run some more tests to try and find what takes most of the time.
* I noticed that speculative execution was turned on, passing (-Dmapred.map.tasks.speculative.execution=false) gives:
|| in-mem-mapred||
| 0h 12m 46s 150 |
it doesn't seem to be the cause of the slowdown.
* How much time does the output takes, this includes computing the oob estimate and outputing the trees and the oob predictions ? I added a special job parameter (debug.mahout.rf.output) when false the mappers don't compute the oob estimates and don't output anything, they just prepare the bags and build the trees. The result is:
|| in-mem-mapred||
| 0h 12m 35s 557 |
actually the output doesn't seem to make much time
* How much time does launching and configuring the MR take, this includes loading the data in all the nodes ? running the *in-mem-mapred* with just 10 trees, thus 1 tree per map, gives:
|| in-mem-mapred||
| 0h 1m 36s 335 |
Starting up the MR doesn't seem to take a lot of time, actually it seems that building the trees *is* what takes most of the time
* Because I'm running a number of maps equal to the number of cluster-nodes, if one maps take 100 minutes and all other maps take only 1 minute, the job still takes 100 minutes to finish. I added a special job parameter (debug.mahout.rf.single.seed), when true all mappers use the same seed thus they all behave similarly. The results are:
|| in-mem-sequential || in-mem-mapred ||
| 0h 40m 39s 829 | 0h 9m 30s 577 |
In the *in-mem-sequential* implementation, each 20 trees take about 4 minutes to be built, but in the *in-mem-mapred* implementation, each map takes 9 minutes to build 20 trees. It looks like building a single tree in a sequential manner is *2x faster* than building the same tree with the cluster !!! I don't have a lot of experience with clusters, is it normal ??? may be 10 instances is just too small to get a good speedup, or may be there is a bug hiding somewhere (I can hear it walking in the code when the moon...)
was (Author: adeneche):
* First of all I implemented the *in-mem-sequential* builder which simulates the execution of many mappers in a sequential manner. Passing the same seed to the *in-mem-mapred* and *in-mem-sequential* implementations generates the same trees with the same output, this should make the comparison easier.
* on a 10 instances cluster (ec2 c1.medium) building 200 trees with KDD10% with a seed=1 gives :
|| in-mem-sequential || in-mem-mapred ||
| 0h 52m 38s 665 | 0h 13m 3s 691 |
its a 4x speedup, I don't know if I should expect a higher speedup, so I run some more tests to try and find what takes most of the time.
* I noticed that speculative execution was turned on, passing (-Dmapred.map.tasks.speculative.execution=false) gives:
|| in-mem-mapred||
| 0h 12m 46s 150 |
it doesn't seem to be the cause of the slowdown.
* How much time does the output takes, this includes computing the oob estimate and outputing the trees and the oob predictions ? I added a special job parameter (debug.mahout.rf.output) when false the mappers don't compute the oob estimates and don't output anything, they just prepare the bags and build the trees. The result is:
|| in-mem-mapred||
| 0h 12m 35s 557 |
actually the output doesn't seem to make much time
* How much time does launching and configuring the MR take, this includes loading the data in all the nodes ? running the *in-mem-mapred* with just 10 trees, thus 1 tree per map, gives:
|| in-mem-mapred||
| 0h 1m 36s 335 |
It seem that building the trees *is* what's taking most of the time
* Because I'm running a number of maps equal to the number of cluster-nodes, if one maps takes 100 minutes and all other maps take only 1 minute, the job still takes 100 minutes to finish. I added a special job parameter (debug.mahout.rf.single.seed), when true all mappers use the same seed thus they all behave similarly. The results are:
|| in-mem-sequential || in-mem-mapred ||
| 0h 40m 39s 829 | 0h 9m 30s 577 |
It looks like building a single tree in a sequential manner is 2x faster than building the same tree with the cluster !!! I don't have a lot of experience with clusters, is it normal ??? may be 10 instances is just too small to get a good speedup, or may be there is a bug hiding somewhere (I can hear it walking in the code when the moon...)
> In-memory mapreduce Random Forests
> ----------------------------------
>
> Key: MAHOUT-140
> URL: https://issues.apache.org/jira/browse/MAHOUT-140
> Project: Mahout
> Issue Type: New Feature
> Components: Classification
> Affects Versions: 0.2
> Reporter: Deneche A. Hakim
> Priority: Minor
> Attachments: mapred_jul12.diff, mapred_patch.diff
>
>
> Each mapper is responsible for growing a number of trees with a whole copy of the dataset loaded in memory, it uses the reference implementation's code to build each tree and estimate the oob error.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-140) In-memory mapreduce Random Forests
Posted by "Deneche A. Hakim (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Deneche A. Hakim updated MAHOUT-140:
------------------------------------
Attachment: inmem_July19_patch.diff
*Changes*
* InMemBuilder can now use a seed to be repeatable
* InMemSequentialBuilder simulates InMemBuilder in a sequential manner, outside of Hadoop
* Arranged many classes to be simpler, and added one or to tests
* BuildForest splitted in two:
** rf.mapred.examples.BuildForest: can launch both the *in-mem-mapred* and *in-mem-sequential* builders
** rf.ref.examples.BuildForest: uses the sequential builder
> In-memory mapreduce Random Forests
> ----------------------------------
>
> Key: MAHOUT-140
> URL: https://issues.apache.org/jira/browse/MAHOUT-140
> Project: Mahout
> Issue Type: New Feature
> Components: Classification
> Affects Versions: 0.2
> Reporter: Deneche A. Hakim
> Priority: Minor
> Attachments: inmem_July19_patch.diff, mapred_jul12.diff, mapred_patch.diff
>
>
> Each mapper is responsible for growing a number of trees with a whole copy of the dataset loaded in memory, it uses the reference implementation's code to build each tree and estimate the oob error.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-140) In-memory mapreduce Random Forests
Posted by "Deneche A. Hakim (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Deneche A. Hakim updated MAHOUT-140:
------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
> In-memory mapreduce Random Forests
> ----------------------------------
>
> Key: MAHOUT-140
> URL: https://issues.apache.org/jira/browse/MAHOUT-140
> Project: Mahout
> Issue Type: New Feature
> Components: Classification
> Affects Versions: 0.2
> Reporter: Deneche A. Hakim
> Assignee: Deneche A. Hakim
> Priority: Minor
> Fix For: 0.2
>
> Attachments: inmem_July19_patch.diff, inmem_Sep29.patch, mapred_jul12.diff, mapred_patch.diff
>
>
> Each mapper is responsible for growing a number of trees with a whole copy of the dataset loaded in memory, it uses the reference implementation's code to build each tree and estimate the oob error.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.