You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Ted Dunning (JIRA)" <ji...@apache.org> on 2008/03/31 02:10:24 UTC
[jira] Created: (MAHOUT-21) Need reference implementation of
Evolutionary Programming
Need reference implementation of Evolutionary Programming
---------------------------------------------------------
Key: MAHOUT-21
URL: https://issues.apache.org/jira/browse/MAHOUT-21
Project: Mahout
Issue Type: New Feature
Reporter: Ted Dunning
I am about to write a sequential version of EP that could be used as a reference implementation.
Parallelization should be relatively straightforward since every member of the population is independently mutated and evaluated.
The implementation will include the recorded step strategy for dense vector states, but will be easily extended to other state types and other strategies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-21) Need reference implementation of
Evolutionary Programming
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-21?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12779218#action_12779218 ]
Sean Owen commented on MAHOUT-21:
---------------------------------
Worth committing? seems so if it's going be worked on over time and provides value. Right now it's just parked in this patch.
> Need reference implementation of Evolutionary Programming
> ---------------------------------------------------------
>
> Key: MAHOUT-21
> URL: https://issues.apache.org/jira/browse/MAHOUT-21
> Project: Mahout
> Issue Type: New Feature
> Reporter: Ted Dunning
> Attachments: MAHOUT-21.patch
>
>
> I am about to write a sequential version of EP that could be used as a reference implementation.
> Parallelization should be relatively straightforward since every member of the population is independently mutated and evaluated.
> The implementation will include the recorded step strategy for dense vector states, but will be easily extended to other state types and other strategies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
RE: [jira] Updated: (MAHOUT-21) Need reference implementation of Evolutionary Programming
Posted by Jeff Eastman <je...@windwardsolutions.com>.
I've looked over Ted's EP implementation and concur with his assessment.
There is some test-specific code that exercises the main classes in a
"reference implementation" sense, but that is already in the test branch.
The rest of the code looks to be very general purpose state change and
mutation stuff that will likely be needed in an MR version.
I've put my reference implementations in my unit tests so that I can use
their outputs to verify the MR version tests once they are running. This
worked well for clustering and might be a good pattern to use generally
going forward. The first unit test is always a test of the reference
implementation, and often requires human inspection of the output for a
sanity check.
Jeff
> -----Original Message-----
> From: Ted Dunning [mailto:tdunning@veoh.com]
> Sent: Tuesday, April 01, 2008 8:40 AM
> To: mahout-dev@lucene.apache.org
> Subject: Re: [jira] Updated: (MAHOUT-21) Need reference implementation of
> Evolutionary Programming
>
>
> My guess is that much of the source code from the reference implementation
> will be used in the MR implementation.
>
> In particular, only a dozen lines of the current implementation actually
> are
> the sequential EP algorithm. The rest has to do with mutation and state
> storage that will be just as useful in the parallel case.
>
>
> On 4/1/08 7:56 AM, "Grant Ingersoll" <gs...@apache.org> wrote:
>
> > Should we have a separate source tree for reference implementations or
> > should we just put them in the main trunk? I can see pros and cons
> > for both sides.
Re: [jira] Updated: (MAHOUT-21) Need reference implementation of
Evolutionary Programming
Posted by Ted Dunning <td...@veoh.com>.
My guess is that much of the source code from the reference implementation
will be used in the MR implementation.
In particular, only a dozen lines of the current implementation actually are
the sequential EP algorithm. The rest has to do with mutation and state
storage that will be just as useful in the parallel case.
On 4/1/08 7:56 AM, "Grant Ingersoll" <gs...@apache.org> wrote:
> Should we have a separate source tree for reference implementations or
> should we just put them in the main trunk? I can see pros and cons
> for both sides.
Re: [jira] Updated: (MAHOUT-21) Need reference implementation of Evolutionary Programming
Posted by Grant Ingersoll <gs...@apache.org>.
Very cool, Ted. I hope to get to this after ApacheCon EU, but others
can feel free to jump on it before then.
Should we have a separate source tree for reference implementations or
should we just put them in the main trunk? I can see pros and cons
for both sides.
Thoughts?
-Grant
On Mar 31, 2008, at 1:04 AM, Ted Dunning (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/MAHOUT-21?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> ]
>
> Ted Dunning updated MAHOUT-21:
> ------------------------------
>
> Attachment: MAHOUT-21.patch
>
> This is a draft of a continuous variable EP optimizer based on
> "Recorded Step Directional Mutation for Faster Convergence" (See http://arxiv.org/abs/0803.3838)
> .
>
>> Need reference implementation of Evolutionary Programming
>> ---------------------------------------------------------
>>
>> Key: MAHOUT-21
>> URL: https://issues.apache.org/jira/browse/MAHOUT-21
>> Project: Mahout
>> Issue Type: New Feature
>> Reporter: Ted Dunning
>> Attachments: MAHOUT-21.patch
>>
>>
>> I am about to write a sequential version of EP that could be used
>> as a reference implementation.
>> Parallelization should be relatively straightforward since every
>> member of the population is independently mutated and evaluated.
>> The implementation will include the recorded step strategy for
>> dense vector states, but will be easily extended to other state
>> types and other strategies.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
[jira] Updated: (MAHOUT-21) Need reference implementation of
Evolutionary Programming
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-21?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Dunning updated MAHOUT-21:
------------------------------
Attachment: MAHOUT-21.patch
This is a draft of a continuous variable EP optimizer based on "Recorded Step Directional Mutation for Faster Convergence" (See http://arxiv.org/abs/0803.3838)
.
> Need reference implementation of Evolutionary Programming
> ---------------------------------------------------------
>
> Key: MAHOUT-21
> URL: https://issues.apache.org/jira/browse/MAHOUT-21
> Project: Mahout
> Issue Type: New Feature
> Reporter: Ted Dunning
> Attachments: MAHOUT-21.patch
>
>
> I am about to write a sequential version of EP that could be used as a reference implementation.
> Parallelization should be relatively straightforward since every member of the population is independently mutated and evaluated.
> The implementation will include the recorded step strategy for dense vector states, but will be easily extended to other state types and other strategies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAHOUT-21) Need reference implementation of
Evolutionary Programming
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-21?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved MAHOUT-21.
-----------------------------
Resolution: Later
> Need reference implementation of Evolutionary Programming
> ---------------------------------------------------------
>
> Key: MAHOUT-21
> URL: https://issues.apache.org/jira/browse/MAHOUT-21
> Project: Mahout
> Issue Type: New Feature
> Reporter: Ted Dunning
> Attachments: MAHOUT-21.patch
>
>
> I am about to write a sequential version of EP that could be used as a reference implementation.
> Parallelization should be relatively straightforward since every member of the population is independently mutated and evaluated.
> The implementation will include the recorded step strategy for dense vector states, but will be easily extended to other state types and other strategies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-21) Need reference implementation of
Evolutionary Programming
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-21?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12779221#action_12779221 ]
Ted Dunning commented on MAHOUT-21:
-----------------------------------
I only think that this would be worth committing if it connect to other aspects of mahout (such as sharing an input format).
As it stands, it is dormant and has no owner (I am swamped). That makes it a candidate for the later treatment.
> Need reference implementation of Evolutionary Programming
> ---------------------------------------------------------
>
> Key: MAHOUT-21
> URL: https://issues.apache.org/jira/browse/MAHOUT-21
> Project: Mahout
> Issue Type: New Feature
> Reporter: Ted Dunning
> Attachments: MAHOUT-21.patch
>
>
> I am about to write a sequential version of EP that could be used as a reference implementation.
> Parallelization should be relatively straightforward since every member of the population is independently mutated and evaluated.
> The implementation will include the recorded step strategy for dense vector states, but will be easily extended to other state types and other strategies.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.