Re: Composing Mahout workflow (Re: Improving Our JIRA State)

[Lance Norskog <go...@gmail.com>]

"You can even use the evolutionary optimizer to do the optimization more
cleanly."
"The hard part is to pass data structures around."

This is one use case- another is to classify data which "classifies well"
v.s. "classifies poorly" as part of fine-tuning your classification jobs.

On Thu, Oct 27, 2011 at 12:25 AM, Ted Dunning <te...@gmail.com> wrote:

> On Wed, Oct 26, 2011 at 10:16 PM, Lance Norskog <go...@gmail.com> wrote:
>
> > What about Groovy? Java does have scripting languages built in. Someone
> > (sorry can't remember) has some patches to make Mahout scala-friendly.
> >
>
> OK (maybe) for scala.
>
> Nix for Groovy.  THe key problem is lack of a serializable closure.
>
> See http://tdunning.blogspot.com/2008/03/hello-world-for-map-reduce.html
>
> The ultimate results were that it was impossible to debug these systems due
> to some of (clever!) hacks I had to use to get things to work.
>
>
> > A use case for "programmable workflow engine" is to run the same
> > classification job 100 times with different tuning parameters, and save
> the
> > confusion matrices for further optimization.  Which of these tools allows
> > this?
> >
>
> That is pretty easy.
>
> You can even use the evolutionary optimizer to do the optimization more
> cleanly.
>
> The hard part is to pass data structures around.
>



-- 
Lance Norskog
goksron@gmail.com