You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by Sean Owen <sr...@gmail.com> on 2009/04/20 14:49:29 UTC
Java Data Mining specification
http://jcp.org/aboutJava/communityprocess/edr/jsr247/index.html
I only just stumbled on this today. Looks like there has already been
a JSR on data mining in Java, and the above is a JSR for version 2.0.
Both seem a bit defunct, but, interesting to see there was some effort
to construct a standard API for exactly the sorts of things Mahout
provides.
I am not suggesting we implement this API -- purely because it looks
like it is not being worked on. But interesting to look at.
Sean
Re: Java Data Mining specification
Posted by Ted Dunning <te...@gmail.com>.
Sure.
I feel like most of these standards are trying to solve a problem that
doesn't entirely exist. I know of two such efforts including
PMML<http://www.thecepblog.com/2009/04/05/predictive-model-markup-language-pmml/>and
JSR-247 <http://jcp.org/en/jsr/detail?id=247>.
PMML's stated intent is to make models more portable between data-mining
systems. In my experience, the only issues in moving most kinds of models
from one system to another come when deploying models from a development to
a production setting. The major issues that I have experienced in trying to
do this generally have to do with bugs in either the development or
production implementation of the model or major differences in the
arithmetic used. Most models used in production data mining settings are
actually very, very simple and re-implementing a model evaluator from
scratch isn't even that hard. This is especially true since you can have
oodles of test data from your development environment just by running the
new implementation against your training data. Generally, people use
logistic regression, neural networks or some variant of decision trees.
These are all dead simple to evaluate. Pretty much all of the magic goes
into the learning algorithms and feature extraction and PMML purposely
addresses neither of these.
JSR-247, on the other hand tries address a broader range of issues. The
real problem with JSR-247 is that it is essentially trying ram Oracle's JDM
down everybody's throat as a Java standard. (See the JDM
API<http://download-uk.oracle.com/docs/cd/B19306_01/datamine.102/b14341/overview-summary.html>for
the original). JDM claims that it provides for the problem of
data-mining "JDBC provides for SQL". The problem is that data mining is
largely an exploratory process that derives insight by the slightly
automated analysis of data. As such, it is much better done in an
exploratory environment with a full range of numerical and graphical
capabilities. Java isn't such a great environment for exploratory work, but
there are large problems where it isn't bad at all. JDM, however, really
doesn't address these situations all that well. In particular, there are
entire packages for Oracle specific implementations of particular algorithms
and only a little bit of stuff for problems such as feature extraction or
data transformation. A great illustration of just how far off point JDM and
JSR-247 are is found in the Mark Hornik's
presentation<http://developers.sun.com/learning/javaoneonline/j1sessn.jsp?sessn=TS-1262&yr=2007&track=3>in
which he shows how you can use hundreds of lines of highly abstracted,
highly reflective code that passes lots of class names around as strings to
do what would take 3 lines in R. Why this is useful, I cannot say.
Overall, I really feel like Weka would provide a better framework, not that
I am a grand fan of it, either.
On Mon, Apr 20, 2009 at 12:30 PM, Otis Gospodnetic <
otis_gospodnetic@yahoo.com> wrote:
>
> Can you elaborate on the nail on the head miss comment please?
>
> Thanks,
> Otis
>
>
>
> ----- Original Message ----
> > From: Ted Dunning <te...@gmail.com>
> > To: mahout-dev@lucene.apache.org
> > Sent: Monday, April 20, 2009 1:34:29 PM
> > Subject: Re: Java Data Mining specification
> >
> > It also is a pretty funky proposal. I can see the point from the
> standpoint
> > of the large vendors, but for us it really doesn't apply all that much.
> >
> > In fact, most of these standardization efforts are not really hitting the
> > nail on the head as far as I am concerned.
> >
> > On Mon, Apr 20, 2009 at 9:23 AM, Otis Gospodnetic <
> > otis_gospodnetic@yahoo.com> wrote:
> >
> > >
> > > I remember Karl Wettin bringing this up a few months ago. There was a
> bit
> > > of discussion, but nothing happened, probably because of the "a bit
> defunct"
> > > part.
> > >
> > >
>
>
--
Ted Dunning, CTO
DeepDyve
111 West Evelyn Ave. Ste. 202
Sunnyvale, CA 94086
www.deepdyve.com
858-414-0013 (m)
408-773-0220 (fax)
Re: Java Data Mining specification
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Can you elaborate on the nail on the head miss comment please?
Thanks,
Otis
----- Original Message ----
> From: Ted Dunning <te...@gmail.com>
> To: mahout-dev@lucene.apache.org
> Sent: Monday, April 20, 2009 1:34:29 PM
> Subject: Re: Java Data Mining specification
>
> It also is a pretty funky proposal. I can see the point from the standpoint
> of the large vendors, but for us it really doesn't apply all that much.
>
> In fact, most of these standardization efforts are not really hitting the
> nail on the head as far as I am concerned.
>
> On Mon, Apr 20, 2009 at 9:23 AM, Otis Gospodnetic <
> otis_gospodnetic@yahoo.com> wrote:
>
> >
> > I remember Karl Wettin bringing this up a few months ago. There was a bit
> > of discussion, but nothing happened, probably because of the "a bit defunct"
> > part.
> >
> >
Re: Java Data Mining specification
Posted by Ted Dunning <te...@gmail.com>.
It also is a pretty funky proposal. I can see the point from the standpoint
of the large vendors, but for us it really doesn't apply all that much.
In fact, most of these standardization efforts are not really hitting the
nail on the head as far as I am concerned.
On Mon, Apr 20, 2009 at 9:23 AM, Otis Gospodnetic <
otis_gospodnetic@yahoo.com> wrote:
>
> I remember Karl Wettin bringing this up a few months ago. There was a bit
> of discussion, but nothing happened, probably because of the "a bit defunct"
> part.
>
>
Re: Java Data Mining specification
Posted by Otis Gospodnetic <ot...@yahoo.com>.
I remember Karl Wettin bringing this up a few months ago. There was a bit of discussion, but nothing happened, probably because of the "a bit defunct" part.
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
----- Original Message ----
> From: Sean Owen <sr...@gmail.com>
> To: Mahout Dev List <ma...@lucene.apache.org>
> Sent: Monday, April 20, 2009 8:49:29 AM
> Subject: Java Data Mining specification
>
> http://jcp.org/aboutJava/communityprocess/edr/jsr247/index.html
>
> I only just stumbled on this today. Looks like there has already been
> a JSR on data mining in Java, and the above is a JSR for version 2.0.
> Both seem a bit defunct, but, interesting to see there was some effort
> to construct a standard API for exactly the sorts of things Mahout
> provides.
>
> I am not suggesting we implement this API -- purely because it looks
> like it is not being worked on. But interesting to look at.
>
> Sean