You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Charles Earl <ch...@me.com> on 2011/10/26 23:34:03 UTC
Monte Carlo toolkits in Mahout?
I'm interested to learn if there is a general monte carlo methods package that has been developed for Mahout or Hadoop. For example, having the capability of GNU scientific computing library. This might be off the machine learning focus of Mahout, but thought there might be overlap. Thanks.
Charles Earl
Re: Monte Carlo toolkits in Mahout?
Posted by Charles Earl <ch...@me.com>.
Thanks.
I have read R has some collab with Clodera. Is there a "mapreduce R" that is stable (apart from streaming).
C
On Oct 27, 2011, at 10:16 AM, Ted Dunning <te...@gmail.com> wrote:
> If you are doing particle event simulation simply using forward inference,
> then map-only tasks are just fine and almost anything will do. Mahout has
> decent random number generators.
>
> If you want modern MCMC codes for reverse inference, I would go elsewhere.
> Radford Neal has some good stuff and Andrew Gelman's group has been doing
> some good work. Both of these are ultimately accessible from R.
>
> On Thu, Oct 27, 2011 at 7:03 AM, Charles Earl <ch...@me.com> wrote:
>
>> Hi Ted,
>> This is mostly for particle event simulation, I'll give the FOAM toolkit as
>> an example
>> http://jadach.home.cern.ch/jadach/Foam/Index.html
>> But I'm trying to determine if there might be commonalities for other
>> domains, options pricing as one.
>> Yes, you are right about GNU SL. I have also wonder whether Colt or
>> Parallel Colt might be worth investigating.
>> Thanks
>> Charles
>>
>> On Oct 26, 2011, at 5:47 PM, Ted Dunning wrote:
>>
>>> We have no significant monte carlo methods in Mahout.
>>>
>>> We do have a reasonable evolutionary optimizer, but randomness isn't the
>>> same as a real Monte Carlo code in the sense of Metropolis-Hastings or
>> Gibbs
>>> sampling.
>>>
>>> What do you mean by "capability of GNU scientific computing library"? I
>>> don't know of any significant MCMC code there either.
>>>
>>> In general, efficient MonteCarlo codes can be tricky to get to work in
>> the
>>> map-reduce paradigm. You can start multiple chains, but that doesn't
>>> necessarily help if all have to spend a long time burning in.
>>>
>>> Can you say more about what you are looking for?
>>>
>>> On Wed, Oct 26, 2011 at 2:34 PM, Charles Earl <ch...@me.com>
>> wrote:
>>>
>>>> I'm interested to learn if there is a general monte carlo methods
>> package
>>>> that has been developed for Mahout or Hadoop. For example, having the
>>>> capability of GNU scientific computing library. This might be off the
>>>> machine learning focus of Mahout, but thought there might be overlap.
>>>> Thanks.
>>>> Charles Earl
>>>>
>>>>
>>
>>
Re: Monte Carlo toolkits in Mahout?
Posted by Ted Dunning <te...@gmail.com>.
If you are doing particle event simulation simply using forward inference,
then map-only tasks are just fine and almost anything will do. Mahout has
decent random number generators.
If you want modern MCMC codes for reverse inference, I would go elsewhere.
Radford Neal has some good stuff and Andrew Gelman's group has been doing
some good work. Both of these are ultimately accessible from R.
On Thu, Oct 27, 2011 at 7:03 AM, Charles Earl <ch...@me.com> wrote:
> Hi Ted,
> This is mostly for particle event simulation, I'll give the FOAM toolkit as
> an example
> http://jadach.home.cern.ch/jadach/Foam/Index.html
> But I'm trying to determine if there might be commonalities for other
> domains, options pricing as one.
> Yes, you are right about GNU SL. I have also wonder whether Colt or
> Parallel Colt might be worth investigating.
> Thanks
> Charles
>
> On Oct 26, 2011, at 5:47 PM, Ted Dunning wrote:
>
> > We have no significant monte carlo methods in Mahout.
> >
> > We do have a reasonable evolutionary optimizer, but randomness isn't the
> > same as a real Monte Carlo code in the sense of Metropolis-Hastings or
> Gibbs
> > sampling.
> >
> > What do you mean by "capability of GNU scientific computing library"? I
> > don't know of any significant MCMC code there either.
> >
> > In general, efficient MonteCarlo codes can be tricky to get to work in
> the
> > map-reduce paradigm. You can start multiple chains, but that doesn't
> > necessarily help if all have to spend a long time burning in.
> >
> > Can you say more about what you are looking for?
> >
> > On Wed, Oct 26, 2011 at 2:34 PM, Charles Earl <ch...@me.com>
> wrote:
> >
> >> I'm interested to learn if there is a general monte carlo methods
> package
> >> that has been developed for Mahout or Hadoop. For example, having the
> >> capability of GNU scientific computing library. This might be off the
> >> machine learning focus of Mahout, but thought there might be overlap.
> >> Thanks.
> >> Charles Earl
> >>
> >>
>
>
Re: Monte Carlo toolkits in Mahout?
Posted by Mathias Herberts <ma...@gmail.com>.
Option pricing can be done in plain MapReduce rather easily (for
example using jQuantLib), not sure there's room in Mahout for that.
Mathias.
Re: Monte Carlo toolkits in Mahout?
Posted by Charles Earl <ch...@me.com>.
Hi Ted,
This is mostly for particle event simulation, I'll give the FOAM toolkit as an example
http://jadach.home.cern.ch/jadach/Foam/Index.html
But I'm trying to determine if there might be commonalities for other domains, options pricing as one.
Yes, you are right about GNU SL. I have also wonder whether Colt or Parallel Colt might be worth investigating.
Thanks
Charles
On Oct 26, 2011, at 5:47 PM, Ted Dunning wrote:
> We have no significant monte carlo methods in Mahout.
>
> We do have a reasonable evolutionary optimizer, but randomness isn't the
> same as a real Monte Carlo code in the sense of Metropolis-Hastings or Gibbs
> sampling.
>
> What do you mean by "capability of GNU scientific computing library"? I
> don't know of any significant MCMC code there either.
>
> In general, efficient MonteCarlo codes can be tricky to get to work in the
> map-reduce paradigm. You can start multiple chains, but that doesn't
> necessarily help if all have to spend a long time burning in.
>
> Can you say more about what you are looking for?
>
> On Wed, Oct 26, 2011 at 2:34 PM, Charles Earl <ch...@me.com> wrote:
>
>> I'm interested to learn if there is a general monte carlo methods package
>> that has been developed for Mahout or Hadoop. For example, having the
>> capability of GNU scientific computing library. This might be off the
>> machine learning focus of Mahout, but thought there might be overlap.
>> Thanks.
>> Charles Earl
>>
>>
Re: Monte Carlo toolkits in Mahout?
Posted by Ted Dunning <te...@gmail.com>.
We have no significant monte carlo methods in Mahout.
We do have a reasonable evolutionary optimizer, but randomness isn't the
same as a real Monte Carlo code in the sense of Metropolis-Hastings or Gibbs
sampling.
What do you mean by "capability of GNU scientific computing library"? I
don't know of any significant MCMC code there either.
In general, efficient MonteCarlo codes can be tricky to get to work in the
map-reduce paradigm. You can start multiple chains, but that doesn't
necessarily help if all have to spend a long time burning in.
Can you say more about what you are looking for?
On Wed, Oct 26, 2011 at 2:34 PM, Charles Earl <ch...@me.com> wrote:
> I'm interested to learn if there is a general monte carlo methods package
> that has been developed for Mahout or Hadoop. For example, having the
> capability of GNU scientific computing library. This might be off the
> machine learning focus of Mahout, but thought there might be overlap.
> Thanks.
> Charles Earl
>
>