You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by marfago <ma...@inwind.it> on 2015/11/28 23:25:59 UTC

Spark and simulated annealing

Hi All,

I would like to implement a simulated annealing algorithm with Spark.
What is the best way to do that with python or scala? Is there any library
already implementing it?

Thank you in advance.
Marco



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-and-simulated-annealing-tp25507.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: Spark and simulated annealing

Posted by marfago <ma...@inwind.it>.
HI,

Thank you for your suggestion.

Is the scipy library (in particular scipy.optimize.anneal function) still
able to leverage the parallelism and distributed calculus offered by Spark?

Marco



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-and-simulated-annealing-tp25507p25537.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: Spark and simulated annealing

Posted by Gylfi <gy...@berkeley.edu>.
1) Start by looking at ML-lib or KeystoneML 

2) If you can't find an impl., start by analyzing the access patterns and
data manipulations you will need to implement. 

3) Then figure out if it fits Spark structures.. and when you realized it
doesn't you start speculating on how you can twist or strong-arm it to fit
:) 

4) You skipped 2) didn't you.. ok so do that now and go back to 3). 



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-and-simulated-annealing-tp25507p25513.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org