You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Edward J. Yoon (JIRA)" <ji...@apache.org> on 2015/07/03 02:13:04 UTC
[jira] [Commented] (HAMA-675) Deep Learning Computation Model
[ https://issues.apache.org/jira/browse/HAMA-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612690#comment-14612690 ]
Edward J. Yoon commented on HAMA-675:
-------------------------------------
Here's new neuron-centric programming interface I propose. I'll start the refactoring first.
{code}
public class ThreeLayerANNExample {
/**
* User-defined sigmoid actiavation function
*/
public static class Sigmoid extends ActivationFunction {
@Override
public double apply(double input) {
return 1.0 / (1 + Math.exp(-input));
}
@Override
public double applyDerivative(double input) {
return input * (1 - input);
}
}
/**
* User-defined cost function
*/
public static class CrossEntropy extends CostFunction {
@Override
public double apply(double target, double actual) {
double adjustedTarget = (target == 0 ? 0.000001 : target);
adjustedTarget = (target == 1.0 ? 0.999999 : target);
double adjustedActual = (actual == 0 ? 0.000001 : actual);
adjustedActual = (actual == 1 ? 0.999999 : actual);
return -adjustedTarget * Math.log(adjustedActual) - (1 - adjustedTarget)
* Math.log(1 - adjustedActual);
}
@Override
public double derivative(double target, double actual) {
double adjustedTarget = target;
double adjustedActual = actual;
if (adjustedActual == 1) {
adjustedActual = 0.999;
} else if (actual == 0) {
adjustedActual = 0.001;
}
if (adjustedTarget == 1) {
adjustedTarget = 0.999;
} else if (adjustedTarget == 0) {
adjustedTarget = 0.001;
}
return -adjustedTarget / adjustedActual + (1 - adjustedTarget)
/ (1 - adjustedActual);
}
}
public static void main(String[] args) throws Exception {
ANNJob ann = new ANNJob();
// set learning rate and momentum weight
ann.setLearningRate(0.1);
ann.setMomemtumWeight(0.1);
// initialize the topology of the model and set the activation function and parallel degree.
ann.addLayer(featureDimension, Sigmoid.class, numOfTasks);
ann.addLayer(featureDimension, Sigmoid.class, numOfTasks);
ann.addLayer(labelDimension, Sigmoid.class, numOfTasks);
// set the cost function to evaluate the error
ann.setCostFunction(CrossEntropy.class);
ann.setMaxIteration(50);
ann.setBatchSize(500);
ann.setInputPath(path);
...
}
}
{code}
> Deep Learning Computation Model
> -------------------------------
>
> Key: HAMA-675
> URL: https://issues.apache.org/jira/browse/HAMA-675
> Project: Hama
> Issue Type: New Feature
> Components: machine learning
> Reporter: Thomas Jungblut
> Assignee: Edward J. Yoon
>
> Jeff Dean mentioned a computational model in this video: http://techtalks.tv/talks/57639/
> There they are using the same idea of the Pregel system, they are defining a upstream and a downstream computation function for a neuron (for cost and its gradient). Then you can roughly tell about how the framework should partition the neurons.
> All the messaging will be handled by the underlying messaging framework.
> Can we implement something equally?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)