[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

[avulanov <gi...@git.apache.org>]

Github user avulanov commented on the issue:

    https://github.com/apache/spark/pull/13617
  
    @JeremyNixon Thanks for the comprehensive list of references!
    
    The internal API of Spark ANN is designed to be flexible and can handle different types of layers. However, only a part of the API is made public. We have to limit the number of public classes in order to make it simpler to support other languages. This forces us to use (String or Number) parameters instead of introducing of new public classes. One of the options to specify the architecture of ANN is to use text configuration with layer-wise description. We have considered using Caffe format for this. It gives the benefit of compatibility with well known deep learning tool and simplifies the support of other languages in Spark. Implementation of a parser for the subset of Caffe format might be the first step towards the support of general ANN architectures in Spark. However, other ANN features are of higher priority for Spark ML right now: https://issues.apache.org/jira/browse/SPARK-15581. In particular, it is Autoencoder and CNN. It would be great if you could help with them. Fo
 r example, review the Autoencoder PR: https://github.com/apache/spark/pull/13621
    
    With regards to the advanced ANN features, we are currently building a package that is supposed to support them https://github.com/avulanov/scalable-ann. Eventually, some of them, might find their place in the main branch. It would be great to collaborate on this effort.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org