You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Yury Babak <y....@gmail.com> on 2018/08/01 14:13:03 UTC

Re: [ML] Machine Learning Pipeline Improvement

Sure, https://issues.apache.org/jira/browse/IGNITE-9158.

Regards,
Yury



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [ML] Machine Learning Pipeline Improvement

Posted by Alexey Zinoviev <za...@gmail.com>.
Dear Manu
it could be a great idea!

Could you please provide any examples of Apache Arrow integration for speed
up ML computation in another ML frameworks, it would be very helpful!

Sincerely yours
      Alexey Zinovyev



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [ML] Machine Learning Pipeline Improvement

Posted by Manu <ma...@hotmail.com>.
Hi, all!

Could be viable to integrate Apache Arrow to improve ML computation using
GPU?
Out of this thread, could be viable to integrate Apache Arrow to improve
Indexing computation using GPU?

Regards

https://rapids.ai <https://rapids.ai>  
https://arrow.apache.org <https://arrow.apache.org>  



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [ML] Machine Learning Pipeline Improvement

Posted by Alexey Zinoviev <za...@gmail.com>.
The prototype of the API will look like that

PipelineMdl mdl = new Pipeline<Integer, Object[]> ()
                       .addFeatureExtractor(featureExtractor)
                       .addLabelExtractor(lbExtractor)
                       .addStage(new EncoderTrainer<Integer, Object[]>()
                           .withEncoderType(EncoderType.STRING_ENCODER)
                           .withEncodedFeature(1)
                           .withEncodedFeature(6))
                       .addStage(new ImputerTrainer<Integer, Object[]>())
                       .addStage(new MinMaxScalerTrainer<Integer,
Object[]>())
                       .addStage(new NormalizationTrainer<Integer,
Object[]>()
                           .withP(1))
                       .addFinalStage(new
DecisionTreeClassificationTrainer(5, 0))
                       .fit(ignite, dataCache);

Also, I've added separate ticket for the update of ParamGrid/CrossValidation
API to support tune hyperparameters not only in final trainers but in
intermideate preprocessing stages too.

https://issues.apache.org/jira/browse/IGNITE-9497

I suggest to add this feature in 2.8 because it doesn't change the current
API of algorithms and has no serialized issues



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/