You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@predictionio.apache.org by Abhimanyu Nagrath <ab...@gmail.com> on 2017/11/22 07:18:52 UTC

Time taken for PIO Training

Hi, I am using pio V0.12.0 (Hbase 1.2.6, Elasticsearch 5.2.1, Spark 2.6.1).
I am using template

>
https://github.com/EmergentOrder/template-scala-probabilistic-classifier-batch-lbfgs

I spawned two servers each having configuration(244 GB RAM, 16 Cores). On 1
server I uploaded 1 million events with 6000 features and on the second
server uploaded 1 million events with 30000 features. It took 24 hrs to
train events with 6000 features whereas dataset with 30000 features set got
trained in 3 hours.

I am not able to get how is this possible that data with 30000 features got
trained so quickly compared to the dataset having less number of features.


Regards,
Abhimanyu