You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@predictionio.apache.org by Daniel O' Shaughnessy <da...@gmail.com> on 2017/01/09 18:45:23 UTC
Ecom recommender adding like event
Hi,
I'm attempting to add a like/dislike event to the ecom recommender ( as per
http://predictionio.incubator.apache.org/templates/similarproduct/multi-events-multi-algos/
).
This method combines the results of 2 separate algorithms in the Serving
class.
When I deploy this using the predictionIO Heroku buildpack it seems to
deploy and run the release command OK but when I check the Heroku logs it
fails here:
*2017-01-09T15:32:44.905195+00:00 app[web.1]: [INFO] [Engine] Using
persisted model*
*2017-01-09T15:32:44.912783+00:00 app[web.1]: [INFO] [Engine] Loaded model
com.dos.sfdc.ECommModel for algorithm com.dos.sfdc.ECommAlgorithm*
*2017-01-09T15:32:45.307111+00:00 app[web.1]: [INFO] [Engine]
Custom-persisted model detected for algorithm com.dos.sfdc.LikeAlgorithm*
*2017-01-09T15:32:45.668678+00:00 app[web.1]: [ERROR] [OneForOneStrategy]
Cannot call methods on a stopped SparkContext.*
*2017-01-09T15:32:45.668690+00:00 app[web.1]: This stopped SparkContext was
created at:*
*2017-01-09T15:32:45.668691+00:00 app[web.1]: *
*2017-01-09T15:32:45.668692+00:00 app[web.1]:
org.apache.spark.SparkContext.<init>(SparkContext.scala:82)*
*2017-01-09T15:32:45.668693+00:00 app[web.1]:
org.apache.predictionio.workflow.WorkflowContext$.apply(WorkflowContext.scala:45)*
*2017-01-09T15:32:45.668694+00:00 app[web.1]:
org.apache.predictionio.workflow.CreateServer$.createServerActorWithEngine(CreateServer.scala:228)*
Anyone come across this before?
Thanks,
Daniel.
Apache PredictionIO + Universal Recommender in an AWS AMI
Posted by Pat Ferrel <pa...@occamsmachete.com>.
If you want to experiment with something quickly, you can try the new Apache PredictionIO + Universal Recommender AMI. It will run any Apache PIO compatible Template and has the UR pre-installed. You just create and start an instance and all services are started on boot. The only thing you start by hand is the EventServer and PredictionServer. This is a single all-in-one setup for a single instance.
https://aws.amazon.com/marketplace/pp/B01N310FF0 <https://aws.amazon.com/marketplace/pp/B01N310FF0>
Re: Ecom recommender adding like event
Posted by Daniel O' Shaughnessy <da...@gmail.com>.
Hi Mars,
I guess my choice of words could have been better here! No problem with the
buildpack, ran exactly as per instructions given.
I have the persistent fs setup but I'll look into it further, seems to fail
on the very last step : (
Thanks for your input.
Pat - sounds good, look forward to the announcement!
On Tue, 10 Jan 2017 at 21:03 Mars Hall <ma...@heroku.com> wrote:
> Hi Daniel,
>
> If you think there's a problem with the PredictionIO buildpack for Heroku,
> please file as issue on its GitHub repo
> https://github.com/heroku/predictionio-buildpack/issues
>
> If it works locally but not on Heroku, my wild guess is that one of your
> algorithms is serializing the model to the filesystem, and it's throwing an
> error when trying to retrieve that model. Heroku does not support
> filesystem persistence between runs (e.g. files committed during
> release/train will not persist to web/deploy)
>
> Perhaps try enabling S3 HDFS for filesystem persistence
> https://github.com/heroku/predictionio-buildpack/blob/master/CUSTOM.md#optional-persistent-filesystem
> Note that with HDFS filesystem path references must be from `/` root, not
> nested in a User ID directory.
>
> *Mars
>
> ( <> .. <> )
>
> > On Jan 10, 2017, at 00:53, Daniel O' Shaughnessy <
> danieljamesdavid@gmail.com> wrote:
> >
> > Hi Pat,
> >
> > Ya I guess it's an issue with the pio heroku buildpack. It seems to
> build and train OK based on the logs but then throws that error when trying
> to deploy.
> >
> > I'm not actually using the similar products template I'm using this:
> https://github.com/apache/incubator-predictionio-template-ecom-recommender
> >
> > I'm trying to extend the ecom template to include 'likes' to try and
> improve the results and using the tutorial as a guideline. I'm using this
> for an app store recommender...Ideally I'd be using the UR but I can't get
> this up on Heroku so far, when AWS approve it I may go down that route for
> hosting eventually.
> >
> > On Mon, 9 Jan 2017 at 20:44 Pat Ferrel <pa...@occamsmachete.com> wrote:
> > AFAIK, the Heroku support is fairly new so not surprising that a
> multiple engine template has trouble. Multiple Algorithms in a Template are
> fairly rare. I’d ask Heroku what is going on.
> >
> > BTW what are you using this for? There are other ways to get similar
> products—some of which may work better. For instance is there a reason for
> multiple algorithms?
> >
> >
> >
> > On Jan 9, 2017, at 10:45 AM, Daniel O' Shaughnessy <
> danieljamesdavid@gmail.com> wrote:
> >
> > Hi,
> >
> > I'm attempting to add a like/dislike event to the ecom recommender ( as
> per
> http://predictionio.incubator.apache.org/templates/similarproduct/multi-events-multi-algos/
> ). This method combines the results of 2 separate algorithms in the Serving
> class.
> >
> > When I deploy this using the predictionIO Heroku buildpack it seems to
> deploy and run the release command OK but when I check the Heroku logs it
> fails here:
> >
> > 2017-01-09T15:32:44.905195+00:00 app[web.1]: [INFO] [Engine] Using
> persisted model
> >
> > 2017-01-09T15:32:44.912783+00:00 app[web.1]: [INFO] [Engine] Loaded
> model com.dos.sfdc.ECommModel for algorithm com.dos.sfdc.ECommAlgorithm
> >
> > 2017-01-09T15:32:45.307111+00:00 app[web.1]: [INFO] [Engine]
> Custom-persisted model detected for algorithm com.dos.sfdc.LikeAlgorithm
> >
> > 2017-01-09T15:32:45.668678+00:00 app[web.1]: [ERROR] [OneForOneStrategy]
> Cannot call methods on a stopped SparkContext.
> >
> > 2017-01-09T15:32:45.668690+00:00 app[web.1]: This stopped SparkContext
> was created at:
> >
> > 2017-01-09T15:32:45.668691+00:00 app[web.1]:
> >
> > 2017-01-09T15:32:45.668692+00:00 app[web.1]:
> org.apache.spark.SparkContext.<init>(SparkContext.scala:82)
> >
> > 2017-01-09T15:32:45.668693+00:00 app[web.1]:
> org.apache.predictionio.workflow.WorkflowContext$.apply(WorkflowContext.scala:45)
> >
> > 2017-01-09T15:32:45.668694+00:00 app[web.1]:
> org.apache.predictionio.workflow.CreateServer$.createServerActorWithEngine(CreateServer.scala:228)
> >
> > Anyone come across this before?
> >
> >
> >
> > Thanks,
> >
> >
> >
> > Daniel.
> >
> >
>
>
Re: Ecom recommender adding like event
Posted by Mars Hall <ma...@heroku.com>.
Hi Daniel,
If you think there's a problem with the PredictionIO buildpack for Heroku, please file as issue on its GitHub repo https://github.com/heroku/predictionio-buildpack/issues
If it works locally but not on Heroku, my wild guess is that one of your algorithms is serializing the model to the filesystem, and it's throwing an error when trying to retrieve that model. Heroku does not support filesystem persistence between runs (e.g. files committed during release/train will not persist to web/deploy)
Perhaps try enabling S3 HDFS for filesystem persistence https://github.com/heroku/predictionio-buildpack/blob/master/CUSTOM.md#optional-persistent-filesystem Note that with HDFS filesystem path references must be from `/` root, not nested in a User ID directory.
*Mars
( <> .. <> )
> On Jan 10, 2017, at 00:53, Daniel O' Shaughnessy <da...@gmail.com> wrote:
>
> Hi Pat,
>
> Ya I guess it's an issue with the pio heroku buildpack. It seems to build and train OK based on the logs but then throws that error when trying to deploy.
>
> I'm not actually using the similar products template I'm using this: https://github.com/apache/incubator-predictionio-template-ecom-recommender
>
> I'm trying to extend the ecom template to include 'likes' to try and improve the results and using the tutorial as a guideline. I'm using this for an app store recommender...Ideally I'd be using the UR but I can't get this up on Heroku so far, when AWS approve it I may go down that route for hosting eventually.
>
> On Mon, 9 Jan 2017 at 20:44 Pat Ferrel <pa...@occamsmachete.com> wrote:
> AFAIK, the Heroku support is fairly new so not surprising that a multiple engine template has trouble. Multiple Algorithms in a Template are fairly rare. I’d ask Heroku what is going on.
>
> BTW what are you using this for? There are other ways to get similar products—some of which may work better. For instance is there a reason for multiple algorithms?
>
>
>
> On Jan 9, 2017, at 10:45 AM, Daniel O' Shaughnessy <da...@gmail.com> wrote:
>
> Hi,
>
> I'm attempting to add a like/dislike event to the ecom recommender ( as per http://predictionio.incubator.apache.org/templates/similarproduct/multi-events-multi-algos/ ). This method combines the results of 2 separate algorithms in the Serving class.
>
> When I deploy this using the predictionIO Heroku buildpack it seems to deploy and run the release command OK but when I check the Heroku logs it fails here:
>
> 2017-01-09T15:32:44.905195+00:00 app[web.1]: [INFO] [Engine] Using persisted model
>
> 2017-01-09T15:32:44.912783+00:00 app[web.1]: [INFO] [Engine] Loaded model com.dos.sfdc.ECommModel for algorithm com.dos.sfdc.ECommAlgorithm
>
> 2017-01-09T15:32:45.307111+00:00 app[web.1]: [INFO] [Engine] Custom-persisted model detected for algorithm com.dos.sfdc.LikeAlgorithm
>
> 2017-01-09T15:32:45.668678+00:00 app[web.1]: [ERROR] [OneForOneStrategy] Cannot call methods on a stopped SparkContext.
>
> 2017-01-09T15:32:45.668690+00:00 app[web.1]: This stopped SparkContext was created at:
>
> 2017-01-09T15:32:45.668691+00:00 app[web.1]:
>
> 2017-01-09T15:32:45.668692+00:00 app[web.1]: org.apache.spark.SparkContext.<init>(SparkContext.scala:82)
>
> 2017-01-09T15:32:45.668693+00:00 app[web.1]: org.apache.predictionio.workflow.WorkflowContext$.apply(WorkflowContext.scala:45)
>
> 2017-01-09T15:32:45.668694+00:00 app[web.1]: org.apache.predictionio.workflow.CreateServer$.createServerActorWithEngine(CreateServer.scala:228)
>
> Anyone come across this before?
>
>
>
> Thanks,
>
>
>
> Daniel.
>
>
Re: Ecom recommender adding like event
Posted by Daniel O' Shaughnessy <da...@gmail.com>.
Hi Pat,
Ya I guess it's an issue with the pio heroku buildpack. It seems to build
and train OK based on the logs but then throws that error when trying to
deploy.
I'm not actually using the similar products template I'm using this:
https://github.com/apache/incubator-predictionio-template-ecom-recommender
I'm trying to extend the ecom template to include 'likes' to try and
improve the results and using the tutorial as a guideline. I'm using this
for an app store recommender...Ideally I'd be using the UR but I can't get
this up on Heroku so far, when AWS approve it I may go down that route for
hosting eventually.
On Mon, 9 Jan 2017 at 20:44 Pat Ferrel <pa...@occamsmachete.com> wrote:
> AFAIK, the Heroku support is fairly new so not surprising that a multiple
> engine template has trouble. Multiple Algorithms in a Template are fairly
> rare. I’d ask Heroku what is going on.
>
> BTW what are you using this for? There are other ways to get similar
> products—some of which may work better. For instance is there a reason for
> multiple algorithms?
>
>
>
> On Jan 9, 2017, at 10:45 AM, Daniel O' Shaughnessy <
> danieljamesdavid@gmail.com> wrote:
>
> Hi,
>
> I'm attempting to add a like/dislike event to the ecom recommender ( as
> per
> http://predictionio.incubator.apache.org/templates/similarproduct/multi-events-multi-algos/ ).
> This method combines the results of 2 separate algorithms in the Serving
> class.
>
> When I deploy this using the predictionIO Heroku buildpack it seems to
> deploy and run the release command OK but when I check the Heroku logs it
> fails here:
>
> *2017-01-09T15:32:44.905195+00:00 app[web.1]: [INFO] [Engine] Using
> persisted model*
>
> *2017-01-09T15:32:44.912783+00:00 app[web.1]: [INFO] [Engine] Loaded model
> com.dos.sfdc.ECommModel for algorithm com.dos.sfdc.ECommAlgorithm*
>
> *2017-01-09T15:32:45.307111+00:00 app[web.1]: [INFO] [Engine]
> Custom-persisted model detected for algorithm com.dos.sfdc.LikeAlgorithm*
>
> *2017-01-09T15:32:45.668678+00:00 app[web.1]: [ERROR] [OneForOneStrategy]
> Cannot call methods on a stopped SparkContext.*
>
> *2017-01-09T15:32:45.668690+00:00 app[web.1]: This stopped SparkContext
> was created at:*
>
> *2017-01-09T15:32:45.668691+00:00 app[web.1]: *
>
> *2017-01-09T15:32:45.668692+00:00 app[web.1]:
> org.apache.spark.SparkContext.<init>(SparkContext.scala:82)*
>
> *2017-01-09T15:32:45.668693+00:00 app[web.1]:
> org.apache.predictionio.workflow.WorkflowContext$.apply(WorkflowContext.scala:45)*
>
> *2017-01-09T15:32:45.668694+00:00 app[web.1]:
> org.apache.predictionio.workflow.CreateServer$.createServerActorWithEngine(CreateServer.scala:228)*
>
> Anyone come across this before?
>
>
> Thanks,
>
>
> Daniel.
>
>
Re: Ecom recommender adding like event
Posted by Pat Ferrel <pa...@occamsmachete.com>.
AFAIK, the Heroku support is fairly new so not surprising that a multiple engine template has trouble. Multiple Algorithms in a Template are fairly rare. I’d ask Heroku what is going on.
BTW what are you using this for? There are other ways to get similar products—some of which may work better. For instance is there a reason for multiple algorithms?
On Jan 9, 2017, at 10:45 AM, Daniel O' Shaughnessy <da...@gmail.com> wrote:
Hi,
I'm attempting to add a like/dislike event to the ecom recommender ( as per http://predictionio.incubator.apache.org/templates/similarproduct/multi-events-multi-algos/ <http://predictionio.incubator.apache.org/templates/similarproduct/multi-events-multi-algos/> ). This method combines the results of 2 separate algorithms in the Serving class.
When I deploy this using the predictionIO Heroku buildpack it seems to deploy and run the release command OK but when I check the Heroku logs it fails here:
2017-01-09T15:32:44.905195+00:00 app[web.1]: [INFO] [Engine] Using persisted model
2017-01-09T15:32:44.912783+00:00 app[web.1]: [INFO] [Engine] Loaded model com.dos.sfdc.ECommModel for algorithm com.dos.sfdc.ECommAlgorithm
2017-01-09T15:32:45.307111+00:00 app[web.1]: [INFO] [Engine] Custom-persisted model detected for algorithm com.dos.sfdc.LikeAlgorithm
2017-01-09T15:32:45.668678+00:00 app[web.1]: [ERROR] [OneForOneStrategy] Cannot call methods on a stopped SparkContext.
2017-01-09T15:32:45.668690+00:00 app[web.1]: This stopped SparkContext was created at:
2017-01-09T15:32:45.668691+00:00 app[web.1]:
2017-01-09T15:32:45.668692+00:00 app[web.1]: org.apache.spark.SparkContext.<init>(SparkContext.scala:82)
2017-01-09T15:32:45.668693+00:00 app[web.1]: org.apache.predictionio.workflow.WorkflowContext$.apply(WorkflowContext.scala:45)
2017-01-09T15:32:45.668694+00:00 app[web.1]: org.apache.predictionio.workflow.CreateServer$.createServerActorWithEngine(CreateServer.scala:228)
Anyone come across this before?
Thanks,
Daniel.