You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@predictionio.apache.org by Pat Ferrel <pa...@occamsmachete.com> on 2016/12/13 21:06:56 UTC
Re: Customizing Recommender engine
The UR has a new Apache PIO compatible repo here: https://github.com/actionml/universal-recommender.git
git clone https://github.com/actionml/universal-recommender.git ur
and proceed. The UR allows boosts or filters by properties. You are using a filter (bias -1), which does not work with the template you are using. Each template defines its own queries, config (engine.json), and input formats. There are a few common ideas but each one will have quite a few differences also and very different features.
On Dec 13, 2016, at 6:20 AM, Hossein Bahrami <h....@live.com> wrote:
Hi, thanks for your reply. I don’t use Universal recommender, actually I use the Recommender Engine (http://predictionio.incubator.apache.org/templates/recommendation/quickstart/ <http://predictionio.incubator.apache.org/templates/recommendation/quickstart/>)
First I tried to use Universal recommender but it was difficulties building the project so I switched to the Recommendation Engine (RE), the RE works fine but don’t know how to customize it to take other events (items’ properties as I described ) into account.
Seems the RE returns very high score recommendations with my data so I guess it’s the choice here for me and just want to do some customization on it to make it predict recommendations per category (i.e property) of items.
Regards
Hossein
From: Magnus Kragelund <ma...@ida.dk>
Sent: Tuesday, December 13, 2016 12:17 PM
To: user@predictionio.incubator.apache.org <ma...@predictionio.incubator.apache.org>
Subject: Re: Customizing Recommender engine
Hi,
Assuming that you are using the Universal Recommender Engine, you should have a look at the "Queries" section here: https://github.com/PredictionIO/template-scala-parallel-universal-recommendation#queries <https://github.com/PredictionIO/template-scala-parallel-universal-recommendation#queries>
Try this request instead, where the "fields" property is used to filter by category
{
"user": "674296",
"num": 10,
"fields": [
{
"name": "categories",
"values": ["CAT1", "CAT2"],
"bias": -1
}
]
}
/magnus
From: Hossein Bahrami <h....@live.com>
Sent: Tuesday, December 13, 2016 10:55:00 AM
To: user@predictionio.incubator.apache.org
Subject: Customizing Recommender engine
Dear all,
I’m new to Predictionio, currently I’m using it and I managed to import (rate, buy) events and getting pretty good results querying it. But now I want to limit the results for items in specific categories. I’ve created events for items’ properties (categories) as well .
I am posting this query to engine but it seems doesn’t care about the categories and returns same result every time.
{ "user": "674296", "num": 10, "categories" : ["CAT2", "CAT1"] }
I’ve imported bellow events
client.create_event(event="rate",
entity_type="user",
entity_id=int(row['userid']),
target_entity_type="item",
target_entity_id=int(row['itemid']),
properties= { "rating" : float(row['rating']) });
client.create_event(
event="buy",
entity_type="user",
entity_id=int(row['userid']),
target_entity_type="item",
target_entity_id=int(row['itemid']), );
client.create_event(
event="$set",
entity_type="item",
entity_id=itemid,
properties= { "categories": itemcats }
);
Could someone give me a solution or hint to how customize this recommender engine to take the categories into account.
Thanks in advance
Hossein
RE: Customizing Recommender engine
Posted by Hossein Bahrami <h....@live.com>.
Hi Pat,
Previous error (" java.lang.NoClassDefFoundError: org/apache/predictionio/controller/EngineFactory) gone after I downloaded the UR temple again under pio 0.9. and now I have below exception. I tried to fix it by using numIteation config tag but seems the UR doesn’t work like the RE and I looked the source code and didn’t find any setting related to numIteration also tried to set "spark.executor.memory": "24g" in sparkConf as well but apparently didn’t worked neither
Could you please advise
Regards
Hossein
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
From: Hossein Bahrami<ma...@live.com>
Sent: Thursday, December 15, 2016 11:07 AM
To: Pat Ferrel<ma...@occamsmachete.com>; user@predictionio.incubator.apache.org<ma...@predictionio.incubator.apache.org>
Cc: Magnus Kragelund<ma...@ida.dk>
Subject: RE: Customizing Recommender engine
Hi Pat,
I downloaded and installed the PredictionIo 9 aml, and pio status returns Ok messages .
[cid:image003.png@01D256C3.2AE41D90]
And I downloaded and built mahout locally with success
Also I downloaded the UR which you shared the link with me (below one), imported data (after pio app new , config the engine.json)
git clone https://github.com/actionml/universal-recommender.git ur
seems everything is fine because the pio build was successful but pio train returns this :
[INFO] [Console$] Using existing engine manifest JSON at /root/workspace/hossein.bahrami/ur2/manifest.json
[INFO] [Runner$] Submission command: /home/PredictionIOAML/vendors/spark-1.6.2/bin/spark-submit --class io.prediction.workflow.CreateWorkflow --jars file:/root/workspac e/hossein.bahrami/ur2/target/scala-2.10/template-scala-parallel-universal-recommendation-assembly-0.5.0-deps.jar,file:/root/workspace/hossein.bahrami/ur2/target/scala-2 .10/template-scala-parallel-universal-recommendation_2.10-0.5.0.jar --files file:/home/PredictionIOAML/conf/log4j.properties,file:/home/PredictionIOAML/vendors/hbase-1. 2.2/conf/hbase-site.xml --driver-class-path /home/PredictionIOAML/conf:/home/PredictionIOAML/lib/postgresql-9.4-1204.jdbc41.jar:/home/PredictionIOAML/lib/mysql-connecto r-java-5.1.37.jar:/home/PredictionIOAML/vendors/hbase-1.2.2/conf file:/home/PredictionIOAML/lib/pio-assembly-0.9.7-aml.jar --engine-id FmvCSLgfRoHLUG1U0PWGLmBbWoYu7g7B --engine-version ae0b6554cfd198c3fb8c4f1b4940e53536cb001c --engine-variant file:/root/workspace/hossein.bahrami/ur2/engine.json --verbosity 0 --json-extractor Both --en v PIO_STORAGE_SOURCES_HBASE_TYPE=hbase,PIO_ENV_LOADED=1,PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta,PIO_FS_BASEDIR=/root/.pio_store,PIO_STORAGE_SOURCES_HBASE_HOME=/ home/PredictionIOAML/vendors/hbase-1.2.2,PIO_HOME=/home/PredictionIOAML,PIO_FS_ENGINESDIR=/root/.pio_store/engines,PIO_STORAGE_SOURCES_LOCALFS_PATH=/root/.pio_store/mod els,PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS,PIO_ST ORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/home/PredictionIOAML/vendors/elasticsearch-1.7.5,PIO_FS_TMPDIR=/root/.pio_store/tmp, PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE,PIO_CONF_DIR=/home/PredictionIOAML/conf,PIO_STORAGE_SOURCES_LOCALFS_TY PE=localfs
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/predictionio/controller/EngineFactory
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at scala.reflect.runtime.ReflectionUtils$.staticSingletonInstance(ReflectionUtils.scala:60)
at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaModuleMirror.instance(JavaMirrors.scala:473)
at io.prediction.workflow.WorkflowUtils$.getEngine(WorkflowUtils.scala:69)
at io.prediction.workflow.CreateWorkflow$.liftedTree1$1(CreateWorkflow.scala:193)
at io.prediction.workflow.CreateWorkflow$.main(CreateWorkflow.scala:192)
at io.prediction.workflow.CreateWorkflow.main(CreateWorkflow.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.predictionio.controller.EngineFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
Regards
Hossein
From: Pat Ferrel<ma...@occamsmachete.com>
Sent: Thursday, December 15, 2016 3:33 AM
To: user@predictionio.incubator.apache.org<ma...@predictionio.incubator.apache.org>
Cc: Magnus Kragelund<ma...@ida.dk>
Subject: Re: Customizing Recommender engine
Yes, sorry but we are in the middle of a major site update now. The version number and repos are the ones I pointed to below.
The UR does require a local Mahout build in v0.5.0. I’ll describe more how to do this on the page you reference, should be up by mid day US pacific time tomorrow. The remark about getting an older version of the UR will not give you something that works with Apache PIO, the older versions are for pre-Apache PIO. I’d recommend that you stick with Apache PIO and use the UR v0.5.0 by downloading and building Mahout.
In any case the fact remains that you cannot use UR queries or data formulation with the other recommenders and they all have different feature sets.
On Dec 14, 2016, at 1:18 AM, Daniel O' Shaughnessy <da...@gmail.com>> wrote:
Hi Hossein,
If you check out http://actionml.com/docs/ur_config you get these instructions:
UR Parameters V0.4.0
Notice: This version requires you to build Mahout v0.13.0-SNAPSHOT from source. If you do not need v0.4.0 features you can stick with v0.3.0 by pulling that tag from the UR repo. See git instructions to pull and checkout a specific tag. To use v0.4.0 You will need to pull the latest master of Mahout v0.13.0 from the Apache Mahout repo<https://github.com/apache/mahout.git> and build it locally on the machine you expect to build the Universal Recommender. We will update the UR as soon as Mahout v0.13.0 is released to avoid this extra build. Also notice that this version is not compatible with Apache PredcitionIO v0.10.0 since it has not been released yet. Continue to use the ActionML version of PredictionIO-0.9.7-aml.
Alternatively you can avoid a mahout build by checking out a previous version of the UR.
On Wed, 14 Dec 2016 at 08:37 Hossein Bahrami <h....@live.com>> wrote:
Hi Pat,
Thanks for your reply, the reason I switched back to the Recommendation Engine is getting error during build.
This is my pio version : 0.10.0-incubating
And I’m getting below error during pio build of the UR template you shared, and also you can find the engine.json as well
##########################################################################
{
"comment":" This config file uses default settings for all but the required values see README.md for docs",
"id": "default",
"description": "Default settings",
"engineFactory": "org.template.RecommendationEngine",
"datasource": {
"params" : {
"name": "sample-handmade-data.txt",
"appName": "ur2",
"eventNames": ["rate", "$set"]
}
},
"sparkConf": {
"spark.serializer": "org.apache.spark.serializer.KryoSerializer",
"spark.kryo.registrator": "org.apache.mahout.sparkbindings.io.MahoutKryoRegistrator",
"spark.kryo.referenceTracking": "false",
"spark.kryoserializer.buffer": "300m",
"es.index.auto.create": "true"
},
"algorithms": [
{
"comment": "simplest setup where all values are default, popularity based backfill, must add eventsNames",
"name": "ur",
"params": {
"appName": "ur2",
"indexName": "urindex",
"typeName": "items",
"comment": "must have data for the first event or the model will not build, other events are optional",
"indicators": [
{
"name": "rate"
},{
"name": "$set",
"maxCorrelatorsPerItem": 50
}
],
"availableDateName": "available",
"expireDateName": "expires",
"dateName": "date",
"num": 4
}
}
]
}
###############################################################################
ERRORS
[INFO] [Console$] [warn] ::::::::::::::::::::::::::::::::::::::::::::::
[INFO] [Console$] [warn] :: UNRESOLVED DEPENDENCIES ::
[INFO] [Console$] [warn] ::::::::::::::::::::::::::::::::::::::::::::::
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] ::::::::::::::::::::::::::::::::::::::::::::::
[INFO] [Console$] [warn]
[INFO] [Console$] [warn] Note: Unresolved dependencies path:
[INFO] [Console$] [warn] org.apache.mahout:mahout-math-scala_2.10:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] [warn] org.apache.mahout:mahout-spark_2.10:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] [warn] org.apache.mahout:mahout-math:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] [warn] org.apache.mahout:mahout-hdfs:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] sbt.ResolveException: unresolved dependency: org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] unresolved dependency: org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] unresolved dependency: org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
[INFO] [Console$] unresolved dependency: org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
[INFO] [Console$] at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:278)
[INFO] [Console$] at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:175)
[INFO] [Console$] at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:157)
[INFO] [Console$] at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:151)
[INFO] [Console$] at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:151)
[INFO] [Console$] at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:128)
[INFO] [Console$] at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:56)
[INFO] [Console$] at sbt.IvySbt$$anon$4.call(Ivy.scala:64)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97)
[INFO] [Console$] at xsbt.boot.Using$.withResource(Using.scala:10)
[INFO] [Console$] at xsbt.boot.Using$.apply(Using.scala:9)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48)
[INFO] [Console$] at xsbt.boot.Locks$.apply0(Locks.scala:31)
[INFO] [Console$] at xsbt.boot.Locks$.apply(Locks.scala:28)
[INFO] [Console$] at sbt.IvySbt.withDefaultLogger(Ivy.scala:64)
[INFO] [Console$] at sbt.IvySbt.withIvy(Ivy.scala:123)
[INFO] [Console$] at sbt.IvySbt.withIvy(Ivy.scala:120)
[INFO] [Console$] at sbt.IvySbt$Module.withModule(Ivy.scala:151)
[INFO] [Console$] at sbt.IvyActions$.updateEither(IvyActions.scala:157)
[INFO] [Console$] at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1318)
[INFO] [Console$] at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1315)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$85.apply(Defaults.scala:1345)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$85.apply(Defaults.scala:1343)
[INFO] [Console$] at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:35)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1348)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1342)
[INFO] [Console$] at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:45)
[INFO] [Console$] at sbt.Classpaths$.cachedUpdate(Defaults.scala:1360)
[INFO] [Console$] at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1300)
[INFO] [Console$] at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1275)
[INFO] [Console$] at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
[INFO] [Console$] at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
[INFO] [Console$] at sbt.std.Transform$$anon$4.work(System.scala:63)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
[INFO] [Console$] at sbt.Execute.work(Execute.scala:235)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
[INFO] [Console$] at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
[INFO] [Console$] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[INFO] [Console$] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[INFO] [Console$] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[INFO] [Console$] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[INFO] [Console$] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[INFO] [Console$] at java.lang.Thread.run(Thread.java:745)
[INFO] [Console$] [error] (*:update) sbt.ResolveException: unresolved dependency: org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] unresolved dependency: org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] unresolved dependency: org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] unresolved dependency: org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] Total time: 4 s, completed Dec 14, 2016 9:16:15 AM
[ERROR] [Console$] Return code of previous step is 1. Aborting.
Regards
Hossein
………………………………
From: Pat Ferrel<ma...@occamsmachete.com>
Sent: Tuesday, December 13, 2016 10:08 PM
To: user@predictionio.incubator.apache.org<ma...@predictionio.incubator.apache.org>
Cc: Magnus Kragelund<ma...@ida.dk>
Subject: Re: Customizing Recommender engine
The UR has a new Apache PIO compatible repo here: https://github.com/actionml/universal-recommender.git
git clone https://github.com/actionml/universal-recommender.git ur
and proceed. The UR allows boosts or filters by properties. You are using a filter (bias -1), which does not work with the template you are using. Each template defines its own queries, config (engine.json), and input formats. There are a few common ideas but each one will have quite a few differences also and very different features.
On Dec 13, 2016, at 6:20 AM, Hossein Bahrami <h....@live.com>> wrote:
Hi, thanks for your reply. I don’t use Universal recommender, actually I use the Recommender Engine (http://predictionio.incubator.apache.org/templates/recommendation/quickstart/)
First I tried to use Universal recommender but it was difficulties building the project so I switched to the Recommendation Engine (RE), the RE works fine but don’t know how to customize it to take other events (items’ properties as I described ) into account.
Seems the RE returns very high score recommendations with my data so I guess it’s the choice here for me and just want to do some customization on it to make it predict recommendations per category (i.e property) of items.
Regards
Hossein
From: Magnus Kragelund<ma...@ida.dk>
Sent: Tuesday, December 13, 2016 12:17 PM
To: user@predictionio.incubator.apache.org<ma...@predictionio.incubator.apache.org>
Subject: Re: Customizing Recommender engine
Hi,
Assuming that you are using the Universal Recommender Engine, you should have a look at the "Queries" section here: https://github.com/PredictionIO/template-scala-parallel-universal-recommendation#queries
Try this request instead, where the "fields" property is used to filter by category
{
"user": "674296",
"num": 10,
"fields": [
{
"name": "categories",
"values": ["CAT1", "CAT2"],
"bias": -1
}
]
}
/magnus
________________________________
From: Hossein Bahrami <h....@live.com>>
Sent: Tuesday, December 13, 2016 10:55:00 AM
To: user@predictionio.incubator.apache.org<ma...@predictionio.incubator.apache.org>
Subject: Customizing Recommender engine
Dear all,
I’m new to Predictionio, currently I’m using it and I managed to import (rate, buy) events and getting pretty good results querying it. But now I want to limit the results for items in specific categories. I’ve created events for items’ properties (categories) as well .
I am posting this query to engine but it seems doesn’t care about the categories and returns same result every time.
{ "user": "674296", "num": 10, "categories" : ["CAT2", "CAT1"] }
I’ve imported bellow events
client.create_event(event="rate",
entity_type="user",
entity_id=int(row['userid']),
target_entity_type="item",
target_entity_id=int(row['itemid']),
properties= { "rating" : float(row['rating']) });
client.create_event(
event="buy",
entity_type="user",
entity_id=int(row['userid']),
target_entity_type="item",
target_entity_id=int(row['itemid']), );
client.create_event(
event="$set",
entity_type="item",
entity_id=itemid,
properties= { "categories": itemcats }
);
Could someone give me a solution or hint to how customize this recommender engine to take the categories into account.
Thanks in advance
Hossein
RE: Customizing Recommender engine
Posted by Hossein Bahrami <h....@live.com>.
Hi Pat,
I downloaded and installed the PredictionIo 9 aml, and pio status returns Ok messages .
[cid:image003.png@01D256C3.2AE41D90]
And I downloaded and built mahout locally with success
Also I downloaded the UR which you shared the link with me (below one), imported data (after pio app new , config the engine.json)
git clone https://github.com/actionml/universal-recommender.git ur
seems everything is fine because the pio build was successful but pio train returns this :
[INFO] [Console$] Using existing engine manifest JSON at /root/workspace/hossein.bahrami/ur2/manifest.json
[INFO] [Runner$] Submission command: /home/PredictionIOAML/vendors/spark-1.6.2/bin/spark-submit --class io.prediction.workflow.CreateWorkflow --jars file:/root/workspac e/hossein.bahrami/ur2/target/scala-2.10/template-scala-parallel-universal-recommendation-assembly-0.5.0-deps.jar,file:/root/workspace/hossein.bahrami/ur2/target/scala-2 .10/template-scala-parallel-universal-recommendation_2.10-0.5.0.jar --files file:/home/PredictionIOAML/conf/log4j.properties,file:/home/PredictionIOAML/vendors/hbase-1. 2.2/conf/hbase-site.xml --driver-class-path /home/PredictionIOAML/conf:/home/PredictionIOAML/lib/postgresql-9.4-1204.jdbc41.jar:/home/PredictionIOAML/lib/mysql-connecto r-java-5.1.37.jar:/home/PredictionIOAML/vendors/hbase-1.2.2/conf file:/home/PredictionIOAML/lib/pio-assembly-0.9.7-aml.jar --engine-id FmvCSLgfRoHLUG1U0PWGLmBbWoYu7g7B --engine-version ae0b6554cfd198c3fb8c4f1b4940e53536cb001c --engine-variant file:/root/workspace/hossein.bahrami/ur2/engine.json --verbosity 0 --json-extractor Both --en v PIO_STORAGE_SOURCES_HBASE_TYPE=hbase,PIO_ENV_LOADED=1,PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta,PIO_FS_BASEDIR=/root/.pio_store,PIO_STORAGE_SOURCES_HBASE_HOME=/ home/PredictionIOAML/vendors/hbase-1.2.2,PIO_HOME=/home/PredictionIOAML,PIO_FS_ENGINESDIR=/root/.pio_store/engines,PIO_STORAGE_SOURCES_LOCALFS_PATH=/root/.pio_store/mod els,PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS,PIO_ST ORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/home/PredictionIOAML/vendors/elasticsearch-1.7.5,PIO_FS_TMPDIR=/root/.pio_store/tmp, PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE,PIO_CONF_DIR=/home/PredictionIOAML/conf,PIO_STORAGE_SOURCES_LOCALFS_TY PE=localfs
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/predictionio/controller/EngineFactory
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at scala.reflect.runtime.ReflectionUtils$.staticSingletonInstance(ReflectionUtils.scala:60)
at scala.reflect.runtime.JavaMirrors$JavaMirror$JavaModuleMirror.instance(JavaMirrors.scala:473)
at io.prediction.workflow.WorkflowUtils$.getEngine(WorkflowUtils.scala:69)
at io.prediction.workflow.CreateWorkflow$.liftedTree1$1(CreateWorkflow.scala:193)
at io.prediction.workflow.CreateWorkflow$.main(CreateWorkflow.scala:192)
at io.prediction.workflow.CreateWorkflow.main(CreateWorkflow.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.predictionio.controller.EngineFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
Regards
Hossein
From: Pat Ferrel<ma...@occamsmachete.com>
Sent: Thursday, December 15, 2016 3:33 AM
To: user@predictionio.incubator.apache.org<ma...@predictionio.incubator.apache.org>
Cc: Magnus Kragelund<ma...@ida.dk>
Subject: Re: Customizing Recommender engine
Yes, sorry but we are in the middle of a major site update now. The version number and repos are the ones I pointed to below.
The UR does require a local Mahout build in v0.5.0. I’ll describe more how to do this on the page you reference, should be up by mid day US pacific time tomorrow. The remark about getting an older version of the UR will not give you something that works with Apache PIO, the older versions are for pre-Apache PIO. I’d recommend that you stick with Apache PIO and use the UR v0.5.0 by downloading and building Mahout.
In any case the fact remains that you cannot use UR queries or data formulation with the other recommenders and they all have different feature sets.
On Dec 14, 2016, at 1:18 AM, Daniel O' Shaughnessy <da...@gmail.com>> wrote:
Hi Hossein,
If you check out http://actionml.com/docs/ur_config you get these instructions:
UR Parameters V0.4.0
Notice: This version requires you to build Mahout v0.13.0-SNAPSHOT from source. If you do not need v0.4.0 features you can stick with v0.3.0 by pulling that tag from the UR repo. See git instructions to pull and checkout a specific tag. To use v0.4.0 You will need to pull the latest master of Mahout v0.13.0 from the Apache Mahout repo<https://github.com/apache/mahout.git> and build it locally on the machine you expect to build the Universal Recommender. We will update the UR as soon as Mahout v0.13.0 is released to avoid this extra build. Also notice that this version is not compatible with Apache PredcitionIO v0.10.0 since it has not been released yet. Continue to use the ActionML version of PredictionIO-0.9.7-aml.
Alternatively you can avoid a mahout build by checking out a previous version of the UR.
On Wed, 14 Dec 2016 at 08:37 Hossein Bahrami <h....@live.com>> wrote:
Hi Pat,
Thanks for your reply, the reason I switched back to the Recommendation Engine is getting error during build.
This is my pio version : 0.10.0-incubating
And I’m getting below error during pio build of the UR template you shared, and also you can find the engine.json as well
##########################################################################
{
"comment":" This config file uses default settings for all but the required values see README.md for docs",
"id": "default",
"description": "Default settings",
"engineFactory": "org.template.RecommendationEngine",
"datasource": {
"params" : {
"name": "sample-handmade-data.txt",
"appName": "ur2",
"eventNames": ["rate", "$set"]
}
},
"sparkConf": {
"spark.serializer": "org.apache.spark.serializer.KryoSerializer",
"spark.kryo.registrator": "org.apache.mahout.sparkbindings.io.MahoutKryoRegistrator",
"spark.kryo.referenceTracking": "false",
"spark.kryoserializer.buffer": "300m",
"es.index.auto.create": "true"
},
"algorithms": [
{
"comment": "simplest setup where all values are default, popularity based backfill, must add eventsNames",
"name": "ur",
"params": {
"appName": "ur2",
"indexName": "urindex",
"typeName": "items",
"comment": "must have data for the first event or the model will not build, other events are optional",
"indicators": [
{
"name": "rate"
},{
"name": "$set",
"maxCorrelatorsPerItem": 50
}
],
"availableDateName": "available",
"expireDateName": "expires",
"dateName": "date",
"num": 4
}
}
]
}
###############################################################################
ERRORS
[INFO] [Console$] [warn] ::::::::::::::::::::::::::::::::::::::::::::::
[INFO] [Console$] [warn] :: UNRESOLVED DEPENDENCIES ::
[INFO] [Console$] [warn] ::::::::::::::::::::::::::::::::::::::::::::::
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] ::::::::::::::::::::::::::::::::::::::::::::::
[INFO] [Console$] [warn]
[INFO] [Console$] [warn] Note: Unresolved dependencies path:
[INFO] [Console$] [warn] org.apache.mahout:mahout-math-scala_2.10:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] [warn] org.apache.mahout:mahout-spark_2.10:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] [warn] org.apache.mahout:mahout-math:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] [warn] org.apache.mahout:mahout-hdfs:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] sbt.ResolveException: unresolved dependency: org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] unresolved dependency: org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] unresolved dependency: org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
[INFO] [Console$] unresolved dependency: org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
[INFO] [Console$] at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:278)
[INFO] [Console$] at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:175)
[INFO] [Console$] at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:157)
[INFO] [Console$] at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:151)
[INFO] [Console$] at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:151)
[INFO] [Console$] at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:128)
[INFO] [Console$] at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:56)
[INFO] [Console$] at sbt.IvySbt$$anon$4.call(Ivy.scala:64)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97)
[INFO] [Console$] at xsbt.boot.Using$.withResource(Using.scala:10)
[INFO] [Console$] at xsbt.boot.Using$.apply(Using.scala:9)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48)
[INFO] [Console$] at xsbt.boot.Locks$.apply0(Locks.scala:31)
[INFO] [Console$] at xsbt.boot.Locks$.apply(Locks.scala:28)
[INFO] [Console$] at sbt.IvySbt.withDefaultLogger(Ivy.scala:64)
[INFO] [Console$] at sbt.IvySbt.withIvy(Ivy.scala:123)
[INFO] [Console$] at sbt.IvySbt.withIvy(Ivy.scala:120)
[INFO] [Console$] at sbt.IvySbt$Module.withModule(Ivy.scala:151)
[INFO] [Console$] at sbt.IvyActions$.updateEither(IvyActions.scala:157)
[INFO] [Console$] at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1318)
[INFO] [Console$] at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1315)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$85.apply(Defaults.scala:1345)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$85.apply(Defaults.scala:1343)
[INFO] [Console$] at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:35)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1348)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1342)
[INFO] [Console$] at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:45)
[INFO] [Console$] at sbt.Classpaths$.cachedUpdate(Defaults.scala:1360)
[INFO] [Console$] at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1300)
[INFO] [Console$] at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1275)
[INFO] [Console$] at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
[INFO] [Console$] at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
[INFO] [Console$] at sbt.std.Transform$$anon$4.work(System.scala:63)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
[INFO] [Console$] at sbt.Execute.work(Execute.scala:235)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
[INFO] [Console$] at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
[INFO] [Console$] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[INFO] [Console$] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[INFO] [Console$] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[INFO] [Console$] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[INFO] [Console$] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[INFO] [Console$] at java.lang.Thread.run(Thread.java:745)
[INFO] [Console$] [error] (*:update) sbt.ResolveException: unresolved dependency: org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] unresolved dependency: org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] unresolved dependency: org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] unresolved dependency: org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] Total time: 4 s, completed Dec 14, 2016 9:16:15 AM
[ERROR] [Console$] Return code of previous step is 1. Aborting.
Regards
Hossein
………………………………
From: Pat Ferrel<ma...@occamsmachete.com>
Sent: Tuesday, December 13, 2016 10:08 PM
To: user@predictionio.incubator.apache.org<ma...@predictionio.incubator.apache.org>
Cc: Magnus Kragelund<ma...@ida.dk>
Subject: Re: Customizing Recommender engine
The UR has a new Apache PIO compatible repo here: https://github.com/actionml/universal-recommender.git
git clone https://github.com/actionml/universal-recommender.git ur
and proceed. The UR allows boosts or filters by properties. You are using a filter (bias -1), which does not work with the template you are using. Each template defines its own queries, config (engine.json), and input formats. There are a few common ideas but each one will have quite a few differences also and very different features.
On Dec 13, 2016, at 6:20 AM, Hossein Bahrami <h....@live.com>> wrote:
Hi, thanks for your reply. I don’t use Universal recommender, actually I use the Recommender Engine (http://predictionio.incubator.apache.org/templates/recommendation/quickstart/)
First I tried to use Universal recommender but it was difficulties building the project so I switched to the Recommendation Engine (RE), the RE works fine but don’t know how to customize it to take other events (items’ properties as I described ) into account.
Seems the RE returns very high score recommendations with my data so I guess it’s the choice here for me and just want to do some customization on it to make it predict recommendations per category (i.e property) of items.
Regards
Hossein
From: Magnus Kragelund<ma...@ida.dk>
Sent: Tuesday, December 13, 2016 12:17 PM
To: user@predictionio.incubator.apache.org<ma...@predictionio.incubator.apache.org>
Subject: Re: Customizing Recommender engine
Hi,
Assuming that you are using the Universal Recommender Engine, you should have a look at the "Queries" section here: https://github.com/PredictionIO/template-scala-parallel-universal-recommendation#queries
Try this request instead, where the "fields" property is used to filter by category
{
"user": "674296",
"num": 10,
"fields": [
{
"name": "categories",
"values": ["CAT1", "CAT2"],
"bias": -1
}
]
}
/magnus
________________________________
From: Hossein Bahrami <h....@live.com>>
Sent: Tuesday, December 13, 2016 10:55:00 AM
To: user@predictionio.incubator.apache.org<ma...@predictionio.incubator.apache.org>
Subject: Customizing Recommender engine
Dear all,
I’m new to Predictionio, currently I’m using it and I managed to import (rate, buy) events and getting pretty good results querying it. But now I want to limit the results for items in specific categories. I’ve created events for items’ properties (categories) as well .
I am posting this query to engine but it seems doesn’t care about the categories and returns same result every time.
{ "user": "674296", "num": 10, "categories" : ["CAT2", "CAT1"] }
I’ve imported bellow events
client.create_event(event="rate",
entity_type="user",
entity_id=int(row['userid']),
target_entity_type="item",
target_entity_id=int(row['itemid']),
properties= { "rating" : float(row['rating']) });
client.create_event(
event="buy",
entity_type="user",
entity_id=int(row['userid']),
target_entity_type="item",
target_entity_id=int(row['itemid']), );
client.create_event(
event="$set",
entity_type="item",
entity_id=itemid,
properties= { "categories": itemcats }
);
Could someone give me a solution or hint to how customize this recommender engine to take the categories into account.
Thanks in advance
Hossein
Re: Customizing Recommender engine
Posted by Pat Ferrel <pa...@occamsmachete.com>.
Yes, sorry but we are in the middle of a major site update now. The version number and repos are the ones I pointed to below.
The UR does require a local Mahout build in v0.5.0. I’ll describe more how to do this on the page you reference, should be up by mid day US pacific time tomorrow. The remark about getting an older version of the UR will not give you something that works with Apache PIO, the older versions are for pre-Apache PIO. I’d recommend that you stick with Apache PIO and use the UR v0.5.0 by downloading and building Mahout.
In any case the fact remains that you cannot use UR queries or data formulation with the other recommenders and they all have different feature sets.
On Dec 14, 2016, at 1:18 AM, Daniel O' Shaughnessy <da...@gmail.com> wrote:
Hi Hossein,
If you check out http://actionml.com/docs/ur_config <http://actionml.com/docs/ur_config> you get these instructions:
UR Parameters V0.4.0
Notice: This version requires you to build Mahout v0.13.0-SNAPSHOT from source. If you do not need v0.4.0 features you can stick with v0.3.0 by pulling that tag from the UR repo. See git instructions to pull and checkout a specific tag. To use v0.4.0 You will need to pull the latest master of Mahout v0.13.0 from the Apache Mahout repo <https://github.com/apache/mahout.git> and build it locally on the machine you expect to build the Universal Recommender. We will update the UR as soon as Mahout v0.13.0 is released to avoid this extra build. Also notice that this version is not compatible with Apache PredcitionIO v0.10.0 since it has not been released yet. Continue to use the ActionML version of PredictionIO-0.9.7-aml.
Alternatively you can avoid a mahout build by checking out a previous version of the UR.
On Wed, 14 Dec 2016 at 08:37 Hossein Bahrami <h.bahrami@live.com <ma...@live.com>> wrote:
Hi Pat,
Thanks for your reply, the reason I switched back to the Recommendation Engine is getting error during build.
This is my pio version : 0.10.0-incubating
And I’m getting below error during pio build of the UR template you shared, and also you can find the engine.json as well
##########################################################################
{
"comment":" This config file uses default settings for all but the required values see README.md for docs",
"id": "default",
"description": "Default settings",
"engineFactory": "org.template.RecommendationEngine",
"datasource": {
"params" : {
"name": "sample-handmade-data.txt",
"appName": "ur2",
"eventNames": ["rate", "$set"]
}
},
"sparkConf": {
"spark.serializer": "org.apache.spark.serializer.KryoSerializer",
"spark.kryo.registrator": "org.apache.mahout.sparkbindings.io.MahoutKryoRegistrator",
"spark.kryo.referenceTracking": "false",
"spark.kryoserializer.buffer": "300m",
"es.index.auto.create": "true"
},
"algorithms": [
{
"comment": "simplest setup where all values are default, popularity based backfill, must add eventsNames",
"name": "ur",
"params": {
"appName": "ur2",
"indexName": "urindex",
"typeName": "items",
"comment": "must have data for the first event or the model will not build, other events are optional",
"indicators": [
{
"name": "rate"
},{
"name": "$set",
"maxCorrelatorsPerItem": 50
}
],
"availableDateName": "available",
"expireDateName": "expires",
"dateName": "date",
"num": 4
}
}
]
}
###############################################################################
ERRORS
[INFO] [Console$] [warn] ::::::::::::::::::::::::::::::::::::::::::::::
[INFO] [Console$] [warn] :: UNRESOLVED DEPENDENCIES ::
[INFO] [Console$] [warn] ::::::::::::::::::::::::::::::::::::::::::::::
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] ::::::::::::::::::::::::::::::::::::::::::::::
[INFO] [Console$] [warn]
[INFO] [Console$] [warn] Note: Unresolved dependencies path:
[INFO] [Console$] [warn] org.apache.mahout:mahout-math-scala_2.10:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] [warn] org.apache.mahout:mahout-spark_2.10:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] [warn] org.apache.mahout:mahout-math:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] [warn] org.apache.mahout:mahout-hdfs:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] sbt.ResolveException: unresolved dependency: org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] unresolved dependency: org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] unresolved dependency: org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
[INFO] [Console$] unresolved dependency: org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
[INFO] [Console$] at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:278)
[INFO] [Console$] at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:175)
[INFO] [Console$] at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:157)
[INFO] [Console$] at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:151)
[INFO] [Console$] at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:151)
[INFO] [Console$] at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:128)
[INFO] [Console$] at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:56)
[INFO] [Console$] at sbt.IvySbt$$anon$4.call(Ivy.scala:64)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97)
[INFO] [Console$] at xsbt.boot.Using$.withResource(Using.scala:10)
[INFO] [Console$] at xsbt.boot.Using$.apply(Using.scala:9)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48)
[INFO] [Console$] at xsbt.boot.Locks$.apply0(Locks.scala:31)
[INFO] [Console$] at xsbt.boot.Locks$.apply(Locks.scala:28)
[INFO] [Console$] at sbt.IvySbt.withDefaultLogger(Ivy.scala:64)
[INFO] [Console$] at sbt.IvySbt.withIvy(Ivy.scala:123)
[INFO] [Console$] at sbt.IvySbt.withIvy(Ivy.scala:120)
[INFO] [Console$] at sbt.IvySbt$Module.withModule(Ivy.scala:151)
[INFO] [Console$] at sbt.IvyActions$.updateEither(IvyActions.scala:157)
[INFO] [Console$] at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1318)
[INFO] [Console$] at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1315)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$85.apply(Defaults.scala:1345)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$85.apply(Defaults.scala:1343)
[INFO] [Console$] at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:35)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1348)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1342)
[INFO] [Console$] at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:45)
[INFO] [Console$] at sbt.Classpaths$.cachedUpdate(Defaults.scala:1360)
[INFO] [Console$] at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1300)
[INFO] [Console$] at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1275)
[INFO] [Console$] at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
[INFO] [Console$] at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
[INFO] [Console$] at sbt.std.Transform$$anon$4.work(System.scala:63)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
[INFO] [Console$] at sbt.Execute.work(Execute.scala:235)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
[INFO] [Console$] at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
[INFO] [Console$] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[INFO] [Console$] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[INFO] [Console$] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[INFO] [Console$] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[INFO] [Console$] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[INFO] [Console$] at java.lang.Thread.run(Thread.java:745)
[INFO] [Console$] [error] (*:update) sbt.ResolveException: unresolved dependency: org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] unresolved dependency: org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] unresolved dependency: org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] unresolved dependency: org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] Total time: 4 s, completed Dec 14, 2016 9:16:15 AM
[ERROR] [Console$] Return code of previous step is 1. Aborting.
Regards
Hossein
………………………………
From: Pat Ferrel <ma...@occamsmachete.com>
Sent: Tuesday, December 13, 2016 10:08 PM
To: user@predictionio.incubator.apache.org <ma...@predictionio.incubator.apache.org>
Cc: Magnus Kragelund <ma...@ida.dk>
Subject: Re: Customizing Recommender engine
The UR has a new Apache PIO compatible repo here: https://github.com/actionml/universal-recommender.git <https://github.com/actionml/universal-recommender.git>
git clone https://github.com/actionml/universal-recommender.git <https://github.com/actionml/universal-recommender.git> ur
and proceed. The UR allows boosts or filters by properties. You are using a filter (bias -1), which does not work with the template you are using. Each template defines its own queries, config (engine.json), and input formats. There are a few common ideas but each one will have quite a few differences also and very different features.
On Dec 13, 2016, at 6:20 AM, Hossein Bahrami <h.bahrami@live.com <ma...@live.com>> wrote:
Hi, thanks for your reply. I don’t use Universal recommender, actually I use the Recommender Engine (http://predictionio.incubator.apache.org/templates/recommendation/quickstart/ <http://predictionio.incubator.apache.org/templates/recommendation/quickstart/>)
First I tried to use Universal recommender but it was difficulties building the project so I switched to the Recommendation Engine (RE), the RE works fine but don’t know how to customize it to take other events (items’ properties as I described ) into account.
Seems the RE returns very high score recommendations with my data so I guess it’s the choice here for me and just want to do some customization on it to make it predict recommendations per category (i.e property) of items.
Regards
Hossein
From: Magnus Kragelund <ma...@ida.dk>
Sent: Tuesday, December 13, 2016 12:17 PM
To: user@predictionio.incubator.apache.org <ma...@predictionio.incubator.apache.org>
Subject: Re: Customizing Recommender engine
Hi,
Assuming that you are using the Universal Recommender Engine, you should have a look at the "Queries" section here: https://github.com/PredictionIO/template-scala-parallel-universal-recommendation#queries <https://github.com/PredictionIO/template-scala-parallel-universal-recommendation#queries>
Try this request instead, where the "fields" property is used to filter by category
{
"user": "674296",
"num": 10,
"fields": [
{
"name": "categories",
"values": ["CAT1", "CAT2"],
"bias": -1
}
]
}
/magnus
From: Hossein Bahrami <h.bahrami@live.com <ma...@live.com>>
Sent: Tuesday, December 13, 2016 10:55:00 AM
To: user@predictionio.incubator.apache.org <ma...@predictionio.incubator.apache.org>
Subject: Customizing Recommender engine
Dear all,
I’m new to Predictionio, currently I’m using it and I managed to import (rate, buy) events and getting pretty good results querying it. But now I want to limit the results for items in specific categories. I’ve created events for items’ properties (categories) as well .
I am posting this query to engine but it seems doesn’t care about the categories and returns same result every time.
{ "user": "674296", "num": 10, "categories" : ["CAT2", "CAT1"] }
I’ve imported bellow events
client.create_event(event="rate",
entity_type="user",
entity_id=int(row['userid']),
target_entity_type="item",
target_entity_id=int(row['itemid']),
properties= { "rating" : float(row['rating']) });
client.create_event(
event="buy",
entity_type="user",
entity_id=int(row['userid']),
target_entity_type="item",
target_entity_id=int(row['itemid']), );
client.create_event(
event="$set",
entity_type="item",
entity_id=itemid,
properties= { "categories": itemcats }
);
Could someone give me a solution or hint to how customize this recommender engine to take the categories into account.
Thanks in advance
Hossein
Re: Customizing Recommender engine
Posted by Daniel O' Shaughnessy <da...@gmail.com>.
Hi Hossein,
If you check out http://actionml.com/docs/ur_config you get these
instructions:
UR Parameters V0.4.0
Notice: This version requires you to build Mahout v0.13.0-SNAPSHOT from
source. If you do not need v0.4.0 features you can stick with v0.3.0 by
pulling that tag from the UR repo. See git instructions to pull and
checkout a specific tag. To use v0.4.0 You will need to pull the latest
master of Mahout v0.13.0 from the Apache Mahout repo
<https://github.com/apache/mahout.git> and build it locally on the machine
you expect to build the Universal Recommender. We will update the UR as
soon as Mahout v0.13.0 is released to avoid this extra build. Also notice
that this version is not compatible with Apache PredcitionIO v0.10.0 since
it has not been released yet. Continue to use the ActionML version of
PredictionIO-0.9.7-aml.
Alternatively you can avoid a mahout build by checking out a previous
version of the UR.
On Wed, 14 Dec 2016 at 08:37 Hossein Bahrami <h....@live.com> wrote:
> Hi Pat,
>
> Thanks for your reply, the reason I switched back to the Recommendation
> Engine is getting error during build.
>
>
>
> This is my pio version : 0.10.0-incubating
>
>
>
> And I’m getting below error during pio build of the UR template you
> shared, and also you can find the engine.json as well
>
> ##########################################################################
>
> {
>
> "comment":" This config file uses default settings for all but the
> required values see README.md for docs",
>
> "id": "default",
>
> "description": "Default settings",
>
> "engineFactory": "org.template.RecommendationEngine",
>
> "datasource": {
>
> "params" : {
>
> "name": "sample-handmade-data.txt",
>
> "appName": "ur2",
>
> "eventNames": ["rate", "$set"]
>
> }
>
> },
>
> "sparkConf": {
>
> "spark.serializer": "org.apache.spark.serializer.KryoSerializer",
>
> "spark.kryo.registrator":
> "org.apache.mahout.sparkbindings.io.MahoutKryoRegistrator",
>
> "spark.kryo.referenceTracking": "false",
>
> "spark.kryoserializer.buffer": "300m",
>
> "es.index.auto.create": "true"
>
> },
>
> "algorithms": [
>
> {
>
> "comment": "simplest setup where all values are default, popularity
> based backfill, must add eventsNames",
>
> "name": "ur",
>
> "params": {
>
> "appName": "ur2",
>
> "indexName": "urindex",
>
> "typeName": "items",
>
> "comment": "must have data for the first event or the model will
> not build, other events are optional",
>
> "indicators": [
>
> {
>
> "name": "rate"
>
> },{
>
> "name": "$set",
>
> "maxCorrelatorsPerItem": 50
>
> }
>
> ],
>
> "availableDateName": "available",
>
> "expireDateName": "expires",
>
> "dateName": "date",
>
> "num": 4
>
> }
>
> }
>
> ]
>
> }
>
>
> ###############################################################################
>
> ERRORS
>
>
>
> [INFO] [Console$] [warn]
> ::::::::::::::::::::::::::::::::::::::::::::::
>
> [INFO] [Console$] [warn] :: UNRESOLVED
> DEPENDENCIES ::
>
> [INFO] [Console$] [warn]
> ::::::::::::::::::::::::::::::::::::::::::::::
>
> [INFO] [Console$] [warn] ::
> org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not found
>
> [INFO] [Console$] [warn] ::
> org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
>
> [INFO] [Console$] [warn] ::
> org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
>
> [INFO] [Console$] [warn] ::
> org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
>
> [INFO] [Console$] [warn]
> ::::::::::::::::::::::::::::::::::::::::::::::
>
> [INFO] [Console$] [warn]
>
> [INFO] [Console$] [warn] Note: Unresolved dependencies path:
>
> [INFO] [Console$] [warn]
> org.apache.mahout:mahout-math-scala_2.10:0.13.0-SNAPSHOT
> (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
>
> [INFO] [Console$] [warn] +-
> com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
>
> [INFO] [Console$] [warn]
> org.apache.mahout:mahout-spark_2.10:0.13.0-SNAPSHOT
> (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
>
> [INFO] [Console$] [warn] +-
> com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
>
> [INFO] [Console$] [warn]
> org.apache.mahout:mahout-math:0.13.0-SNAPSHOT
> (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
>
> [INFO] [Console$] [warn] +-
> com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
>
> [INFO] [Console$] [warn]
> org.apache.mahout:mahout-hdfs:0.13.0-SNAPSHOT
> (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
>
> [INFO] [Console$] [warn] +-
> com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
>
> [INFO] [Console$] sbt.ResolveException: unresolved dependency:
> org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not found
>
> [INFO] [Console$] unresolved dependency:
> org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
>
> [INFO] [Console$] unresolved dependency:
> org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
>
> [INFO] [Console$] unresolved dependency:
> org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
>
> [INFO] [Console$] at
> sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:278)
>
> [INFO] [Console$] at
> sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:175)
>
> [INFO] [Console$] at
> sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:157)
>
> [INFO] [Console$] at
> sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:151)
>
> [INFO] [Console$] at
> sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:151)
>
> [INFO] [Console$] at
> sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:128)
>
> [INFO] [Console$] at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:56)
>
> [INFO] [Console$] at sbt.IvySbt$$anon$4.call(Ivy.scala:64)
>
> [INFO] [Console$] at
> xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
>
> [INFO] [Console$] at
> xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78)
>
> [INFO] [Console$] at
> xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97)
>
> [INFO] [Console$] at xsbt.boot.Using$.withResource(Using.scala:10)
>
> [INFO] [Console$] at xsbt.boot.Using$.apply(Using.scala:9)
>
> [INFO] [Console$] at
> xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58)
>
> [INFO] [Console$] at
> xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48)
>
> [INFO] [Console$] at xsbt.boot.Locks$.apply0(Locks.scala:31)
>
> [INFO] [Console$] at xsbt.boot.Locks$.apply(Locks.scala:28)
>
> [INFO] [Console$] at sbt.IvySbt.withDefaultLogger(Ivy.scala:64)
>
> [INFO] [Console$] at sbt.IvySbt.withIvy(Ivy.scala:123)
>
> [INFO] [Console$] at sbt.IvySbt.withIvy(Ivy.scala:120)
>
> [INFO] [Console$] at sbt.IvySbt$Module.withModule(Ivy.scala:151)
>
> [INFO] [Console$] at
> sbt.IvyActions$.updateEither(IvyActions.scala:157)
>
> [INFO] [Console$] at
> sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1318)
>
> [INFO] [Console$] at
> sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1315)
>
> [INFO] [Console$] at
> sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$85.apply(Defaults.scala:1345)
>
> [INFO] [Console$] at
> sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$85.apply(Defaults.scala:1343)
>
> [INFO] [Console$] at
> sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:35)
>
> [INFO] [Console$] at
> sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1348)
>
> [INFO] [Console$] at
> sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1342)
>
> [INFO] [Console$] at
> sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:45)
>
> [INFO] [Console$] at
> sbt.Classpaths$.cachedUpdate(Defaults.scala:1360)
>
> [INFO] [Console$] at
> sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1300)
>
> [INFO] [Console$] at
> sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1275)
>
> [INFO] [Console$] at
> scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
>
> [INFO] [Console$] at
> sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
>
> [INFO] [Console$] at sbt.std.Transform$$anon$4.work(System.scala:63)
>
> [INFO] [Console$] at
> sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
>
> [INFO] [Console$] at
> sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
>
> [INFO] [Console$] at
> sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
>
> [INFO] [Console$] at sbt.Execute.work(Execute.scala:235)
>
> [INFO] [Console$] at
> sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
>
> [INFO] [Console$] at
> sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
>
> [INFO] [Console$] at
> sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
>
> [INFO] [Console$] at
> sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
>
> [INFO] [Console$] at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
> [INFO] [Console$] at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>
> [INFO] [Console$] at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
> [INFO] [Console$] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>
> [INFO] [Console$] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>
> [INFO] [Console$] at java.lang.Thread.run(Thread.java:745)
>
> [INFO] [Console$] [error] (*:update) sbt.ResolveException: unresolved
> dependency: org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not
> found
>
> [INFO] [Console$] [error] unresolved dependency:
> org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
>
> [INFO] [Console$] [error] unresolved dependency:
> org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
>
> [INFO] [Console$] [error] unresolved dependency:
> org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
>
> [INFO] [Console$] [error] Total time: 4 s, completed Dec 14, 2016 9:16:15
> AM
>
> [ERROR] [Console$] Return code of previous step is 1. Aborting.
>
>
>
> Regards
>
> Hossein
>
> ………………………………
>
>
>
> *From: *Pat Ferrel <pa...@occamsmachete.com>
> *Sent: *Tuesday, December 13, 2016 10:08 PM
> *To: *user@predictionio.incubator.apache.org
> *Cc: *Magnus Kragelund <ma...@ida.dk>
>
>
> *Subject: *Re: Customizing Recommender engine
> The UR has a new Apache PIO compatible repo here:
> https://github.com/actionml/universal-recommender.git
>
> git clone https://github.com/actionml/universal-recommender.git ur
>
> and proceed. The UR allows boosts or filters by properties. You are using
> a filter (bias -1), which does not work with the template you are using.
> Each template defines its own queries, config (engine.json), and input
> formats. There are a few common ideas but each one will have quite a few
> differences also and very different features.
>
>
>
> On Dec 13, 2016, at 6:20 AM, Hossein Bahrami <h....@live.com> wrote:
>
> Hi, thanks for your reply. I don’t use Universal recommender, actually I
> use the Recommender Engine (
> http://predictionio.incubator.apache.org/templates/recommendation/quickstart/
> )
>
> First I tried to use Universal recommender but it was difficulties
> building the project so I switched to the Recommendation Engine (RE), the
> RE works fine but don’t know how to customize it to take other events
> (items’ properties as I described ) into account.
>
> Seems the RE returns very high score recommendations with my data so I
> guess it’s the choice here for me and just want to do some customization on
> it to make it predict recommendations per category (i.e property) of items.
>
> Regards
> Hossein
>
> *From: *Magnus Kragelund <ma...@ida.dk>
> *Sent: *Tuesday, December 13, 2016 12:17 PM
> *To: *user@predictionio.incubator.apache.org
> *Subject: *Re: Customizing Recommender engine
>
> Hi,
> Assuming that you are using the Universal Recommender Engine, you should
> have a look at the "Queries" section here:
> https://github.com/PredictionIO/template-scala-parallel-universal-recommendation#queries
>
> Try this request instead, where the "fields" property is used to filter by
> category
>
> {
> "user": "674296",
> "num": 10,
> "fields": [
> {
> "name": "categories",
> "values": ["CAT1", "CAT2"],
> "bias": -1
> }
> ]
> }
>
> /magnus
>
> ------------------------------
> *From:* Hossein Bahrami <h....@live.com>
> *Sent:* Tuesday, December 13, 2016 10:55:00 AM
> *To:* user@predictionio.incubator.apache.org
> *Subject:* Customizing Recommender engine
>
> Dear all,
>
> I’m new to Predictionio, currently I’m using it and I managed to import
> (rate, buy) events and getting pretty good results querying it. But now I
> want to limit the results for items in specific categories. I’ve created
> events for items’ properties (categories) as well .
>
> I am posting this query to engine but it seems doesn’t care about the
> categories and returns same result every time.
>
> { "user": "674296", "num": 10, "categories" : ["CAT2", "CAT1"] }
>
> I’ve imported bellow events
>
> client.create_event(event="rate",
> entity_type="user",
> entity_id=int(row['userid']),
> target_entity_type="item",
> target_entity_id=int(row['itemid']),
> properties= { "rating" : float(row['rating']) });
>
>
> client.create_event(
> event="buy",
> entity_type="user",
> entity_id=int(row['userid']),
> target_entity_type="item",
> target_entity_id=int(row['itemid']), );
>
>
> client.create_event(
> event="$set",
> entity_type="item",
> entity_id=itemid,
> properties= { "categories": itemcats }
> );
>
> Could someone give me a solution or hint to how customize this recommender
> engine to take the categories into account.
>
> Thanks in advance
> Hossein
>
>
RE: Customizing Recommender engine
Posted by Hossein Bahrami <h....@live.com>.
Hi Pat,
Thanks for your reply, the reason I switched back to the Recommendation Engine is getting error during build.
This is my pio version : 0.10.0-incubating
And I’m getting below error during pio build of the UR template you shared, and also you can find the engine.json as well
##########################################################################
{
"comment":" This config file uses default settings for all but the required values see README.md for docs",
"id": "default",
"description": "Default settings",
"engineFactory": "org.template.RecommendationEngine",
"datasource": {
"params" : {
"name": "sample-handmade-data.txt",
"appName": "ur2",
"eventNames": ["rate", "$set"]
}
},
"sparkConf": {
"spark.serializer": "org.apache.spark.serializer.KryoSerializer",
"spark.kryo.registrator": "org.apache.mahout.sparkbindings.io.MahoutKryoRegistrator",
"spark.kryo.referenceTracking": "false",
"spark.kryoserializer.buffer": "300m",
"es.index.auto.create": "true"
},
"algorithms": [
{
"comment": "simplest setup where all values are default, popularity based backfill, must add eventsNames",
"name": "ur",
"params": {
"appName": "ur2",
"indexName": "urindex",
"typeName": "items",
"comment": "must have data for the first event or the model will not build, other events are optional",
"indicators": [
{
"name": "rate"
},{
"name": "$set",
"maxCorrelatorsPerItem": 50
}
],
"availableDateName": "available",
"expireDateName": "expires",
"dateName": "date",
"num": 4
}
}
]
}
###############################################################################
ERRORS
[INFO] [Console$] [warn] ::::::::::::::::::::::::::::::::::::::::::::::
[INFO] [Console$] [warn] :: UNRESOLVED DEPENDENCIES ::
[INFO] [Console$] [warn] ::::::::::::::::::::::::::::::::::::::::::::::
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] :: org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [warn] ::::::::::::::::::::::::::::::::::::::::::::::
[INFO] [Console$] [warn]
[INFO] [Console$] [warn] Note: Unresolved dependencies path:
[INFO] [Console$] [warn] org.apache.mahout:mahout-math-scala_2.10:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] [warn] org.apache.mahout:mahout-spark_2.10:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] [warn] org.apache.mahout:mahout-math:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] [warn] org.apache.mahout:mahout-hdfs:0.13.0-SNAPSHOT (/root/workspace/hossein.bahrami/ur2/build.sbt#L17-38)
[INFO] [Console$] [warn] +- com.actionml:template-scala-parallel-universal-recommendation_2.10:0.5.0
[INFO] [Console$] sbt.ResolveException: unresolved dependency: org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] unresolved dependency: org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] unresolved dependency: org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
[INFO] [Console$] unresolved dependency: org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
[INFO] [Console$] at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:278)
[INFO] [Console$] at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:175)
[INFO] [Console$] at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:157)
[INFO] [Console$] at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:151)
[INFO] [Console$] at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:151)
[INFO] [Console$] at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:128)
[INFO] [Console$] at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:56)
[INFO] [Console$] at sbt.IvySbt$$anon$4.call(Ivy.scala:64)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97)
[INFO] [Console$] at xsbt.boot.Using$.withResource(Using.scala:10)
[INFO] [Console$] at xsbt.boot.Using$.apply(Using.scala:9)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58)
[INFO] [Console$] at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48)
[INFO] [Console$] at xsbt.boot.Locks$.apply0(Locks.scala:31)
[INFO] [Console$] at xsbt.boot.Locks$.apply(Locks.scala:28)
[INFO] [Console$] at sbt.IvySbt.withDefaultLogger(Ivy.scala:64)
[INFO] [Console$] at sbt.IvySbt.withIvy(Ivy.scala:123)
[INFO] [Console$] at sbt.IvySbt.withIvy(Ivy.scala:120)
[INFO] [Console$] at sbt.IvySbt$Module.withModule(Ivy.scala:151)
[INFO] [Console$] at sbt.IvyActions$.updateEither(IvyActions.scala:157)
[INFO] [Console$] at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1318)
[INFO] [Console$] at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1315)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$85.apply(Defaults.scala:1345)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$85.apply(Defaults.scala:1343)
[INFO] [Console$] at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:35)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1348)
[INFO] [Console$] at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1342)
[INFO] [Console$] at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:45)
[INFO] [Console$] at sbt.Classpaths$.cachedUpdate(Defaults.scala:1360)
[INFO] [Console$] at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1300)
[INFO] [Console$] at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1275)
[INFO] [Console$] at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
[INFO] [Console$] at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
[INFO] [Console$] at sbt.std.Transform$$anon$4.work(System.scala:63)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
[INFO] [Console$] at sbt.Execute.work(Execute.scala:235)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
[INFO] [Console$] at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
[INFO] [Console$] at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
[INFO] [Console$] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[INFO] [Console$] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[INFO] [Console$] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[INFO] [Console$] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[INFO] [Console$] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[INFO] [Console$] at java.lang.Thread.run(Thread.java:745)
[INFO] [Console$] [error] (*:update) sbt.ResolveException: unresolved dependency: org.apache.mahout#mahout-math-scala_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] unresolved dependency: org.apache.mahout#mahout-spark_2.10;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] unresolved dependency: org.apache.mahout#mahout-math;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] unresolved dependency: org.apache.mahout#mahout-hdfs;0.13.0-SNAPSHOT: not found
[INFO] [Console$] [error] Total time: 4 s, completed Dec 14, 2016 9:16:15 AM
[ERROR] [Console$] Return code of previous step is 1. Aborting.
Regards
Hossein
………………………………
From: Pat Ferrel<ma...@occamsmachete.com>
Sent: Tuesday, December 13, 2016 10:08 PM
To: user@predictionio.incubator.apache.org<ma...@predictionio.incubator.apache.org>
Cc: Magnus Kragelund<ma...@ida.dk>
Subject: Re: Customizing Recommender engine
The UR has a new Apache PIO compatible repo here: https://github.com/actionml/universal-recommender.git
git clone https://github.com/actionml/universal-recommender.git ur
and proceed. The UR allows boosts or filters by properties. You are using a filter (bias -1), which does not work with the template you are using. Each template defines its own queries, config (engine.json), and input formats. There are a few common ideas but each one will have quite a few differences also and very different features.
On Dec 13, 2016, at 6:20 AM, Hossein Bahrami <h....@live.com>> wrote:
Hi, thanks for your reply. I don’t use Universal recommender, actually I use the Recommender Engine (http://predictionio.incubator.apache.org/templates/recommendation/quickstart/)
First I tried to use Universal recommender but it was difficulties building the project so I switched to the Recommendation Engine (RE), the RE works fine but don’t know how to customize it to take other events (items’ properties as I described ) into account.
Seems the RE returns very high score recommendations with my data so I guess it’s the choice here for me and just want to do some customization on it to make it predict recommendations per category (i.e property) of items.
Regards
Hossein
From: Magnus Kragelund<ma...@ida.dk>
Sent: Tuesday, December 13, 2016 12:17 PM
To: user@predictionio.incubator.apache.org<ma...@predictionio.incubator.apache.org>
Subject: Re: Customizing Recommender engine
Hi,
Assuming that you are using the Universal Recommender Engine, you should have a look at the "Queries" section here: https://github.com/PredictionIO/template-scala-parallel-universal-recommendation#queries
Try this request instead, where the "fields" property is used to filter by category
{
"user": "674296",
"num": 10,
"fields": [
{
"name": "categories",
"values": ["CAT1", "CAT2"],
"bias": -1
}
]
}
/magnus
________________________________
From: Hossein Bahrami <h....@live.com>>
Sent: Tuesday, December 13, 2016 10:55:00 AM
To: user@predictionio.incubator.apache.org<ma...@predictionio.incubator.apache.org>
Subject: Customizing Recommender engine
Dear all,
I’m new to Predictionio, currently I’m using it and I managed to import (rate, buy) events and getting pretty good results querying it. But now I want to limit the results for items in specific categories. I’ve created events for items’ properties (categories) as well .
I am posting this query to engine but it seems doesn’t care about the categories and returns same result every time.
{ "user": "674296", "num": 10, "categories" : ["CAT2", "CAT1"] }
I’ve imported bellow events
client.create_event(event="rate",
entity_type="user",
entity_id=int(row['userid']),
target_entity_type="item",
target_entity_id=int(row['itemid']),
properties= { "rating" : float(row['rating']) });
client.create_event(
event="buy",
entity_type="user",
entity_id=int(row['userid']),
target_entity_type="item",
target_entity_id=int(row['itemid']), );
client.create_event(
event="$set",
entity_type="item",
entity_id=itemid,
properties= { "categories": itemcats }
);
Could someone give me a solution or hint to how customize this recommender engine to take the categories into account.
Thanks in advance
Hossein