You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Daniel Mühlbachler <da...@muehlbachler.org> on 2011/02/24 15:46:37 UTC
SlopeOne MemoryDiffStorage processOneUser???
Hello,
I have a problem with the memory caching recommender. Due to performance
issues I must choose a memory caching recommender but we need to reload
the cache only for one user (I don't explain why).
After investigating the MemoryDiffStorage for the SlopeOne recommender,
I found the method "private long processOneUser(long averageCount, long
userID)" that processes only one user - right? In fact, my idea is that
I am making this method public and add this method to the WebService
class but what is this averageCount variable? Does it matter if I choose
a random number or must that be a special one?
Moreover, where can I find the WebService class files and how can I
compile them again to get a *.jws file so that I can add this method to
the WebService?
Thanks in advance!
Best regards,
Daniel Mühlbachler
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Ted Dunning <te...@gmail.com>.
Take a look at jconsole. It will show you how much heap your program is
actually using. You probably can look at that and just give the program a
small multiple of how much space it is using. To tune it more carefully
will require that you study how the java garbage collector works in more
detail and make measurements on your program under realistic conditions.
2011/2/26 Daniel Mühlbachler <da...@muehlbachler.org>
> yes, that's right. thx for that tip but a general question: how must i
> calculate the heap size for a perfect performance? should i use the smallest
> possible & working amount of memory or as high as possible and how can i
> calculate the highest possible one? (can it only be max a specific
> percentage of the available memory?)
>
> thanks!
>
>
>
> Am 26.02.2011 um 20:46 schrieb Ted Dunning <te...@gmail.com>:
>
> > 2011/2/26 Daniel Mühlbachler <da...@muehlbachler.org>
> >
> >> update:
> >> now it works. It seems as if the java heap size of 3GB was too high for
> >> tomcat. limiting it to 1GB and it works again! :-)
> >>
> >
> > Sounds like you are using a 32 bit JVM.
> >
> > If you later need more memory you should check on a 64 bit jvm.
>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Daniel Mühlbachler <da...@muehlbachler.org>.
yes, that's right. thx for that tip but a general question: how must i calculate the heap size for a perfect performance? should i use the smallest possible & working amount of memory or as high as possible and how can i calculate the highest possible one? (can it only be max a specific percentage of the available memory?)
thanks!
Am 26.02.2011 um 20:46 schrieb Ted Dunning <te...@gmail.com>:
> 2011/2/26 Daniel Mühlbachler <da...@muehlbachler.org>
>
>> update:
>> now it works. It seems as if the java heap size of 3GB was too high for
>> tomcat. limiting it to 1GB and it works again! :-)
>>
>
> Sounds like you are using a 32 bit JVM.
>
> If you later need more memory you should check on a 64 bit jvm.
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Ted Dunning <te...@gmail.com>.
2011/2/26 Daniel Mühlbachler <da...@muehlbachler.org>
> update:
> now it works. It seems as if the java heap size of 3GB was too high for
> tomcat. limiting it to 1GB and it works again! :-)
>
Sounds like you are using a 32 bit JVM.
If you later need more memory you should check on a 64 bit jvm.
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Daniel Mühlbachler <da...@muehlbachler.org>.
update:
now it works. It seems as if the java heap size of 3GB was too high for
tomcat. limiting it to 1GB and it works again! :-)
another thing: am I right that my context declaration is connectionpooled?
Am 26.02.2011 12:44, schrieb Sean Owen:
> I don't know enough about your code to guess at an explanation. I don't know
> whether you were actually building the diffs before as it sounds like you
> were looking to build it manually or something.
>
> I am not sure if 3GB is enough, or if it is, whether you want to spend that
> much memory on diffs. You could start with a lower number like 10M.
>
> 2011/2/26 Daniel Mühlbachler<da...@muehlbachler.org>
>
>> Ok, that's true because I in my preferenceTable I have about 650 000
>> entries/ratings (~24MB) but it's curious that it worked before perfectly
>> without any problem and I have nothing changed on my system except the svn
>> update and recompilation. It also takes approximately as long as before to
>> build the diffs.
>> So, you think if I say 100 million entries should be kept about 3GB of heap
>> size is enough?
>>
>>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Daniel Mühlbachler <da...@muehlbachler.org>.
OK, I will try it with 100M. Nevertheless, the diffs get built by the
recommender startup automatically. (standard behaviour - I took the
grouplens example recommender and just modified the data model)
Am 26.02.2011 12:44, schrieb Sean Owen:
> I don't know enough about your code to guess at an explanation. I don't know
> whether you were actually building the diffs before as it sounds like you
> were looking to build it manually or something.
>
> I am not sure if 3GB is enough, or if it is, whether you want to spend that
> much memory on diffs. You could start with a lower number like 10M.
>
> 2011/2/26 Daniel Mühlbachler<da...@muehlbachler.org>
>
>> Ok, that's true because I in my preferenceTable I have about 650 000
>> entries/ratings (~24MB) but it's curious that it worked before perfectly
>> without any problem and I have nothing changed on my system except the svn
>> update and recompilation. It also takes approximately as long as before to
>> build the diffs.
>> So, you think if I say 100 million entries should be kept about 3GB of heap
>> size is enough?
>>
>>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Daniel Mühlbachler <da...@muehlbachler.org>.
OK, I just tried to limit it to 100M. After deploying the new war file
to tomcat I thought it would be goof to just make this call:
RecommenderServlet?userID=1&debug=true
So, I get the following error:
*exception*
javax.servlet.ServletException: org.apache.mahout.cf.taste.common.TasteException: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
org.apache.mahout.cf.taste.web.RecommenderServlet.init(RecommenderServlet.java:76)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
java.lang.Thread.run(Thread.java:662)
*root cause*
org.apache.mahout.cf.taste.common.TasteException: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel.getNumThings(AbstractJDBCDataModel.java:590)
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel.getNumItems(AbstractJDBCDataModel.java:545)
org.apache.mahout.cf.taste.impl.recommender.slopeone.MemoryDiffStorage.<init>(MemoryDiffStorage.java:101)
org.muehlbachler.filmspiration.recommender.FilmspirationRecommender.createRecommender(FilmspirationRecommender.java:65)
org.muehlbachler.filmspiration.recommender.FilmspirationRecommender.<init>(FilmspirationRecommender.java:45)
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
java.lang.Class.newInstance0(Class.java:355)
java.lang.Class.newInstance(Class.java:308)
org.apache.mahout.cf.taste.web.RecommenderSingleton.<init>(RecommenderSingleton.java:51)
org.apache.mahout.cf.taste.web.RecommenderSingleton.initializeIfNeeded(RecommenderSingleton.java:42)
org.apache.mahout.cf.taste.web.RecommenderServlet.init(RecommenderServlet.java:74)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
java.lang.Thread.run(Thread.java:662)
*root cause*
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
com.mysql.jdbc.Util.handleNewInstance(Util.java:407)
com.mysql.jdbc.Util.getInstance(Util.java:382)
com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1013)
com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987)
com.mysql.jdbc.SQLError.createSQLException(SQLError.java:982)
com.mysql.jdbc.SQLError.createSQLException(SQLError.java:927)
com.mysql.jdbc.ConnectionImpl.throwConnectionClosedException(ConnectionImpl.java:1213)
com.mysql.jdbc.ConnectionImpl.checkClosed(ConnectionImpl.java:1200)
com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4203)
org.apache.commons.dbcp.DelegatingConnection.prepareStatement(DelegatingConnection.java:295)
org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.prepareStatement(PoolingDataSource.java:318)
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel.getNumThings(AbstractJDBCDataModel.java:576)
org.apache.mahout.cf.taste.impl.model.jdbc.AbstractJDBCDataModel.getNumItems(AbstractJDBCDataModel.java:545)
org.apache.mahout.cf.taste.impl.recommender.slopeone.MemoryDiffStorage.<init>(MemoryDiffStorage.java:101)
org.muehlbachler.filmspiration.recommender.FilmspirationRecommender.createRecommender(FilmspirationRecommender.java:65)
org.muehlbachler.filmspiration.recommender.FilmspirationRecommender.<init>(FilmspirationRecommender.java:45)
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
java.lang.Class.newInstance0(Class.java:355)
java.lang.Class.newInstance(Class.java:308)
org.apache.mahout.cf.taste.web.RecommenderSingleton.<init>(RecommenderSingleton.java:51)
org.apache.mahout.cf.taste.web.RecommenderSingleton.initializeIfNeeded(RecommenderSingleton.java:42)
org.apache.mahout.cf.taste.web.RecommenderServlet.init(RecommenderServlet.java:74)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
java.lang.Thread.run(Thread.java:662)
So, although it worked before too that's my resource context (specified in<tomcat-conf-dir>/context.xml):
<Resource name="jdbc/filmspiration" auth="Container" type="javax.sql.DataSource"
username="user"
password="passwd"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/filmspiration"
maxActive="15"
maxIdle="7"/>
Am 26.02.2011 12:44, schrieb Sean Owen:
> I don't know enough about your code to guess at an explanation. I don't know
> whether you were actually building the diffs before as it sounds like you
> were looking to build it manually or something.
>
> I am not sure if 3GB is enough, or if it is, whether you want to spend that
> much memory on diffs. You could start with a lower number like 10M.
>
> 2011/2/26 Daniel Mühlbachler<da...@muehlbachler.org>
>
>> Ok, that's true because I in my preferenceTable I have about 650 000
>> entries/ratings (~24MB) but it's curious that it worked before perfectly
>> without any problem and I have nothing changed on my system except the svn
>> update and recompilation. It also takes approximately as long as before to
>> build the diffs.
>> So, you think if I say 100 million entries should be kept about 3GB of heap
>> size is enough?
>>
>>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Sean Owen <sr...@gmail.com>.
I don't know enough about your code to guess at an explanation. I don't know
whether you were actually building the diffs before as it sounds like you
were looking to build it manually or something.
I am not sure if 3GB is enough, or if it is, whether you want to spend that
much memory on diffs. You could start with a lower number like 10M.
2011/2/26 Daniel Mühlbachler <da...@muehlbachler.org>
> Ok, that's true because I in my preferenceTable I have about 650 000
> entries/ratings (~24MB) but it's curious that it worked before perfectly
> without any problem and I have nothing changed on my system except the svn
> update and recompilation. It also takes approximately as long as before to
> build the diffs.
> So, you think if I say 100 million entries should be kept about 3GB of heap
> size is enough?
>
>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Daniel Mühlbachler <da...@muehlbachler.org>.
Ok, that's true because I in my preferenceTable I have about 650 000
entries/ratings (~24MB) but it's curious that it worked before perfectly
without any problem and I have nothing changed on my system except the
svn update and recompilation. It also takes approximately as long as
before to build the diffs.
So, you think if I say 100 million entries should be kept about 3GB of
heap size is enough?
Am 26.02.2011 12:22, schrieb Sean Owen:
> You probably have a lot of items then, and need to limit the size of the
> diff storage or else it will run out of memory. For example, if you have 1M
> items, the table in theory has up to 500B entries. Just pass maybe 100
> million as the last parameter to the MemoryDiffStorage constructor to limit
> how many entries are kept.
>
> 2011/2/26 Daniel Mühlbachler<da...@muehlbachler.org>
>
>> OK, but I already did a
>> export CATALINA_OPTS="-Xms3072m -Xmx3072m"
>> and I get the same error. Last time everything ran with only 1024M.
>>
>>
>> Am 26.02.2011 12:10, schrieb Sean Owen:
>>
>> Well, there's your error!
>>> java.lang.OutOfMemoryError: Java heap space
>>>
>>> Clearly you need to increase the heap size.
>>>
>>> 2011/2/26 Daniel Mühlbachler<da...@muehlbachler.org>
>>>
>>> my catalina.out log (starting tomcat& making the query)
>>>>
>>>>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Sean Owen <sr...@gmail.com>.
You probably have a lot of items then, and need to limit the size of the
diff storage or else it will run out of memory. For example, if you have 1M
items, the table in theory has up to 500B entries. Just pass maybe 100
million as the last parameter to the MemoryDiffStorage constructor to limit
how many entries are kept.
2011/2/26 Daniel Mühlbachler <da...@muehlbachler.org>
> OK, but I already did a
> export CATALINA_OPTS="-Xms3072m -Xmx3072m"
> and I get the same error. Last time everything ran with only 1024M.
>
>
> Am 26.02.2011 12:10, schrieb Sean Owen:
>
> Well, there's your error!
>>
>> java.lang.OutOfMemoryError: Java heap space
>>
>> Clearly you need to increase the heap size.
>>
>> 2011/2/26 Daniel Mühlbachler<da...@muehlbachler.org>
>>
>> my catalina.out log (starting tomcat& making the query)
>>>
>>>
>>>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Daniel Mühlbachler <da...@muehlbachler.org>.
OK, but I already did a
export CATALINA_OPTS="-Xms3072m -Xmx3072m"
and I get the same error. Last time everything ran with only 1024M.
Am 26.02.2011 12:10, schrieb Sean Owen:
> Well, there's your error!
>
> java.lang.OutOfMemoryError: Java heap space
>
> Clearly you need to increase the heap size.
>
> 2011/2/26 Daniel Mühlbachler<da...@muehlbachler.org>
>
>> my catalina.out log (starting tomcat& making the query)
>>
>>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Sean Owen <sr...@gmail.com>.
Well, there's your error!
java.lang.OutOfMemoryError: Java heap space
Clearly you need to increase the heap size.
2011/2/26 Daniel Mühlbachler <da...@muehlbachler.org>
> my catalina.out log (starting tomcat & making the query)
>
>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Daniel Mühlbachler <da...@muehlbachler.org>.
my catalina.out log (starting tomcat & making the query)
Feb 26, 2011 5:42:31 AM org.apache.coyote.http11.Http11Protocol init
INFO: Initializing Coyote HTTP/1.1 on http-8080
Feb 26, 2011 5:42:31 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 792 ms
Feb 26, 2011 5:42:31 AM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Feb 26, 2011 5:42:31 AM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.28
Feb 26, 2011 5:42:31 AM org.apache.catalina.startup.HostConfig
deployDescriptor
INFO: Deploying configuration descriptor host-manager.xml
Feb 26, 2011 5:42:31 AM org.apache.catalina.startup.HostConfig
deployDescriptor
INFO: Deploying configuration descriptor ROOT.xml
Feb 26, 2011 5:42:32 AM org.apache.catalina.startup.HostConfig
deployDescriptor
INFO: Deploying configuration descriptor manager.xml
Feb 26, 2011 5:42:32 AM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive filmspiration_org.war
Feb 26, 2011 5:42:32 AM org.slf4j.impl.JCLLoggerAdapter warn
WARNING: You are not using ConnectionPoolDataSource. Make sure your
DataSource pools connections to the database itself, or database
performance will be severely reduced.
Feb 26, 2011 5:42:32 AM org.slf4j.impl.JCLLoggerAdapter info
INFO: Starting MySQL data model with these settings:
dataSourceName = jdbc/filmspiration
table = ratedMovies
userColumn = userID
itemColumn = movieID
preferenceColumn = rating
timeColumn = time
Feb 26, 2011 5:42:32 AM org.slf4j.impl.JCLLoggerAdapter info
INFO: Building average diffs...
Feb 26, 2011 5:42:52 AM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-8080
Feb 26, 2011 5:42:52 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 20854 ms
Feb 26, 2011 5:43:06 AM
org.apache.axis.configuration.EngineConfigurationFactoryServlet
getServerEngineConfig
SEVERE: Unable to find config file. Creating new servlet engine config
file: /WEB-INF/server-config.wsdd
Feb 26, 2011 5:43:06 AM org.apache.axis.utils.JavaUtils
isAttachmentSupported
WARNING: Unable to find required classes (javax.activation.DataHandler
and javax.mail.internet.MimeMultipart). Attachment support is disabled.
Feb 26, 2011 5:43:06 AM
org.apache.axis.transport.http.AbstractQueryStringHandler processAxisFault
INFO: AxisFault:
AxisFault
faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException
faultSubcode:
faultString: java.lang.IllegalStateException: Not initialized
faultActor:
faultNode:
faultDetail:
{http://xml.apache.org/axis/}stackTrace:java.lang.IllegalStateException:
Not initialized
at
org.apache.mahout.cf.taste.web.RecommenderSingleton.getInstance(RecommenderSingleton.java:35)
at RecommenderService.<init>(RecommenderService.java:43)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at java.lang.Class.newInstance0(Class.java:355)
at java.lang.Class.newInstance(Class.java:308)
at
org.apache.axis.providers.java.JavaProvider.makeNewServiceObject(JavaProvider.java:385)
at
org.apache.axis.providers.java.JavaProvider.getNewServiceObject(JavaProvider.java:235)
at
org.apache.axis.providers.java.JavaProvider.getServiceObject(JavaProvider.java:105)
at
org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:287)
at
org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
at
org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:454)
at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
at
org.apache.axis.transport.http.QSMethodHandler.invokeEndpointFromGet(QSMethodHandler.java:129)
at
org.apache.axis.transport.http.QSMethodHandler.invoke(QSMethodHandler.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.axis.transport.http.AxisServlet.processQuery(AxisServlet.java:1226)
at
org.apache.axis.transport.http.AxisServlet.doGet(AxisServlet.java:249)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at
org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
{http://xml.apache.org/axis/}hostname:filmspiration.com
{http://xml.apache.org/axis/}isRuntimeException:true
java.lang.IllegalStateException: Not initialized
at org.apache.axis.AxisFault.makeFault(AxisFault.java:101)
at
org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:333)
at
org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
at
org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:454)
at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
at
org.apache.axis.transport.http.QSMethodHandler.invokeEndpointFromGet(QSMethodHandler.java:129)
at
org.apache.axis.transport.http.QSMethodHandler.invoke(QSMethodHandler.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.axis.transport.http.AxisServlet.processQuery(AxisServlet.java:1226)
at
org.apache.axis.transport.http.AxisServlet.doGet(AxisServlet.java:249)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at
org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.IllegalStateException: Not initialized
at
org.apache.mahout.cf.taste.web.RecommenderSingleton.getInstance(RecommenderSingleton.java:35)
at RecommenderService.<init>(RecommenderService.java:43)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at java.lang.Class.newInstance0(Class.java:355)
at java.lang.Class.newInstance(Class.java:308)
at
org.apache.axis.providers.java.JavaProvider.makeNewServiceObject(JavaProvider.java:385)
at
org.apache.axis.providers.java.JavaProvider.getNewServiceObject(JavaProvider.java:235)
at
org.apache.axis.providers.java.JavaProvider.getServiceObject(JavaProvider.java:105)
at
org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:287)
... 28 more
localhost log file (startup because the query doesn't get logged there):
Feb 26, 2011 6:04:40 AM org.apache.catalina.core.ApplicationContext log
SEVERE: StandardWrapper.Throwable
java.lang.OutOfMemoryError: Java heap space
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.rehash(FastByIDMap.java:283)
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.growAndRehash(FastByIDMap.java:272)
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.put(FastByIDMap.java:176)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.MemoryDiffStorage.processOneUser(MemoryDiffStorage.java:386)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.MemoryDiffStorage.buildAverageDiffs(MemoryDiffStorage.java:320)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.MemoryDiffStorage.<init>(MemoryDiffStorage.java:119)
at
org.muehlbachler.filmspiration.recommender.FilmspirationRecommender.createRecommender(FilmspirationRecommender.java:65)
at
org.muehlbachler.filmspiration.recommender.FilmspirationRecommender.<init>(FilmspirationRecommender.java:45)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at java.lang.Class.newInstance0(Class.java:355)
at java.lang.Class.newInstance(Class.java:308)
at
org.apache.mahout.cf.taste.web.RecommenderSingleton.<init>(RecommenderSingleton.java:51)
at
org.apache.mahout.cf.taste.web.RecommenderSingleton.initializeIfNeeded(RecommenderSingleton.java:42)
at
org.apache.mahout.cf.taste.web.RecommenderServlet.init(RecommenderServlet.java:74)
at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1173)
at
org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:993)
at
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4350)
at
org.apache.catalina.core.StandardContext.start(StandardContext.java:4659)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:546)
at
org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:905)
at
org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:740)
at
org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:500)
at
org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at
org.apache.catalina.core.StandardHost.start(StandardHost.java:785)
Feb 26, 2011 6:04:40 AM org.apache.catalina.core.StandardContext
loadOnStartup
SEVERE: Servlet /filmspiration_org threw load() exception
java.lang.OutOfMemoryError: Java heap space
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.rehash(FastByIDMap.java:283)
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.growAndRehash(FastByIDMap.java:272)
at
org.apache.mahout.cf.taste.impl.common.FastByIDMap.put(FastByIDMap.java:176)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.MemoryDiffStorage.processOneUser(MemoryDiffStorage.java:386)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.MemoryDiffStorage.buildAverageDiffs(MemoryDiffStorage.java:320)
at
org.apache.mahout.cf.taste.impl.recommender.slopeone.MemoryDiffStorage.<init>(MemoryDiffStorage.java:119)
at
org.muehlbachler.filmspiration.recommender.FilmspirationRecommender.createRecommender(FilmspirationRecommender.java:65)
at
org.muehlbachler.filmspiration.recommender.FilmspirationRecommender.<init>(FilmspirationRecommender.java:45)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at java.lang.Class.newInstance0(Class.java:355)
at java.lang.Class.newInstance(Class.java:308)
at
org.apache.mahout.cf.taste.web.RecommenderSingleton.<init>(RecommenderSingleton.java:51)
at
org.apache.mahout.cf.taste.web.RecommenderSingleton.initializeIfNeeded(RecommenderSingleton.java:42)
at
org.apache.mahout.cf.taste.web.RecommenderServlet.init(RecommenderServlet.java:74)
at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1173)
at
org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:993)
at
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4350)
at
org.apache.catalina.core.StandardContext.start(StandardContext.java:4659)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:546)
at
org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:905)
at
org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:740)
at
org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:500)
at
org.apache.catalina.startup.HostConfig.start(HostConfig.java:1277)
at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at
org.apache.catalina.core.StandardHost.start(StandardHost.java:785)
Other logs are not available and the same error occurs if I am trying to
estimate a preference which is already set in the MySQL database.
Am 26.02.2011 11:51, schrieb Sean Owen:
> Hmm, I am guessing that is due to some other error, which is preventing
> things from initializing. Look in your logs to find a previous error?
>
> 2011/2/26 Daniel Mühlbachler<da...@muehlbachler.org>
>
>> Sorry for my late answer but I jadn't the time to test it but after
>> recompiling, repackagin and redeploying I get this error after trying to
>> estimate a preference:
>>
>>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Sean Owen <sr...@gmail.com>.
Hmm, I am guessing that is due to some other error, which is preventing
things from initializing. Look in your logs to find a previous error?
2011/2/26 Daniel Mühlbachler <da...@muehlbachler.org>
> Sorry for my late answer but I jadn't the time to test it but after
> recompiling, repackagin and redeploying I get this error after trying to
> estimate a preference:
>
>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Daniel Mühlbachler <da...@muehlbachler.org>.
Sorry for my late answer but I jadn't the time to test it but after
recompiling, repackagin and redeploying I get this error after trying to
estimate a preference:
Feb 26, 2011 5:43:06 AM org.apache.axis.utils.JavaUtils
isAttachmentSupported
WARNING: Unable to find required classes (javax.activation.DataHandler
and javax.mail.internet.MimeMultipart). Attachment support is disabled.
Feb 26, 2011 5:43:06 AM
org.apache.axis.transport.http.AbstractQueryStringHandler processAxisFault
INFO: AxisFault:
AxisFault
faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException
faultSubcode:
faultString: java.lang.IllegalStateException: Not initialized
faultActor:
faultNode:
faultDetail:
{http://xml.apache.org/axis/}stackTrace:java.lang.IllegalStateException:
Not initialized
at
org.apache.mahout.cf.taste.web.RecommenderSingleton.getInstance(RecommenderSingleton.java:35)
at RecommenderService.<init>(RecommenderService.java:43)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at java.lang.Class.newInstance0(Class.java:355)
at java.lang.Class.newInstance(Class.java:308)
at
org.apache.axis.providers.java.JavaProvider.makeNewServiceObject(JavaProvider.java:385)
at
org.apache.axis.providers.java.JavaProvider.getNewServiceObject(JavaProvider.java:235)
at
org.apache.axis.providers.java.JavaProvider.getServiceObject(JavaProvider.java:105)
at
org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:287)
at
org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
at
org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:454)
at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
at
org.apache.axis.transport.http.QSMethodHandler.invokeEndpointFromGet(QSMethodHandler.java:129)
at
org.apache.axis.transport.http.QSMethodHandler.invoke(QSMethodHandler.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.axis.transport.http.AxisServlet.processQuery(AxisServlet.java:1226)
at
org.apache.axis.transport.http.AxisServlet.doGet(AxisServlet.java:249)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at
org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
{http://xml.apache.org/axis/}hostname:filmspiration.com
{http://xml.apache.org/axis/}isRuntimeException:true
java.lang.IllegalStateException: Not initialized
at org.apache.axis.AxisFault.makeFault(AxisFault.java:101)
at
org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:333)
at
org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
at
org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:454)
at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
at
org.apache.axis.transport.http.QSMethodHandler.invokeEndpointFromGet(QSMethodHandler.java:129)
at
org.apache.axis.transport.http.QSMethodHandler.invoke(QSMethodHandler.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.axis.transport.http.AxisServlet.processQuery(AxisServlet.java:1226)
at
org.apache.axis.transport.http.AxisServlet.doGet(AxisServlet.java:249)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at
org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.IllegalStateException: Not initialized
at
org.apache.mahout.cf.taste.web.RecommenderSingleton.getInstance(RecommenderSingleton.java:35)
at RecommenderService.<init>(RecommenderService.java:43)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at java.lang.Class.newInstance0(Class.java:355)
at java.lang.Class.newInstance(Class.java:308)
at
org.apache.axis.providers.java.JavaProvider.makeNewServiceObject(JavaProvider.java:385)
at
org.apache.axis.providers.java.JavaProvider.getNewServiceObject(JavaProvider.java:235)
at
org.apache.axis.providers.java.JavaProvider.getServiceObject(JavaProvider.java:105)
at
org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:287)
... 28 more
Nevertheless, the recommender builds the diffs successfully. I also
tried to pruge the tomcat server and install it again but that doesn't
help neither.
Another information: I got the error with the disabled attachment
support with the old version too and it worked perfectly!
Am 24.02.2011 19:08, schrieb Sean Owen:
> Oops that's a small bug. It was not clearing the estimate for an item that
> is already known to a user in this case. Try the change I just committed to
> SVN.
>
> 2011/2/24 Daniel Mühlbachler<da...@muehlbachler.org>
>
>> Ok if it should be so, why doesn't it work?
>> I made the following:
>> * estimatePreference userID=1 itemID=1 -> 6....
>> * setPreference userID=1 itemID=1 value=4.0
>> * estimatePreference userID=1 itemID=1 -> 6.... and not 4.0
>>
>> So, how can I change that behaviour?
>>
>>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Sean Owen <sr...@gmail.com>.
Oops that's a small bug. It was not clearing the estimate for an item that
is already known to a user in this case. Try the change I just committed to
SVN.
2011/2/24 Daniel Mühlbachler <da...@muehlbachler.org>
> Ok if it should be so, why doesn't it work?
> I made the following:
> * estimatePreference userID=1 itemID=1 -> 6....
> * setPreference userID=1 itemID=1 value=4.0
> * estimatePreference userID=1 itemID=1 -> 6.... and not 4.0
>
> So, how can I change that behaviour?
>
>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Daniel Mühlbachler <da...@muehlbachler.org>.
Ok if it should be so, why doesn't it work?
I made the following:
* estimatePreference userID=1 itemID=1 -> 6....
* setPreference userID=1 itemID=1 value=4.0
* estimatePreference userID=1 itemID=1 -> 6.... and not 4.0
So, how can I change that behaviour?
Am 24.02.2011 um 18:28 schrieb Sean Owen <sr...@gmail.com>:
> Setting a preference clears the cache for that user already. This shouldn't
> be an problem.
>
> CachingRecommender has nothing to do with diffs. There is no caching of
> diffs in that sense.
>
> 2011/2/24 Daniel Mühlbachler <da...@muehlbachler.org>
>
>> Nope, I tried that before and in my opinion the problem is the
>> CachingRecommender. I need a caching one because of the performance and
>> runtime. In fact, I must update or even clear the cache for the user who
>> updates his preferences. Information: I use a new CachingRecommender(new
>> SlopeOneRecommender(...), ...)
>> Additionally (I don't know if this matters), I use a MySQLDataModel but no
>> MySQLDiffStorage because it takes too long to build the diffs (after 3 hours
>> it even hadn't finished!).
>> So, all in all, if you say the SlopeOneRecommender does it itself, my
>> problem is the CachingRecommender that caches all diffs if they are called
>> once until the cache gets cleared.
>>
>>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Sean Owen <sr...@gmail.com>.
Setting a preference clears the cache for that user already. This shouldn't
be an problem.
CachingRecommender has nothing to do with diffs. There is no caching of
diffs in that sense.
2011/2/24 Daniel Mühlbachler <da...@muehlbachler.org>
> Nope, I tried that before and in my opinion the problem is the
> CachingRecommender. I need a caching one because of the performance and
> runtime. In fact, I must update or even clear the cache for the user who
> updates his preferences. Information: I use a new CachingRecommender(new
> SlopeOneRecommender(...), ...)
> Additionally (I don't know if this matters), I use a MySQLDataModel but no
> MySQLDiffStorage because it takes too long to build the diffs (after 3 hours
> it even hadn't finished!).
> So, all in all, if you say the SlopeOneRecommender does it itself, my
> problem is the CachingRecommender that caches all diffs if they are called
> once until the cache gets cleared.
>
>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Daniel Mühlbachler <da...@muehlbachler.org>.
Nope, I tried that before and in my opinion the problem is the
CachingRecommender. I need a caching one because of the performance and
runtime. In fact, I must update or even clear the cache for the user who
updates his preferences. Information: I use a new CachingRecommender(new
SlopeOneRecommender(...), ...)
Additionally (I don't know if this matters), I use a MySQLDataModel but
no MySQLDiffStorage because it takes too long to build the diffs (after
3 hours it even hadn't finished!).
So, all in all, if you say the SlopeOneRecommender does it itself, my
problem is the CachingRecommender that caches all diffs if they are
called once until the cache gets cleared.
Am 24.02.2011 17:42, schrieb Sean Owen:
> I see well if you are using SlopeOneRecommender, it already updates
> the diffs correctly for you, when you call setPreference() for
> example. If you do it that way it all works. There should be no need
> to expose and update the diffs separately.
>
> 2011/2/24 Daniel Mühlbachler <daniel@muehlbachler.org
> <ma...@muehlbachler.org>>
>
> Ok but if I want to call removeItemPref I must know my preference
> but to get that preference I need runtime. However, what happens
> if I remove a preference and want to get the preference after that
> again (so it should recalculate the user's preferences).
> Another thing: if I want to call updateItemPref there is no way to
> specify an user. Only itemID and a prefDelta. I understand why
> there is no userID but what's prefDelta? And the biggest issue in
> that case is the runtime because we have many queries at the same
> time that must be processed as quickly as possible. In fact, if I
> update all preferences is there a timeout for all other queries?
> Ok, let's specify again what I have/need:
> A website that interacts with my recommender and there will be
> many many queries at the same time (huge site). In fact, a user
> can rate movies and if a user does this we need a method that
> rebuilds at least the user's recommendations. (All other
> recommendations from other users that get affected by that change
> are not important - only the user's ones because we are doing a
> complete refesh every day.) For that, we can't have a high timeout
> (e.g. refresh needs about 20-25 seconds). So the perfect way would
> be only to update an user's preferences or the use of the
> updateItemPref method (if this method can provide that things).
>
> btw... thanks again for your help with my MySQL Connector and
> Tomcat! :-)
>
>
>
> Am 24.02.2011 17:15, schrieb Sean Owen:
>
> It's just a count maintained across calls to the method which
> tracks how
> many entries have been stored in the data structures. This is
> so it can stop
> growing the data size once a limit has been reached.
>
> It would not work to expose this method directly, as-is, no.
> Instead I think you want to reflect your updates by calling
> updateItemPref()
> and/or removeItemPref() for all the changes that your user makes.
>
> 2011/2/24 Daniel Mühlbachler<daniel@muehlbachler.org
> <ma...@muehlbachler.org>>
>
> Hello,
>
> I have a problem with the memory caching recommender. Due
> to performance
> issues I must choose a memory caching recommender but we
> need to reload the
> cache only for one user (I don't explain why).
> After investigating the MemoryDiffStorage for the SlopeOne
> recommender, I
> found the method "private long processOneUser(long
> averageCount, long
> userID)" that processes only one user - right? In fact, my
> idea is that I am
> making this method public and add this method to the
> WebService class but
> what is this averageCount variable? Does it matter if I
> choose a random
> number or must that be a special one?
> Moreover, where can I find the WebService class files and
> how can I compile
> them again to get a *.jws file so that I can add this
> method to the
> WebService?
>
> Thanks in advance!
> Best regards,
> Daniel Mühlbachler
>
>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Sean Owen <sr...@gmail.com>.
I see well if you are using SlopeOneRecommender, it already updates the
diffs correctly for you, when you call setPreference() for example. If you
do it that way it all works. There should be no need to expose and update
the diffs separately.
2011/2/24 Daniel Mühlbachler <da...@muehlbachler.org>
> Ok but if I want to call removeItemPref I must know my preference but to
> get that preference I need runtime. However, what happens if I remove a
> preference and want to get the preference after that again (so it should
> recalculate the user's preferences).
> Another thing: if I want to call updateItemPref there is no way to specify
> an user. Only itemID and a prefDelta. I understand why there is no userID
> but what's prefDelta? And the biggest issue in that case is the runtime
> because we have many queries at the same time that must be processed as
> quickly as possible. In fact, if I update all preferences is there a timeout
> for all other queries?
> Ok, let's specify again what I have/need:
> A website that interacts with my recommender and there will be many many
> queries at the same time (huge site). In fact, a user can rate movies and if
> a user does this we need a method that rebuilds at least the user's
> recommendations. (All other recommendations from other users that get
> affected by that change are not important - only the user's ones because we
> are doing a complete refesh every day.) For that, we can't have a high
> timeout (e.g. refresh needs about 20-25 seconds). So the perfect way would
> be only to update an user's preferences or the use of the updateItemPref
> method (if this method can provide that things).
>
> btw... thanks again for your help with my MySQL Connector and Tomcat! :-)
>
>
>
> Am 24.02.2011 17:15, schrieb Sean Owen:
>
> It's just a count maintained across calls to the method which tracks how
>> many entries have been stored in the data structures. This is so it can
>> stop
>> growing the data size once a limit has been reached.
>>
>> It would not work to expose this method directly, as-is, no.
>> Instead I think you want to reflect your updates by calling
>> updateItemPref()
>> and/or removeItemPref() for all the changes that your user makes.
>>
>> 2011/2/24 Daniel Mühlbachler<da...@muehlbachler.org>
>>
>> Hello,
>>>
>>> I have a problem with the memory caching recommender. Due to performance
>>> issues I must choose a memory caching recommender but we need to reload
>>> the
>>> cache only for one user (I don't explain why).
>>> After investigating the MemoryDiffStorage for the SlopeOne recommender, I
>>> found the method "private long processOneUser(long averageCount, long
>>> userID)" that processes only one user - right? In fact, my idea is that I
>>> am
>>> making this method public and add this method to the WebService class but
>>> what is this averageCount variable? Does it matter if I choose a random
>>> number or must that be a special one?
>>> Moreover, where can I find the WebService class files and how can I
>>> compile
>>> them again to get a *.jws file so that I can add this method to the
>>> WebService?
>>>
>>> Thanks in advance!
>>> Best regards,
>>> Daniel Mühlbachler
>>>
>>>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Daniel Mühlbachler <da...@muehlbachler.org>.
Ok but if I want to call removeItemPref I must know my preference but to
get that preference I need runtime. However, what happens if I remove a
preference and want to get the preference after that again (so it should
recalculate the user's preferences).
Another thing: if I want to call updateItemPref there is no way to
specify an user. Only itemID and a prefDelta. I understand why there is
no userID but what's prefDelta? And the biggest issue in that case is
the runtime because we have many queries at the same time that must be
processed as quickly as possible. In fact, if I update all preferences
is there a timeout for all other queries?
Ok, let's specify again what I have/need:
A website that interacts with my recommender and there will be many many
queries at the same time (huge site). In fact, a user can rate movies
and if a user does this we need a method that rebuilds at least the
user's recommendations. (All other recommendations from other users that
get affected by that change are not important - only the user's ones
because we are doing a complete refesh every day.) For that, we can't
have a high timeout (e.g. refresh needs about 20-25 seconds). So the
perfect way would be only to update an user's preferences or the use of
the updateItemPref method (if this method can provide that things).
btw... thanks again for your help with my MySQL Connector and Tomcat! :-)
Am 24.02.2011 17:15, schrieb Sean Owen:
> It's just a count maintained across calls to the method which tracks how
> many entries have been stored in the data structures. This is so it can stop
> growing the data size once a limit has been reached.
>
> It would not work to expose this method directly, as-is, no.
> Instead I think you want to reflect your updates by calling updateItemPref()
> and/or removeItemPref() for all the changes that your user makes.
>
> 2011/2/24 Daniel Mühlbachler<da...@muehlbachler.org>
>
>> Hello,
>>
>> I have a problem with the memory caching recommender. Due to performance
>> issues I must choose a memory caching recommender but we need to reload the
>> cache only for one user (I don't explain why).
>> After investigating the MemoryDiffStorage for the SlopeOne recommender, I
>> found the method "private long processOneUser(long averageCount, long
>> userID)" that processes only one user - right? In fact, my idea is that I am
>> making this method public and add this method to the WebService class but
>> what is this averageCount variable? Does it matter if I choose a random
>> number or must that be a special one?
>> Moreover, where can I find the WebService class files and how can I compile
>> them again to get a *.jws file so that I can add this method to the
>> WebService?
>>
>> Thanks in advance!
>> Best regards,
>> Daniel Mühlbachler
>>
Re: SlopeOne MemoryDiffStorage processOneUser???
Posted by Sean Owen <sr...@gmail.com>.
It's just a count maintained across calls to the method which tracks how
many entries have been stored in the data structures. This is so it can stop
growing the data size once a limit has been reached.
It would not work to expose this method directly, as-is, no.
Instead I think you want to reflect your updates by calling updateItemPref()
and/or removeItemPref() for all the changes that your user makes.
2011/2/24 Daniel Mühlbachler <da...@muehlbachler.org>
> Hello,
>
> I have a problem with the memory caching recommender. Due to performance
> issues I must choose a memory caching recommender but we need to reload the
> cache only for one user (I don't explain why).
> After investigating the MemoryDiffStorage for the SlopeOne recommender, I
> found the method "private long processOneUser(long averageCount, long
> userID)" that processes only one user - right? In fact, my idea is that I am
> making this method public and add this method to the WebService class but
> what is this averageCount variable? Does it matter if I choose a random
> number or must that be a special one?
> Moreover, where can I find the WebService class files and how can I compile
> them again to get a *.jws file so that I can add this method to the
> WebService?
>
> Thanks in advance!
> Best regards,
> Daniel Mühlbachler
>