You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Diogo Matos <di...@gmail.com> on 2006/08/30 10:37:16 UTC

Can't use SnowballAnalyzer

Hi All,

I'm trying to use the SnowballAnalyzer and for some strange reason i cannot.
I got the following error message in the logs file:

org.apache.solr.core.SolrException: Error instantiating class class
org.apache.lucene.analysis.snowball.SnowballAnalyzer
    at org.apache.solr.core.Config.newInstance(Config.java:213)
    at org.apache.solr.schema.IndexSchema.readAnalyzer(IndexSchema.java:466)
    at org.apache.solr.schema.IndexSchema.readConfig(IndexSchema.java:294)
    at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:67)
    at org.apache.solr.core.SolrCore.<init>(SolrCore.java:189)
    at org.apache.solr.core.SolrCore.getSolrCore(SolrCore.java:170)
    at org.apache.solr.servlet.SolrServlet.init(SolrServlet.java:74)
    at javax.servlet.GenericServlet.init(GenericServlet.java:211)
    at org.apache.catalina.core.StandardWrapper.loadServlet(
StandardWrapper.java:1105)
    at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java
:932)
    at org.apache.catalina.core.StandardContext.loadOnStartup(
StandardContext.java:3917)
    at org.apache.catalina.core.StandardContext.start(StandardContext.java
:4201)
    at org.apache.catalina.core.ContainerBase.addChildInternal(
ContainerBase.java:759)
    at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java
:739)
    at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:524)
    at org.apache.catalina.startup.HostConfig.deployDirectory(
HostConfig.java:904)
    at org.apache.catalina.startup.HostConfig.deployDirectories(
HostConfig.java:867)
    at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java
:474)
    at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1122)
    at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java
:310)
    at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(
LifecycleSupport.java:119)
    at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1021)
    at org.apache.catalina.core.StandardHost.start(StandardHost.java:718)
    at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1013)
    at org.apache.catalina.core.StandardEngine.start(StandardEngine.java
:442)
    at org.apache.catalina.core.StandardService.start(StandardService.java
:450)
    at org.apache.catalina.core.StandardServer.start(StandardServer.java
:709)
    at org.apache.catalina.startup.Catalina.start(Catalina.java:551)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:294)
    at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:432)
Caused by: java.lang.InstantiationException:
org.apache.lucene.analysis.snowball.SnowballAnalyzer
    at java.lang.Class.newInstance0(Class.java:335)
    at java.lang.Class.newInstance(Class.java:303)
    at org.apache.solr.core.Config.newInstance(Config.java:211)
    ... 33 more


Does any one used it before?
Thank you
Diogo

Re: Can't use SnowballAnalyzer

Posted by Diogo Matos <di...@gmail.com>.
Erik,

Thank for yout prompt response. I've changed the code as you said and it's
working fine now.

Diog


On 8/30/06, Erik Hatcher <er...@ehatchersolutions.com> wrote:
>
> I'd be surprised if the SnowballAnalyzer can work directly, as its
> constructor requires the language parameter.  I see SnowballAnalyzer
> mentioned in a comment in the example schema.xml, but there is no
> specification for language.   My guess is you'll need to construct
> your own Analyzer that uses the SnowballFilter with a hardcoded
> language name (or passed in as a parameter some other way).
>
>         Erik
>
>
> On Aug 30, 2006, at 4:37 AM, Diogo Matos wrote:
>
> > Hi All,
> >
> > I'm trying to use the SnowballAnalyzer and for some strange reason
> > i cannot.
> > I got the following error message in the logs file:
> >
> > org.apache.solr.core.SolrException: Error instantiating class class
> > org.apache.lucene.analysis.snowball.SnowballAnalyzer
> >    at org.apache.solr.core.Config.newInstance(Config.java:213)
> >    at org.apache.solr.schema.IndexSchema.readAnalyzer
> > (IndexSchema.java:466)
> >    at org.apache.solr.schema.IndexSchema.readConfig
> > (IndexSchema.java:294)
> >    at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:67)
> >    at org.apache.solr.core.SolrCore.<init>(SolrCore.java:189)
> >    at org.apache.solr.core.SolrCore.getSolrCore(SolrCore.java:170)
> >    at org.apache.solr.servlet.SolrServlet.init(SolrServlet.java:74)
> >    at javax.servlet.GenericServlet.init(GenericServlet.java:211)
> >    at org.apache.catalina.core.StandardWrapper.loadServlet(
> > StandardWrapper.java:1105)
> >    at org.apache.catalina.core.StandardWrapper.load
> > (StandardWrapper.java
> > :932)
> >    at org.apache.catalina.core.StandardContext.loadOnStartup(
> > StandardContext.java:3917)
> >    at org.apache.catalina.core.StandardContext.start
> > (StandardContext.java
> > :4201)
> >    at org.apache.catalina.core.ContainerBase.addChildInternal(
> > ContainerBase.java:759)
> >    at org.apache.catalina.core.ContainerBase.addChild
> > (ContainerBase.java
> > :739)
> >    at org.apache.catalina.core.StandardHost.addChild
> > (StandardHost.java:524)
> >    at org.apache.catalina.startup.HostConfig.deployDirectory(
> > HostConfig.java:904)
> >    at org.apache.catalina.startup.HostConfig.deployDirectories(
> > HostConfig.java:867)
> >    at org.apache.catalina.startup.HostConfig.deployApps
> > (HostConfig.java
> > :474)
> >    at org.apache.catalina.startup.HostConfig.start(HostConfig.java:
> > 1122)
> >    at org.apache.catalina.startup.HostConfig.lifecycleEvent
> > (HostConfig.java
> > :310)
> >    at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(
> > LifecycleSupport.java:119)
> >    at org.apache.catalina.core.ContainerBase.start
> > (ContainerBase.java:1021)
> >    at org.apache.catalina.core.StandardHost.start(StandardHost.java:
> > 718)
> >    at org.apache.catalina.core.ContainerBase.start
> > (ContainerBase.java:1013)
> >    at org.apache.catalina.core.StandardEngine.start
> > (StandardEngine.java
> > :442)
> >    at org.apache.catalina.core.StandardService.start
> > (StandardService.java
> > :450)
> >    at org.apache.catalina.core.StandardServer.start
> > (StandardServer.java
> > :709)
> >    at org.apache.catalina.startup.Catalina.start(Catalina.java:551)
> >    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >    at sun.reflect.NativeMethodAccessorImpl.invoke(
> > NativeMethodAccessorImpl.java:39)
> >    at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:25)
> >    at java.lang.reflect.Method.invoke(Method.java:585)
> >    at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:294)
> >    at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:432)
> > Caused by: java.lang.InstantiationException:
> > org.apache.lucene.analysis.snowball.SnowballAnalyzer
> >    at java.lang.Class.newInstance0(Class.java:335)
> >    at java.lang.Class.newInstance(Class.java:303)
> >    at org.apache.solr.core.Config.newInstance(Config.java:211)
> >    ... 33 more
> >
> >
> > Does any one used it before?
> > Thank you
> > Diogo
>
>

Re: Can't use SnowballAnalyzer

Posted by Chris Hostetter <ho...@fucit.org>.
: constructor requires the language parameter.  I see SnowballAnalyzer
: mentioned in a comment in the example schema.xml, but there is no
: specification for language.   My guess is you'll need to construct

Whooops ... i just changed that example so as not to misslead people.

FYI: the SnowballFilter uses reflection, so it's not recommended for
perofrmance...

http://incubator.apache.org/solr/docs/api/org/apache/solr/analysis/SnowballPorterFilterFactory.html



-Hoss

Re: Can't use SnowballAnalyzer

Posted by Erik Hatcher <er...@ehatchersolutions.com>.
I'd be surprised if the SnowballAnalyzer can work directly, as its  
constructor requires the language parameter.  I see SnowballAnalyzer  
mentioned in a comment in the example schema.xml, but there is no  
specification for language.   My guess is you'll need to construct  
your own Analyzer that uses the SnowballFilter with a hardcoded  
language name (or passed in as a parameter some other way).

	Erik


On Aug 30, 2006, at 4:37 AM, Diogo Matos wrote:

> Hi All,
>
> I'm trying to use the SnowballAnalyzer and for some strange reason  
> i cannot.
> I got the following error message in the logs file:
>
> org.apache.solr.core.SolrException: Error instantiating class class
> org.apache.lucene.analysis.snowball.SnowballAnalyzer
>    at org.apache.solr.core.Config.newInstance(Config.java:213)
>    at org.apache.solr.schema.IndexSchema.readAnalyzer 
> (IndexSchema.java:466)
>    at org.apache.solr.schema.IndexSchema.readConfig 
> (IndexSchema.java:294)
>    at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:67)
>    at org.apache.solr.core.SolrCore.<init>(SolrCore.java:189)
>    at org.apache.solr.core.SolrCore.getSolrCore(SolrCore.java:170)
>    at org.apache.solr.servlet.SolrServlet.init(SolrServlet.java:74)
>    at javax.servlet.GenericServlet.init(GenericServlet.java:211)
>    at org.apache.catalina.core.StandardWrapper.loadServlet(
> StandardWrapper.java:1105)
>    at org.apache.catalina.core.StandardWrapper.load 
> (StandardWrapper.java
> :932)
>    at org.apache.catalina.core.StandardContext.loadOnStartup(
> StandardContext.java:3917)
>    at org.apache.catalina.core.StandardContext.start 
> (StandardContext.java
> :4201)
>    at org.apache.catalina.core.ContainerBase.addChildInternal(
> ContainerBase.java:759)
>    at org.apache.catalina.core.ContainerBase.addChild 
> (ContainerBase.java
> :739)
>    at org.apache.catalina.core.StandardHost.addChild 
> (StandardHost.java:524)
>    at org.apache.catalina.startup.HostConfig.deployDirectory(
> HostConfig.java:904)
>    at org.apache.catalina.startup.HostConfig.deployDirectories(
> HostConfig.java:867)
>    at org.apache.catalina.startup.HostConfig.deployApps 
> (HostConfig.java
> :474)
>    at org.apache.catalina.startup.HostConfig.start(HostConfig.java: 
> 1122)
>    at org.apache.catalina.startup.HostConfig.lifecycleEvent 
> (HostConfig.java
> :310)
>    at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(
> LifecycleSupport.java:119)
>    at org.apache.catalina.core.ContainerBase.start 
> (ContainerBase.java:1021)
>    at org.apache.catalina.core.StandardHost.start(StandardHost.java: 
> 718)
>    at org.apache.catalina.core.ContainerBase.start 
> (ContainerBase.java:1013)
>    at org.apache.catalina.core.StandardEngine.start 
> (StandardEngine.java
> :442)
>    at org.apache.catalina.core.StandardService.start 
> (StandardService.java
> :450)
>    at org.apache.catalina.core.StandardServer.start 
> (StandardServer.java
> :709)
>    at org.apache.catalina.startup.Catalina.start(Catalina.java:551)
>    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>    at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:39)
>    at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:25)
>    at java.lang.reflect.Method.invoke(Method.java:585)
>    at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:294)
>    at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:432)
> Caused by: java.lang.InstantiationException:
> org.apache.lucene.analysis.snowball.SnowballAnalyzer
>    at java.lang.Class.newInstance0(Class.java:335)
>    at java.lang.Class.newInstance(Class.java:303)
>    at org.apache.solr.core.Config.newInstance(Config.java:211)
>    ... 33 more
>
>
> Does any one used it before?
> Thank you
> Diogo

Re: Can't use SnowballAnalyzer

Posted by Bertrand Delacretaz <bd...@apache.org>.
On 8/30/06, Diogo Matos <di...@gmail.com> wrote:

> ...org.apache.solr.core.SolrException: Error instantiating class class
> org.apache.lucene.analysis.snowball.SnowballAnalyzer
>     at org.apache.solr.core.Config.newInstance(Config.java:213)...

It looks like SnowballAnalyzer cannot be used in that way, as it
doesn't have a no-parameter constructor. As a quick workaround to try
it, you could create a class that inherits from SnowballAnalyzer and
hardcodes the required parameter.

See http://lucene.apache.org/java/docs/api/org/apache/lucene/analysis/snowball/SnowballAnalyzer.html

-Bertrand

Re: Doc add limit, im experiencing it too

Posted by sangraal aiken <sa...@gmail.com>.
I sent out an email about this a while back, but basically this limit
appears only on Tomcat and only when Solr attempts to write to the response.


You can work around it by splitting up your posts so that you're posting
less than 5000 (or whatever your limit seems to be) at a time. You DO NOT
have to commit after each post. I recently indexed a 38 million document
data base with this problem and although it took about 8-9 hours it did
work... I only commited every 100,000 or so.

-Sangraal

On 9/6/06, Michael Imbeault <mi...@sympatico.ca> wrote:
>
> Old issue (see
> http://www.mail-archive.com/solr-user@lucene.apache.org/msg00651.html),
> but I'm experiencing the same exact thing on windows xp, latest tomcat.
> I noticed that the tomcat process gobbles memory (10 megs a second
> maybe) and then jams at 125 megs. Can't find a fix yet. I'm using a php
> interface and curl to post my xml, one document at a time, and commit
> every 100 document. Indexing 30000 docs, it hangs at maybe 5000. Anyone
> got an idea on this one? It would be helpful. I may try to switch to
> jetty tomorrow if nothing works :(
>
> --
> Michael Imbeault
> CHUL Research Center (CHUQ)
> 2705 boul. Laurier
> Ste-Foy, QC, Canada, G1V 4G2
> Tel: (418) 654-2705, Fax: (418) 654-2212
>

Re: Doc add limit, im experiencing it too

Posted by Chris Hostetter <ho...@fucit.org>.
: Old issue (see
: http://www.mail-archive.com/solr-user@lucene.apache.org/msg00651.html),
: but I'm experiencing the same exact thing on windows xp, latest tomcat.

did you notice the followup thread from sangraal where he mentioned that
he'd narrowed the problem down to both using Tomcat and submitting docs
containing CDATA? ...

http://www.nabble.com/Add-doc-limit---Follow-Up-tf2186440.html#a6048436

: I noticed that the tomcat process gobbles memory (10 megs a second
: maybe) and then jams at 125 megs. Can't find a fix yet. I'm using a php
: interface and curl to post my xml, one document at a time, and commit
: every 100 document. Indexing 30000 docs, it hangs at maybe 5000. Anyone

interesting ... you may not be hitting the same problem, Sangraal
specificly said he only encountered this bug when submitting a large
number of docs in a single request -- when you say it jams at 125 what do
you mean? ... are you sure you aren't just getting an OutOfMemory error?




-Hoss

Doc add limit, im experiencing it too

Posted by Michael Imbeault <mi...@sympatico.ca>.
Old issue (see 
http://www.mail-archive.com/solr-user@lucene.apache.org/msg00651.html), 
but I'm experiencing the same exact thing on windows xp, latest tomcat. 
I noticed that the tomcat process gobbles memory (10 megs a second 
maybe) and then jams at 125 megs. Can't find a fix yet. I'm using a php 
interface and curl to post my xml, one document at a time, and commit 
every 100 document. Indexing 30000 docs, it hangs at maybe 5000. Anyone 
got an idea on this one? It would be helpful. I may try to switch to 
jetty tomorrow if nothing works :(

-- 
Michael Imbeault
CHUL Research Center (CHUQ)
2705 boul. Laurier
Ste-Foy, QC, Canada, G1V 4G2
Tel: (418) 654-2705, Fax: (418) 654-2212