You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Andrew Nagy <an...@villanova.edu> on 2007/04/10 17:41:01 UTC
Schema validator/debugger
Does anyone have a good method of debugging a schema?
I have been struggling to get my new schema to run for the past couple
of days and just do not see anything wrong with the XML document.
Thanks
Andrew
Re: Schema validator/debugger
Posted by Yonik Seeley <yo...@apache.org>.
On 4/11/07, Ryan McKinley <ry...@gmail.com> wrote:
> > Off topic a bit, Has anyone set forth to build a new admin interface for
> > SOLR? I build a lot of admin interfaces for my day job and would love
> > to give the admin module a bit of a tune-up (I won't use the term overhaul).
>
> i think we definitely need an updated admin interface, yes!
I agree. We've never put much emphasis on the admin pages, and they
could really use more functionality and a fresher look.
> Ideally, we could get rid of all the JSP and do everything with
> requestHandlers + XSLT.
Seems rather pointless if the look or functionality isn't improved.
I'd personally find it much harder to debug or enhance pages that try
to do *everything* in XSLT.
Where it makes sense to export info programatically, a request handler
can be used, but that's certainly not in all the cases.
> Then everything could work with only a JRE -
AFAIK, all of the newest servlet containers can run off a JRE. When
Jetty 6.1 becomes stable, we should use that in the example.
-Yonik
Re: Schema validator/debugger
Posted by Andrew Nagy <an...@villanova.edu>.
Ryan McKinley wrote:
>>
>> Off topic a bit, Has anyone set forth to build a new admin interface for
>> SOLR? I build a lot of admin interfaces for my day job and would love
>> to give the admin module a bit of a tune-up (I won't use the term
>> overhaul).
>>
>
> i think we definitely need an updated admin interface, yes!
>
> Ideally, we could get rid of all the JSP and do everything with
> requestHandlers + XSLT. Then everything could work with only a JRE -
> it would not require a JDK and all the jsp jars. AND this would let
> other clients have access to the same info with JSON/xml etc
>
> check:
> http://issues.apache.org/jira/browse/SOLR-162
>
> This puts a lot of the functionality as handlers, but still needs an
> XSLT front end
Yes, Erik Hatcher showed this off a bit at a conference I was at in
February. I was thinking about building off of these xml files. I do
quite a bit with XSLT in my day job, so I would love to play around with
it. I think this is something that is desperately needed in SOLR, I
know it would make my life a lot easier :)
I will take a stab at some of this soon ...
Andrew
Re: Schema validator/debugger
Posted by Ryan McKinley <ry...@gmail.com>.
>
> Off topic a bit, Has anyone set forth to build a new admin interface for
> SOLR? I build a lot of admin interfaces for my day job and would love
> to give the admin module a bit of a tune-up (I won't use the term overhaul).
>
i think we definitely need an updated admin interface, yes!
Ideally, we could get rid of all the JSP and do everything with
requestHandlers + XSLT. Then everything could work with only a JRE -
it would not require a JDK and all the jsp jars. AND this would let
other clients have access to the same info with JSON/xml etc
check:
http://issues.apache.org/jira/browse/SOLR-162
This puts a lot of the functionality as handlers, but still needs an
XSLT front end
Re: Schema validator/debugger
Posted by Andrew Nagy <an...@villanova.edu>.
Greg Ludington wrote:
>> I just installed SOLR-75 patch and the "schema browser" is able to view
>> the schema perfectly. When I used the default schema with SOLR,
>> everything is fine, but when I replace my schema ... it's throws this
>> NullPointerException ....
>
> One thing to note is that the "schema browser" as currently in SOLR-75
> just transforms the xml file -- it does not represent any of the
> actual parsing of the configuration by Solr upon startup, and so would
> not be able to catch any errors Solr might encounter in the schema.
Sure, but it's an easier way for my non-textual brain to grasp the
schema file :)
Nice job, BTW.
Off topic a bit, Has anyone set forth to build a new admin interface for
SOLR? I build a lot of admin interfaces for my day job and would love
to give the admin module a bit of a tune-up (I won't use the term overhaul).
Re: Schema validator/debugger
Posted by Greg Ludington <gl...@gmail.com>.
> I just installed SOLR-75 patch and the "schema browser" is able to view
> the schema perfectly. When I used the default schema with SOLR,
> everything is fine, but when I replace my schema ... it's throws this
> NullPointerException ....
One thing to note is that the "schema browser" as currently in SOLR-75
just transforms the xml file -- it does not represent any of the
actual parsing of the configuration by Solr upon startup, and so would
not be able to catch any errors Solr might encounter in the schema.
-Greg
Re: Schema validator/debugger
Posted by Chris Hostetter <ho...@fucit.org>.
: <pingQuery>
: qt=dismax&q=solr&start=3&fq=id:[* TO *]&fq=cat:[* TO *]
: </pingQuery>
:
: Perhaps we should change it to something simple by default??? "q=solr"?
looking back at the log, i made it something complicated to demonstrate
the odd format requirements (similar to a URL ... but without URL escaping
... but it's in XML so you have to XML escape the ampersands)
http://svn.apache.org/viewvc?view=rev&revision=486904
(We should really come up with a better syntax for "pingQuery"
that supports the NamedList style way of specifying SolrParams in configs)
-Hoss
Re: Schema validator/debugger
Posted by Andrew Nagy <an...@villanova.edu>.
Yonik Seeley wrote:
> Oh wait... Andrew, were you always testing via "ping"?
>
> Check out what the ping query is configured as in solrconfig.xml:
>
> <pingQuery>
> qt=dismax&q=solr&start=3&fq=id:[* TO *]&fq=cat:[*
> TO *]
> </pingQuery>
>
> Perhaps we should change it to something simple by default??? "q=solr"?
>
> That solves the Jetty failure mystery... so it looks like you either
> have a tomcat setup problem, or a Solr bug that only shows under
> tomcat.
Yes, this is the problem! Good catch :) I have been testing via ping.
However this still does not solve my original problem ... I will dig a
bit more and see what I can find.
Thanks
Andrew
Re: Schema validator/debugger
Posted by Yonik Seeley <yo...@apache.org>.
Oh wait... Andrew, were you always testing via "ping"?
Check out what the ping query is configured as in solrconfig.xml:
<pingQuery>
qt=dismax&q=solr&start=3&fq=id:[* TO *]&fq=cat:[* TO *]
</pingQuery>
Perhaps we should change it to something simple by default??? "q=solr"?
That solves the Jetty failure mystery... so it looks like you either
have a tomcat setup problem, or a Solr bug that only shows under
tomcat.
-Yonik
On 4/12/07, Chris Hostetter <ho...@fucit.org> wrote:
> : Ahhh, are you using the dismax handler? I was using the standard
> : request handler with a simple query "solr"
>
> That would explain the "SolrException: undefined field text" he got using
> Jetty, but not the orriginal NullPointerException.
>
> that still worries me....
>
> -Hoss
Re: Schema validator/debugger
Posted by Chris Hostetter <ho...@fucit.org>.
: Ahhh, are you using the dismax handler? I was using the standard
: request handler with a simple query "solr"
That would explain the "SolrException: undefined field text" he got using
Jetty, but not the orriginal NullPointerException.
that still worries me....
-Hoss
Re: Schema validator/debugger
Posted by Yonik Seeley <yo...@apache.org>.
On 4/12/07, Andrew Nagy <an...@villanova.edu> wrote:
> Yonik Seeley wrote:
> > I dropped your schema.xml directly into the Solr example (using
> > Jetty), fired it up, and everything works fine!?
> Okay, I switched over to Jetty and now I get a different error:
> SEVERE: org.apache.solr.core.SolrException: undefined field text
Ahhh, are you using the dismax handler? I was using the standard
request handler with a simple query "solr"
The dismax handler in the example solrconfig.xml has a lot of default
params that depend on certain fields (since it searches across
fields):
<requestHandler name="dismax" class="solr.DisMaxRequestHandler" >
<lst name="defaults">
<str name="echoParams">explicit</str>
<float name="tie">0.01</float>
<str name="qf">
text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
</str>
<str name="pf">
text^0.2 features^1.1 name^1.5 manu^1.4 manu_exact^1.9
</str>
<str name="bf">
ord(poplarity)^0.5 recip(rord(price),1,1000,1000)^0.3
</str>
<str name="fl">
id,name,price,score
</str>
<str name="mm">
2<-1 5<-2 6<90%
</str>
<int name="ps">100</int>
<str name="q.alt">*:*</str>
</lst>
</requestHandler>
-Yonik
Re: Schema validator/debugger
Posted by Andrew Nagy <an...@villanova.edu>.
Ryan McKinley wrote:
> With a clean checkout, you can run:
> $ ant example
> $ cd example
> $ java -jar start.jar
>
> and things work OK.
>
> But, when you delete all but the two fields, you get an exception
> somewhere?
Well, I was working from my own directory, not the example directory. I
can give that a try ... but all I did was copy everything from the
example dir to a different dir.
But, basically ... yes.
I will delete all instances of solr on my server and start from scratch
... i must have something screwy somewhere.
Re: Schema validator/debugger
Posted by Ryan McKinley <ry...@gmail.com>.
With a clean checkout, you can run:
$ ant example
$ cd example
$ java -jar start.jar
and things work OK.
But, when you delete all but the two fields, you get an exception somewhere?
On 4/12/07, Andrew Nagy <an...@villanova.edu> wrote:
> Yonik Seeley wrote:
> > I dropped your schema.xml directly into the Solr example (using
> > Jetty), fired it up, and everything works fine!?
> Okay, I switched over to Jetty and now I get a different error:
> SEVERE: org.apache.solr.core.SolrException: undefined field text
> >
> > Are you sure you are using the stock solrconfig.xml, etc?
> Yep.
>
>
> I apologize ahead of time if I am being blatantly stupid about this. :)
>
Re: host logging options (was Re: Schema validator/debugger)
Posted by Chris Hostetter <ho...@fucit.org>.
: > SEVERE: org.apache.solr.core.SolrException: undefined field text
: As someone who has used both Jetty and Tomcat in production (and has
: come to prefer Tomcat), what are my choices to get the "undefined field
: xxx" error in the catalina log files (or is it stashed somewhere I'm
: overlooking?)
i think this depends largely on the tomcat logging configuration ... i
don't know much about tomcat's default logging config, or wheter you've
customized the logging levels, but as i recall from when i did some Solr
testing to make sure the JNDI and plugin stuff worked, i *think* Tomcat
was using seperate log files for basic logging vs "serious" errors
(controlled via logging levels) ... but i may be wrong.
-Hoss
host logging options (was Re: Schema validator/debugger)
Posted by Walter Lewis <le...@hhpl.on.ca>.
Andrew Nagy wrote:
> Yonik Seeley wrote:
>> I dropped your schema.xml directly into the Solr example (using
>> Jetty), fired it up, and everything works fine!?
> Okay, I switched over to Jetty and now I get a different error:
> SEVERE: org.apache.solr.core.SolrException: undefined field text
As someone who has used both Jetty and Tomcat in production (and has
come to prefer Tomcat), what are my choices to get the "undefined field
xxx" error in the catalina log files (or is it stashed somewhere I'm
overlooking?)
Walter Lewis
Re: Schema validator/debugger
Posted by Andrew Nagy <an...@villanova.edu>.
Yonik Seeley wrote:
> I dropped your schema.xml directly into the Solr example (using
> Jetty), fired it up, and everything works fine!?
Okay, I switched over to Jetty and now I get a different error:
SEVERE: org.apache.solr.core.SolrException: undefined field text
>
> Are you sure you are using the stock solrconfig.xml, etc?
Yep.
I apologize ahead of time if I am being blatantly stupid about this. :)
Re: Schema validator/debugger
Posted by Yonik Seeley <yo...@apache.org>.
I dropped your schema.xml directly into the Solr example (using
Jetty), fired it up, and everything works fine!?
Are you sure you are using the stock solrconfig.xml, etc?
-Yonik
On 4/12/07, Andrew Nagy <an...@villanova.edu> wrote:
> So maybe I am doing something really dumb, but for testing purposes I
> took a copy of the example schema.xml file and removed all of the fields
> except for 2 to start with something really basic and it is throwing the
> nullpointerexception again.
>
> I attached the file if it is of any help
>
> I am using a fresh copy of the nightly build.
>
> Ryan McKinley wrote:
> >>
> >> Would it help if I posted my schema file?
> >>
> >
> > yes
Re: Schema validator/debugger
Posted by Andrew Nagy <an...@villanova.edu>.
So maybe I am doing something really dumb, but for testing purposes I
took a copy of the example schema.xml file and removed all of the fields
except for 2 to start with something really basic and it is throwing the
nullpointerexception again.
I attached the file if it is of any help
I am using a fresh copy of the nightly build.
Ryan McKinley wrote:
>>
>> Would it help if I posted my schema file?
>>
>
> yes
Re: Schema validator/debugger
Posted by Ryan McKinley <ry...@gmail.com>.
>
> Would it help if I posted my schema file?
>
yes
Re: Schema validator/debugger
Posted by Andrew Nagy <an...@villanova.edu>.
Andrew Nagy wrote:
> Ryan McKinley wrote:
>> If the example schema.xml works, can you try adding a little bit of
>> your schema at a time?
> Yeah, that is my last resort. I guess I have no choice!
So I did this, and I ended up removing all of my fields and copyfields
with still no luck. I took a fresh copy of the example schema, replaced
my fields and copyfields and it still the error persists.
One thing I noticed is that in the admin it goes from "Solr Admin
(example)" to "Solr Admin (unknown)" which makes me think it is having a
hard time parsing the schema file?
Would it help if I posted my schema file?
Thanks!
Andrew
Re: Schema validator/debugger
Posted by Andrew Nagy <an...@villanova.edu>.
Yonik Seeley wrote:
> On 4/11/07, Andrew Nagy <an...@villanova.edu> wrote:
>> > If the example schema.xml works, can you try adding a little bit of
>> > your schema at a time?
>> Yeah, that is my last resort. I guess I have no choice!
>
> That certainly is strange... Sounds like you definitely hit some sort
> of Solr bug, but I don't see how an error in schema parsing can be
> lost w/o it going into a log somewhere. Did you check the other
> Tomcat log files?
The only tomcat log file that has any mention of an error is catalina.out
>
> Perhaps try putting your schema.xml into the example distributed with
> Solr (jetty based) and see what happens.
I can try this. Will jetty create a log file somewhere?
Thanks
Andrew
Re: Schema validator/debugger
Posted by Yonik Seeley <yo...@apache.org>.
On 4/11/07, Andrew Nagy <an...@villanova.edu> wrote:
> > If the example schema.xml works, can you try adding a little bit of
> > your schema at a time?
> Yeah, that is my last resort. I guess I have no choice!
That certainly is strange... Sounds like you definitely hit some sort
of Solr bug, but I don't see how an error in schema parsing can be
lost w/o it going into a log somewhere. Did you check the other
Tomcat log files?
Perhaps try putting your schema.xml into the example distributed with
Solr (jetty based) and see what happens.
-Yonik
Re: Schema validator/debugger
Posted by Andrew Nagy <an...@villanova.edu>.
Ryan McKinley wrote:
> Are you using the example solrconfig.xml? The stack trace looks like
> an error finding the solr.home index directory, that is configured in
> solrconfig.xml, not schema.xml
Yeah, I noticed that too ... but I don't understand why it can't find
the home. I have the data home set to ./solr/data and I always start
tomcat from the parent directory.
>
> If the example schema.xml works, can you try adding a little bit of
> your schema at a time?
Yeah, that is my last resort. I guess I have no choice!
Thanks
Andrew
Re: Schema validator/debugger
Posted by Ryan McKinley <ry...@gmail.com>.
Are you using the example solrconfig.xml? The stack trace looks like
an error finding the solr.home index directory, that is configured in
solrconfig.xml, not schema.xml
If the example schema.xml works, can you try adding a little bit of
your schema at a time?
On 4/11/07, Andrew Nagy <an...@villanova.edu> wrote:
> Andrew Nagy wrote:
> > Ryan McKinley wrote:
> >> What errors are you getting? Are there exceptions in the log when it
> >> starts up?
> > Just a null pointer exception.
> >
> > I added a few fields to my schema, and then replaced my solr war file
> > with the latest build (see my message from a week ago). It wouldn't
> > work, so I assumed something wrong with the latest build. I then
> > reverted back to my 1.1.0 copy and am getting these errors. The XML
> > is valid, but that is all I can test.
>
> I just installed SOLR-75 patch and the "schema browser" is able to view
> the schema perfectly. When I used the default schema with SOLR,
> everything is fine, but when I replace my schema ... it's throws this
> NullPointerException ....
>
> Any ideas?
>
> Andrew
>
Re: Schema validator/debugger
Posted by Andrew Nagy <an...@villanova.edu>.
Andrew Nagy wrote:
> Ryan McKinley wrote:
>> What errors are you getting? Are there exceptions in the log when it
>> starts up?
> Just a null pointer exception.
>
> I added a few fields to my schema, and then replaced my solr war file
> with the latest build (see my message from a week ago). It wouldn't
> work, so I assumed something wrong with the latest build. I then
> reverted back to my 1.1.0 copy and am getting these errors. The XML
> is valid, but that is all I can test.
I just installed SOLR-75 patch and the "schema browser" is able to view
the schema perfectly. When I used the default schema with SOLR,
everything is fine, but when I replace my schema ... it's throws this
NullPointerException ....
Any ideas?
Andrew
Re: Schema validator/debugger
Posted by Andrew Nagy <an...@villanova.edu>.
Ryan McKinley wrote:
> What errors are you getting? Are there exceptions in the log when it
> starts up?
Just a null pointer exception.
I added a few fields to my schema, and then replaced my solr war file
with the latest build (see my message from a week ago). It wouldn't
work, so I assumed something wrong with the latest build. I then
reverted back to my 1.1.0 copy and am getting these errors. The XML is
valid, but that is all I can test.
Here is a snapshot of my logs, everything starts up fine - but then I
click on "ping" and get a server error:
INFO: Deploying web application archive solr.war
Apr 10, 2007 11:35:23 AM org.apache.solr.servlet.SolrServlet init
INFO: SolrServlet.init()
Apr 10, 2007 11:35:23 AM org.apache.solr.servlet.SolrServlet init
INFO: No /solr/home in JNDI
Apr 10, 2007 11:35:23 AM org.apache.solr.servlet.SolrServlet init
INFO: user.dir=/office/local
Apr 10, 2007 11:35:23 AM org.apache.solr.core.Config getInstanceDir
INFO: Solr home defaulted to 'solr/'
Apr 10, 2007 11:35:23 AM org.apache.solr.core.SolrConfig initConfig
INFO: Loaded SolrConfig: solrconfig.xml
Apr 10, 2007 11:35:23 AM org.apache.solr.core.RequestHandlers <init>
INFO: adding requestHandler standard=solr.StandardRequestHandler
Apr 10, 2007 11:35:23 AM org.apache.solr.core.RequestHandlers <init>
INFO: adding requestHandler dismax=solr.DisMaxRequestHandler
Apr 10, 2007 11:35:23 AM org.apache.solr.core.RequestHandlers <init>
INFO: adding requestHandler partitioned=solr.DisMaxRequestHandler
Apr 10, 2007 11:35:23 AM org.apache.solr.core.RequestHandlers <init>
INFO: adding requestHandler instock=solr.DisMaxRequestHandler
Apr 10, 2007 11:35:23 AM org.apache.solr.core.SolrCore <init>
INFO: Opening new SolrCore at solr/, dataDir=solr/data
Apr 10, 2007 11:35:23 AM org.apache.solr.schema.IndexSchema readConfig
INFO: Reading Solr Schema
Apr 10, 2007 11:35:23 AM org.apache.solr.schema.IndexSchema readConfig
INFO: Schema name=example
Apr 10, 2007 11:35:23 AM org.apache.solr.schema.IndexSchema readConfig
INFO: default search field is title
Apr 10, 2007 11:35:23 AM org.apache.solr.schema.IndexSchema readConfig
INFO: query parser default operator is OR
Apr 10, 2007 11:35:23 AM org.apache.solr.schema.IndexSchema readConfig
INFO: unique key field: id
Apr 10, 2007 11:35:23 AM org.apache.solr.servlet.SolrUpdateServlet init
INFO: SolrUpdateServlet.init() done
Apr 10, 2007 11:35:24 AM org.apache.coyote.http11.Http11BaseProtocol start
INFO: Starting Coyote HTTP/1.1 on http-8081
Apr 10, 2007 11:35:24 AM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:8009
Apr 10, 2007 11:35:24 AM org.apache.jk.server.JkMain start
INFO: Jk running ID=0 time=0/25 config=null
Apr 10, 2007 11:35:24 AM org.apache.catalina.storeconfig.StoreLoader load
INFO: Find registry server-registry.xml at classpath resource
Apr 10, 2007 11:35:24 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 2659 ms
Apr 10, 2007 11:35:40 AM org.apache.solr.core.SolrException log
SEVERE: java.lang.RuntimeException: java.lang.NullPointerException
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:363)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:259)
at
org.apache.solr.request.SolrQueryRequestBase.getSearcher(SolrQueryRequestBase.java:152)
at
org.apache.solr.request.DisMaxRequestHandler.handleRequest(DisMaxRequestHandler.java:231)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:595)
at org.apache.jsp.admin.ping_jsp._jspService(ping_jsp.java:75)
at
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:332)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:314)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:264)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.lang.NullPointerException
at java.io.File.<init>(File.java:194)
at
org.apache.lucene.store.FSDirectory.getDirectory(FSDirectory.java:149)
at
org.apache.lucene.store.FSDirectory.getDirectory(FSDirectory.java:131)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:126)
at
org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:85)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:354)
... 25 more
Re: Schema validator/debugger
Posted by Ryan McKinley <ry...@gmail.com>.
What errors are you getting? Are there exceptions in the log when it starts up?
On 4/10/07, Andrew Nagy <an...@villanova.edu> wrote:
> Does anyone have a good method of debugging a schema?
> I have been struggling to get my new schema to run for the past couple
> of days and just do not see anything wrong with the XML document.
>
> Thanks
> Andrew
>