You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Peter A. Kirk" <pk...@alpha-solutions.dk> on 2010/09/14 01:00:42 UTC
Field names
Hi
is it possible to issue a query to solr, to get a list which contains all the field names in the index?
What about to get a list of the freqency of individual words in each field?
thanks,
Peter
Re: solr.DateField: org.apache.solr.common.SolrException: Error while
creating field
Posted by Erick Erickson <er...@gmail.com>.
If you're using Javas SimpleDateFormat, try enclosing your Z in the format
string with single quotes, like:
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'");
HTH
Erick
On Tue, Sep 14, 2010 at 4:54 PM, h00kpublic@gmail.com <
h00kpublic@googlemail.com> wrote:
> hi... i am using solr for indexing local files (solrj) and indexing
> crawled nutch-documents...
> i have configured the fieldtype
>
> <fieldType name="date" class="solr.DateField" omitNorms="true"/>
>
> and use this type by field
>
> <field name="metadata_last_modified" type="date" indexed="true"
> stored="true"/>
>
> the pattern for date is how described in DateField.java:
>
> yyyy-MM-dd'T'HH:mm:ssZ
>
> i need this date for sorting my search results... but i always get the
> exception below...
> can someone tell me whats wrong or give an alternative resolution?
>
> best regards marcel :)
>
> SEVERE: org.apache.solr.common.SolrException: Error while creating field
> 'metadata_last_modified{type=date,properties=indexed,stored,omitNorms}' from
> value '2010-09-14T22:29:24+0200'
> at org.apache.solr.schema.FieldType.createField(FieldType.java:242)
> at org.apache.solr.schema.SchemaField.createField(SchemaField.java:94)
> at
> org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:204)
> at
> org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:277)
> at
> org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:60)
> at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:139)
> at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:69)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1323)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:337)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:240)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:842)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> Caused by: org.apache.solr.common.SolrException: Invalid Date
> String:'2010-09-14T22:29:24+0200'
> at org.apache.solr.schema.DateField.parseMath(DateField.java:166)
> at org.apache.solr.schema.DateField.toInternal(DateField.java:136)
> at org.apache.solr.schema.FieldType.createField(FieldType.java:240)
> ... 28 more
>
>
RE: Re: solr.DateField: org.apache.solr.common.SolrException: Error while creating field
Posted by Dennis Gearon <ge...@sbcglobal.net>.
Can you give us a scencario:
1/ Like a OOP sequence diagram, "Thishappens, that happens, now that"
2/ Where you see it useful?
Isn't it possible to convert before storing/after retrieving?
Couldn't a timezone offset (or .... local timezone designation) be stored as a separate field to filter/sort on?
Dennis Gearon
Signature Warning
----------------
EARTH has a Right To Life,
otherwise we all die.
Read 'Hot, Flat, and Crowded'
Laugh at http://www.yert.com/film.php
--- On Tue, 9/14/10, Markus Jelsma <ma...@buyways.nl> wrote:
> From: Markus Jelsma <ma...@buyways.nl>
> Subject: RE: Re: solr.DateField: org.apache.solr.common.SolrException: Error while creating field
> To: solr-user@lucene.apache.org
> Date: Tuesday, September 14, 2010, 2:18 PM
> It would be a nice feature if Solr
> supports queries with time zone support on an index where
> all times are UTC. There is some chatter about this in
> SOLR-750 but i haven't found an issue that would add support
> for time zone queries.
>
>
>
> Did i do a lousy search or is the issue missing as of yet?
>
> -----Original message-----
> From: Yonik Seeley <yo...@lucidimagination.com>
> Sent: Tue 14-09-2010 22:58
> To: solr-user@lucene.apache.org;
>
> Subject: Re: solr.DateField:
> org.apache.solr.common.SolrException: Error while creating
> field
>
> On Tue, Sep 14, 2010 at 4:54 PM, h00kpublic@gmail.com
> <h0...@googlemail.com>
> wrote:
> > SEVERE: org.apache.solr.common.SolrException: Error
> while creating field
> >
> 'metadata_last_modified{type=date,properties=indexed,stored,omitNorms}'
> from
> > value '2010-09-14T22:29:24+0200'
>
> Different timezones are currently not allowed - you must
> UTC (hence
> the "Z" timecode).
>
> -Yonik
> http://lucenerevolution.org Lucene/Solr Conference,
> Boston Oct 7-8
>
RE: Re: solr.DateField: org.apache.solr.common.SolrException: Error
while creating field
Posted by Markus Jelsma <ma...@buyways.nl>.
It would be a nice feature if Solr supports queries with time zone support on an index where all times are UTC. There is some chatter about this in SOLR-750 but i haven't found an issue that would add support for time zone queries.
Did i do a lousy search or is the issue missing as of yet?
-----Original message-----
From: Yonik Seeley <yo...@lucidimagination.com>
Sent: Tue 14-09-2010 22:58
To: solr-user@lucene.apache.org;
Subject: Re: solr.DateField: org.apache.solr.common.SolrException: Error while creating field
On Tue, Sep 14, 2010 at 4:54 PM, h00kpublic@gmail.com
<h0...@googlemail.com> wrote:
> SEVERE: org.apache.solr.common.SolrException: Error while creating field
> 'metadata_last_modified{type=date,properties=indexed,stored,omitNorms}' from
> value '2010-09-14T22:29:24+0200'
Different timezones are currently not allowed - you must UTC (hence
the "Z" timecode).
-Yonik
http://lucenerevolution.org Lucene/Solr Conference, Boston Oct 7-8
Re: solr.DateField: org.apache.solr.common.SolrException: Error while
creating field
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Tue, Sep 14, 2010 at 4:54 PM, h00kpublic@gmail.com
<h0...@googlemail.com> wrote:
> SEVERE: org.apache.solr.common.SolrException: Error while creating field
> 'metadata_last_modified{type=date,properties=indexed,stored,omitNorms}' from
> value '2010-09-14T22:29:24+0200'
Different timezones are currently not allowed - you must UTC (hence
the "Z" timecode).
-Yonik
http://lucenerevolution.org Lucene/Solr Conference, Boston Oct 7-8
solr.DateField: org.apache.solr.common.SolrException: Error while
creating field
Posted by "h00kpublic@gmail.com" <h0...@googlemail.com>.
hi... i am using solr for indexing local files (solrj) and indexing
crawled nutch-documents...
i have configured the fieldtype
<fieldType name="date" class="solr.DateField" omitNorms="true"/>
and use this type by field
<field name="metadata_last_modified" type="date" indexed="true"
stored="true"/>
the pattern for date is how described in DateField.java:
yyyy-MM-dd'T'HH:mm:ssZ
i need this date for sorting my search results... but i always get the
exception below...
can someone tell me whats wrong or give an alternative resolution?
best regards marcel :)
SEVERE: org.apache.solr.common.SolrException: Error while creating field
'metadata_last_modified{type=date,properties=indexed,stored,omitNorms}'
from value '2010-09-14T22:29:24+0200'
at org.apache.solr.schema.FieldType.createField(FieldType.java:242)
at org.apache.solr.schema.SchemaField.createField(SchemaField.java:94)
at
org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:204)
at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:277)
at
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:60)
at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:139)
at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:69)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1323)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:337)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:240)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:842)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: org.apache.solr.common.SolrException: Invalid Date
String:'2010-09-14T22:29:24+0200'
at org.apache.solr.schema.DateField.parseMath(DateField.java:166)
at org.apache.solr.schema.DateField.toInternal(DateField.java:136)
at org.apache.solr.schema.FieldType.createField(FieldType.java:240)
... 28 more
RE: Field names
Posted by "Peter A. Kirk" <pk...@alpha-solutions.dk>.
From: Simon Willnauer [simon.willnauer@googlemail.com]
Sent: Tuesday, 14 September 2010 17:47
To: solr-user@lucene.apache.org
Subject: Re: Field names
>On Tue, Sep 14, 2010 at 1:39 AM, Peter A. Kirk <pk...@alpha-solutions.dk> wrote:
>> <result name="response" numFound="9" start="0">
>> <doc>
>>
>> So it only finds 9?
>
>Since the "gb" term says 18 occurrences throughout the index I suspect
>you updated you docs once without optimizing or indexing a lot of docs
>so that segments are merged. Try to call optimize if you can afford it
>and see if the doc-freq count goes back to 9
Yes - thanks!
Re: Field names
Posted by Simon Willnauer <si...@googlemail.com>.
On Tue, Sep 14, 2010 at 1:39 AM, Peter A. Kirk <pk...@alpha-solutions.dk> wrote:
> Fantastic - that is exactly what I was looking for!
>
> But here is one thing I don't undertstand:
>
> If I call the url:
> http://localhost:8983/solr/admin/luke?numTerms=10&fl=name
>
> Some of the result looks like:
>
> <lst name="fields">
> <lst name="name">
> <lst name="topTerms">
> <int name="gb">18</int>
>
> Does this mean that the term "gb" occurs 18 times in the name field?
Yes that is the Doc Frequency of the term "gb". Remember that deleted
/ updated documents and their terms contribute to the doc frequency
until they are expunged from the index. That either happens through a
segment merge in the background or due to an explicit call to
optimize.
>
> Because if I issue this search:
> http://localhost:8983/solr/select/?q=name:gb
>
> I get results like:
> <result name="response" numFound="9" start="0">
> <doc>
>
> So it only finds 9?
Since the "gb" term says 18 occurrences throughout the index I suspect
you updated you docs once without optimizing or indexing a lot of docs
so that segments are merged. Try to call optimize if you can afford it
and see if the doc-freq count goes back to 9
simon
>
> What do the above results actually tell me?
>
> Thanks,
> Peter
>
> ________________________________________
> From: Ryan McKinley [ryantxu@gmail.com]
> Sent: Tuesday, 14 September 2010 11:30
> To: solr-user@lucene.apache.org
> Subject: Re: Field names
>
> check:
> http://wiki.apache.org/solr/LukeRequestHandler
>
>
>
> On Mon, Sep 13, 2010 at 7:00 PM, Peter A. Kirk <pk...@alpha-solutions.dk> wrote:
>> Hi
>>
>> is it possible to issue a query to solr, to get a list which contains all the field names in the index?
>>
>> What about to get a list of the freqency of individual words in each field?
>>
>> thanks,
>> Peter
>>
RE: Field names
Posted by "Peter A. Kirk" <pk...@alpha-solutions.dk>.
Fantastic - that is exactly what I was looking for!
But here is one thing I don't undertstand:
If I call the url:
http://localhost:8983/solr/admin/luke?numTerms=10&fl=name
Some of the result looks like:
<lst name="fields">
<lst name="name">
<lst name="topTerms">
<int name="gb">18</int>
Does this mean that the term "gb" occurs 18 times in the name field?
Because if I issue this search:
http://localhost:8983/solr/select/?q=name:gb
I get results like:
<result name="response" numFound="9" start="0">
<doc>
So it only finds 9?
What do the above results actually tell me?
Thanks,
Peter
________________________________________
From: Ryan McKinley [ryantxu@gmail.com]
Sent: Tuesday, 14 September 2010 11:30
To: solr-user@lucene.apache.org
Subject: Re: Field names
check:
http://wiki.apache.org/solr/LukeRequestHandler
On Mon, Sep 13, 2010 at 7:00 PM, Peter A. Kirk <pk...@alpha-solutions.dk> wrote:
> Hi
>
> is it possible to issue a query to solr, to get a list which contains all the field names in the index?
>
> What about to get a list of the freqency of individual words in each field?
>
> thanks,
> Peter
>
Re: Field names
Posted by Ryan McKinley <ry...@gmail.com>.
check:
http://wiki.apache.org/solr/LukeRequestHandler
On Mon, Sep 13, 2010 at 7:00 PM, Peter A. Kirk <pk...@alpha-solutions.dk> wrote:
> Hi
>
> is it possible to issue a query to solr, to get a list which contains all the field names in the index?
>
> What about to get a list of the freqency of individual words in each field?
>
> thanks,
> Peter
>