You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by solr-user <so...@hotmail.com> on 2012/08/08 18:35:42 UTC
question re lucene spatial toolkit aka LSP aka spatial4j
Hi all. I tried posting this in the Solr users group but didnt get any
replies so thought I would try the Lucene group.
hopefully someone is using the lucene spatial toolkit aka LSP aka spatial4j,
and can answer this question
we are using this spatial tool for doing searches. overall, it seems to
work very well. however, finding documentation is difficult.
I have a couple of questions:
1. I have a geohash field in my solr schema that contains indexed geographic
polygon data. I want to find all docs where that polygon intersects a given
lat/long. I was experimenting with returning distance in the resultset and
with sorting by distance and found that the following query works. However,
I dont know what distance means in the query. i.e. is it distance from
point to the polygon centroid, to the closest outer edge of the polygon, its
a useless random value, etc. Does anyone know??
http://solrserver:solrport/solr/core0/select?q=*:*&fq={!v=$geoq%20cache=false}&geoq=wkt_search:%22Intersects(Circle(-97.057%2047.924%20d=0.000001))%22&sort=query($geoq)+asc&fl=wkt_trimmed,area_name,latitude,longitude,dist:query($geoq,-1),loc_city,loc_state
2. some of the polygons, being geographic representations, are very big (ie
state/province polygons). when solr starts processing a spatial query (like
the one above), I can see ("INFO: Building Cache [xxxxxx]") it fills in some
sort of memory cache
(org.apache.lucene.spatial.strategy.util.ShapeFieldCache) of the indexed
polygon data. We are encountering Java OOM issues when this occurs (even
when we trimmed the polygons and boosted the mem to 7GB). I know that some
of the polygons can have more than 2300 points, but heavy trimming isn't
really an option due to level of detail issues.
Can we control this caching, or the indexing of the polygons, in any way to
reduce the memory requirements?? Is the cache size dependant on the area of
the polygons being indexed or ???
any help appreciated
p.s. here is an example of the error message I get:
INFO: Building Cache [187249]
Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError:
GC ov
erhead limit exceeded
at java.io.BufferedOutputStream.<init>(Unknown Source)
at java.io.BufferedOutputStream.<init>(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown
Sou
rce)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown
Sour
ce)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source
)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.lang.Thread.run(Unknown Source)
Aug 8, 2012 9:29:00 AM org.apache.solr.common.SolrException log
SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: GC
overhead
limit exceeded
at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilt
er.java:456)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte
r.java:284)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:3
99)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.jav
a:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:1
82)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:7
66)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHand
lerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.
java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:1
52)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:54
2)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpCo
nnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.
java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.j
ava:582)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at
com.vividsolutions.jts.geom.GeometryFactory.createPoint(GeometryFacto
ry.java:285)
at
com.vividsolutions.jts.geom.GeometryFactory.createPoint(GeometryFacto
ry.java:277)
at
com.googlecode.lucene.spatial.base.shape.JtsEnvelope.getCenter(JtsEnv
elope.java:193)
at
org.apache.lucene.spatial.strategy.prefix.PointPrefixTreeFieldCachePr
ovider.readShape(PointPrefixTreeFieldCacheProvider.java:41)
at
org.apache.lucene.spatial.strategy.prefix.PointPrefixTreeFieldCachePr
ovider.readShape(PointPrefixTreeFieldCacheProvider.java:26)
at
org.apache.lucene.spatial.strategy.util.ShapeFieldCacheProvider.getCa
che(ShapeFieldCacheProvider.java:64)
at
org.apache.lucene.spatial.strategy.util.CachedDistanceValueSource.get
Values(CachedDistanceValueSource.java:56)
at
org.apache.lucene.queries.function.FunctionQuery$AllScorer.<init>(Fun
ctionQuery.java:120)
at
org.apache.lucene.queries.function.FunctionQuery$FunctionWeight.score
r(FunctionQuery.java:95)
at
org.apache.lucene.search.FilteredQuery$1.scorer(FilteredQuery.java:15
6)
at
org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQue
ry.java:318)
at
org.apache.solr.search.FilterImpl$FilterSet.iterator(SolrIndexSearche
r.java:2218)
at
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(Con
stantScoreQuery.java:133)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:506)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:279)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearch
er.java:1394)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearche
r.java:1269)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.jav
a:384)
at
org.apache.solr.handler.component.QueryComponent.process(QueryCompone
nt.java:420)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(Sea
rchHandler.java:204)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandl
erBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1550)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter
.java:442)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte
r.java:263)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:3
99)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.jav
a:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:1
82)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:7
66)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHand
lerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.
java:114)
--
View this message in context: http://lucene.472066.n3.nabble.com/question-re-lucene-spatial-toolkit-aka-LSP-aka-spatial4j-tp3999889.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: question re lucene spatial toolkit aka LSP aka spatial4j
Posted by "David Smiley (@MITRE.org)" <DS...@mitre.org>.
I responded to the solr-user thread.
-----
Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
--
View this message in context: http://lucene.472066.n3.nabble.com/question-re-lucene-spatial-toolkit-aka-LSP-aka-spatial4j-tp3999889p4000025.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org